GitLab Incubation Engineering, 29 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IncEng MLOps Update - 2022-08-29

Description

ML Experiment Tracking: What it is, what we are not doing, what we are doing and current progress

GitLab ML Experiment Tracking SSoT: https://gitlab.com/groups/gitlab-org/-/epics/8560
This update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/61
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

A

Hello, everyone and welcome to another update for occupation engineering envelopes and today we're going to talk about machine learning, experiment tracking. Let me first go and explain exactly what is machine learning experience tracking before we move on. So when a data scientist creates a machine learning model, there are many things that can influence the final performance, some of them, including parameters, the algorithm being used for learning. The algorithm itself, is a hyper parameter that we called, or even the data used and over the progress of the project.

A

It's common for data scientists to lose track of all the possible configurations that they used for a specific trial that they had so they create a model, but they don't know how to replicate that because they lost track of what was the specific configuration they had. So experiment tracking is all about this.

A

It's about helping users have open data scientists to keep to log the all the parameters used for a specific trial or for a specific candidate and an experiment is a set of comparable candidates that they have uh an experiment can be many things, uh but in gitlab it could be a merge request. It could be a commit, a commit that will trigger an experiment or just they explore like a data scientist exploring on their local environment as well, so the biggest alternative, the the most common solution for this right now is ammo flow.

A

I have spoken about this dim already, so it's a common uh open source tool uh very, very popular, and we want to help users that already use mflow to use gitlab as well. So let me just start by saying what we are not doing. We are not packaging emma flow inside gitlab.

A

This was discussed as a potential solution, but it wouldn't lead to the best user experience for our self-managed customers and the complexity of deploying on the way gitlab does would not be worth it. So what we are doing is that we are implementing the tracking server native to git lab. So when you go to your project project page, you will find an a and a tab for experiments and over there you can track your experience, so I'm employing. Why is git laby?

A

It's git lab expert tracking, but we are providing full support for emflo rest api. So if you use mflow client, you just need to swap the tracking your eye to gitlab and that's it that's all you need to do to use gitlab. Instead, um we on that note progress. We are ready. We have now nmr for the three first endpoints uh create experiment, get experiment. I gotta experiment, my name, we're focusing on demo flow compatibility right now and we automated the documentation for the difference.

A

So here you have the same uh request being called both on gitlab and mflow and the difference between the results. So you know what you expect when you switch between one and another up next implement the remaining eight endpoints uh to make this script work. This is our mvp make this script uh work. This is example trip of mfo track. They use mfo client to track uh an experiment.

A

There are eight remaining uh endpoints to do so and, furthermore, the front end necessary and the documentation we are on track for 15.5.

A

If, okay in case you have any feedback you want to discuss, you want to know where we are going. The this epic 8560 is our source of truth uh and we will post a link on below on this video. uh This is where discussion will happen and where we will post all directions and updates that was it for today. Thank you very much.