GitLab MLOps Open Meetings

8 Sep 2023

This talk will be given in person at the OSS Summit 2023

What is MlOps, and why is it a thing in the first place?
What are the unique challenges faced when developing features powered by Machine Learning?
How is GitLab building the MLOps Platform?
What is LLMOps

1 participant
31 minutes

mlops

mlopsis

devops

mlml

mlflow

developing

explain

management

versioning

envelops

21 Jul 2023

Current status on the development of GitLab Model Registry: where are we and where do we want to go.

- All Updates: https://gitlab.com/groups/gitlab-org/-/epics/9423#note_1437373082
- Epic: https://gitlab.com/groups/gitlab-org/-/epics/9423

1 participant
10 minutes

model

models

registry

motor

tooling

gitlab

versioning

candidate

mlflow

relative

1 May 2023

An exciting update with the release of Model experiments to all users, and the next step on the MlOps @Gitlab journey: Model Registry!

Demo: https://youtu.be/uxweU4zT40c
Documentation: https://docs.gitlab.com/ee/user/project/ml/experiment_tracking/#machine-learning-model-experiments
Feedback: https://gitlab.com/groups/gitlab-org/-/epics/9341
Handbook: https://about.gitlab.com/handbook/engineering/incubation/mlops/

1 participant
9 minutes

tracked

experiment

machine

models

mlflow

manage

implement

mode

demo

gitlab

1 May 2023

A demo of the recently released features Model experiments and MlFlow integration on GitLab

Documentation: https://docs.gitlab.com/ee/user/project/ml/experiment_tracking/#machine-learning-model-experiments
Training Code: https://gitlab.com/gitlab-org/incubation-engineering/mlops/mlflow_experiment
Feedback: https://gitlab.com/groups/gitlab-org/-/epics/9341

1 participant
4 minutes

gitlab

demo

experiments

logged

machine

model

ammo

flow

mflo

backend

22 Mar 2023

- What is experiment tracking?
- Demo of the feature and user experience at 15.10
- What's next for 15.11

Epic: https://gitlab.com/groups/gitlab-org/-/epics/9341
Feedback Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/381660
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
11 minutes

mlflow

gitlab

experiment

model

tracking

lab

repository

machine

metadata

evolving

6 Feb 2023

A demo of the implemented features for Gitlab Model Experiment Tracking and the MLFlow integration.

Epic: https://gitlab.com/groups/gitlab-org/-/epics/9341

1 participant
3 minutes

gitlab

model

ammo

user

tutorial

runs

project

flow

pico

tracking

6 Feb 2023

Iterating on UX Review, efforts on Incubation foundations, starting closed testing

- Showcase: https://www.youtube.com/watch?v=E8GhtXXm_O0
- Feedback Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/381660
- This Update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/67
- All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
4 minutes

incubation

model

implementing

experiment

tracking

interactions

gitlab

qualitative

experience

ml

2 Feb 2023

Epic: https://gitlab.com/groups/gitlab-org/-/epics/9341
Feedback Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/381660
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
13 minutes

experiment

experiments

replicate

mlops

tracking

implementing

hyper

machine

incubating

kitlab

17 Jan 2023

Epic: https://gitlab.com/groups/gitlab-org/-/epics/9341
Feedback Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/381660
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
9 minutes

experiment

experiments

incubation

tracking

implementing

gitlab

model

replicate

demo

machine

20 Dec 2022

Jupyter Diffs Stats and progress on MLFLow Integration

- Request for testing: https://forms.gle/pALYkCFbz9jZmfXU7
- Feedback Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/381660
- This Update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/64
- All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
3 minutes

mlflow

mlops

tracking

gitlab

jupiter

features

filtering

machine

integration

runs

18 Nov 2022

New Features, MVP is complete, priority now is kickstarting testing and early user support

- Request for testing: https://forms.gle/pALYkCFbz9jZmfXU7
- Feedback Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/381660
- This Update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/63
- All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
4 minutes

testers

gitlab

experiments

incubation

features

improving

machine

watching

managed

docs

7 Nov 2022

GitLab ML Experiment Tracking SSoT: https://gitlab.com/groups/gitlab-org/-/epics/8560
Feedback Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/381660
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/meta/-/issues/16

1 participant
2 minutes

mlops

gitlab

tracking

machine

experience

metadata

important

platform

mlap

iterating

10 Oct 2022

ML Experiment Tracking Early Demo

GitLab ML Experiment Tracking SSoT: https://gitlab.com/groups/gitlab-org/-/epics/8560
Frontend Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/370480
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
3 minutes

experimenter

demoflow

mlflow

demo

model

machine

gitlab

circulation

experience

user

15 Sep 2022

Further progress on MLFlow. Expect internal testing in two weeks, external testing in a month

GitLab ML Experiment Tracking SSoT: https://gitlab.com/groups/gitlab-org/-/epics/8560
Api Compatibility Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/370478
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
1 minute

ammo

mlflow

flow

incubation

endpoints

testing

model

mlaps

run

gitlab

29 Aug 2022

ML Experiment Tracking: What it is, what we are not doing, what we are doing and current progress

GitLab ML Experiment Tracking SSoT: https://gitlab.com/groups/gitlab-org/-/epics/8560
This update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/61
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
4 minutes

experiment

mflow

gitlab

tracking

machine

managed

mfo

demo

lab

emflo

12 Aug 2022

Kickstarting ML Experiment Tracking

ML Experiment Tracking Epic: https://gitlab.com/groups/gitlab-org/-/epics/8560
This update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/60
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
2 minutes

gitlab

machine

mvp

tracking

mflow

experiments

integrates

updates

implement

mlaps

28 Jul 2022

MLOps JTBD and Started work on MLFlow

MLOps JTBD: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/58
gitlab-mlflow: https://gitlab.com/gitlab-org/incubation-engineering/mlops/gitlab-mlflow
This update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/59
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
3 minutes

mlops

gitlab

iterate

july

machine

expected

incubation

mlaps

improving

planning

11 Jul 2022

Strong growth in Jupyter Notebook diffs in June, 8 MRs hitting 15.2, MLFlow Vision

MLFlow Integration Vision: https://www.youtube.com/watch?v=V4hos3VFeC4
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
4 minutes

updates

gitlab

notebook

process

devs

increasing

releasing

mlflow

collaborating

megabytes

5 Jul 2022

As we gear up to start exploring an integration between MLFlow and GitLab, we want to talk about why we are doing this in the first place, and where do we want to go

Epic for feedback: https://gitlab.com/groups/gitlab-org/incubation-engineering/mlops/-/epics/9

1 participant
8 minutes

mflow

registry

mflo

myproject

mlflow

ammo

machine

docker

model

manage

23 Jun 2022

New data from ServicePing, Consolidation of Jupyter Notebook Diffs, Current Challenges

All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
4 minutes

improvements

gitlab

data

managed

commits

users

installations

maintainers

benchmarking

bing

2 Jun 2022

Continuous growth on comments for Notebook Diffs, Rendering Images on notebook diffs, Next Steps

All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
4 minutes

notebooks

gitlab

features

updates

jupiter

diffing

version

latest

important

rendering

11 May 2022

Notebooks seem to have caused and acceleration on the growth of comments on Notebooks, Finding on MLFlow, general blockers

All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16
This Update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/54

1 participant
5 minutes

gitlab

managed

project

repository

ml

machine

registry

users

flow

incubation

22 Apr 2022

Usage, progress and Backlog update

All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16
This Update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/52

1 participant
4 minutes

improvements

gitlab

devops

incubation

process

backlog

collaboration

repository

important

notebooks

30 Mar 2022

This week we talk more about pipelines, what we have worked on so far and why they are so important for MLOps.

Pipelines with Stubbed Jobs: https://gitlab.com/groups/gitlab-org/-/epics/7681
Citer: https://gitlab.com/gitlab-org/incubation-engineering/mlops/poc/citer
Glyter: https://gitlab.com/gitlab-org/incubation-engineering/mlops/glyter/-/tree/poc/glyter

All updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16
This Update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/48

1 participant
19 minutes

pipeline

scientists

gitlab

project

workflows

incubation

process

development

documentation

melops

10 Mar 2022

Let's check what we we have accomplished so far on our four objectives.

- Toggleable Notebook Diffs on Commit page
- Citer
- Planning for Pipeline with Stubbed Jobs
- Hands On Machine Learning Bookclub

Text version: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/47

1 participant
11 minutes

gitlab

mlms

objectives

lab

asylum

understanding

sag

repository

iterating

notebooks

21 Feb 2022

Today we talk about Citer, a POC we are working on to make the experience of iterating on pipelines within GitLab faster

1 participant
11 minutes

pipelines

git

process

pipeline

repository

iterating

execution

commit

ci

production

3 Feb 2022

Today we show a preview of the new feature to swap between rendered and raw diffs on the commit page, which will soon be merge and also added to the MR page.

All updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16
This Update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/44

1 participant
2 minutes

version

markdown

diffs

sag

commit

transformed

render

comments

clear

feature

20 Jan 2022

- Text Version and Links: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/43
- All updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
7 minutes

envelopes

gitlab

updates

diffs

implemented

push

analytics

project

incubation

important

16 Jan 2022

Current state of MLFLow Integration

- Request for testing: https://forms.gle/pALYkCFbz9jZmfXU7
- Feedback Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/381660
- This Update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/65
- All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
4 minutes

tracking

experiments

mlops

mlflow

gitlab

data

jupiter

mflo

implement

help

5 Jan 2022

Happy new year! In the first update of the year, we are taking the time not to talk about what we did, but what are the things we working on and where do we want to get with them.

Update Issue: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/42

User Personas: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/40

Rendered Notebook Diffs: https://gitlab.com/groups/gitlab-org/-/epics/7194

Glyter: https://gitlab.com/gitlab-org/incubation-engineering/mlops/glyter/-/tree/poc

Knowledge Repo: https://github.com/airbnb/knowledge-repo

1 participant
20 minutes

research

gitlab

prototyping

mission

representation

future

mellops

pipelines

improvements

software

24 Dec 2021

- User Personas
- Progress on Jupyter Notebook Diffs

All updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
5 minutes

users

personas

research

gitlab

mlaps

important

implementing

kickstarter

production

models

14 Dec 2021

This week: Running Jupyter Notebooks as Gitlab Pipelines,

Feedback and Ideas for Jupyter Support: https://gitlab.com/gitlab-org/gitlab/-/issues/343024

Feedback and Ideas for Pipeline Experimentation: https://gitlab.com/groups/gitlab-org/incubation-engineering/mlops/-/epics/6

All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

1 participant
10 minutes

glitter

gitlab

implementing

jupiter

scientist

tooling

improvements

sag

ml

production

2 Dec 2021

Part 2 of Exploring GitLab Pipelines for Hyperparameter Optimization

Last time, we created a very simple Hyperparemeter Optimization pipeline, that runs the trials sequentially. However, that becomes quickly impractical, since in real world use cases training a single model can take hours, days or even weeks. This week, let's explore how to run those trials in parallel using Dynamic Parent-Child pipelines!

Code: https://gitlab.com/gitlab-org/incubation-engineering/mlops/hyperparameter-tuning-exploration/-/tree/part_2_running_trials_in_parallel
Exploring GitLab Pipelines for Hyperparameter Optimization Series:
https://gitlab.com/groups/gitlab-org/incubation-engineering/mlops/-/epics/6

1 participant
13 minutes

hyperparameter

optimization

hyper

intensive

pipeline

emelops

gitlab

minutes

machine

editors

26 Nov 2021

This time, we explore how to create a very simple pipeline to perform Hyperparameter Optimization, using SKLearn's GridSearchCV and posting the results to the MR.

Codebase: https://gitlab.com/gitlab-org/incubation-engineering/mlops/hyperparameter-tuning-exploration

Exploring Hyperparameter Optimization with GitLab: https://gitlab.com/groups/gitlab-org/incubation-engineering/mlops/-/epics/6

1 participant
15 minutes

hyper

gitlab

optimization

pipeline

hyperparameter

model

lifecycle

synthetic

machine

parameter

22 Nov 2021

We are starting a series of videos exploring wether GitLab Pipelines can be used for Data Science use cases. To kick this off, we go through what is Hyperparameter Optimization.

Hyperparameter Optimization with GitLab Epic: https://gitlab.com/groups/gitlab-org/incubation-engineering/mlops/-/epics/6

Incubation Engineering MLOps Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16