GitLab Incubation Engineering, 20 Aug 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Incubation Engineering MLOps -- Weekly Demo August 20th 2021

Description

https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/17

A

Good evening, everyone, my name, is eduardo. I am the single engineer for ml ops at gitlab, and today I will be walking you through the weekly demo for mlaps uh for august 20th 2021.. This is our first weekly demo. uh So uh welcome to the experience.

A

I guess um I've created this issue where I'll be highlighting I'll, be posting every weekly demo on uh as a issue, and they are gathered uh on this larger issue uh that, uh if you want to know more about envelopes within gitlab, just subscribe to that watch that issue and we'll get updates, um moving back to what's important, um so we're starting now with the mellop's, gitlab and I'll understand a little bit. What needs to be done? How can we position ourselves?

A

How can we become the best buddies of machine learning engineers and data scientists? um How can we leverage our uh our strengths into improving user experience, uh and one thing that pops out everywhere is uh the experience with jupiter notebooks on gitlab is not on par with expectations of our users.

A

Group notebooks are key to to the workflow of both machine learning engineers and data scientists, both as developing machine learning models, but as also delivering insights within companies, and it a lot of the features of gitlab, don't really work with uh japanese books, for example, code reviews. They don't do well so this week we spent quite a lot of time on exploring what needs to be done, the current state uh of of jupiter books for to do so.

A

We uh we went and created a uh a repository uh just a second a repository uh called ipine beams, uh mr sample, and here we have uh one simple notebook which is renderly rendered quite well over here, not too much, but over here on the merge request. We explore how gitlab renders tips on the situation.

A

So, for example, if I have um if I just rerun the the notebooks, if I look at the notebook in a different order because of the file system, because the weight structure um and the format it's just like it gives up uh it's a json file internally that is rendered to the html.

A

uh So the user experience is not the past. So if you come over here and now, look at the smaller one and another one, um if you look at the changes over here now, it renders, but this doesn't look like a jupiter notebook at all.

A

This was requested many times in many places in many issues and many.

A

Entry points, but what we did uh so we summarized everything on this issue uh he over here. So this summarizes everything that we found out. We also explored github and we also explored uh reviewing b. So if reviewnb is external is a third party that renders diffs for jupiter notebooks, and this is what it should look like on gitlab.

A

If we worked on it now now, reviewing b works on top of github it attaches to github and renders diffs on github pull requests.

A

There was a an exploration, a talk with reviewing b on how to make it work on gitlab, and there were many blockers uh and this never happened, but uh so what we are doing now with pika, picked up a uh issue from three years ago.

A

um We revived it and uh we are now uh adding all the things that need to be done, all the mvcs that we need to work on uh to get this done and we'll be working on uh looking at them more thoroughly from now on. So let's go back over here uh and that's uh for dr notebook.

A

uh Additionally, what we are doing like I said we are still exploring how to position ourselves. We are doing a a survey of all the all the different vendors uh from a lapse. It's not an easy task. Every day there are three or four new vendors uh appearing. uh It's a blooming industry that is not really well defined, yet it's very uh young.

A

The definitions and the requirements are not really clear so every day, there's new ideas and things coming up. We are currently saving this findings on the internal on an internal document for now, uh because I'm adding personal notes that might not be public, uh but for gitlab uh colleagues, uh you can find the link over there so up next for uh the mlaps, we will dive deep on the jupiter notebook experience.

A

um It is clear that it is a key feature that our users expect from us, that uh that it is giving them problems and uh that we can differentiate ourselves by delivering this to them.

A

So that's it for this, this weekly demo and next we will share more of what we're doing cheers.

A