GitLab Incubation Engineering, 11 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IncEng MLOps Update - 2022-07-11

Description

Strong growth in Jupyter Notebook diffs in June, 8 MRs hitting 15.2, MLFlow Vision

MLFlow Integration Vision: https://www.youtube.com/watch?v=V4hos3VFeC4
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

A

Hello, everyone and welcome to another envelopes update for occupational engineering. My name is eduardo and today is july 11th.

A

um Well, let's kick start by talking about usage, uh so I've been working forevers on uh triple notebook, tiffs and the usage continues to grow wildly made when june 22nd was the peak of uh of comments left on jupiter's book devs and it was 25 larger than the last peak in march, and it was three times more than in june last year, so from june last year to june this year the growth was above 200, and this is quite insane, so people are really uh warming up to this feature.

A

uh It's uh it's really nice to see this happening progress right now. So for 15.2 we are releasing a total of eight, mrs along the the distribution and, while most of them are bug fixes, but notably we are moving. The entire code base that we had for generating the divs into the main gitlab code base in a way that um it's not forgotten somewhere in some lost repo. No every change now goes through the regular uh review process.

A

We guarantee the same quality and so on so forth, um and the most notable thing that we did on this uh this release is that generating the notebook devs now is 25. Is 12 percent faster and takes eight times less memory. So what you make a notebook used to take, I don't know 120 megabytes of memory, and now it's taking 17 megabytes of memory, it's quite insane the difference.

A

We achieved that by leveraging by collaborating with og and open source library, we collaborated with the maintainers to implement some changes, um and then we moved ever most of the heavy stuff into uh ruby extension. So most of the stuff is now running on c and it's it's a lot better right. Now it's much faster uh and you use a lot less memory. It was one of the problems we were having with uh notebook devs.

A

Next we released a video. uh I released a video on explaining how I want to go about integrating gitlab animal flow. How do I want to deploy this, and how do I want to release this? So just explaining my goals with this exploration, my vision um to get feedback? To put the word out there that we're doing this and the reception was great already. Actually um I had customers uh sending me email about this uh asking when is this beginning or whether or not uh the views are quite nice.

A

um So I'm quite happy with this. It's actually uh one of the fastest videos that I got feedback on. So that's very interesting. People are actually excited about this one. uh The video is eight minutes. I put a link over there. uh Well, you can watch uh if you, if you have the time to do so and for the next steps we are on euro python between doubling between july 13th and 16th.

A

So if you were there and you want to talk about gitlab, you want to talk about mlflow, you want to talk about anything or grab a beer. Reach me out I'll, be probably the guy walking around with a kit lab t-shirt so easy to spot.

A

Furthermore, after that we have one leftover issue for drip player def, which is increasing the size of the diffs. We already already spoke with the stakeholders.

A

We already have a plan on how to do this in a way that doesn't uh create problems later on, and so this is going to be the next, uh mr, that I've been working on and along that get our hands dirty with ammo flow. We already been speaking too much and it's time to actually write code for this and that's what I had for today.

A

So if you have any feedback, you want to watch the previous updates, the link for the the link for the for all my issues, all my updates is on the description below. Thank you very much and have a great day.