GitLab MLOps, 17 Sep 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Incubation Engineering MLOps -- Weekly Demo September 17h 2021

Description

https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16

A

Good afternoon, everyone, my name is eduardo, and this is the update for the incubation engineering envelopes for september 17th, um two things on the last week. First of all, first one uh we finished well not finished, but we are. We made really good progress with our markdown converter.

A

So uh right now our converter generates a really clean, uh markdown version of the notebook that we can already use to um to do differing, and this is what we're going to use soon. I want to start integrating in the beginning, even though images are embedded into the marketer itself, uh the code review uh viewer will not display them. Yet uh this will come eventually, but at least we'll be able to code with you already python code and uh errors and yeah markdown.

A

This will be much easier with this new version, um so this is on the ipython ipad b to markdown uh repository. uh If anybody is interested to look at that, it's a go. It's a very small global library uh that we are now be integrated into gitlab itself for code review um yeah. So that would be the next step and the second thing that we did.

A

uh We don't want to implement everything from a lot of the one or we can't there's way too much to implement on, but we do want gitlab to play well uh with most envelope solutions out there, so we want that it work well with whatever tool our user.

A

Our users are already already have on their architecture, so we want to look at them and see okay. How can gitlab do the best job along them and to be able a little bit to kind of rank some candidates, so we could explore uh integrations uh what I did. I looked at the top, uh so I looked at the top.

A

Open wrappers on github and I just used the api. I picked up the stars to look a little bit, what the machine learning landscape look like, and the thing is surprisingly, I then went here and ranked them early, so whether it's a framework, whether it's some learning resources.

A

So let me just uh on future over here so, for example, there's a lot of different classifications, but most of it out of the top 200 uh rebels.

A

I think it's almost percent, like forty percent, are learning resources so either tutorials or uh uh rebels that collect links uh about other resources and they are not really useful for us. But what we are looking here is that the frameworks uh at the uh repo's that do with mlabs and the repos that deal with autumnal. So these are the things that we need to to look at and see. How can we better support them? For example, for as expected, the top one is tensorflow.

A

There's a lot to be done over here, so provide a quick view into a tensorboard, for example. It's something that we could do within gitlab uh during the pipeline of doing ci, cd of training pipelines. So this is where at least we are ranking a short list of candidates. You should explore note that this doesn't include uh private tools, so this looks open, github repository. It doesn't include any private code base. So, for example, I'm not looking.

A

This doesn't include uh the big vendors like google cloud or aws, we'll have to look at different ways to to write them out, uh or just name them at the beginning, but I think it's a good short list. So if you look, uh for example, the ones we really care about, it should be uh applications. We don't care data, sensors, learning, no, perhaps library, monitoring, ultima yeah. So here we have like the first, the top rated, uh the top start it's ray.

A

uh Then we have battle netron, uh shrimlet, some other extremely there's more visualization, but still part of meta machine learning. I would call it um and the little down here you have ml flow, you have cube flow and, of course, the frameworks to build up one, so pi torch natural flow.

A

These are the ones that we have to give good support. Some of them are on the model creation step which is tensorflow most of the frameworks. So when you create a model, how can we give better support and the others are after the model is created. So, for example, uh how can we do model integrated demo float or model versioning from motor registry, or what can we use for prediction service as well? And how can we make sure that plays well with these different legal pieces on the machine architecture?

A

And this was it for this week um on the next week now we are happily working on the integration to uh with ipi and b2md and gitlab, so that we can actually finally have jupiter notebooks code reviews uh available to the public, and I think there will be a huge, huge milestone for us, an early big, big early in win and I'm very excited about it and we'll see each other next weekend.

A