Add a meeting Rate this page

A

Teddy Nickerson hi.

B

There, hey all.

A

The things that children tell you all good old, safe, yeah.

B

Find that promise here Hanks and things with you there.

A

Exactly you are in which part of us I.

B

I'm, actually on a small island just off the coast of France, while you are and okay, okay.

A

Okay, for some reason, I thought you were in us yeah.

B

Well, I I'm in Jersey, but I mean old Jersey.

B

What does it contain? There I stopped us for in the afternoon.

A

Okay, okay, so next week, so internally I mean just click on on Christian myself. You know be a being having a lot of discussions internally with the teams, there's quite aggressively. You know the work is going on on to the tech, tour and cue flow pipeline side. We have actually engaged you know a couple of Techtron folks.

A

From the community as well alright, so let's see you know if folks don't join here, we can probably you know unless you have something we can. You know, kill the call. Let's wait for a couple of more minutes and because you know in general we are having quite a lot of meetings and I'm now going to force. You know a couple of these folks from the Google side on a subsequent call right. So that's there's what I want to do. Yeah.

B

Okay,.

A

You see the announcement from Google where they actually launched pure flow pipe lands and production onto the public cloud.

B

No I haven't seen them.

A

Okay, let me see if I have the link.

A

So the thing is, you know we haven't yet seen Tecton productized by Google, but people of my client has been.

A

So Frances not as impacted I'm on the link I'm pinging and in the check I.

A

Ho doing is it OS OVH.

C

Hey guys, this is omar, so was it just joining you guys, first time, journalist how the stake was interested in seeing what you guys are doing. Yeah.

B

Welcome.

A

I'm.

C

With apple okay,.

A

So I just ping the link in the chat right so teri for you right in general, and so all my running one of the things we do discuss. You know some of the stakeholders or not there. So there are two ways you know this. This ml object goes. One is essentially where we, you know, proceed through a technical charter which is essentially you know the the integration needed on the current CI sealy pipelines right to make them more ml also rated, and the second is what 30 leads, which is.

A

Essentially, you know the roadmap envisioned for what does ml ops mean right, so both these groups sort of work independently. A lot of the work happens. You know oxide, and this mostly becomes a melting point of you know. Getting the folks together, so I think the link which I paint was essentially.

A

If you look at it, so that's what is you know this year in general in the industry? Right have been this emergence of. You know cloud platform, pipelines right and is is going to become, you know more and more prominent right. So obviously, if you see war, Google ended up, launching is the cue flow pipelines as a service right.

A

The two key things to note in this is there you know, then the they've taken the route that you know. It's not only just the cue flow pipeline SDK, but the tf-x SDK as well, because tf-x just on its own, is not that hugely known but tensorflow, as we all know is you know a gorilla and they want to make sure you know the end. Users coming from the tensorflow side also have a way to you know, run different components of tensorflow, whether it's the data validation model analysis.

A

You know using the tf-x sdk onto this platform and over a period of time right. What will happen is you know both the cue flow pipeline sdk as well as the tf-x sdk? They are going to merge into one now. I think we have discussed quite a few times right that the pipeline's are taking us from A to B, but from ml ops perspective right. The whole notion of.

A

Lineage tracking is very important, so they have done. You know quite a bit of work in terms of you know ensuring that you know what we are showing here. It's not generic. You know CIC D pipeline, but you know, if you look at generating the graphs, etc four different steps of the pipeline, then you know the artifact and the lineage tracking. So if you can see.

A

Yeah so, like you know all the artifacts which are be getting produced as part of when the pipelines are running right, they are versioned and stored, and then you can actually you know in the end, also run through the lineage of everything being produced.

A

Let me kill this slack. It's just acting oh yeah yeah and be able to trace back right back. You know where what was the data set? You know used to train the models. You know what was a tensorflow version without put of different components right, so very, very rich, lenient, Explorer, also inbuilt now so and so I think in general, you know there is. There is a very strong adoption for this in the community and obviously for us.

A

You know the core effort is to see you know if we can get to this stage, but replace the auto engine and in the commas, with checked on right. So that's strategically, given that there is a lot of investment in Techtron from Red Hat from IBM and even Google right. So one of the things which we do want to understand is that checked on as a project. You know oriented from Google's was initially right and so wanna see you know their perspective as well on in general, the longer term.

A

This whole engine is based on checked on under the covers right doing the the native see ICD work yeah, so.

C

How does that? How does that play when you have like are going flux, we're trying to combine forces and does that affect this in any way, with similar work? That's being done in combination?

C

Sorry, what's the question, so we have like you know like in this yeah it's in a cloud native CIC space. We look, we see we have like you know we have we've flux and Argo and Tecton and a couple other tools. So following kind like they get ops flow with our go and we've they kind of combine forces they're trying to contort their work together into a singular kind of repository. There's some ongoing work with that I think there's another sig for that.

C

So what I was wondering is, if that has an effect on how the agro workflow engine operates now or if there's any, are you looking at this kind of changes or if they're, even changes worth talking on.

A

Yeah.

A

So what others say, you're saying is, is effectively discussing the inclusion. I mean I didn't so, where do Joanna's from like? What's your charter, okay, so.

C

Just a little background once also Omar, so I'm representing cloud technologies at Apple, so of Arts in technology or kind of like our chart. Let's look at multi cloud kind of strategies for CI CD, and a lot of my work is also focusing on the intelligence side of CI CD. So I saw you guys have this second hmmm ops and I was interested to see because we're also we're looking at mobile cloud for a lot of the services and operations as well force other stuff for doing so interesting.

B

To see what's.

C

Being done in that space, I guess.

B

You're, probably I'm more interested in in in the roadmap side of things at the at the moment, then perhaps we can have a quick chat about that and help to explain what we're doing the the idea really is is.

B

Whilst obviously there are a number of teams involved within the CD foundation that already have CD platforms that are existing and out there and being extended to to incorporate machine learning aspects.

B

We we feel it's really important that we actually get a solid grasp on on on what ml ops really needs to do and what the customer demand is because right right now, if you look at the way things are developing in the space they're they're being driven in a typical fashion by you know, we can do x and y. So, let's just add a bit of machine learning to that and see what happens, but things have not really been driven from full-on understanding of what the problem space is.

B

What the challenges are that the customers are facing in that area and what the longer-term roadmap is going to need to look like, especially for supporting larger customers. We're really the machine. Learning aspects need to need to be pushed right down into silicon, and you know where you're looking to develop tooling, that can then get pushed down into. You know individual components that become very cheap to deliver and easy to mass-produce.

B

So so the idea was to put together a road map to capture as many of these requirements as possible and start to paint a picture of what's possible today, what's expected in the future, and what we're gonna need to do to to get everybody aligned in delivering to to those capabilities.

B

So really, the idea is very much an open collaboration here. We're encouraging everybody to to get involved, read and contribute to the document start to to inform everyone of what the challenges are in the individual spaces that they're working in and then over time. We'll start to draw that, together into some key pieces of technology, that we need to develop collaboratively to to move everybody forwards in the right direction.

A

If you're looking at in general, you know the if you are at a point in the journey where you are sort of looking at you know, de I, don't know if, for example, have you standardized with an apple on the underlying CI CD pipelines? And then you know what you're probably looking at?

A

Is you know what you standardize on is is also the one which becomes the basis for your ml ops or the one we know which you can actually expose for their machine learning engineers and data scientists trying for machine learning needs, that's probably the intent on your side. Looking at this, yes yeah.

C

For sure that's the beginning pieces is I mean from is identifying kind of what would be kind of a best practices CD pipeline for and the thing is there's so many different use cases right. There's some machine learning. These cases there's just average migrations in digital transformation. In these cases that need to be done so a lot of these things they require various different different tool sets to be able to accomplish them. So what I'm looking is looking at kind of a future direction?

C

It doesn't require any kind of monolithic software stack, I'm jabbing at spinnaker, but it's pretty much. You know because I was able to accomplish CICE through just stitching containers together a couple years back right. It wasn't something that was incredibly complex to kind of fathom, but given the nature of how people want to do deployment patterns and different features like that, then you know you kind of have to look at what can I do in terms of intelligence to handle these kind of different patterns for deployments.

C

So then, I'm just basically running around the circles of CI CD pieces and seeing what's there and what I, what I'm really really want to tackle I mean I kind of see whether the whole pieces, but I'm really looking at, is just sharing.

C

This with you guys is that I see a lot of a lot of complexity being added to our kind of domain as we move forward, and this added complexity speaks to the amount of knowledge that needs to be consumed in order for one to become a proficient in these technologies, let's say so moving forward.

C

So the problem is that is like in my world, I see that there's enough actionable intelligence being emitted off of infrastructure pipelines, users repos that I could assemble an intelligent way to start determining how how why and when I should be making deployments into which cloud. So that's kind of the frame of where I'm, like my mind, is that right now and now, I can kind of see where cube flow as a service right.

C

Something like that would give me kind of the platform start to look at how I can achieve this kind of this end goal that I'm looking for you.

A

Know so I think couple of things right equation. One thing is, you know, is: is Apple I mean and based on you know, I've been having. You know, chat with a couple of folks within Apple as well. It seems Apple is also you know heavily standardizing on through right.

A

So there is a lot of work going on with an apple right where your underlying platform is becoming communities, and if that is, is the route I think what your thunking is is a right way if you're looking at two things, one, if you look at you, know the cube native space, which is what they call you know the technologies which essentially are born, because you know humanities has become the ubiquitous as standard. So from that perspective you know pipelines like checked Ron, Jenkins, X right.

A

They were essentially catered towards that right, so Tecton specifically is essentially as cube native as it gets. In terms of you know, everything is architected as a custom resource humanities, customer resource using cuban ADC or DS except raw. So within that particular space Tecton is becoming popular for the folks. You know who are standardizing on criminate.

A

He says then underlying platform and, although you know the pre-existing pipelines which are very popular like air flow in the machine learning space air flow or you know, Apache being except raw, they are being made to work on top of Cuban IDs.

A

No, but it's not the same, as you know, when you look at the right architectural way of putting something on Cuban, IDs I think checked on has that advantage, because you know it rose after the Cuban eighties prominence now, if you're looking at a machine learning space on form of communities, so that's where yes, definitely chill flow is getting a lot of adoption and our charter specifically within this, so I came to dock.

A

In the mean you know what we have been looking at technically has been so like electron is you know for us sort of like now becoming you know the standardized ICD pipeline, so both I mean public cloud DevOps services as well as open. If standardized on Tecton as the underlying ICD pipeline. Now for the machine learning pipeline, you made a decision to go with queue, flow pipelines right.

A

So there was a lot of back-and-forth and discussion between airflow is your flow pipelines versus life I in team and again by virtue of us being a cube native company. At this point where everything is running on top of humanity's, the decision has been to bhowmick through flow pipelines. But what we want to do is, you know, not: consume cure flow pipelines, as is, but essentially so. This is, you know the current stage where cure flow pipeline is and what we essentially want to do.

A

Is we basically flow pipeline, some form of Tecton right, so that, essentially, you know ensures that you know the underlying standardization around Tecton is the CICE pipeline is consumed, reused right. So that's the word. One of the you know work which we are driving as part of this say, guess. Well we're. Essentially you know. How do we take this? So we have started. You know, obviously the first phase, which we are almost getting completed, Paul's around. You know the compilation, so you can take.

A

You know the cue flow pipeline SDK, which is exposed to the data scientist and machine learning, ingenious and compile back to a tech, town output right and then there are other phases so I think, depending on from where you are coming from, you know the first decision always becomes. There is not standardization around see ICD pipeline, that's the first one you probably would want to make, and that will then help you decide.

A

You know the the upper layer right so now, once we have standardized on a CICU pipeline, you probably would want to reuse that when you're looking at the machine, learning use cases or ETL use cases, you know etc. Right.

A

So and I think the the point which I was making you know earlier in the call was you know so this work, which we have been doing now. You know the goal is to engage Google heavily I mean Google is aware of it, the the core repo is it to the cue flow repository itself, where we re basing the whole of cue flow pipeline to run off Tecton. But now you know we want to essentially push it to the point where it becomes.

A

You know the core part of it, because yeah for us I, think at this point, Tecton in general, is becoming the strategy and we want to make sure you know rest on the use cases on top of it, whether they are machine learning, whether they're ETL oriented we are building on top of checked on rather than replacing Techtron and bringing specialized set.

B

Of another.

A

Set of pipelines, yeah.

C

And if interested, is he honest progresses and see how we could definitely because I have a couple of peers, he's undergone GCP right now or something nice to give it a kind of unit test. Great.

A

Cool anything else teddy or anyone else want to bring up otherwise,.

B

Again, yeah a couple of quick updates from from me: they've been a few additions to to the challenges table and in the roadmap I'm just going through the review process on that and I'll publish those, hopefully in a day or so also we are arranging a second meeting for Asia Pacific, so so there will be a meeting time, which will be a few hours after this one so that everyone in in in those time zones will will be able to also collaborate on on this and I will publish those details shortly. Okay,.

A

Make sense Thanks.

A

Thanks everyone, you know we can steal five minutes back. Thank you. You guys appreciate it. Thank you. Geez.

B

You.
youtube image
From YouTube: CDF SIG MLOps Meeting 2020-03-26

Description

No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).