Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Building MLOps POCs and Sandbox Environments Using k3s and Argo - Sergio Mendez

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Building MLOps POCs and Sandbox Environments Using k3s and Argo - Sergio Mendez, Universidad San Carlos de Guatemala

This talk presents the challenges that corporations have when they evaluate technologies for data pipelines automation or for MLOps workflows, this includes processes like ETL, ML model generation and ML model deployments. At the same time, this talk shows how open source technologies can help to reduce implementation time and infrastructure costs for MLOps systems. The talk also presents the advantages of using k3s as a lightweight Kubernetes for POCs, the lessons learned at work when trying to implement MLOps using Scikit Learn for ML Models and Apache Spark for ETL processes. As a complement for k3s the talk describes how Argo can replace or complement pipelines designed with Apache Airflow and how these types of technologies prepare companies for Edge Computing. At the end of the session the speaker presents a short demo showing a basic MLOps workflow implementation using k3s and Argo and showing how k3s and Argo interact with each other to deploy ML models ready to be used.

A

Welcome to the cubicon at cloud9com, europe 2021.

A

This talk is about building envelopes, pocs and sandbox environment using ktris and nargo. I am sergio mendez a little bit about me. I am a priority system, professor. I am clown native enthusiast. I really like to play with cloud native technologies.

A

I do some devops at jello.

A

I am the organizer of clownery guatemala, I am a link or the hero.

A

Let's get start with some cartoon first showing how we depend on technology and these things and how machine learning is going to substitute or works in the future basic concepts.

A

You have to see some basic concepts before jump to the technical part.

A

Poc approvals concepts, pocs used to evaluate different kind of technologies, solution and context, and the learning good goals and that kind of things, so we use a poc to compare different technologies in order to solve a problem, so we experiment and do things to determine which technology is better to solve that problem.

A

A sandbox environment is pretty similar than production, but it's just for testing purposes, so you can run trusted software without risk. There is a control environment and can be used for testing pocs and you can have access to some similar data from production.

A

Each computing in the other side refers that. Let's say that you are close to the source of data, so you want to process the information there. Let's say that you are close to the data and you want to process information with your smartphone.

A

Let's say it could be possible, uh but the that doesn't mean that the cloud is going to be disappeared. That means that the cloud is coming to you, because you are going to process the things with your own devices and niche computing reference to the distribute geographic computing that things some use. Cases for age computing could be machine, learning, iot applications, data processing games or any workload.

A

Talking about devops devops refers to the collaboration between the operations and the developers and this kind of workflows, where you are going to build tests really software in a really fast way, with ci cd pipelines and that kind of things a pipeline. That is a really common concept using on devops. um Here is a concept that shows what is a data pipeline, but it's pretty similar that let's say that it's a series of steps that maybe the output of one asset could be the input from another step.

A

So it's really awesome to understand that pipeline. In that way, the console.

A

Envelops in in the other side, refers more to the automation of the model generation, talking more about how to design build and manage these processes that you are generating a model getting predictions in order to push your ml power software in some way.

A

This diagram refers pretty pretty close to the way there are some diagrams on internet about devops and refers really really nice, but in the context of envelopes, envelopes contains the data, the model and the code. Let's say that the data change you have to regenerate the model. Maybe you can do some modifications to your code. Maybe you you want to regenerate your model too, and this this includes the whole cycle of the envelopes in general.

A

So there are difference between devops and envelopes: let's classify it. There are different users for devops. You are going to see developers, operators, let's say sre devops cloud engineers envelopes, guys data scientists develops engineers and that kind of things so devops focus more than developers and ml ops more than the science team spartan data.

A

So they are pretty pretty different. The artifacts that devops generates- or they are getting touch- are more like github repositories or give repository source files made with multiple programming languages and envelopes is more focused on machine learning. Models and data sets devops use different technologies, common to hear that people mention front-ends, front-end frameworks, back-end frameworks, languages, databases like bjs, titan, java, mysql, etc.

A

In mlaps context, the people mentioned more models about data and technologies like python. Scikit learn tensorflow that are libraries to generate machine learning models, jupiter notebooks for the experiments, apache spark for data processing and the whole ecosystem of hadoop, and why not airflow?

A

It has different goals. Devops looks more for the software, deploys some application on the cloud and have some quality assurance for the software fast deployments, cicd pipelines, but envelopes is trying to automate the model generation. The predictions the inference and optimize that kind of workflows.

A

They are like, let's say, two kind of experiments like the static experiments that are ready to be passed to production environment. So the code is not going to change anymore, it's really stable. So maybe you want to schedule this kind of experiments periodically we claim only yearly.

A

There are dynamic experiments that are under development by data scientists. The code is constantly changing it's unstable and that kind of things.

A

There are some benefits for envelopes in the context of the containers, so the containers can bring you package. The logic of the experiments can freeze. The library versions can give you that portability for the experiments, and maybe you can share these containers with experiments across the different teams of data scientists and why not to implement that kind of githubs to generate these models and these experiments or how to make these experiments.

A

Let's remember that mlaps is not only technology. It's people. This experiment is data. You have to organize that that people, that experiments and data in order to get valuable predictions and in order to implement successfully envelopes, environment or culture there are some technologies for poc. Some machine learning pipelines that you can use k2s, for example, is a certified kubernetes distribution that can use erm architecture that is ready for iot and age computing.

A

It has support for intel architecture too.

A

The amazing thing is that k2s includes the whole components of the regular kubernetes clusters. In only one binary, it has several different backends, like mysql, etc, includes ingress, branstaller, helm, support network support, continuity for the containers.

A

And please remember that if you are using a erm architecture or devices, you have to recompile your libraries or applications if needed in order to support that architecture, and then you will be ready for edge computing, because this kind of architecture so common videos on this context of face computing and compile or performs this kind of workflows.

A

So the companies right now are migrating to age computing, to reduce goals for their workflows.

A

Argo ground source in this site is a piece of software to create this automation, these pipelines and is designing to run on kubernetes.

A

There are different features for argo. uh You can execute that step inside that container, your workflow or your dac.

A

Can be defined in a file like a simple directive, a cyclic graph, pretty simple syntax using demo. Argo is kubernetes native cloud agnostic. um You can use argo workflows for different things. Instead of machine learning just to implement a regular ci cd with less complexity.

A

Argo cd in the other side is a complement for our workflows. You have to use argo cd just for continuous delivery and deployment, and it's designer for cuber names.

A

This supports different formats of configuration like hemp, chart, customize json files demo, or why not your custom plugins for your custom configurations?

A

Don't forget argo events and rollouts. It's part of the whole argo ecosystem to define a really really nice ci, cd or envelopes workflows.

A

You can use k3s as a lightweight environment, plus our goal on the edge or maybe for pocs.

A

Argo could replace apache airflow, maybe beans, luigi ml flow or other ml solutions like sagemaker or maybe could be a complement for your current solution.

A

The benefits is that, with this kind of technologies, you are going to spend less money for pocs instead of a cluster. Why not a virtual machine with k3s um using k3s and argo undergo workflows and cd? They are prepared for age, computing and iot. They are cloud agnostic, pretty lightweight and high scalable, and they are no bender locking because you are using open source, the demonstration we are using argo workflows and argo cd installed on k3s onenote cluster.

A

Let's say it's not a cluster and we are going to execute some simple pipelines and we are going to show the code that we are using for that.

A

The architecture of this demo is like their first step for the basic envelopes pipeline is reading. A csv file with the scores of students are is going to predict the final score um in the extraction.

A

Transformational, loading step step.

A

Number one is reading this information cleaning, this information deleting columns generating a processed csv file that is reading for the step number two for the training and generate this model with that file, and this is going to generate this model and these files are going to be uploaded to a bucket on google cloud plus temporal storage and the step of the deployment is going to read this model and it's going to call argo cd that detects a helm chart that creates a deployment using this model to expose this model as an api, with an endpoint with an ingress controller and for the inference.

A

We are going to access this endpoint to get these predictions and generate a final csv file with these predictions. Let's call it the inference.

A

So, let's move to that part.

A

So right now, let's.

A

Let me start our good workflows: let's access the ui.

A

So right now it's.

A

A

Yeah so right now we have the argo workflows. Here's the dashboard, the things there are several current workflows that were executed in the past. Here is argo city. You have to configure your repository on github that detects the help chart and you have to create an application and.

A

Here this is going to show your deployment regular deployment with the reference revisions and argo is going to perform that deployment and, let's execute a very simple pipeline from our.

A

A

So in this site, let's move to the repository.

A

Let's say: argo workflows has a cli, that's pretty pretty awesome.

A

Let's clean the cli, so it's going to show you the steps here and in the command line interface.

A

So it's really really nice.

A

um So you can't visualize this thing if there are workflows right now here, let's reload the this part. Basically, it's going to show you, let's zoom a little bit this. uh This step is for getting two steps that is running parallel.

A

It's going to show the log here and basically call a whale container with a custom message in the logs, so each step has a wrong log with a custom message.

A

Here shows the same thing and yeah. That's a pretty basic basic pipeline and me show you the.

A

The basic envelopes workflow that we already have.

A

Here, okay, here is the whole envelopes pipeline that is going to execute the process that we explain in the architecture of the demo. So let's call it q q com test, one.

A

We are going to execute this part.

A

So this thing maybe will need more space to show training and these things.

A

And maybe it's going to need a little bit of more space here, showing the whole process. It's executing the etl process, the model, the training and these things so right now is showing a real time missing the part of the inference right now, the other steps are in green crawlers, so are in progress right now, yeah. So it's done.

A

Let me open here so run real perfectly, and we can see how this thing calls. Let's call it the test number two: how this thing goes.

A

Argo cd, so right now, is in the ideal process.

A

So right now is in the training. The next step is to deploy the model.

A

So, let's move here it's going to change in some moments: yeah right now, it's changing it's replacing the old bots and that kind of things. So right now is refreshing. The things.

A

Yeah so right now it's moving. Everything looks fine, he's deleting pots, updating the things and yeah. That's it. It's ready.

A

uh Let's say that, for example, we can execute another another.

A

Let's call it the test tree, I don't know if I executed.

A

um Yeah, it's this part.

A

It's going to execute the same part and in our god you are going to see like a kind of history.

A

Well, right now is trying to deploy another. Let's say right: if the data change or the goal or something like that.

A

Right now it's changing the things and what, if there's a mistake or there's a need that returned to that previous version, so here's um an option in the history rollback and you can roll back the previous version right now. It's in the part number three, the cubans test cubic contest too. Let's roll back to that to that version right now is doing that rollback.

A

A

Yes, he's doing the things.

A

Right now I I can show you then a little bit the code of the things. Well, we have here the helm chart. This is a report that I can share with you. People, the part of the pipelines, is the envelopes simple pipeline here, so you define the variables and these things this is optional. If you want to use token ferrargo, so you parameterize the things you include the different steps, the tl, the training, the serve the inference on each step.

A

You have to create that kind of template really similar like tecton, and so you call a container and send parameters on values above get, for example, and you have to parametrize the whole thing, and let me show you another part of the code well here in the part of the containers. um There's the atl process.

A

That is basically python code.

A

Let's see a little bit the index here, it's a pretty basic python code that do the cleaning thing with using pandas and that kind of libraries.

A

A

Let's move to the model training, maybe it's like uploading. The things.

A

Downloading the processed file and generating the model and uploading the model here this part, all the experiments are based on the scores of students and in the part of the inference, is calling the endpoint that this parameter is here in the function of prediction.

A

So, in the end, it's going to generate this inference file that we can see here on our repo.

A

Let's see here in our storage or cubicle, you can 21 project on google account, so here's the inference file.

A

Yes, the initial file, the model and the processor.

A

And the health chart is, in this part.

A

This helm, folder is going to be read by argo in order to deploy the thing and that's it. That's the general argo workflows, argo, cd and k3s.

A

Some of the resources that I use for this demo is the official websites k3s and argo workflows and cd, the slides. If you want to see my slides um the repository at github, you can replicate this whole experiment and understand, step by step, my personal email and I'm in the social networks uh sergioram gpl, and thank you very much for the opportunity.