Cloud Native Computing Foundation Kubernetes AI Day EU 2022, 19 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Building AutoML Pipelines With Argo Workflows and Katib - Andrey Velichkevich + Johnu George

Description

Building AutoML Pipelines With Argo Workflows and Katib - Andrey Velichkevich, Apple + Johnu George, Nutanix

The fairly recent field of Automated Machine Learning (AutoML) provides the richness of powerful algorithms for model selection and hyperparameter (HP) tuning – one of the most important steps of the MLOps lifecycle. Katib is a popular Kubernetes native open source project to perform AutoML. Katib can tune HPs for models written in any framework such as Tensorflow, PyTorch, MXNet, and Scikit learn. To find the best HPs, metrics are evaluated after a model training step. Usually, the model training is a complex process which includes data preprocessing, data validation, actual training, and many more. This whole lifecycle can be represented by a workflow dependency graph by specifying dependencies between model operations. Argo Workflows provides a great container-native workflow engine to orchestrate jobs on Kubernetes, which makes it an ideal candidate for Katib Experiments. This talk will demonstrate how Argo Workflows natively integrates in Katib infrastructure.

A

Hi everyone, I'm andre, I'm a soldier here, engineer at apple and also I am the co-chair of ultimate training working group in qcload.

B

I'm jono a staff engineer at nutanix leading efforts in training and automal in queue flow. Today we will be presenting on how to build automate pipelines with argo, workflows and cat tip.

B

Before diving into the main topic, we'll just talk about general, automated machine learning area or automl shot.

B

So this main area has it's very vast and has lots of research happening and it can be considered as as like multiple sub domains, for example, hyper parameter, optimization or hp, tuning neural architecture, search of nas feature engineering model, compression data, preparation, data validation, part.

B

So when we consider, from a component's point of view, we could think as like multiple components, so one is the main configuration space. So when we talk about configuration, uh different configurations extending from what we have discussed in the previous slide, uh the feature engineering domain uh deals with features. Hyper-Parameter tuning deals with hyper parameters, neural architecture, search deals with uh model architectures right and once we have training.

B

Data passed on to this new metrics are emitted from models based on the configuration that we have passed on and using these metrics optimizers uh based on each environment. Right like it, creates new configurations, and this loop continues till some success criteria is formed. So that is the main, uh the core, the automal field and the end result is. You would basically get a well optimized ml model.

B

So coming to catip, so that's the the main focus in the stock. So it's a kubernetes native, open source project for general automl needs, and this is like one of the core uh queue flow component, so it can be deployed independent uh from uh the the main queue flow insulation or if it's by default, it is tied with the queue flow. uh The main deployment um from all the domains that we have talked about in automate supports hyperparameter tuning neural architecture, search with early stopping mechanism and outer box.

B

It supports almost all popular automal algorithms, uh for example, bayesian hyperband, tp, cma, etc, and the platform is very extensible in the sense that users can deploy on custom automl algorithm during runtime without even restarting the deployment and catem is agnostic to ml framework and programming languages, which means user. Can user is free to write a code in any programming, language or ml framework, say whether it is tensorflow python or psycat xt, booster, whatever it is, and since it is kubernetes native, takes advantage of the underlying kubernetes infrastructure.

B

So so it is portable, so it can be deployed on prem or on cloud or in the hybrid nature mode, and we can basically tune any custom resource in kubernetes using uh cad tip and since it is baked with q flow, it also uh supports um advanced training configurations like distributed training uh through queue flow operators like um the the give flow, the pi dots operator or the uh tensorflow operator. So by default you you, you can use that or you can basically deploy it uh total uh totally independent from the keyflow installation.

B

So this is one very high level, uh cad architecture. So, from a user point of view, he creates an experiment in the form of a spec or a standard kubernetes manifest like yaml for in yaml format and the experiment controller, which is responsible for the experiment. Customer resource picks it up and create suggestion. Custom results, the suggestion controller, looking at the user spec figures out, which is the algorithm that that user is interested in it talks to the right algorithm service and provides produces a set of hyper parameters now against each set of hyper parameters.

B

The trial controller creates trials and you could think trial as a worker job, so this can be distributed in case of a distributed training or it can be a single port training as well, so each trial produces metrics, which we have talked about in the earlier slide right, which is the evaluation metrics and they are stored in an external metrics db. The experiment controller looks at these metrics and figures out whether the experiment has to be continued or not so in. In short, the experiment or the optimization look continue. Still.

B

Your success criteria is met or uh the use of budget is satisfied and it contains once it is once it is completed. User gets to know the best hyper parameters for that experiment, and this can be used for finding the best model of that onto andrei for uh the arc overflow integration.

A

Thank you jonah, so before jumping to uh implementation, how we use argo workflow in calip. I just want to briefly talk about what is argo workflow.

A

So, basically, is the container range of reconstructed system built on top of commerce, and I would say, like it's de facto platform, uh which widely used across all the industries and especially in kubeflow, we're already using cargo in kubeflow pipelines and kdp, and for us it's very important, to find the orchid searching tool to be able to run out more jobs because in the next slides I will explain you how we integrating this internally in our infrastructure.

A

So argo has a lot of different projects that, under using argo underneath and it's very powerful, to be able to run this workflow engine inside other crds, so speaking about kelp integration. uh So, on the previous architecture, diagram, as you can see like we, our trials spawns the workers, and since we uh trying to build the architecture itself by providing the abstractions on top of the worker, our users can deploy almost everything which has the specific states.

A

And this particular example. We can integrate our clothes directly to our child's jobs and users be able to run their training code inside the one argo workflow, so why it is actually useful for us, because by using by running training job as the argument flow, we can just perform very complex amount by clients inside a training job. We can. Our users can do like the pre-processing step, post-processing step also, they can like train the model in a multiple different ways.

A

For example, it's very useful in the multi-object object, optimization and also since argo like provides a great uis and great artifact tracking users can also track the metrics results and analyze them in the different various workflows.

A

It's also like very uh efficient if you need to like train your model on a different data set all the different like volume, resources and all of this can be easily integrated inside kdp infrastructure and just analyzing the results and making your hyper parameter optimization.

A

So, speaking about integration steps, how you can integrate argo workflows inside kdp, it's very straightforward before I can just explain you. I just want to say that any command is 3d can be implemented in calip as long as this crd can create support and also can allow allows you to inject the side cars, because our metrics collector is just a sidecar container and also has the succeed of trailer studies. So, as jonah mentioned before, this rd can be like everything we have an examples for job job fighter jobs.

A

So all of the flow training corp distributed training operators and also have an example for tecton pipeline or arguable flow, because kubernetes job sometimes does not fit all of your needs. uh If you want to run your training pipeline and if you want to integrate the argo flow inside get it, you just need to simply update your container arguments inside your credit controller and also you should have the contributive control of cholesterol airbag, to be able to give the particular access for cadet controller and to be able to reconcile the argo workflow resources.

A

So after just embedding this step, you just need to restart your creative controller, and after that you can easily integrate algorithms, educated experiment and think about how you speak about how you gonna define your argo flow in your kind of experiment and have an api called trial template where our users usually define the child template and define all the apis required to run. This template during the experiment run so in this template user have to define four different apis.

A

So the first is primary port labels, which identify uh the port and identify the labels of the port which matrix collector will be injected. Because imagine like you in uh in argo workflow, for example, you can have it two different parts and each port can do a different uh workflow operation like pre-processing, step and actual training, so our matrix collector should needs to be injected injected only in the training step- and I will show you this in the demo as well and also like in the primary container name.

A

You need to define the the name of the container which matters collector nest uh will be wrapped and also the successful failure condition in the json format and this condition when your resources met. This condition uh controller will automatically reconcile this customer source during the experiment run and speaking about how actually users define the argo workflow inside experiment spec. We have the child spec api, and here users can define the whole workflow inside their experiment and, if you're familiar with argoflow.

A

This is very simple workflow that we're going to use I'm going to show you this in the demo. So in this workflow we have two simple steps. So the first one is pre-processing.

A

Where is just we simply bring the random value and just uh getting this value by uh by randomly dividing the 6 000 on 10 and 100, and then we're getting this value and just pushing it to the next step, which is more actual mode of training, and we can see in this example that uh we have the output parameter from the first step. So this one and the input parameter from our actually optimization algorithm, so which it's learning rate so we're going to pass the learning rate from calip algorithm and also we're going to pause.

A

The number of examples from our previous step and these two steps we're going to run our so each trial are going to run the whole workflow, which has like these two different steps and we're going to use this example in our like in our java, so yeah, and with that I just want to jump to the demo and just to show you how you can easily run the artwork flows inside kind of experiment.

A

So I'm going to use exactly the same experiment as I defined to you. uh So this is the. This is the example that we're going to use. I just want to copy that to our content, so this is the kdpi that we have so our users can define a new experiment, uh so the experiments, so they can define all of the apis by using these uis, such as metadata trial threshold. So, for example, how many trials they want to run the parallel. What is the maximum number of trials?

A

What is the maximum number of failed trials? Also, very important users can define the objective where they just specify which metrics they want to optimize and what kind of additional metrics they want to collect. So in this particular example, we're going to tune the validation accuracy in the meantime, we're going to collect train accuracy from our training process.

A

uh So this is the search algorithms. As jonah mentioned, we support like a lot of like novel automal algorithms. Here uh users can easily define them based on their needs. Also, they can specify the early stopping techniques, um and this is the hyper parameters that the users also can specify.

A

They can change the domain, the search space definition, the different types of hyper parameters and also like this is the matrix collector, where users can uh use the different apis for methods collect and how they want to analyze the metrics from the training process, uh and this is the child template. So it's again it's.

A

This is the very important- and I just explained you how users can use the child template api to be able to just define the argo workflow inside the experiment run, so we're going to pass the whole yaml inside this edit form and we're going to change only the maximum number of trials to eight and we're going to use two trials in parallel.

A

So again, as I mentioned before, each trial just running the argo workflow and we're going to run two trials in parallel, so two algorithm parallel for each evaluation run so after creating the experiment, user can click create. We can see the status of experiment if we jump to another namespace, we can analyze. Other are the users experiment, um so this is the optimal trial that the users can also like analyze and understand the best hyper parameters um they can click to the experiment, with some results we're going to impact this later.

A

So if we jump to argon namespace our experiment currently creating and we're going to jump to a terminal to just analyze, the results from our experiment run. So in my name space. Currently, I'm running the argo flow container. Sorry, argo workflow controller, which is basically reconcile all of the workflows and also the credit controller and the uis for argo and kev. So, first of all, we want to check the trials, so once experiment is creating.

A

Two trials also have been created and they're currently in the running state, and we also can uh check the the workflows in the argument space. So, as we can see here, the two trials have been created and two workflows also have been created.

A

So if we discard, if we're going to describe one of the trials, we are going to see the some of the interesting results. So here we can see the learning rate this is generating by our kdp algorithm and we're using just a random algorithm here, but we only tuned learning creators I mentioned before, and we can check the ports. So it's inside our argo, argo namespace.

A

We see that each workflow creates two different ports, so the first part is data processing, port and the second part is actual model training and we so if we're going to analyze the results, for example from like from this trial. um First of all, we can check the the locks from our pre-processing step, so our processing step just generating the value like 150 1538.

A

We're going to pass this. This value to the second step, which is actual model model training, um and this value is being used during the training run and also we pass the learning crate uh inside our training job. So this is the simple mnist example, just a simple um cnn model, and during the train run after the training is finished.

A

We just our metric matrix, collector, actually parse, all these results and send them to the db to make the new prediction and the new actually uh generating new hyper parameters and which is very cool like since we're using argo underneath under our trial job. We can use all of the richness of the argo, like all of the features from argo workflows. For example, we can jump to the argo ui and we can analyze all this workflow, how they're running what is the, how the value passed between the workflows?

A

What is the current status, and also we can jump to the particular workflow to see the whole of the steps. So, as I mentioned, it's very simple two-step workflow, uh but we can analyze the input parameters, output parameters.

A

We can just check the locks and of course you can like you can define very sophisticated workflow here you can have a different volumes which your workflow will be using and you can again, as I mentioned before, uh you can have like the multiple training uh training optimization during your uh like hyper running tuning up run.

A

You can check the manifest uh from from this ui, uh so you can see that learning crate just was generated and was substituted by argo by by edit controller, um and we need to wait until all of the workflows will be finished, because all of the trials also will be finished, and then we can just see the results.

A

So if we, if we're going to jump to the kdpi here, our users can analyze the experiment run, they can see the the experiment name, their current status, the best trial, the current best hyper parameters, um some information about running trials, some information about experiment conditions, for example. If they're going to click to the child page here, they can see that all the child, all of the statuses of the childs and what is the best trial, we're also highlighting them for the user.

A

uh So, as you can see like it's automatically, adding new and new trials and users also can click the child to see the uh the metrics that our matrix collector um just parsed from the training. From from the training step and again as since we just uh getting called two metrics. We see like the validation, curious and training accuracy in this in this graph.

A

um So it depends on the depends on the results. The best hyper parameters are generated and if you click to the details, you're going to see like more detail information about the experiment and, of course, if you're like familiar with kubernetes, you can always like see the whole yaml for the experiment run for the more detailed information.

A

So again, this is a very simple example, but it's very powerful, because using argo in calip has a lot of advantages, as I mentioned before, and users can definitely like define the very sophisticated workflows inside the optimization step, and it helps a lot a lot for us to make some like again, some multi-objective optimization, which just a simple distributed distributed front, doesn't support it. So it's very efficient and uh it has a lot of like potential in case of like uh in case of hyper parameter optimization.

A

So, once the experiment is finished, we can see the best the best trial, which is generating this learning, crate and users, can just take this learning crate and put them in the in the final model and just push this model to production.

A

So this is a very simple example that I really want to show you how you can easily integrate uh the argo workflows in calip and again, as I said like, it has a lot of potential uh and it's like a way like in a very early stage of adopting the workflows inside optimal experiments uh and just before we wrap up. I really want to mention very important things about community, so all of this would not be possible without our great community, and I really encourage you to follow this guide.

A

To run this uh argo workflow examples um by yourself inside you can even use the your local kind cluster. You don't need any on-premise or like public um clouds, to be able to run this example. uh If you want to like join our regular meetings, please feel free to check these links from argor, paul and cadip. We have like regular regular meetings.

A

Also, you can check out github repositories and if you want to j, if you have any questions, please feel free to ask us. On the slack channel. We have the keyflow slack channel. We have the argo post channel, so I'm going to have it like to answer any of your questions and if you're using kd, please update the adopters list.

A

We really want to have the interaction with our customers to just understand the um the problems that we have in our current infrastructure and if you want to contribute, please check the developer's guides the our issues with this label. You can also submit your proposal and if you want to learn more about kelly, please feel free to check our presentation list and with that. Thank you. So much for listening us we're more than happy to answer all of your questions.

A