Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Efficient Model Exploring and Continuous Delivery With Polyaxon + Kubeflow - Shotaro Kohama

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Efficient Model Exploring and Continuous Delivery With Polyaxon + Kubeflow - Shotaro Kohama, Mercari Inc

Machine learning projects are highly iterative. A project begins with an experiment phase, a productionization phase, an operation phase, and the next experiment phase. The smooth transition from the experimental stage to the productionization phase is the key to accelerate the iterations. Polyaxon is a cloud-native machine learning experiments platform that allows developers to run parallel and scalable hyperparameter tuning job in a declarative way. KubeflowPipelines is a workflow engine for machine learning pipelines. This talk shows how the Machine Learning Platform team at Mercari helps accelerate machine learning projects by Polyaxon and KubeflowPipelines.

A

Hi welcome to my call, I'm shotalo, I'm a machine learning platform engineer at bell party. Today, I'm going to talk about efficient, modern, exploring and continuous daily value, polyaccent and the cubic row. Let's get started.

A

First of all, I will explain what melton is mentally is a customer to customer marketplace where customers can sell their items to the other customers? The balcony is available in the us and japan. Today, I'm going to talk about the meltdown u.s massive learning platforms award. One of our unique challenges is pricing for sellers. If the item price is too expensive that it takes time to draw studio.

A

On the other hand, if the item is too affordable, the seller cannot make a profitable sales to help to solve this problem. We use these machine learning models. We call that system applies guidance system. That class guidance system consists of the two features. One is the applied suggestion apply. Suggestion feature recommends a viable price range when the customer makes a listing on our marketplace, and the second feature is smart pricing.

A

When the customer make a listing on our marketplace, customer account. Besides, the blue applies, and smart plus in the future, gradually and automatically update that item flies to the floor flies and then that the item will be promoted to the potential buyers to design item price lunch and floor plans. We use this machine learning model.

A

Our machine learning model takes the item, title description in category and so on and the suggested prices we use machine, learning, practical components behind this price guidance system. So, in the latter part of this presentation, I will dive into that machine. Learning component.

A

Here is the today's agenda. First, I will talk about the general machine learning project life cycle and then I will explain that how we use the polyaxon and the cube flow pipeline of melatonin. Lastly, I will talk about how we build the machine learning platform to utilize that uh polyaccinto pipeline efficiency.

A

Let's move on to the first part, what's in learning development time, life cycle.

A

Generally speaking, machine learning projects are highly iterative.

A

First, we need to design the task to solve the sml, and once we design that task, then we'll start the collecting training data and we got the enough training data. We can start the model exploring phase through the model exploring phase. We decide that our first model to deploy to the production and then we can start the development microservice to deploy our model and integrate the microservice with the other systems.

A

uh Once we we finished, deploying the model to the production, and then we can get the actual feedback from the production process, and then we can go back to the correcting database. We can train the ml model with the actual feedback data and that this iteration will keep going other machine learning platform perspective.

A

How we can accelerate this iteration is the key to the success of the ml project and we believe we can be able to accelerate these iterations by automating manual processes, with the open source, ml, ops and devops tools.

A

At melcalli, we use these open source tools for acceleration for modern exploration phase we use polyaction, for the accent is a one of our panel obstacles to support the scalable and visible organics foundation and for continuous training. We use cubicle pipeline fibopro pipeline allows us to manage them and to end in their workflows. On top of the kubernetes and for continuous daily body we use spin anchor.

A

The spinaker is able to trigger a deployed pipeline when the new image pushed to the registry so that we use skin maker for continuous delivery.

A

In this talk, I will focus on the polyaxon and the cubic flow pipeline parts, because in cubicle north america, 2020 snap in team gave a great presentation about how we can use the skin maker for ml ops. So we use a similar way to get blown that ml microscientist to production so that I will focus on the poliox and the cuticle parts in the later presentation.

A

Let's move on to the polyacin section.

A

Polyacin is an ml obstacle to support the scalable and reproduce moderate discrimination. The polyacin has a yaml specification called the polyaccent file on the polyester file. We can define the steps to build the docker image for training and also we can define the commands to run the model training and also we can put what parameter we will explore as a hyper permutation.

A

And polyaccent has a ui to visualize the results of hyperpermutation jobs. As you can see on the left below image on this right, we can see the what parameters were the best as a one, metrics and audi accent file has all the information to the produce that experiment that other engineer can easily declutter that experiment and also it allows us to take over a project easily.

A

That's a big benefit.

A

Here are the steps to run the hyperparameter tuning job of the polyaccent first, we'll define a polyaxon file and next we'll create or modify the folder to train the model to decide what the liability or you know, for the architecture we will use.

A

Third step is to upload the multi-poly excel file and the code with polioxin cli, and then the polyexin will build our docker image to run the training call and it will uh schedule the hyper perimeter tuning job in particular. That's not how to run the polyester phone job in this role. My favorite point is that holy accent helps us to build a docker image.

A

With this feature, a developer doesn't have to build a docker image manually or doesn't have to wait for continuous integration. Every time you modify the call, it sounds a subtle thing, but it's for me. It's very important to keep my concentration in the development as well.

A

This feature prevents interaction from happening in the document. So that's my favorite part at the military u.s. We have been using koliakson for more than two years with less than 10. You know developers we created 175 projects and we've done about 87, 000 experiments and also we have metadata around this experiment on the poly accent and we can see the what parameters used for this experiment and that's amazing.

A

Next is cube. Pro pipeline cubicle pipeline is a quantum open source workflow engine and it which is especially for machine learning workflow comparing to the other workflow engine qb pro flow pipeline has a metadata store with the metadata store to throw pipeline stores outputs of the each stage so that we can uh store the uh what parameter we used in the training instead, and also we can store. The results of evaluation stems.

A

Cubic flow pipeline. Ui has the teacher to visualize that matrix so that we can visualize the confusion matrix of the evaluation step on that web ui.

A

Cube flow pipeline also has a python sdk. With that person sdk we can write the workflow as their python ddl and also that python sdk provides a way to define the reusable component.

A

So we can create one component to share the among the pipelines.

A

At military u.s, we use cube flow pipeline to achieve continuous training and continuous daily value.

A

Cubicle pipeline automates, the manual operations regarding the modern extra collaboration phase and the deploying ml model to production, for example, in the one month workflow the first step submit to the policy job to get the training job and next step is to submit to the policy to get a training model and the third step to submit the policy job for model evaluation and next step.

A

If the matrix looks good, then the pro pipeline will create the docker image to solve the new triangle model and push it to the docker registry, and then spinaker will detect the new token image, and then it will belong to the production.

A

That's the way we achieve the continuous training and deployment.

A

With this approach, we can avoid early implementation in transition from the modern exploration phase to the continuous training phase. For example, we don't have to modify the phone to access the training data or we don't have to modify the call toward metadata. Something like that. I think. As a result, here we can say the developing time.

A

The last three as a platform team, we built our tools and the setup configures integration to be able to write such and to end the workflow easily, and I feel these three points on this slide are very important for the efficient development in the last section. I will highlight these three points, especially the first one is monolith or cubicle pipelines.

A

We build our monolith to manage the multiple project, cubicle pipelines, to share the best practices and knowledges and second point is we define the yaml manifest to manage the resources on the cubicle pipeline and for reaction, and it enables us to achieve without infrastructure, asphalt with resource management that point in the cubicle pipeline component to submit the polio symbol.

A

We built a custom component to submit a policy job from a suitable pipeline. So with this component, every developer doesn't have to write a code to robin uh for the action and submit that job and yeah. That's the point: let's deep dive into them, 1.1.

A

And here is the structure of the model level and we put the python package to the share. The lightweight components. Light weight component means uh uh one node way to create a cube flow pipeline component.

A

Tube pro pipeline sdk has a feature to convert a python function to the qb4 component. We don't have to write a docker image and uh yaml file for the component.

A

With that feature, we built it for the accent cube, draw pipeline component and also with that python package, we can share the util functions right, setting up memory and cpu resources or, and also we can share the constant values for and variable names or secret values easily, and we set up that continuous integration with this monologue.

A

When we push a call to this plant gift hub and then test continuous integration will detect the change of pipeline, it automatically compile and upload it to the development browser.

A

At that time, continuous integration will use a blank name and the last commit hash. With this way, we can standardize the versions of the typical pipelines.

A

Now we have a development plaster and production cluster and only while merging that liquid into the main blanch, the ci will upload to the production process with that we can keep that only for the past the code review on the production process.

A

That's the one benefit.

A

Next thing is project manifest for flow and reaction.

A

We define this table specification for resource management.

A

Cio will create the cubicle python experiment and polyjackson project with this camera file, and then we don't have to manually create these resources of the ui, and then we can keep the consistency between development and production environment.

A

This manifest also has the owner's skill with this field. We can manage the permission for quality and it enables each team to upload the full request to modify the negative code.

A

That's the way we use this project, one.

A

The second thing is for reaction to python component: these are the steps how to grow pipeline for the action component works at the first phase and any the container from the private repository. That's that secret, which is stored like in the hub token and then main container log into the polyaccent user, with a secret. You could destroy the polyacid token in the third step.

A

The main container submits adjust to the polyxen through the polyacid api and then the main container tails, the lungs until the job ends and last the main container fetch the results of the job and the outputs that are data to nexus.

A

Here is the example of the arcade pro pipeline. We use for the continuous scheduling and training light site figure shows the actual compile the cubicle pipeline. We have a blog post about the how we use them for continuous training and deployment that, if you are interested in that, please take a look.

A

That's it here's the takeaway from my presentation. First, one is reaction, suites the model exploration in a scalable and relatively visible way, monolith and the continuous integration for cubicle pipeline works well for us, and it allows us to keep high efficiency and consistency.

A

The custom q pro pipeline component for production enabled us moving forward from model exploration phase to continuous training phase seamlessly.

A

Thank you for listening. Let's move on to the q live q monday session bye.