Red Hat OpenShift Edge | OpenShift, 28 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitOps for Manufacturing Edge

Description

See how GitOps can help speed up software develop and release cycles and optimize their roll-out across hundreds of manufacturing plants.

Learn more at openshift.com/edge

A

In the following demonstration, we have a closer look at git ops for manufacturing edge.

A

We will see actually how we can speed up this software development and release cycles for applications that span from central data center to manufacturing plants and line data servers, and this means on one hand the software development and roll out another hand also optimizing. The whole lot of configurations across multiple manufacturing plans.

A

All this with an approach, that's called git ops, where we have all the different changes managed in git and using processes like pull, requests and deployments to hold out the right configuration to the right side in a very controlled manner.

A

The context is here a simplified application for monitoring and production environment, the application spans across three levels from top to bottom, the central data center, the factory data center and the line data server, and we will see in the first demo flow how an otops manager can change the configuration of sensors uh using a github's approach and in the second demo flow. We have a look at the developer and especially rolling out software change within with a pipeline and doing the correct, versioning and doing this controlled within github's approach.

A

uh During the demonstration. The dashboard of the sensor can be monitored by the machine operator.

A

Let's have first look at the application.

A

You can see the dashboard for the which is used by the machine operator to observe various parameters and, as you can see, the the metrics holding in real time the technology beneath is deployed on redhead openshift and let's have a quick look at the pots running.

A

First, you see, uh for example, the pod for the line dashboard for that dashboard. Actually, in one open shift environment uh in a different kind of project, you also see the imports for the for the messaging. That means for the amq and pushing the data to the real-time ui and on a completely different cluster.

A

You see actually the two parts for the machine: sensors.

A

Now we are going to change the configuration of the sensor using and get ops approach. As you can see here, we have two sensors, but one sensor sending temperature and vibration data and the other sensor only vibration data. We are jumping directly into github. For for that change, and in this directory you see the settings for specific deployment.

A

The deployment is here, the production deployment and the settings here are really specific to this production environment, but we can fine-tune these because the generic settings are defined in templates.

A

Here you see the templates for machine sensors and, for example, the properties which are generic and for all sensors and with the approach we have here with the directory structure and with the concepts of using customize to tune the manifests, we can only do or we can do the changes really uh in the fine grain manner without duplicating a lot of configurations.

A

Now, let's do the actual change we are simply putting here. This attribute sensor temperature, enable and true or make a a quick comment here, attempt to true and commit the change, and we are committing this here now directly into master and the pure githubs approach. We would create a pull request or create do this in a branch and create a pull request, but let's do this now directly into into master.

A

We can now also jump to argo, cd and argo cd is observing the changes in a regular interval. We can force a sink so that we don't need to wait. Oh it's already. Jumping in you see the changes here now. It uh actually only taught the change here and then deploys the new configuration to the to the pods.

A

This will be picked up, pods will be restarted and we will see also here in a second already immediately how this extra line is coming up. The extra charge is coming up for the temperature data, so that means with this change um and the monitoring or the approach of argo cd and monitoring the files in the git repository and forcing that to the actual operative deployment in the production environment.

A

We deployed this change automatically.

A

In the second part, I'd like to discuss actually the rollout of software changes using openshift pipelines.

A

This will involve here on one hand, the openshift, the pipelines. That means the tectum project to to build the software to test the software, pushing then containers to clay and then rolling these containers out again, we assume here that the software change has been already made and we will drill down into the pipeline.

A

This is actually an example, or this is the pipeline, how to roll out the consumer component.

A

It has several steps which I would like to explain briefly in the uh first step here, uh the the containers are built and then pushed into the local regulatory registry. We are using here semantic versioning.

A

That means under another next steps we are having, then the the version file counted to one level up and with that also tagging the newly created containers in the local registry, because this is the githubs approach we are also taking now uh the the manifests and to declare which image should be developed, deployed and we're doing this in this step here to actually uh change the configuration the github's configuration in on github.

A

That would be a change like this here, where you see here the customize file and then the image and the text. That means we are taking the image here so that the new version would be deployed in this test environment as a first step uh and then running uh actually informing argo cd to deploy this change.

A

So it would pick up the customize file and deploying our new containers and then having that deployed in the in this testing environment here doing various tests for for the sensor, consumer and end-to-end test, and if everything runs, okay, uh we can now or the pipeline pushes the image to to quay because from the central quake repository, uh we would feed this into into the production environments.

A

So, on one once this is done, we also modify the git ops repository for for the production environment. That would be the same step like before, and in this time we can also have a closer look here. uh It would simply go ahead and change the customers file with the the customization file with the within your image tags and then, and at the last step it would create actually the pull request, the pull request to say: look, the changes have made correctly, and now we can.

A

uh Somebody can now say: okay, I want to haul in this exchange for production this uh now. This is the pull request that can be viewed uh of the changes of the configuration. That means exactly saying which container should run where and approving this pull request and merging that in argo, cd then, would pick that up and then also enforcing the change for the production environment.

A

So we looked at two git ups use cases for manufacturing edge on one hand, rolling out a configuration change for software sensors that has been then deployed through and for enforcing the manifest with our city to the right location and, secondly, a tecton pipeline, where we built new containers, test containers, put the right versions into the manifests and then rolling out that to the test environment, but also rolling that out or creating a pull request to hold this out into a production environment.

B

In this video I'm going to demonstrate one of the many ways you can extend the get ops model to your machine learning workflow as part of the get ops for manufacturing edge demo, we have sensors deployed at the edge data centers that are using machine learning to detect when anomalies occur.

B

In this scenario, I want to show you one of the many ways in which a data scientist can develop a model at the core data center then update multiple edge data centers simultaneously, with one commit to a get ops repository.

B

Here we are utilizing the open data operator to provide a data scientist with the tools that they need to develop their machine learning models on openshift.

B

Here we have a jupiter notebook with a model that a data scientist has just updated and now needs to make it available for all of our edge data.

B

B

In this cell, you can see we have serialized this model, so they can be packaged and deployed in our updated image for model serving.

B

A git repository that will provide the source for our image that will be deployed to the models at the edge.

B

Once the updated model has been committed to the repository, we will execute an open shift pipeline that will orchestrate our cicd process before we release this model into production.

B

The first task will build the model serving container image locally, using a source to image, build process and automatically determine the next available image tag build number for the release.

B

The next few tasks will perform our ci test to verify that the model deploys and runs successfully in our staging environment.

B

Once the ci test executes successfully, the pipeline will push the new release of the container to an image registry that can be accessed by the edge data centers. Finally, the pipeline will update the get ops repository by creating a pull request that will specify what version of our model needs to be deployed at the edge.

B

Now that the container image and get ops repository have been updated, all of the edge data centers running argo cd will detect the change and update their models. Accordingly, this example shows one of the many ways you can extend the git ops model to your preferred data science workflow.

B

If you would like to learn more or try this out on your own, please look for more information on our website at opendatahub.io.