Cloud Native Computing Foundation KubeCon + CloudNativeCon China 2019 (Shanghai), 5 Jul 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Continues Integration Practices for Zephyr Upstream - Hake Huang, NXP Semiconductor Inc

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Continues Integration Practices for Zephyr Upstream - Hake Huang, NXP Semiconductor Inc

Continues integration(CI) is getting popular in many open source projects. There are many successful open source projects which can support CI. However, how to integrate existing open source projects and build an efficient CI pipeline is getting importance, as the project is getting much more complex. Hereinafter, we present how NXP zephyr Devops system works for NXP upstream augment testing. The proposed system contains several subsystems based on several popular open source projects, including building an internal cloudy build system with zephyr Docker file, and linking this Docker image with Jenkins Docker, a scheduler system for multiply board testing as well as virtualization validation system. and some glue scripts. With this system setting, we build up a CI system with minimum supporting resources and high reliable quality tracking and validation for upstream patches

https://sched.co/Nrv0

A

Good morning, everyone so.

A

My topic today is continuous integration practices on the further upstream I myself as come from an SP semi conductor. So some people may wonder as a chip company. Why would you use some cloud or talk--I technologies? The mainstream technology is used in chip is mainly in CI or the develop ops.

B

A

This is the abstract of my speech. We will adopt some open source, Dockers anagen, to put them together, wear some glue script, and then we will have some like components and huh and somehow wares to install deploy the system. So based our experience, we can build such a stair system with minimum supporting resources and high reliable quality tracking and can also makes the test available after five minutes.

B

A

That, for yourself it has a shippable platform. As the theorem works shippable, it is not public available. So that means it is not possible for customization or for general users, because license is required and also it is very hard to dynamically change. The tester sense, and also you will find that a lot of the testers, scrape or tester staff will be mixed in the same gate. Red boom.

A

This is actually not allowed, because when we release a pair to customers, we do not want to expose the underlying layers or some of the unnecessary stuff for users and for tester code. It is now in very good quality so because it is a kind of commercial version. So if you want you have the develop of architecture. To connect to such a commercial version will make some troubles, even if you have some support from the company ISM, it is actually will cause more problems and also as a chip maker or as the board maker.

A

What is the key issue is for shippable. You cannot run on the real world. You can only do it on simulator or emulator. So if you new, simulator or emulator, to maintain a yong-chol is assuming simrita is idiom, but if to realize the CI, it will take more effort and more time than doing the test on the board. So that means, if just to do some basic checks on the simulator. A lot of issues will happen on a real border, so this problem will have deep impacts on the real applications, qualities like a lifecycle.

B

A

You really well, we are establishing a cloud architecture. We need to take user case into consideration first, so today's user case is how to upstream ESPE. So our user case is. We will have the upstream task and then we will do look at the beginning. Do some local function test and then we will generate a pull request and this pull request. We'll have some integration test and after the integration test has been completed, then we will finish the upstream.

A

So this is a standard user case. If you have any questions, you can interrupt me at any time. So, based on the previous user case, we will have some requirements from the user case, so the first is a test on request. So that means there will be not a specific test. A set. The test side is based on the program's. You are going to develop, so this test is on request and a second is: we will use common and user-friendly technologies.

A

That means it doesn't require very tough learning curve to learn some different technologies or typical languages. So that means you have easy access to the technologies which can be used in your programs and the whole.

A

Invocation process will be stateless because you have a lot of tasks to run. You need to set up the board, you need to run 100 cases and, for example, if to be exaggerating, you need to test your apps in more than 1,000 boards. So that means, if it has a state, you need to switch from one state to another state. So that will requires human intervention. Then it will require a lot of human efforts to maintain. So that is why we require a shtetl, stateless process.

A

That is as long as you send a simple request, and then you will get what you want. You do not need to care about the process.

A

B

A

Based only Yuda case requirement analysis, we will come up with a system level requirements which include first, we need to have a system to configure the tester scope and the second is: we will try to adopt open source framework and also we need to have a task scheduler to make it stateless process and also we need a reliable flash mechanism.

A

For example, if you want to flash, you need to have some equipment to control your board, so that means, once you have flash on a program whether it is successful or not, so it needs to be reliable and also it needs to be developed, a dedicated delay and also we need to have the result processor, because we think that you can develop any apps, you want and they test a process edan. It is from the signals to charge the result, so that will reduce the limitations for our developers.

A

And, last and not least, we need to have a storage system to store the locks. So what we are using is a ftp talker, so that means for the test results. We can store it in or in the talkers.

A

So this is the architecture of the CI system. There are two layers, one is the in user interface and the second is the servers Liam. So all the servers are transparent to you to you, tourism and what we have used them is centric YouTube user interface and where the pollution talker to track all the servers and to get the input from different servers and the the wine, the middle, the binary documents and the the Austrian log, we will put it on the servers. So such kind of architecture is based on our requirement.

A

Analysis, and these are the talkers which we are using. One is the Jenkins Dockers. We are using it for CS system with database and we have a FTP token. We also have a dev, you talkin, so all the compilation is real last, where the debute token, and also we have a task queue token. So well, you adopt open-source framework is better to read less code customized framework because for open-source there will be a lot of input from outside and it's lab span is very long.

A

So if you have created some new frameworks that will lead to a very complex system which is difficult to understand, so it will take long time for you to explain to the others why you do like that, and the deployment of the whole system will become very simple. We need a SCM system, either it is github or bitbucket, and then we do it a local cloud or rant some cloudy easiest and on the cloud, the easiest you can deploy or talkers. Then we need a flash machine and then for our board.

A

It is supported by pyro CD and then gets the locks where you are T. So the whole process is very simple: nothing, the one stone for cloudy easy Allison. We can get open so stone or if we get very clear request them, we can get some other solutions by depends on the evolution of your program and next is something special, because if you glue all the open source stuff together, you need to know how to glue them together.

A

So this is about how to connect the Dockers and some suggesting using IP tables to use the IP associated communications. But after our analysis, we think that it's not necessary to use the big data. We can use the darker shared volumes. That means, for all token, it is to be mounted with the same volumes. So that means it can be shared in one PC and the inter operations of tourism. You can use stalkers exact to use the amount for each docker container.

A

Once you have mounted that note or stalkers will copy the function from one container to the other container ISM, so all will be sent via socket. So from this we can optimize the interoperable operation system and there is a concept of convention over configuration. So that means once you have defined a convention, it is a convention and it will be very high efficient for practice.

A

So another thing is we have mentioned: we need to test to establish a tested set. If you have studied the Jenkins pablum fail, you will find that.

B

A

All the theater systems- shippable durable it have the peplum scheming, so we hope all developers do not need to learn the schema. So that means it's necessary for us to develop some simple codes. For example, if developers want to have a test, then we just need them to know what they are going to test the. What is that has the route as to the compilation as to how to write how to render log? We do not need to know that, but for check-ins you need to consider whole process.

A

You need to define the setup, define, run and define how to get a result. So we have developed some script as long as you well, the apps is, you can run the script and generate the pipeline fire. You.

B

Just need to define what you want to test, so you only need to have the knowledge of Yama file.

B

You just need to know about the overall structure of Yama Yama is very popular and easy to understand grandma compared with SML and jenkins. It has a lot of advantages, so you, if you need to have a local file system, I, really recommend you to use that.

B

So we used Yama's customization, because it doesn't a cloud include or exclude that's why we set up an additional layer so that we can and do some basic calculations on the board and the overall system would be scalable, because in llamó you have to use a script and you need to use specific grammar.

B

All these things is included in the script, so you just need to include them, and the next Espada flash system I have said that if you want to have building development or development of any boards, you have to customize your program because we are chip supplier. How do you program?

B

Nobody else would know we need to provide a framework to any user.

B

We have user interface programs. We have also developed py MCU TK. This is a talk kid to program. All our company's chips as supports I are coyote DB and.

B

Mgo expresso, so we can call the commandant of these systems and you can also have double check after download, and you can also use in J link or J bug. Is that all the functions? All these things are included in the script and is already open source.

B

After all these defined processes, we can complete a whole set of ESP upstream process. There will be two parts, one is many of the other is automated for a manual part. You just need to develop the program on your branch and do your own tests and then create an type line file. You just need to define what you want to test the functions of your development.

B

What kind of testing needs to be done, and then you can add that to the poor request and for the poor request will be assigned to SDM Jenkins trigger all this will be done, and a blue ocean will trigger a test request according to this and generate build command and the build command will automatically upload the image to the FTP, and then we all have a rank, random and then pour binary and operate on the board and then send it to the Jenkins and upload the test on locked to the server and the user will receive an email notification or any other notification.

B

You define to inform you that the test is done, so this is an example or configuration within this example. You only only need to do three scenes first unit 2 in phone. Where is the kid with the kid and then you need to define the regular compression and then what is the name of the has to file? You only need to do these three scenes very simple.

B

When the test is done, it will create such a report. If you're familiar with in Jenkins Lotion pipeline horizontally, it means the sequence show and the other what code Direction means it's been done simultaneously, so you can define how many and build can be least in parallel, and you can also define the relationships with each other. So if it is passed it you will see a green mark. If it fails, you can see a red mark.

B

So this is the log from blue ocean some tricky part, because we are using URL to look place, the storage method, so the Jenkins is only storing the URL. It will not store the test result in the Davis, so we can reduce the burden to the Davis and similarly with the log in the lower part, we just put the URL and we just inform about the test result. If this fails, you need to check out the lock.

B

You just need to link through the URL, which is also reducing the load for the database, so these are the benefits of using the system for every four rest, we can do that on real board and the test scope can be customized by the original developer, because you are the developer, you generate the feature, you would know what other things that need to be tested, but of course, regression tests can run for all the known cases for.

B

The other, since you can customize yourself for the whole process, it's still is a sink. You just need to send a pull request. Then your duties are done. You just need to wait for the results you can switch to other tests. You don't need to care. What are the other interventions? You need to do? It's not necessary.

B

You don't need to learn many things. This is the best part of that, because for a developer they need to focus on their projects. If it is for the bottom-line software, they just need to know.

B

They don't need to know how the cloud system is structured, it's not necessary, so you just need to link three things. First, you need to happen basic dark knowledge, knowledge which takes about one hour and in bank it, and then you also need to understand some simple Yama's schema. Llamas emo is also very simple. It takes only about 30 minutes, so we don't need to have menu maintenance for the whole process.

B

This is very good because operating on a stable open-source architecture, we are combining everything together in a very rational way. You don't have many things to do so. Only when you have more demands, you need to duplicate the system. The benefit of darker is that you just need to expand it, so we use original configuration when you need to have more darker the documents you just need to set up your naming rules. You don't need to do many other things.

B

I've said that if you don't need have Hardware scaling, it takes only about five minutes to expand the setting.

B

So these are the future works that will be done. Zephyr is an open-source toast, so its purpose is to highest and six six evaluation. So all the part should reach 1%. We haven't progress. We can used in GDP to get all the coverage of data and have real-time analysis, because only on real boards, you can run all the tests to cases and do coverage cases in simulation. It's very difficult because it's not possible to simulate all coz.

B

This is what we are going to do. Next many users wants to get the test. The status of the boards, instead of just seeing the blocks, were reporting. They want to know about the the boards on different LTS, whether it's stable, whether it can pass some tests, whether they can use it. That way, they need to have intestine mint tennis systems, because Java is using benchmark Azrael. So our next action would be bringing our test reports and tested IO and combine them together, and this is a first working group.

B

So we're going to open all the scripts.

B

Basically, that's all for my parts. If you have any questions about the verbs or zephyr you can raise your question now. We still have some time.

B

We still have five minutes.

B

B

So you cannot add it any time you can add, because the Jenkins can allow extra note. You just need to add IP. That means just you need to do that in the interface. Yes, it has back-end UI. We call that API you can operate at the backend. You can add a note there.

B

So these are a random notes. These are the notes for the physical machine. You can also have the notes for the virtual machines. My second question during the Commission when it's dark.

A

B

Possible to enter some PPV to see how do you enter that? Because for every machine it's running on the darker? If you want to do the debugging, you can login onto the server and to see the actual stated of the server in the log.

B

So actually there's no authentication control. You can see everything. There is some control. If it is machine, you need to turn on their authentication for the end users. They don't want to have the authentication control it's difficult to control, because normally we want to have authentication, because we don't know what the users are doing.

B

We don't want users to do something because we want the system to be transparent. You don't need to worry that we have done something wrong, because we have provided all the locks to you and it's very easy to see that in the log, because, if we're making the system we're doing the same thing as you were, we're providing the makes command lines, I understand it's very difficult to balance right. Thank you. I.

B

Want to ask you, you said that the user can go to the specific notes for debugging like jay-z, and you can go into the container. Is that safe? If I give you the authentication, you'll be adjusted user? So that means that not everyone can have that, so you need to manually apply for that. Yes, actually, we have considered about the CI system. Our initial purpose is that what the developers are doing and we will just follow suit and we'll just provide the command lines to you. So there will be nothing in transparency.

B

The only difference is the machines doing things and humans are doing things. We don't want to have an artificial intervention if.

B

We open source. That is because we have done a lot of tests we can ensure, is reliable. You can trust that these modules have very high reliability for you. You don't need to a doubt about any issues brought by our system. You can replicate that the same thing, so there are no new problems caused by that. So we don't need the user to debug what if the user wants to add a new test case, it's very simple in your PR. You just need to create an pipeline file and inform you about the debugging path.

B

I mean if we add some test files, the environment is different and we can do any adjustment because for a typical case of Zephyr, it's not possible to have different environment. If it is in Custance, you won't be able to enter the Zephyr because you can only use the SIMEX can control method. The other masters wouldn't work, I mean.

A

So when you are running a total tester when downloading it, so if you are going to write your script, then you can't judge. That means our gift, a lot to you and then you can charge based on a log whether you pass or not, but sometimes the log is.

B

A

Where you are so, if there is, some issues needs to use some Qi tag to the bath. We suppose that it shouldn't be fixed on the interface Leon. Otherwise, your CAD package, the cabbage out of the cabbage I.

A

Have a question for Jenkins from your slides: the test case is running invisible, a running in parallel, so for the master it did like kind.

B

A

For the jacquelyn's itself it has the load balance. It depends on the course of the CPU. If you have two calls, then you can have two cases to run and then, if you have 16 calls, you can have all the sixteen calls in a border. So if the task has already gone beyond the extreme of the master, then it will just wait there with Auto mode automatically, so that is in Jenkins spiritum.

A

Well doing the execution, you cannot conclude a board, so what we done is why we have the task of Q, so we have used to server server to generate it, has killed and then put it in program machines. Does the program will tell the pastor kill that the last one has been finished and then we will have the next a to run, so that will make the whole process stateless.

A

Any more questions, if no thank you.