keptn User Groups, 28 Sep 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: k6 Integration Service for Keptn - GSoC Project Demo

Description

Google Summer of Code (GSoC) is a global, online program focused on bringing new contributors into open source software development. GSoC contributors work with an open source organization on a summer programming project under the guidance of mentors. GSoC contributors accepted into the program work on projects to improve or enhance the Keptn project and its ecosystem. During the mid-term evaluation our mentees will present their projects. In this session Jainam Shah, a GSoC contributors in Keptn, will present her project: "k6 Integration Service for Keptn".

K6 is an open-source tool widely used for load testing. With Keptn’s Quality gates, we can automate the process of evaluating the results of the test and also monitor them. Keptn already supports JMeter, Locust, Litmus Chaos, and others. This project will enable users to use K6 for performance testing. Further goals include adding SLI and SLO compliance using Prometheus which K6 supports natively.

More about Keptn in GSoC and the projects: https://github.com/keptn/community/blob/main/mentorship/gsoc/2022/projects/k6-integration-service/README.md

A

um One button: the button: okay, real life, so hello uh today is September 28th and we have the final presentation of uh the k6 integration service project by h m. uh So it's a Google summer of code product. uh It was quite a negative summer for students in the uh Captain organization, so three projects got successful according to the mentors and we are doing presentations. So today is one of the presentations um and lately you can find all the materials on our on our YouTube channel. You can also join our slug.

A

If you have any questions and I think we can start the presentation, then.

B

C

Thank you so yeah. This is my screen visit.

A

B

C

So yeah hello, I, am a sub contributor and I worked on the k6 integration for Captain product so and my mentors were Adam Gardner.

C

Let's talk a brief about cases, so k6 is a widely accepted, uh Cloud native tool, which is used for load testing, and it has very good. uh It has very good resource utilization, so even with 21 load generator, we can get thousands of user virtual users and k6 is also code driven, so k6 uses uh k6 uses JavaScript as a scripting language and k6 also has support for Native support for Prometheus, Diner, trace and data token. So yeah. Let's take a look at the k6 scripts first.

C

So this is how a general basic k6 script will look like. So over there we can Define the number of virtual users. So k6 works with the concept of virtual users, which run your test scripts, and so virtual users are, you can say, like a parallel while true Loop, so yeah uh so yeah and the script is written in JavaScript as es6 models.

C

Thank you yeah. So after the k6 runs so k610 sends a small summary to the output. The two standard output stdm so for the so I I mean. If you want to do any granular analysis or uh on time series data or anything, then we can also stream the live test data to any external data sources too so, but by default. K6 provides this test summary, so that contains some basic summary statistics like uh like Main and P95 values and Max values for different different basic mathematics, yeah, so yeah.

C

So cases on k6 also has a k6 thresholds uh feature. So thresholds are basically a pass or failed criteria for your testimately. So if a test Matrix doesn't uh doesn't meet the required expectation, then it will fail. So k6 users often use this thresholds to codify their slos, so yeah, so so thresholds can uh can be set on any of the test Matrix. So, for example, we can have that cases. We can have some thresholds like that.

C

95 percentile of the request should have response time below 200 milliseconds, and only one percentage error request should return error. So this kind of the this kind of thresholds we can add so coming to k6 extension, so uh k6 extensions help us allow us to extend the use case of cases. So, while k6 also provides the ability to give to summarize the Matrix in skdo, but sometimes we need the data to return to be written to uh to uh some external Source, like Prometheus or dynasties, so and for a better visualization of the test.

C

So k6 has a feature of j6 extension, so we can just download this x k6 tool and build it using uh build the k6 binary uh returning book so yeah that we can do so. This integration will add uh so Captain plus k6 integration will add a load testing Tool uh to already existing tools of set of tools in Captain, so like jmeter, and also with k6 quality gate evaluation and the k6 captains, quality, gate, evaluation and k6 support to write Matrix to external platforms like Prometheus. We can also plug this things.

C

The glove the two features together and we can also do the politicate evaluation on The Matrix written by k6, so yeah, and also it will enable the community users to migrate from using either of the tools so yeah. So the Milestones of this projects were mainly first. We should be able to run a standard k6 using job executor service and uh also after that, the next step was to run a k6 extension and Export The Matrix to Prometheus via captain and after that so yeah.

C

So this was the work which was done in the phase one along with the side, along with the site demo of POC, for using custom service for custom Captain service for k6. But later on, we decided that job executor service will be sufficient to do all the to basically will be sufficient for all the requirements of the project. So we decided to go with job executor service only and so for phase. Two.

C

The remaining step was that we should be able to do quality gate evaluation using the Prometheus Matrix that k6 had written and after that documenting the tutorials.

C

So let's take a look at how the workflow for k6 and Captain k6 and quality gate evaluation in Captain would look like so uh you know. So there is an evaluation task in the ship yeah.

C

So evaluation task is a result task in Captain and it will be used for Quality gate evaluation and we can add this added to our sequence after test to automatically automatically trigger the evaluation using the quality weights in Captain after the test of k6 has been executed, and uh so what will happen after this, so the k6 will write the write. The Matrix to promise so k6 is Prometheus. Extension will allow us to do that, and we will. We can also add tags corresponding to that Matrix, so we want to by first.

C

So all the Matrix will be written in the same with the same key in Prometheus, but to differentiate the test between test results between a project and a service of Captain. So for that we can use the we can use the tags feature that is there in case. It's Prometheus extension to do that and after that, the Prometheus service of Captain will fetch this Matrix. So that is the next step of getting the SLI values. So Lighthouse service of Captain is uh responsible for Quality gate evaluation and what this service does it?

C

The service uh listens to the uh it listens. To the evaluation.triggered cloud event and upon receiving it, it internally calls the get CLI dot triggered Cloud event, so we will have to configure the SLI provider in this case being Prometheus, and uh what this, what the Prometheus service will receive.

C

This get SLI dot, triggered Cloud event and give us the SLI metrics that we have specified, and on top of that, we will have a SLO config, so like Lighthouse service will use this SLI values and check whether the uh whether or whether all the criteria have been met for the quality rate evaluation. So this is how the workflow will look like. So, let's take a look at how the at the demo of the of this service so yeah.

C

So yeah, as mentioned earlier, so we will also add the another task, another task in the sequence of our Shipyard offer project. So we will add this evaluation task, which will be responsible for the which will be responsible for the quality, get evaluation, so it will get triggered after the k6 test. Execution has been done successfully and uh after that, so this k6 will run on this script and we will get the results corresponding to it and yeah.

C

So this is the one and over here while we are running the k6 test in drawback security service. So in the conflict of that job executor service, we will add a we will add a tag corresponding to which job and uh corresponding to which corresponding to which Captain service project and Captain State Captain stage.

C

This task is so this will allow us to bifurcate the so differentiate between the two different uh The Matrix in Prometheus between two different different services and its test Matrix, so that we will add it over here and the next step will be the Get Next. Step will be that for this project we will have to add a config map in the promises config map in the lighthouse service, which will tell Lighthouse service that will listen, uh get get the Matrix from Prometheus and uh yeah.

C

So for what that, what we will do is that we will add a config map to the lighthouse service in in the captain name space with probe with giving the data source of SLI provider as promises. So we can also do this by using the captain configure monitoring command and uh yeah. So once we have this after that, we will create a SLI sli.aml file, but that file will do this. It file will require a Prometheus, Matrix uh Prometheus expressions and it will create and will Define the slis that we want to give.

C

So in this case, what I am doing is that I am getting the P95 request.

C

Duration of the of the test, and uh over here I, am also providing the service and job and service project and stage name, so that will fetch the correct Matrix from the Prometheus and once we have the slis over here we have to mention using the Prometheus Expressions, so we can add, uh add even more complex queries like doing uh taking the average or doing any any kind of mathematical operation like some or some or anything on the Matrix, and after that, once we have the Matrix so corresponding to that SLI.

C

We can Define the objectives of our of our of our quality gate evaluation and we can give us the total score and uh total score for passing the evaluation, so over here I have kept the kept warning as a 500 millisecond, so this Matrix should be less than 500 milliseconds and the past criteria is uh uh one one second, so yeah over this way. We can provide the config of the criterias of pass and warning in the slo.aml file, so yeah.

C

Let's take a look at the at what happens and at what happens at each state, and so over here I have triggered a new sequence corresponding to this test, my service sequence, with for micro for the service micro services. So over here we can see that there are. There will be three different, different uh stages, so sequences, so one is one is test, so test will be doing the k6 evaluation and over in the k6 evaluation we already provide the we already provide this uh image of k6 Prometheus.

C

So this is the docker image for writing for k6 Prometheus extension, and it will give us that output will be k6. Matrix k6 Matrix, two promises remote end point, so it will write the Matrix to Prometheus and over here we can also see the summary of summary of the test results in the stdo and after that, once it is done. What will happen is that it will call the evaluation sequence and the evaluation sequence will in turn call the get SLI in sequence.

C

So over here this Prometheus service would have started and it will try to fetch the data from the from Prometheus. So over here we can see that the Matrix says Matrix is K k6 HTTP, request, P95 duration and the value is 271.67 and which can be also matched over here as 261.65 from the output, and also we can verify that from the flow meters itself. So yeah we are.

C

We are fetching the correct Matrix over here and then, once once, we have fetched the slis we can get the so SLO Lighthouse service will trigger the evaluation of the evaluation on that SLI. So over here both have been passed correct correctly, so we have the score of also 1000 and the result is 100 and result is also passed.

C

So this way we'll- and we can also do the analysis of how the tests have been performing over the time over the time period of period of time and how how is the airflow going on so yeah? This is the demo for the doing quality, gate evaluation on The Matrix of Matrix. That k6 is writing and yeah, apart from that. So for the for the purpose of the writing, the tutorial, so we have split the tutorials into three different different parts.

C

First is the running a standard k6, and once we have a standard k6 running in job executor service using Captain, we can move further into using k6 extension in job executor service and, after that, uh once that part 2 is also done. We can use the quality gate, evaluations for on the exported, Matrix, so yeah. This tutorials have also been documented foreign.

B

C

Yeah, thank you. Everyone and special thanks to Adam and president for mentoring.

B

Me any questions.

A

Thanks a lot for the presentation, so we don't have questions in the chat, um but we still have pepper on the call who is a mentor. So maybe you would like to share some feedback about the project.

D

Foreign, yes, um I, want to share that. We are pretty happy with the result and with Janam work, so we started the project with uh let's integrate captioning and cases together, and we have gone the extra mile of all the type of integration that we initially envisioned and okay, the project from my site has been a pleasure working with China with Ada, so a very I I would say that is very nice from the product perspective, so the two products are very aligned.

D

Okay, it's about defining um expectations for your loaded Captain, without defining also the quality day that you need to so in. It interacts pretty well from both sides, and the collaboration has been very nice, so very easy to work with Dynam. He had done everything that was initially that even more so I hope that the community find enter on this integration and that people use it on their day-to-day work.

D

A

So one question about: uh what is your vision for future evolution of the project um So? Currently we have k6 integration service, but obviously there are so many potential features there so, which ones would you like to see? Next in such an integration service.

B

C

Actually so there were two different different things, so one was k6 thresholds, which will already be I mean it will already be so I I mean it will already be evaluated by a job executor service itself and my mother was quality, get evaluation, so we were able to eat. From the I mean we were able to read The Matrix from projectors and use it over here. So I don't think there are any other I mean. On top of my mind, there are no any more features, but there may be perfect food.

B

D

I, don't have more feature on top of my mind, I think we covered the three UK that we want in from the beginning and, as we have been talking in the past, I think it's more about providing clear instruction now, explaining a user how to use and every time that you put that product on the hand of the people. You start seeing more requests. So I said that we have done everything even more and it's more about explaining the community how to use the product, and probably they will come with new ideas.

A

A

And yeah I guess it's a good time to promote the service, so we have quite a lot of users doing cover those kinds of quality, gigs users running gym, meter and I. Think that there's still so many people interested in q6 which keep promoting this integration and Gathering back from users.

A

So yeah thanks a lot for this project. Okay, it was within us to see how it evolves and your deliverables were on time, which is super good yeah I was rather on.

A

um I was looking after the project, but I definitely didn't intervene at all. I was just following and yeah I hope it was a good experience for you, so General feedback I shared at the previous session that we should ensure that projects collaborate more with the wider community so that you get the experience full community, not just a lot of isolated experience in a particular integration service, and we will need to put more Frameworks in place to make it happen in the future.

A

And then, if you could share your feedback about how we could do better next year with Jason Captain, uh what we're missing, how we could have helped? You it would be a really valuable feedback.

C

uh Yeah so I mean my experience was also very good and I guess. Captain was also doing piece of I mean it was a juice of organization for first time in Jesus, so I guess it was pretty even and what my one feedback was that maybe if we can, you know also allow the the developers to contributors who join the core development meetings.

C

So I see there is only one developer meeting that, but if they are so, let's say, I am working on uh job executor service right, so maybe, if I could, just you know, get a more in I mean just a more uh information on what job executive service development is happening, or that kind of thing so I mean the core project.

A

B

Have a question purely curiosity: um if I go to our Integrations page right now, there is a k6 load, testing integration up there, um but it says was updated six months ago or eight months ago. So this is not your work right. This is something else.

C

uh Yeah in that artifact.

C

In that artifact.

C

uh I mean there is this artifact Link in that there is a k6x k6 integration, but that is an outdated one. So uh Adam has probably already raised the issue in the uh in that uh Adam has already raised the an issue in that case of Captain contracts that artifact Hub repository to add me to that repository and code and code owner so notation. So after that I will I mean we will link it with the updated and resources. Oh.

B

Good okay yeah makes complete as I sense, I I, don't know the details of this, but I looked at it. It's like. Oh, we have k6 except it has been updated for eight months and yet we know it's just at all this wonderful work so cool, so we will be seeing your integration up there soon. It sounds like yeah yeah, oh and I'm, supposed to check the video when I'm talking, oh leg's, gonna, kill me.

B

A

B

A

But thanks a lot for the presentation, excellent.

B

Presentation so.

A

B

A

Heard really good things all.

B

A

B

Actually about you so.

A

Yeah, if you're interested I think that I should be still a good opportunity to engage with the captain Community.

A

um There are many other programs I believe that in key six there is also a lot of opportunities for Integrations and contributions at the moment. So you're more than welcome to stay in open source, Community yeah. If she used, we will be happy to help you.

B

So thanks a lot for the.

A

Presentation, thank you and um so I don't see any additional questions in the chat, so I think we can wrap up and yeah for those who are looking for other videos, you can find them on YouTube and we will be publishing common mode in the future.

A

So thanks a lot good.

B

A

You so we'll be.

B

Seeing more of you Janine you're, not going away yeah excellent news, excellent news.

A

Thanks for that,.

B

A