keptn User Groups, 18 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Automating & evaluating load testing with Locust and Keptn - Keptn User Group - May 18th, 2021

Description

Learn how to use Locust load testing and how to write powerful load tests in a simple way. We will then use Keptn to have the tests automatically executed as part of our CD and Keptn Quality Gates to evaluate the impact of the tests on our applications. We even build a self-service platform for developers to be able to trigger deployment, tests, and evaluations for new version of their applications.

Presenters:
Lars Holmberg, Redshirt
Jürgen Etzlstorfer, Dynatrace/Keptn

Timecodes:
00:00 Introduction
01:35 What is Locust?
25:26 Locust + Keptn
31:56 Demo of Locust + Keptn
47:55 Q&A and closing remarks

Meeting notes: https://docs.google.com/document/d/1Om9pj16hGKP_w2vUaH-7Cp0ffEIj-Oe3IezeVCpFYAM/edit#

Learn more: https://keptn.sh
Get started with tutorials: https://tutorials.keptn.sh
Join us in Slack: https://slack.keptn.sh
Star us on Github: https://github.com/keptn/keptn
Follow us on Twitter: https://twitter.com/keptnProject
Sign up to our community newsletter: https://keptn.sh/community/newsletter/

A

Hi everyone and welcome to this captain user group. Today we are going to talk about how to automate and evaluate load testing with locust and, of course, captain.

A

uh It's my pleasure also to have today here lars holmberg hi lars hi, and uh he will give us an introduction to locust uh what it is, how you can use it or what it can do and then I will show you how to actually use locust in inside captain how to use the the locust integration we have built in captain and how you can automate and evaluate your load testing. So the agenda for today is lost will give us this introduction.

A

What is locus? Maybe you've heard of locust before. Maybe you have not heard of locusts, uh so we will get an introduction to locusts and then we will see how to use locust and service level objective based quality gates in captain how you can even build, let's say a self-service platform for developers to trigger quality evaluations with locust and captain.

A

We will have a demo prepared and uh yeah actually, at the end, uh you're very much invited uh to give you to give it a try yourself. So with this, I'm already handing over to lars to show us what is locust and what it can do for you. So this is actually a part for later.

B

A

I stopped sharing.

B

All right, so um I hope you can all see my screen. uh My name is lars holmberg and I've been a performance tester for about 15 years. I've worked with j-meter and loadrunner and then some more j-meter and lots with locust in the last five years, or so. uh I have been a contributor to locus for a long time and I am currently the main maintainer of locust. So I'm the one doing most of the code, reviews and stuff like that and yeah. I live in stockholm uh and I work as an independent contractor.

B

uh I have worked with locust at and at the swedish state owned gaming company called svenskaspiel, where I am right now. My company is called redshirt consulting, so I give you red shirt today, but without further ado. Let's start looking at locust, locust was named after the swarming grasshopper, the the thing that basically attacks and eats everything uh in big swarms uh and it was created in 2011.

B

So it's been around for quite a while. It was originally made by a guy called heyman another swede and he uh made it for use at the at the gaming studio called dice. Make yourself battlefield if you've heard of that, um but right now it's uh used by lots of other places as well. So it's used by clarinet, it's used by salando delivery. Hero sounds caspial, uh also seen it mentioned in it by microsoft.

B

uh Talking about like how to run locust on azure and stuff like that, so it's been it's all over the place, um but why? Why should we use locust?

B

I would say the main reason for using locust is that you get to write user scenarios in just regular, plain python code. You can use any programming constructs. You don't have to learn any weird gui to do stuff in uh and you can maintain your tests as you would any other source code and get you can diff. If you run into some issues stuff like that, is really useful.

B

um It's distributed and scalable. uh It might not be the fastest load testing tool if you're counting like requests per second, but it is a very low overhead for the number of users and you can scale it infinitely. Really, uh you can. You can simulate hundreds of thousands of users, um so so it's quite scalable you can use it to test any protocol really.

B

Oh sorry, there are implementations for the protocols that I mentioned here, but if you want to test some special service which is not http, you can definitely add that with very low effort and you can easily easily modify behaviors and add new things into locust using hooks like if you want to send your data to a certain database or if you want to, I don't know, do some setup or tear down before or after your tests. There are hooks available for that all right.

B

Let me show you the most basic locust file, so let me make that a little bit bigger uh so, like I said you can do anything in a locust file that you can in python, uh so you can import any modules you like and just run it.

B

Here we import something called http user, which is, uh if you use loadrunner it's something similar to virtual users or in jmeter thread groups.

B

So a user defines a behavior which you can then run in locust, so our user called demo user has a task, and tasks are something that users can run. uh It's called hello world and what it does is that it uses the client which is an http um session. It's built on python requests framework, so it's like the normal python http client and we use it to call to end points before I show you this running I'll.

B

Just do another quick example: another task uh with higher weight, so this three here means that this task will be executed three times as much as this one, and it also does a small loop here just to show you how you can use regular programming constructs in in the locus file.

B

uh Also, there is an onstart method which will be called before this user does any tasks so that, if you want to log in or if you want to get some, I don't know get some data for that specific user.

B

You can do that. um I yeah, I think um I can show you just running it once in the debugger, because that's something you can do as well, um so you can see like there's some just ignore this text for now, but yeah. So you can. If you, if you have like an issue in your script or if you're, while you're developing your script, you sh, you can definitely use uh the debugger as you would any other python script or application.

B

So you can see what is go what's going on and like this user doesn't have a lot of data. But if, if you put like- I don't know uh user id as an instance variable for your user, you would be able to investigate that here in the debugger.

B

But uh let's, let's look at how you actually run locus like in a normal case.

B

uh So you do it something like this and say oh, which which locust file do I want to run, and where do I want to direct my load? Can.

A

You zoom in here also yeah.

B

Sure sure, actually I'm not gonna yeah, it's gonna deter. Thank you. So, um let's open the web interface.

B

And let's simulate 100 users and let's spawn them quite slowly, so we can see what happens so, as I showed you like, the the test that's being run here, it just does a few things. It just does four different types of requests. It does log in and then it does hello and world and item and I've sort of I have a small uh service that I'm calling, which is a fake service, and I've introduced the performance bug into it, so that this one will actually be quite slow if you increase the load.

B

So let's see how it goes.

B

So, as I mentioned, this is this: is the web ui for locust? You can use it or you can just skip it and do everything from command line. uh This is this. Web ui is good for for like analysis, uh but in reality I don't actually use it that much because we have a different backend.

B

We send all our logs into a time. Scaled database then use grafana to to visualize everything instead of looking at this this web gui. But as you can see here, we have all of all the requests listed and you can see that this one is actually quite slow and it's even getting worse as we go along.

B

So let's have a look at the charts. Just give us some more um info here. You can see the ramp up. Oh sorry, I've gotta, I can just yeah just so. um You can see the number of requests, which is the main graph here they are. The number of requests are increasing as we ramp up the load. You see here now we're at 73 users, and this is the total request rate, but you can also see that the response times are increasing. As well so this is a this is a bad result.

B

uh This system does not perform very well, and here you can see the wrap happening.

B

Okay, um yeah and now it's finished ramping up it's at 100 users and uh yeah. um There are other tabs here as well, but they're, mostly for when you get errors which we don't have right now.

B

What we should see after a while here is that this should start to level off, because the load is no longer increasing, but it's just stable.

B

Okay, uh any questions so far, uh I um I know I'm sort of rushing, but I really.

A

There is sorry sorry there is actually a question coming in about a q, a what's your best practice on data-driven testing, meaning. How would I pull data uh that I need for my test? Is there anything in locus that helps me query my test data.

B

So um there are several ways to handle test data in locus the most basic one is just use some sort of csv file, of which you can read uh directly. I can. I can show you.

B

This was something I was gonna mention anyway uh locus plugins is sort of uh some additions to locust that we have built at sanskrit, and uh one of those things is. It contains two readers and they allow you to read test data from uh from a csv file or from mongodb.

B

We actually use this one more, but this is the easiest one to get started. So if you just have a csv file, let's say with social security numbers, uh you would just yeah you just instantiate it and then just read lines from that.

B

Does that answer the question.

A

I think so, uh if there is any follow-up question, uh please uh just let us know in the chat or in the q a uh and if there are more questions on other topics. uh Also, please use the chat or q a function.

A

We will forward into.

B

Hours, cool yeah and here's another example of how how you use the reader, but it's basically the same. Just just read your data as you would normally using python or one of these pre-made ones that will help you get started cool.

B

um Just uh let me know if there are more questions, because I can't really see that right now, um but uh let's close the web gui for now and look at how you would run it from command line, uh because if you want to do uh continuous integration or running this uh automatically, there won't be a person looking at the gui. So we need to run it from command line.

B

So um there's a there's a reasonably new feature in locus: it's been around for maybe a year or something that you can configure locus, not just by command line parameters, but by using a configuration file you can also do it actually using environment variables.

B

But let's look at the configuration file, so uh this configuration file will uh point locus to the same host that I mentioned known command line earlier. It will select the uh this uh test on this locust file. It will run it in headless, so no web gui and it will just launch immediately and ramp up to 100 users and add two more users per second until it reaches this number and also add a runtime limit. So after 30 seconds it will just stop.

B

Actually let me leave that 60 seconds so now, if I just run focused like this, it will run using those settings and output this data to command line instead of the web key.

B

So let's just leave it running.

B

You can see the same behavior here where, where this item is reasonably fast in the beginning before we start ramping up but as we add more and more users, it just starts to break down.

B

B

A

You can already see it's getting slower and slower.

A

And what we can also see here is like the number of faith requests. It would be directly available also by the by a locust.

A

But it seems this is not the problem here.

B

Yeah, so so, um as we finish here, we get sort of a fi one final report, and we also get uh a detailed report of the response time percentiles. So, as you can see the the uh the response times for everything, except for this one, where I've act actually sort of. Oh, what's the word uh intentionally made it slow?

B

Everything else is just fine right, um yeah just quickly. I can show you the the way it works. When you send the data to to a timescale database, uh we have uh grafana. That shows a view. This is just a screenshot because I didn't have a good test run in place, uh but here you can see like how to visualize a test run, and this is what you this you can do after the test was run. So it's it's quite convenient reporting it to to something.

B

I think uh I mean there are lots of options here. You can rely on on the system, monitoring that you have in place that you have prometheus or something uh that's also, that's also quite useful. uh The good thing here is like, like you, get some aggregated and uh specific data on like what failed. If something failed, yeah cool.

A

I think there is a question coming in related to this, so I will take the last question first, which is the data storage also pluggable? uh Could I send the data to another database as well.

B

Yes, yes- uh and this is uh this- is exactly what we do here, um so we're using something called uh time scale listener uh which sends the data into time scale um uh so yeah. Basically, that's that uses. I can show you just uh the the core of this. Actually, let me just show you the the simplest listener there is, which is the print print listener, which is uh hooks into the the request uh event in locust.

B

uh You can see where that's done. That's done here.

B

So basically you just say: oh whenever there is a request trigger this function, and then it does a little bit of magic, but mostly just prints the the uh the failing request, uh so uh that will give you to standard out what went wrong basically, but the the uh the the more real um listener here, which is time, scale works in a similar fashion.

B

It just writes data to a time scale database, but but as you as what I wanted to show with the print listing was basically just it's very easy to plug in some sort of uh other database. There's also, I I believe there are uh prometheus endpoints, uh but I am not sure I haven't used those myself. So I can't really say how good they are, but then locust itself will would expose a prometheus endpoint.

B

If that's the word either. Maybe it's called something else: I'm not an expert on prometheus anything else. There was another question.

A

um Yeah, we can take it right now. The question um are the: is there some kind of concept of user profiles uh simulating different connection, speed, different browser types, uh even maybe some kind of user behavior? um Is this possible.

B

Yeah, so there are um so the the um if you're talking about like simulating different types of users, uh what you do is just you define another class call whatever. Let's call him slow user and then single task that just well. He does hello world. But then he.

B

And he sleeps for five seconds.

B

So if you were to run this locus file now you would get 50 slow users and 50 demo users, uh but there's not a built-in way to simulate uh slow network connections or anything like that. uh You can do that using uh something like um yeah. You would have to use like some linux tool uh firewall to to simulate that what you can do is, if you use locus to run uh full browsers, which is sort of a rare scenario, because because it doesn't really scale very well, but if you do that, then you can.

B

You can instruct chrome to to do things slowly. Let me show you an example of that.

B

B

B

It yeah here so so, if you run, if you run webdriver and run a full browser, you can you can use the built-in network emulation there, but this is not really uh like. I said it's not scalable. uh You can run, I would say at most some 50 uh 50 clients per per core.

B

So so it's it's uh yeah and it's not very because selenium is not very um stable under load. uh It's not very useful. I I've only used it a couple of times uh and then maybe as a sort of something to run next to the load generation to to do some particular measurements or something. This is also part of locuspike's, this webdriver client. So that's where you find it: okay, cool.

A

Cool also yeah, I think at the moment there are no more questions. Okay,.

B

um Do you want to continue.

A

B

If there are any more questions later, I'm happy to answer uh also, uh if, if you have questions later, uh just come to the locust io slack or we actually monitor the the uh where I actually monitor the uh stack overflow. uh So if, if you post a question on stack overflow, then probably you'll get an answer there as well.

B

A

Cool thanks so much for um for all this demo already, and I will be continuing here so and I will be more uh diving into how you can now use locust with captain in combination with captain and uh I'm not the expert on on locust, but it was actually quite easy to use, and I know we have some of them in some of the contributors to the local integration. Also on this uh on this call today or in this captain user group, so everyone feel free to also ask the questions to lars.

A

If what are the best practices to improve this integration uh or uh how we can extend it. So what we've built um with the captain team and the captain uh contributors is a locust integration that you can use together with captain and uh it will listen for test trigger events uh if you're familiar with capital, you already know character, will orchestrate all your tools uh and each time there is a triggered event. You can kind of react to this kind of events for locusts.

A

It makes a lot of sense to listen for test, trigger events and locus will then start its action so to install it. It's basically a one-line installation for the locust service and the local service with will come with the locus binary already included, so that the part that we've already seen where lars will show uh was showing us to start a locus test with locust and then minus f file, name or just locus with the configuration file.

A

This is actually part of this whole integration, so the only thing that needs to be done is to install the locust service, which is the integration for locust for captain and then adding your locus tests that you already have. You add them to captain how to do this, uh it's uh the same as as you would always do it also with some other testing files that you have done. Maybe in the past with jmeter, um you could just use the captain cli and do a captain add resource and in this example, the resources locust file.pi.

A

I think that's kind of the um the default name for a locust file, but it can be any file name. Of course, you will also add.

B

A

Project which service- I just made it a little bit uh abbreviated here and made it a little bit shorter.

A

So now you have the locus integration, you have your notice tests added and you can then trigger a delivery, workflow or deliver a sequence in captain when the captain starts the delivery um starts the tests does the evaluation and can promote the artifact or roll it back.

A

So, with these kind of three steps, you're all set, and you can make use of locust already so what you get with captain, in addition that to the locust to the locust results that we have already seen in the demo from lars, is you get the full power of the captain quality gate? So it's uh you can make use, of course, of the response time.

A

The error rate, the throughput all this kind of metrics, that you might also see as an outcome of lopes, but I think, what's also very important- is that you can compare automatically or actually captain will compare automatically the outcome of one test run to a previous test run or to multiple previous test runs, uh and this is where the captain quality gates uh come into play.

A

So if you define your slos, then captain after locust run, captain will use data from prometheus trace other sources to do the quality, gate, evaluation and compare it to previous runs or compare it to a fixed threshold. So both is possible and uh in the it's basically the step that you would do after you execute the load test with lowest.

A

What's also possible is uh to provide really a self-service platform for developers. uh I will also show this in the demo, but instead of doing the full, continuous delivery from let's say development and to hardening and production, um you could also use just one sequence in captain and only use this sequence for a performance test for kind of a self-service performance test to start a deployment with your new um version of your say, shopping, cart or I don't know which service lars was using.

A

That had already this performance degradation, but just use this version and trigger this performance test sequence and captain will make sure to do a deployment to trigger the tests uh with a predefined uh test strategy, or the test strategy can define here in the sequence and to also do the evaluation so with this, uh and there is no promotion to the next stage. There is no rollback defined. So with this you can always make sure um to use locus, plus captain uh for uh evaluating the performance of the service.

A

uh You don't have to set up anything. uh You can just use this as part of a stage that you might already have, and you introduce a new sequence in this stage, and then you can have this.

A

This testing and evaluation, and one thing that has also been added just recently from the captain community or by the captain community, is the support for uh lopez configurations. So the default way uh to execute a locust test. uh Also in captain, is, if there's just one file called locust file, dot. Pi, then character will pick up this file. You will execute it as part of the testing task.

A

um If there's also a locus.com file, captain can pick this code file up and can use can make use of this file and also related to other specification files that we or configuration files that you might already be familiar with in captain is a dot conf.tml file where you can easily map a test strategy to a file that should be executed.

A

So I just put here a very simple locus.conf.dml file where it says if it's the test strategy, if the test strategy is functional, the script is to be executed, is a basic tie file and there's also configuration that should be used, but the configuration is not mandatory. So, for example, in this case, a functional light test strategy could mean you would only only just use this file or on. On the other hand, if you don't um have the script as part of the configuration file here, uh you saw this also in the demo from lars.

A

If the script is also part of the locus.conf, then this could also um then this is also possible. So there's multiple ways how to configure which file which test file is actually executed.

A

There are instructions in the github repository of the local service and you will find your way uh how to write this locus.com yaml file in which tests are actually executed.

A

So with this, uh let me show you a little bit of a demo how to uh how to run this and how to use it so uh yeah. This is the locust service. It's on captain sandbox slash local service, and here you can already um you have all the instructions how to install it, how to use it, how to do the configuration part. So I use this in uh the latest version, uh but actually I made a small change that I will um push also um soon to the git repository.

A

So you can also make use of it.

A

What I already did is a couple of different runs in preparation for today, so I did a couple of um deliveries of my service and I also did a couple of just performance tests, the kind of sales service, but let us start with, uh let's start, maybe with the delivery, so I will trigger, as you might already be familiar with, I will trigger a new captain delivery sequence and I start with a captain trigger delivery for my stock shop project and for my shopping cart service. I will just use this uh image name here.

A

It's the cards from the captain examples and in the version 12.1. So this is the version I'm going to use in my demo, and uh this event is now sent to captain captain will pick it up. Let me just reload. I can open for quite a while um captain will now pick up this event and we'll start the delivery sequence. So right now the appointment is going on character will make sure to deploy this by the helm service and then afterwards we will see the tests that are going to be executed.

A

So let us have a look which tests are going to be executed in which tests I have already added and where they are stored in the captain hit repository. So we do have our captain lopez. Let me start at the very beginning.

A

I do have here my git repository with all the information that captain needs. I have my shipyard file as a description for my for my whole project and I have three different branches: dev staging and production and let's have a look what's happening in the dev branch. So I do have my shopping cart here, and I have also my locust file here or my locust folder with a basic dot pi.

A

So basically, in my uh yeah, I I I've just used the um the conf yaml from the uh from the demo uh and what will be executed is my basic.I and the basic.pi is not doing a lot of things. It's basically only checking on the or just sending a request on some uh some endpoints, the the route and the help endpoint. So it's doing a couple of get requests.

A

um What we can see here in the in captain is also that the local service started- and uh this is just one very small improvement I made once once it fetches the file that is going to be executed. It's also printing the file link here and where um which service is going to be tested, so um first it will indicate local services starting and once it's figured out which file to use in this case it's the functional test strategy, so it was mapped to the basic.pi.

A

It also sends an update here, so this test will run now for about one or two minutes. Once the test is finished and the test is successful, then this will be also indicated here in the bridge and captain can move on to the evaluation.

A

So in this case, yes, the tests are finished. Captain is already starting the evaluation. So in this case there is not a lot of things going on in the evaluation, because the basic check is actually checking if the service is available. If I get a positive response from no because if not it would, it would be stopped here, and since we got the positive response from locust, we can move it on to the next stage. So it's now being deployed in staging, and there are a couple of more tests running.

A

We can also have a look which tests are running in staging by just switching into the staging branch here. So I'm just switching here into staging- and I can see here- I don't have a lot of configuration.

A

So I take the default settings of locust and since the locust file dot pi is the default way, how to name the test, file and notice. uh If this file is available, captain would just pick up this file.

A

So in this case I have uh again a couple of get requests against my micro service, and I have also one post request to add items in the shopping cart and with the other requests on the items I can fetch. So there will be again a couple of minutes of those testing going on with um either posting items uh into the uh into the shopping, cart or clearing the items from the shopping cart, uh as we saw in the demo from lars.

A

We could also have a wait on this tasks, to maybe add more items in the shopping cart uh with higher probability, adding items to the shopping, cart than maybe doing the health check, but this can be. This can be improved. So the main part of this um of this file is actually, if you don't need any configuration file, then it's the only way, um or the only thing you would need to do- is to have a file. That's called locust file.pi and add it to the to little repository of capture in capital.

A

B

Might already see.

A

Tests are going on, uh it might take maybe another minute or two for the tests finished, but in this case we will also see an evaluation and we have a quality gate based on on service level objectives.

A

The data here is coming from promethors, so at the moment we are not using the data that is coming directly from locust, uh so this is something that has to be improved uh within captain itself to also make use of the data. This locust itself is providing, but we are using the data that is stored in prometer, so our microservice, the shopping cart here, is monitored by promethois.

A

All the data is stored in commuters and uh in the captain quality gate captain reaches out to prometheus fetches the data and does the quality evolution based on slos.

A

So we will see this quality gate regulation just in a minute.

A

In the meantime, we can have a look at our quality gate itself, so our quantity gate is, as you know, in uh described in the slo file and I've just added here, a couple of different uh objectives, like checking for the response time that 95th percentile, but also the 90th percentile for the 5th percentile, the error rate, the throughput and also once very specific response time.

A

That is only for adding of the items, uh since this might be a crucial transaction for, um for my shopping cart, uh I only want to have this and um the queries how to clear all this data is actually defined in the sli file, as you also might know which is also stored here, and we can maybe in the meantime we can have a look in the promethos folder. We have the sli file and all um the different queries are defined here.

A

B

See there is no.

A

Indicator for the error rate or for the wrist for the throughput, because this is the default way, the built-in way that we're going to use- and this comes with with the prometers slide service or captain anyway.

A

So let us go back to captain. We see evaluation is already finished and we can have a look at what is actually the outcome. um So, having a look here at this evaluation, we can see a list, a breakdown of all of our sli's that have been evaluated and we can see the actual value and the criteria needed to pass.

A

um What we can see here is that it's also, for example, here um we are not satisfying the response time for the 95th percentile. It's actually satisfying a criteria, it's below the 600 milliseconds. We need it to be it's actually it's very fast, so this wouldn't be a problem, um but we are also comparing it to a previous run that has been uh that has been passed. uh The previous successful run and it's fast and slower than the an increase of 10.

A

So this is something that captain can bring also to the table instead of having only the evaluation of locust, but we can also compare it to previous runs and then have a look: okay, it's actually slower than an increase of 10 percent or 25 uh or, um however, you wanted to build, uh but compare it to other ones and uh and also had a combination between relative and fixed thresholds.

A

So nevertheless we we scored enough points uh for it to be able to pass on to the next stage. So we needed at least 75 points for um to get a warning in the morning means it's allowed to move on to the next stage, and we received this score since there was no errors. Throughput was fine, and so the response time for the others indicated um a passing result, so why this is now deployed in production and tested in production.

A

uh Let us have a look in how you how we can build kind of this self-service um platform, and uh I see there are more questions coming in. But let me just start this, so let us have a look on how to how to trigger this, or maybe first have a look on how I built it here.

A

uh I was already kind of saying this in the in on the slides, but here in my shipyard file I have depth staging and production at the end, but in staging I also define a task sequence for a performance test and I want to use all the configuration of staging, but I don't want to move it all to promote it to production or uh need it to go through the f environment with all the testing. I just want to have this um staging configuration for my performance test. Let's say so.

A

I define the deployment a test in an evaluation with some properties and I can trigger this task sequence uh by just sending a cloud event to captain and trigger exactly those three tasks. I've already prepared the cloud event here. It's uh triggering my performance test in staging it's. uh It will use a specific version of my shopping cart and I have all the information I need.

A

So let me now send this to the uh to captain, and I think I have it somewhere here. So I'm sending this out to captain captain will of course pick it up and start this exact performance, performance test sequence with a deployment test and evaluation.

A

So um let us have a look here. Let me just uh oh, I don't even have to reload, so we can see performance test is started. We will start the deployment directly in staging and then execute the tests and do that evaluation. We can also give it a try with a couple of different versions to see which are the slow versions which are the fastest versions, or we could also take a look at some previous test runs.

A

I did earlier today, but while this is running, let me just find the q a window and we can also answer some questions here.

A

um There is one question: a very good question is the report of the locus test execution available in the captain's bridge. That's a very good question. So if we take a look here, uh maybe I have a look at this one.

A

No, not not even here, but I had some previous runs where the test execution was failing. If the test execution is failing, you will get the full. um Basically, it's not the stack trace, but the full output of low cost by the test was failing. You would get it directly into the captain's bridge for successful runs. We right now do not include the full output, but you could also include it. What I did just in a kind of intermediate version is to also include what is the test strategy.

A

What is the the file that is now actually executed? What is the url, where captain's sending the traffic to uh where locust is sending the traffic to so? I just included this here. um This can be improved and we are also open here for uh suggestions from the community. So if you have, if you want to join us here in improving the locust service, um feel free to reach out or open an issue or request on github repository.

A

Let me have a look if there are more questions coming in. I think there is there's more.

A

So are there any plans to attach more than one file to the captain, locust run? um I'm not sure. If I understand correctly. Maybe that means, if, uh if test runs, can be um that there can be one test run after the other, and this is um this is for sure possible. um The only thing you need to do is you just need to define, or you can define more test tasks.

A

You can even name them differently uh and have different tasks in your captain shipyard, and then locust can also react on this and start tests there, so that that is for sure also possible to run multiple tests and not only one test.

A

So I'm not sure if more questions are coming in, but while we were waiting, we can see in this case a delivery or a performance test. Sequence finished with a specific version of my shopping cart, where you did the deployment, the test and an evaluation. um In this case, the operation was completely successful, so it was very fast. um Also, the the adding of the items was faster than the previous run.

A

So in this case uh you can provide your developers a kind of a self-service platform where they can just execute um performance test runs where locus tests are triggered and they would just use the lowest files that you already have and have them executed by captain, and then they also get the slow base, quality integration.

A

B

A

Just uh if there are any more questions, I'm happy to answer them, but just let me go back uh to the slides for just a second to kind of uh wrap it up here and then open up for more um questions and answers.

A

So um if you want to give it a try, um go to locust.io to learn everything about low curse or just watch the video again where lars was introducing on our locust and showing us how to write tests.

A

um If you want to use the locust service, it's available on captain sandbox on github uh by a localist service, and you will find the full installation instructions there and also configuration options. um If you just want to learn a little bit more about kept and how to use captain, you can go to tutorials.captain.sh.

A

um There is right now there is no dedicated locus tutorial on captain.sh. uh If there is any demand, we are happy also a dedicated locus tutorial there, uh but you can also just replace your um j-leader files that you had with uh with the logos files. So there is no need to rewrite the whole shipyard.

A

um You would just remove the cheap meter service and use a local service and handle this test, so this is uh in this way. You can already experience this. uh If you have any questions um around locust, then please uh join the slack channel of locust. If you have any questions around the integration join uh dislike of captain and we're also happy to help you there.

A

So with this, um give it a try yourself, uh either by the locus binary, to get started with writing tests, and once you have written your tests, you finalize this. This part give it a try and use the locust service in captain, um so I'm opening up for more q, a I'm not sure.

A

If I answered the question from earlier, um if I answered it correctly, please let me know in the chat or in the q a um if, if this answered the question or if you have any more questions uh to ours, then I think we can also take them. Take them here.

A

Actually, I would have a question on uh on some kind of best practices how to how to write or structure some locust files. Is there any kind of guidance uh on the sizing of a locust file on uh how many tasks you should have in uh in one locust file when you want to start one set of tests and when you want to start another set of tests um like doing smoke, testing or doing these kind of things in separate files or separate steps in your whole city process? So do you have any?

A

uh Can you comment on this best practices.

B

Sure I mean um it's hard to to name any best practices in general. I guess like when you're load testing your your first step is usually just warming up your system uh running a single user or very few users. Just to make sure that caches are warm and system is, is active and then start a proper load test a little while later, but in terms of how to structure the tests.

B

um I guess uh what we do at sanskrit spiel, for instance, is that we have a base user class, uh which has uh wrapper methods for for things like uh making rest calls like uh all the rest calls that we make may have like a certain header or uh or we want to treat the responses in a specific way.

B

For instance, we check for a specific place in the um response json for an error message, and if there is an error message, we automatically fail that particular requests. We don't have to do that explicitly check for that. Every time we make a call. uh So that's a useful thing where you just you, just define a base user class which you could put in a separate file and just import it. uh The one thing to remember there, that is, that you need to market abstract, uh because if you don't then locals will feel like.

B

Oh, this is another user that I'm supposed to run and then just make a this. When you say you want to run 100 users, it might run 50 of your your base class user, which you didn't really intend to uh yeah. um I I guess um it's really hard to say like in general uh best practices, but but I hope that gives you some some idea.

B

So so maybe one thing to add: being a passionate python developer myself.

A

You don't have best practices in low cost, because you have best practices in python and you you cannot think of low cost of something like chem meter, where you create a jmx file. You really write python code according to the best practices of python, and then this just looks nice and it works.

A

That's the best practice right write the code in python.

B

Yeah, I absolutely agree uh the one, the the thing that we've standardized, both in in locust since uh a couple of versions back as well as at svenska, spiel and jmeter plugins, and that stuff is to use uh pylint and uh black for auto formatting, which makes it really uh like it's very convenient. It's very easy to read other people's code. It's very easy to well. I guess that applies for any python project really, but um that works very well for us.

A

B

That, actually, uh that was a christian.

A

um Who was uh commenting on the python and uh big shout out on on here uh to christian and dominic, and tito also helped in developing the the locust integration. So I think this was really here a kind of community uh achievement which is really great, and I was just the one presenting the outcome of the locust integration. So it was not was not done by myself, but uh really.

B

Cool to see this, it.

A

B

A

And I have to say it was quite simple: to build the service itself using the service template and also reading up the low cost documentation, so those two just go hand in hand very nicely. um So yeah, as you say, a huge team effort, and I think uh so the locust um website is really a great resource to learn how to write locust files. uh I kind of made my way through just by taking a look at a couple of examples to write or update a couple of um files there and yeah.

A

I know that the locust integration that we've built is already used in production, um I'm not sure if anyone here on the call would want to comment on this. Also, um if not maybe we can then uh see this in one of the next captain user groups or develop meetings. How this service is already doing a lot of load testing in in our continuous delivery.

A

Yeah are there.

B

A

More questions from the community feel free also to raise the hand I am happy to unmute or allow you to unmute.

A

Okay, if not, I would like to thank you lars again for joining us here in the captain user group. It was really great to learn what locust is, how we can use it, how to write our files, how to reach out in case of questions.

A

Thank you so much for joining um and yeah thanks. Everyone also for joining. uh If there is, if there are any more uh questions. um Oh is there any other question coming in? No, no, if there.

B

Are any more questions, uh please.

A

Join us in slack, don't hesitate to reach out and we will have our next captain user group meeting on the third tuesday in june, so uh in about one month's time, and we will hear a captain adoption story uh from a large company, so stay tuned uh okay, so we are uh already at the top of the hour. Thank you so much lars and happy load testing everyone and see you all next time.

A

A