gRPC Community, 26 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: gRPC May Community Meetup/ Demo: Performance Testing gRPC Services by Mostafa Moradian

Description

gRPC has gained a lot of attention as well as a wide adoption since it was first released. With the release of k6 v0.29.0, a native client for gRPC communication is introduced. In this talk, Mostafa Moradian went through the basics of performance testing in general and load testing gRPC services in particular.

A

I'm mustafa moradian, as you may have heard by now, I'm a senior software engineer at k6, which is a graph owner labs company.

A

I am presenting performance testing grpc services today, I'm mostly or generally talking about k6 but we'll.

B

Also show a few secrets.

C

Of testing grpc services.

A

um Yes, so I have a few years of experience. I am a senior software engineer. Also a security lead. I've been with the company for about three years now.

A

My main area of expertise is python, but I also code in any other language to get the job done and I'm also still learning. So, uh if you want to reach me out, I'm available on social media and also on my website so feel free to reach out with questions with. You know, suggestions feedback, whatever you feel like.

B

Okay, um I'm I'm gonna, I'm gonna present um performance.

A

Testing in general and then talk about k6 and then we'll I'll have a small demo uh after the demo. I will talk about k6 features after that. I'm.

B

Going to introduce grpc and k6 show you.

A

A few scripts for performance testing, grpc services, and then I will finish my slides with questions and answers.

D

C

What is performance testing.

B

Performance testing.

C

Is basically testing a system to see how it performs and behaves.

A

Under various conditions,.

C

Usually on the load to fulfill non-functional requirements, so this is kind of a somewhat cryptic sentence, so I'm gonna decrypt it by explaining.

A

Every single word in sentence so.

C

First of all, performance testing is a general term that that that refers to this specific sentence so load testing. I will talk about load testing a little bit later, but load testing is a sub is a way to do performance testing, so they are not equal.

C

um Just just so. You know. While we are doing the testing, we are testing a system. So what is the system system? Is we call it a system under test? So in our scenario today.

A

I'm gonna be using grpc bin, which much like http bin is a is a testing system is a.

C

um Is a test system for the grpc protocol.

A

I'm sure you you've heard about it, uh so our system.

B

Under test today is grpc bin.

C

To c means to observe, to monitor, to, to see metrics, to see logs, so seeing is that how it performs and behaves is basically to see what the normal behavior of a system or the system on the test is and how the how the behavior on the load, stress or soap testing is which I'll.

A

Explain a little bit later.

C

Various conditions are normal conditions, so in a normal condition, you make a few requests and the system just accepts a.

A

Request processes them and returns them back determines the response.

C

But when you put load, then you see how it can handle that load and see if it sustains the load.

B

um A little bit further, you can increase the load a little bit over the threshold and and basically.

C

Stress test the system: you can keep running the test for a few hours and you're soaked testing the system. So these are the terms.

A

You will be hearing from me a few times when talking in this talk.

D

B

Usually stress testing.

C

Is is, is used for putting more load on the system to see if it can handle more load than we expect and soak.

A

Testing is used to test the system over a long period of time to detect.

C

Slowdowns and memory licks of sorts if they exist.

C

Last but not least, we have to fulfill some requirements which are non-functional requirements. Our non-functional requirements that we want to fulfill are responsiveness.

A

Stability scalability, reliability. You know speed.

C

And these kind.

D

A

D

C

So while we are doing the performance testing, we are basically measuring certain qualities with quantities, so we are quantizing those qualities to be able to test them. So, for example, if we want to measure reliability, we should measure latency. We should measure response time failure rate. You know all those metrics all those indicators for us.

A

To understand, if our system is reliable enough or not.

A

All in all, it's all about slis and slos and slas.

C

So we probably have an agreement with the customer, the team next door, our boss, or some stakeholder in the system. So this is our sla, so the agreement and for this agreement to be fulfilled like, for example,.

A

uh Imagine that we have a service, a grpc service, an http service or anything, a microservice, a borg, something.

C

And we want to deliver 500 milliseconds less than 500 milliseconds for the users to be happy so.

B

What should we aim for? What are the objectives? What are our slos, so the.

C

Objective objectives we want to achieve should be lower than our slas, otherwise we will easily hit our sla like.

A

In a few seconds of testing, so, for example, if your sla is like.

B

500 milliseconds, uh your objective should.

C

Be 300 milliseconds and what you're measuring in this whole scenario or the numbers, the indicators, the sli's.

A

Or like, for example, latency so latency.

B

Is defined differently in different systems, but usually.

C

How long does it take for the system to process and.

A

Return, the response after you make a request. This is the latency.

C

It could be other metrics, but this is just.

A

An example to show you how this sla, slo and slis come into play when you are doing performance testing.

C

Okay enough with the performance testing, so let's see what is k6.

C

B

Actually is modern reliability, testing tool for developer happiness. I will talk about every single word in this briefly.

C

So it is written in go and.

A

uh It's scriptable in javascript I'll show you a sample script of k6, so, for example, this is the script of k6.

D

I don't know if you can see that very well, but I.

B

Hope so so uh it has a few imports. It imports some modules.

C

Which are from the k6 api, you set a few options: the number of virtual users, the duration of the test,.

A

And you have a default function.

C

That defines which requests.

A

To make and how to check for them like the.

C

A

B

You saw this script is written in javascript, the syntax uh supported syntax is uh ecmascript, 2015 and es6.

B

C

An easy to use interface and a lot of apis to work with it. It's easily.

A

Integrable with cis and.

B

Monitoring tools and stuff has a lot of outputs. uh It is extensible in go via.

A

Xk6, so this is k6.

B

A

As you can see, it's already mature- and this is xk6 so.

C

You should use xk6 to build extensions for k6 that.

A

Takes your extension in and uh builds it and gives you a binary, so let me, for example, give you an example.

B

Of that x uh k6, so this.

C

Is this is an extension I wrote to load test kafka so in.

A

Order to build this, you just need to.

A

Install xk6 and let.

C

Xk6 build the binary for you and there you have it.

A

Let me just show you in a few moments how this is done so, for example, imagine this and then I just.

C

I I have already installed xk6, so I just want to build xk6 kafka. The latest version we just hit enter.

B

It just goes and fetch the resource and builds.

A

The binary for us as simple as that and if I just ls into the directory, you will see that I have a k6 installed.

B

So it's developed version.

A

Because I we built it ourselves and then the latest version, almost the latest version.

B

So that's the extension.

B

C

Also, a gpl license so free and open source there you have it.

B

It can be used for load testing.

A

So testing you know stairs testing all sorts of different testing. I've told you about. It can also be used for performance and performance testing and synthetic monitoring and yeah. We also have a few extensions to help with chaos and reliability. Testing.

B

So, for example, we have xk6 browser to do the browser testing. As I told you about it,.

A

Browser automation and everything, and we also have xk6 chaos if I'm not mistaking.

D

It with something else, yes,.

C

So this is for chaos testing, so an extension to do chaos, testing.

A

We also have a lot of things for kubernetes and stuff if you're looking into those.

D

C

It supports http, 1.1 and 2. It supports web sockets and grpc internally, but then you can have there.

A

Are a lot of extension to choose from, for example, the one I've shown you you can? You can use my extension to load this kafka, your consumers and producers.

A

And and and last um you can also.

C

B

C

It doesn't support browser.

A

Api or node.js apis, but you can bundle a lot of modules that do not use such apis.

A

If they, if they do, I suppose um you can also bundle some because they're shims and wrappers and stuff to deal with those apis, but all in all, you can bundle a few uh modules and it also, we also wrote a lot of modules ourselves. So if you go to the x k6 documentation, you will see that there are a few internal modules: how to bundle modules, how to import modules, how to import remote like modules from https, endpoints and stuff. So that's that.

C

So now's the demo time: let's do a demo.

D

Okay, um let's see what we have here.

B

C

So this is the same exact ripple that I just shown. So this is the repo. If you want to access the code. This is public.

D

uh Let me go back, so I I do have k6 installed. You can.

A

B

Install it, uh let me show.

D

B

C

Easily install it, you can you either use, docker or or you can just download the binary or.

A

There are also like app.

B

Dev packages.

A

B

Packages available.

A

uh So I used apt to install it, so let me see.

D

A

Yes, there I have it. The latest version.

B

A

D

D

C

Let me first go through the script and then talk about running.

D

C

So any script in in k6 has a life cycle, a life cycle.

A

In k6 means that.

C

There are four steps in the code, so there is the init code, there is the setup code, there is the default function and the teardown. I just show you a.

B

Few of them, so these are the modules.

C

You have to import, so this is the http module and by the way, all of these modules are available here.

D

Let me show you.

B

uh We have the javascript api, so you.

A

Can just go to the k6 api and see all the modules and everything like crypto data for for using shared arrays, encoding execution html. So the http, while I say like.

C

K6 http this it is here.

A

Like uh you can like go to the get parameter and see what parameters it accept and the response it just returns and then for the response you can just see what's in the response, so that you can check if, if the things you're looking for is in the response back.

C

To the code, so these are the imports.

B

C

Outside the default function, what is outside the default function is called init code. Whatever you run here will be shared across all the virtual users. So what is the virtual use? I will. I will talk about virtual user in a few after the demo.

B

But uh what is a virtual user? It's basically.

D

A way for uh the code to.

C

Be able to initialize something so, for example, if I go to the grpc script.

A

You will see that I'm initial initializing, the the grpc client outside the virtual user code, which is.

D

Here? Why? Because.

A

I want to share this client instance.

C

Across all the virtual users, that's why that's it, and then we have a.

B

Lot of options to set here, I just shown two for the sake of easiness, so.

C

The virtual users or users, if you want to call it that.

C

I'm going to use 50 virtual users and I'm going to.

A

Run this test for the duration of 10 seconds.

B

C

A virtual user do so a virtual user.

B

Each virtual user executes.

C

This default function or any other function that you define.

B

A

Talk talk about it in the scenario section.

C

And we'll make a few requests here: I'm showing http requests.

A

So uh I'm just like initializing the variable to receive the response and I'm using get to get the test server of k6, and then I also imported check. So check is a way.

C

In k6 to assert assert responses that you get from k6, for example, we are asserting that inside the response, the status is 200..

C

Okay, you can assert many things, it's and, and unlike assert, it doesn't stop the test from running it just records the number of time it succeeded the assertion and number of times it felt the assertion. So the difference with assertion is that so.

D

Let me just save this script and run it: okay, okay,.

B

uh If I run k6, you will see the commands and supports the.

A

Available commands are here, you can play with them and there are a lot of flags so in.

C

Order to run this.

A

C

A

D

Http script.js.

B

And we we see that the logo is printed. A few.

A

Execution context variables the scenario that it that it is using.

B

And then you will see the progress you've seen it.

A

It's done and then you see this message, so you see a green message because basically.

C

uh This check verified that all the requests that we sent had the 200 http.

A

Status code in the response and.

C

A

A lot of other metrics, so we've we have sent uh here, I can show you. We have sent 5006 requests.

C

In just 10 seconds, these 50 views we've gone through the default function.

A

2500 times so this is called iteration, so the number of.

C

Times this loops keeps going so per each loop. This.

A

Code is executed, which is called an iteration um and then.

C

This is the amount of data we received. This is what we sent. This is the time it took for.

B

The http blocking to.

C

Receive the data or to send or whatever.

A

This is the time it took for connecting, while sending the request.

C

This is the total duration, including connecting uh blocked receiving and sending and tls handshake, and probably waiting.

D

If I'm not mistaken, um yeah and.

C

Yeah the expected response. We expected response from all of the messages because we basically checked for the response. We we had the option to discard all the responses, but that's another topic.

A

That's for testing under huge amount of load, for example millions of requests per second. Then you, you don't care about the response, because you know you're bashing. The system.

C

And then, by the way you can you visualize all this data.

A

And send it to influx db, prometheus and a lot of stats, the telegraph and on the system.

C

This is the total duration of each iteration took to send.

A

Those requests- and these are the number of views.

C

And views max, so we used all the virtual users in the pool of virtual users. We have.

A

That's that I will answer the questions. After the talk, we will have a q a session.

B

C

Now, you've probably seen a lot of features of k6 you've seen the test lifecycle.

A

I just shown you uh there's the init default function, you know vu code, we call it and then options and imports and the turn down. So I didn't show you the teardown, because it's.

C

Not needed for http you've seen a lot of metrics. These are the metrics. These are the built-in metrics you see. These are called built-in metrics and then you can also.

B

B

Custom metrics k6 support four different types of metrics, so.

C

Counters gauges rates and.

B

Trends and you can import these metrics.

A

You see like you, want to have a trend and then you import it and you add, for example, your timing of your your request.

C

A

Of your request and then record these uh and then send it somewhere uh like, for example, like this.

B

You see it, it records everything.

A

You can also tag those metrics, and these are the metric types supported.

D

There are also checks that I talked about.

A

D

Basically, if I go back okay.

C

There's something very interesting, at least for me, to support something called thresholds, so these are pass fail. Criterias, basically,.

D

A

Slides ago I shown you that we need our services to be responsive under 500 milliseconds.

C

So this is how you set those thresholds.

B

A

All these thresholds are set on the script using thresholds that is supported in k6.

B

And this is the syntax in the options, if.

D

Let me go to my own script, so.

D

B

So grpc request duration is an internal uh metric.

A

For uh recording the duration, it takes for a grpc request to you, know, be invoked and then receive.

C

A

C

We created a threshold in the options and we said: okay, we want the threshold, so p5 59, so 95 of requests or basically, we allow five percent of the requests to be above 300 milliseconds and the 95 percent should be always below 300.

A

C

We also set this to abort if the. If there is more than five percent of failing of thresholds and.

A

We give it one second to uh once again.

C

Delay to abort the evaluation, because it takes time for this threshold to be calculated, because basically it takes time for the request to go and a response comes back, especially if you're making a lot of requests and.

A

Then I also set another threshold to say.

C

The you've probably seen this five nines.

A

It's a very famous one, so we say the five nine of system is this, so you can say that p five five nines should be less than 500 milliseconds.

A

B

About thresholds, and then there are modules I've already shown.

D

B

uh But I can also show.

D

You a few others. There are js libraries.

B

Provided by us like expect.

A

Httpx recently released and aws also recently released, for example, with.

B

A

Aws library, you can access s3 and access your pockets and storage and stuff.

C

You can also store secrets in.

B

In secrets, manager in aws and retrieve your secrets from there, instead of.

C

Passing it in the script and a few different.

B

Things uh also, there is the chai js chair session library, uh the port of it for k6. uh Can you can use it.

A

To do bdd tdd style sessions, for example,.

B

You you import this to describe and expect and then.

A

You set a threshold and say: okay describe this the response and expect this status to equal 200. This is how bdd works, behavior, driven development or test driven development, whatever.

B

C

This is how you have the things, so it internally uses check to expect.

B

K6 also supports tags and groups, so you can tag your metrics while you're sending them to um your. You can tag metrics scenarios, groups.

C

You basically can tag everything almost um and tags are very helpful when you are using.

B

A monitoring solution.

D

A

Like send a few information like.

C

uh Anything that you can tag on your metric.

A

And then filter.

C

Your metrics on your monitoring solution by using these tags.

D

Then there are groups that you.

C

Can use to group requests and then have them tagged all at.

A

Once and then there is a new metrics added called.

B

Group duration,.

B

Which is how long it takes for the whole group of requests. For example, you are testing your uh logging login uh microservice.

C

So you group all the endpoints in one group and then it shows.

A

You how long it takes for you.

B

To send a login request, receive a response and then.

A

Send another request to verify the user, for example, or send another request to verify the profile of the user or do something else with the profile.

D

C

Are a few system tags.

A

Like your name, group check and stuff.

A

C

You can also define your own tags uh like this.

A

When you're making a request, you can like say that, okay, this is my tag, and this is the value of my tag, uh and this will be uh shown on your request. Let me just.

D

Quickly, show you how how it works.

D

Let's see if I can yes, I'm attack, oh no, you can tag everything.

D

If yeah, let's hope this is correct,.

A

Let's run the script again and see what happens.

D

You stop the test.

A

Okay, I stopped test, so everything is gone into. Haywire.

B

But the tags are applied, I I'm pretty sure.

A

But they are not shown in the cli. As far as I know,.

D

A

B

Go to the next.

D

C

The next feature is scenarios. uh Scenarios is a way to describe how you want your load.

B

Because because k6 has a.

A

C

To share iteration to share.

A

Vu's uh to do constant g cross rate to do constant arrival rate to like buffer request, if, if the system cannot handle it and so on and so forth,.

D

A

Can use this to define a few scenarios, for example like this, for example, here we have two scenarios and each scenario starts at a certain time and it does something at that time. So.

D

B

Then you also have access to environment variables, so k6 can accept environment variables on the command line.

A

Or you can include your system environment variables and you can deal with a lot of configurations and you also have access to execution context.

B

Variables, which are basically what you are you running on what.

A

Iteration, are you executing and stuff.

B

uh To the good part.

C

So for testing grpc uh k6 has built-in support for uni unary requests.

B

Support for event, loop is added in version, 37 uh version.

A

uh Now it's version 38.3.

B

But we are slowly but surely.

A

Moving towards adding streaming and all those protocols that need this streaming and event, loops and stuff.

C

We would be very happy to see your contributions on issue number 2020.

B

Which is a support for grpc streaming.

D

C

That's that and let's go see some demos, some more demos.

B

Okay, for the sake of this demo, I just use grpc bin. So let me show you yeah, so I I just um cloned grpc bin. It.

A

Is also a sub module in the repository I um I.

D

Am like uh you, I've.

C

Stored these scripts on, and uh I've already built this.

B

Thing but I can build it again. I think fancy happens other than building a binary and you can run the grpc bin, which is a test api.

D

Like this, let.

B

D

Just quickly show you, your pc bin.

C

Yes, this is your pc bin for those.

B

A

What it is so it's a test service test service like http bin, and it has a few services and you can use it to test over insecure and secure grpc and you can use docker to run it. But since I have go install I just built built it and just ran it.

B

And what I'm going to do is to first explain.

D

C

The script works in terms of grpc, so we should first import grpc from the net module that grpc module.

B

And then we also uh import sleep from k6, I'm using sleep basically because normal users.

C

A

Delay in like, for example, clicking on some button and then like scrolling down and clicking on another thing. So.

C

D

Is used to wait, you know.

B

C

So that you don't bash the system like crazy, you should initialize the client, the grpc client, and then you should load from this directory.

B

The hello protocol, so let me quickly show you so. This is also available as a sub module, so the hello hello service.

A

And the hello protocol, so, as you can see, there are.

D

Four uh endpoints, I would say.

B

This we are interested in, say, hello, which is a hello request, uh is an optional string greeting the first field, and then it returns.

A

The response, which is a required response and the field is called reply.

D

If I go back yes.

B

I'm gonna use five views for now.

C

And the duration of three seconds just to show you how it works, the default.

B

Function here you see it's, it's a closure in this default.

C

Function so every virtual user go through this loop, so it connects.

A

To this address in plain text and then.

C

There is some data we want to send and then we invoke this endpoint, which is say hello in the hello that hello service with this data, so an unary request to just receive something I just wrote.

C

I use console.log.

B

To see what's inside the message for demonstration purposes, but it's.

C

It is not usually recommended to console.log, while you're testing a system, because it generates hundreds of.

A

Thousands of messages per second and your system will be unresponsive.

C

B

Quickly and then I should close the client on each iteration. I can.

A

um Just do the close on teardown, but I just keep to this example and.

C

Then I wait, I send the request receive a response and then wait for one. Second.

D

Okay, let's go here, k6 run.

B

D

And max from it: okay, five, five: five: okay, you see.

C

A

Basically, because we are running for three seconds and we have five virtual.

C

Users so three to five and we are waiting for a second, so each second, we are sending five requests from five virtual users, and this is the amount of data that we sent and received.

C

This is the duration it took for the grpc.

B

Request and response, so the average is 903 microseconds. The minimum.

A

Median max p90 and p95 each.

B

Iteration took one second exactly um kind of obvious, and we.

A

Traded 15 times because.

B

Like you know three seconds, uh each iteration.

A

One second, you know three to five. Five views is 50., so.

D

That's that now I'm going to show you a more slightly more advanced script.

C

B

A few other options, so I'm using.

D

A metric, a rate.

C

I'm introducing checks, groups and slip from k6.

B

I'm not using I'm not loading the protocol file because protobuf files, because I'm gonna use reflection for the server.

A

To decide what I'm, sending and and and serialize and be serialized based on that, I.

B

Also introduce uh okay, so we are using 50 virtual users. We are going to run.

C

B

C

Seconds there are two thresholds.

B

That I've already shown you for the request, duration. I I want the p59 to be uh less than.

A

B

200 milliseconds and abort, if, if it goes above, um I'm gonna set it to something.

C

A

Like I know that it will fail on 10 seconds uh fingers crossed.

D

A

Yeah p, nine five five lines is.

B

Less than 500 milliseconds, I introduced the rate um for the number of requests, so it records uh here.

A

Okay, I will explain the rate a little bit later.

B

uh We are making the same connection, we are.

C

Grouping our requests- and we call the group say hello to bert. The data is the same per each loop. We are, we are having our own loop that creates 100 requests, it invokes the say, hello 100. This is not needed, you can use virtual users and iterations and.

B

Stuff, I'm just showing for for the demo purposes, I'm just showing so that you can have an idea.

C

Of how we can heated thresholds, this is this is definitely not needed. Just let k6 do the job, for you increase the number of views and stuff instead of doing this, this is an anti-pattern. Don't do it so.

B

This is a rate metric that I've just.

B

Initialized here and I'm.

C

Checking if the response here, the status of the response is okay, so it records the number of failed and succeeded, requests and show us at the end, and then I also check if the response status is okay, that's a separate thing from the rate metric and I also check the message to see if the reply that I've shown you is hello. Birth.

A

It also checks, for you know, data type and stuff, but that's.

D

Out of the question you can introduce a sleep per request.

C

So 100 slips, that's not a good idea, probably you can introduce sleep per group and then at the end you can also sleep per virtual user per iteration. I would say so uh then I will close the client after making these many requests.

B

C

Test and see if we can hit the.

D

Threshold, yes, okay and there we go okay. Let's wait.

B

D

10 seconds, I hope it fails, it doesn't so that's that.

D

C

The request duration was so fast, it didn't fail.

A

The average is.

B

One millisecond.

A

B

Should go below millisecond.

C

Or increase the number of virtual users.

A

So let me increase it to.

B

uh 5000 virtual, this is this is not a good idea.

A

If you have a huge script, yes, because it uses a lot of memory, uh I hope my system doesn't crash fingers crossed.

B

Okay took a few seconds to um yes.

C

There we go, I managed to fail, it.

D

Okay, uh we have five thousand views.

C

A

500 yeah, I'm mistakenly saying 500, maybe.

C

And it took three seconds to bash the service and get a response.

A

That is above 10 milliseconds.

D

A

That's it and yeah. The p, 90 and 95 is like two seconds, so that's huge, that's very above 500 milliseconds and also even 10 milliseconds.

B

B

D

Let's go back uh yes, it concludes my presentation and I am happy to answer your question and thanks for listening. Thank you mustafa. um Thank you all for uh for being here and please. If you have any question, please, um okay, so paul! Please go ahead.

D

That was an accident. Sorry, no.

C

Worries no worries, I'd be happy.

D

To answer your questions by the way, do you have any question paul, okay, so milty.

D

What a wonderful presentation looks like a very interesting interesting tool, indeed, indeed, of course, and let me see if there's anyone.

C

If, if you want to know more about k6, you can always consult the documentation, but we also have a k6.

A

C

A

There, the last one.

B

Was uh on last friday, with paul leandro and.

A

Mihail we joined together to answer people.

D

People's questions.

A

D

A

On youtube so just follow there.

C

And I'd be happy to answer all of your questions. Even after the meeting.