Cloud Native Computing Foundation PromCon Online EU 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: Load testing your Prometheus Remote Write endpoints - Daniel González Lopes, k6.io

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Lightning Talk: Load testing your Prometheus Remote Write endpoints - Daniel González Lopes, k6.io

"Remote write storage is supported by several systems (Cortex, Thanos, M3DB, InfluxDB) and since 2.25.0 by Prometheus itself.

In this lightning talk, you will learn how to load test these systems using the open-source tool k6 and its fresh new xk6 remote write extension. Also, we'll briefly talk about additional use cases and how to export the results to Prometheus."

A

Hi welcome to love testing your promise here, remote buy timeframes. My name is daniel. I'm a tribute in here and on my free time, excellent, maintain rafanatenko. You can reach me on some places online. So if you want please go ahead so promisius offers this removed white api that lets you send samples from a prometheus instance to some external backend.

A

There are a lot of compatible backends. Two of the most prominent ones, are cortex and thanos. Also, it's interesting that prometheus itself supports remote, quite signs 2.25, it's experimental feature, but it's something very good so some weeks ago. Actually this is why this torque exists. I deployed cortex in microservices and mouse on top of kubernetes, and I did this for learning purposes and the setup was great and I thought everything was running smoothly, but I have some questions.

A

I wasn't sure if I had configured all the components properly and I didn't know if I have provisions enough resources in government to handle some basic laws. So what I wanted to do is I wanted to generate some loads on this ingestion path and to understand how cortex will behave and how all the components will will work. So I picked my favorite load testing tool for the job and that tool is physics.

A

Kisis is an open source tool written in gold that lets you create your tests as codes in javascript. Even the configuration can be called, so you can count everything on a repository or hit repository.

A

I want to say, as a disclaimer that I work for kisses for the company behind it, but I have to say that I'm here not because I'm sponsored or something like that, it's just because I like the tool- and I think that it's useful for some user spaces.

A

So I thought okay, I can create a test in e6 using javascript for testing my remote brightness points out. That's it, but I'm a little lazy and I thought, but I don't want to write that much javascript, so yeah here it comes disclaimer. This stuff is experimental, but cool. Keep that in mind.

A

Earlier this year, k6 developers added support for key 6 extensions, and this means that you can write, go code, use your code models and get that code for javascript from your test. So when I saw this, I went ahead and I created a bunch of extensions and one of them was for remote white.

A

So this extension is really basic, but we will see how to use it um yeah. I hope you like it so for using the extension you have to create your own k6 binary. This is very easy. You just have to download the x kisses helper and once you have that you put the version of q6 that you want to use as a base and put the name of the stencils that you want to bundle onto this new binary.

A

Once we have the binary, we can create our text. As you can see, I'm importing my remote write extension in javascript, I'm instantiating, a new client pointing to my remote right, packets, and once I have that I can use the store now methods. This method is used for sending a sample to these remote right vacant.

A

Some levels with some value right and storing this this sample also I'm checking that the response from from this request is 200, because I want to see if everything went great properly, I'm going to run this for 10 seconds and with 50 producers.

A

When I run this, k6 is going to give me a lot of information on how my test my really basic test, performs I get some basic metrics generated by k6, but also custom metrics, that my extension generates like remote request, duration and remote write, request total.

A

So probably you are thinking okay, daniel. It's handy to get this summary, but I would like to see how these metrics perform over time to understand how my test behaves and keysig support. Is you can ship your metrics in real time to some backend and to charge them whatever you want? The thing is that promise use is not yet supported.

A

So how does the future look like for the remote write extension? I mean it's tiny, it works and it lets you model almost any scenario that you want. You can write your javascript codes and model it as any other test. I would like to refactor it a bit to use the official remote right, client but yeah. On the other hand, about exporter or the output super misuse, it's something that we don't have.

A

There are ideas to what the support to promise you. There is like a related issue if you want to hack this from tokyo, so you think that it's interesting, please, foreign comment. What you will expect of this integration. um Yeah just give a like on the issue, if you find it useful, on the other hand, side, one class that this can be implemented as an x-kisses output extension, someone from community are really added one for an influx db version two, so maybe when the stock is released, we already have one for probably use. You know.

A

Thank you a lot for for coming to my presentation here at chromeprocom, and I hope you have fun bye.