Grafana Mimir Community, 25 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Grafana Mimir Community Call 2022-08-25

Description

Grafana Mimir Community Call from 25th of August 2022. We have discussed changes in upcoming Grafana Mimir 2.3 release and new usage stats feature.

Agenda:
- 00:00 - Introduction
- 00:30 - Mimir 2.3 release
- 13:25 - Usage stats

Join us on the last Thursday of every month. Please see the linked document for meeting notes, agenda and details on how to join: https://docs.google.com/document/d/1E4jJcGicvLTyMEY6cUFFZUg_I8ytrBuW8r5yt1LyMv4/view

A

So yeah uh is the recording on yes all right. So welcome to the mimir community call. um We have an attached agenda to the call, and should I share that. Should I share my screen.

A

Oh, you sure didn't link okay,.

A

So we will talk about uh two bigger topics here: one is the upcoming mirror 2.3 release where for mimir, which for which the release candidate should be out by now. I think uh maybe someone can confirm.

A

It's not okay, it's not out yet, but.

B

We will have a reason.

C

To do it soon,.

A

C

And uh publish the release candidate today, yes, okay, so it will be published by the end of the day.

A

Okay and so let's go through some of the improvements uh coming out in in 2.3, um there's this uh ingest matrix in open termite format um with someone care to elaborate I'm more into the hunt, part of things.

C

A

Can spend a couple.

C

Of words in it, uh so the open telemetry collector actually support uh multiple protocols.

C

So we already supported the open telemetry collector using uh the prometheus remote threat protocol, which is basically the same problem used by prometheus or the grafana agent to remote write the metrics to mimir, but in mimir230 we are going to introduce the native otlp prodigal. So the open, the native open, telemetry protocol um support in mimir as well. um The support will be experimental, uh but.

A

Yeah, uh that would basically.

C

A

Second way or a.

C

Second, uh probable supported for remote writing to member from the open, telemetry collector.

A

Thank you and next up is increased, instant query performance. I've don't see, maybe marco, if you could take this one as well. I don't see that.

C

Yeah sure so in mimir we use several queer acceleration techniques other than various caching layers. We already support splitting range queries by interval like a three days. Query is split into three one day: queries each and then the result is aggregated together or we also support queryshot query. Sharding is supported for range and instant queries as well, while the instant sorry, while the time splitting is support, was supported, adjust for range queries in mimi 2.30.

C

We are introducing an experimental support for splitting instant queries by time as well. Similarly to how it works for range queries. um It's still experimental. uh There are some known issues, um so our main suggestion is to not use uh this feature in production. Yet, but it's something we could iterate uh over the time.

A

B

I'm wondering how can it work similarly for instant query, if you don't have any time range, can you explain that.

C

B

C

Similarly, I mean conceptually similarly, so when you run an instant query, you still, you can still select a range of samples for each series, which is the range vector selector, uh the one in the square.

A

C

Like a square bracket, five minute or one hour six hour whatever it is the way the instant query is splitting by time works is we use a technique? Similarly, to query shading, where we rewrite the query into multiple partial queries, um each one, you know, processing uh a subset of the time range um and then we run an aggregation on top of the of these partial queries.

B

A

All right uh next topic is tenon federation for metadata queries.

B

I can mention shortly one: this is about a weho metadata api in mimir for fetching like help and uh the type of the metric, and now it works for confederation, feature as well or it works together with confederation feature, so it actually queries all alternates and joins the unions, the the results together and sends the output. So that's the improvement.

A

um When you say all tenants, you mean the tenants that are passed in the scope, org id or like all oil tenants.

B

Only those that are actually in the header in http header of the query right.

A

Thank you and simpler object, store configuration. um I have actually started to document this, but if somebody is more knowledgeable in the implantation speak up now,.

A

So this is basically, if you look at the mimir configuration there are a couple of places where you need to set up your object, store bucket access and then credentials and so on. This is the block storage ruler, config art manager and for enterprise. It's the admin api and it has all. It is always been an area where we get many questions on.

A

You know how to configure this and whether I truly need to copy paste, for example, the url for the object store at every place and access key to every place, and now there is a feature where you can have a common where we have a common section in the document uh configuration uh where you can set uh those things that are common across your all. Your buckets basically object, store, buckets and that will hopefully simplify the the configuration um and uh we'll be including this in the json nut and helm.

A

Diploma methods uh a bit later, but for nights and it's a mimir uh feature that you can use already all right um next thing is debian and rpm packages for mimir coming to contribution.

A

Anyone familiar with.

B

The yeah, it is what it says so it's packages we have packages for both our arm 64 and amd 64 platforms. We don't use them. So if you run into problems, please report them. They are experimental but yeah. They should be usable because there is a test checking that they actually install correctly and they start linear.

B

So hopefully they work.

A

Cool and uh import historic data- this is this is the backfill. If I'm correct, I don't know if I ever want to say a few words about this.

D

Yes, that should be uh backfill as it's also known as so. Basically in mimir 2.3, we've added experimental support for uploading historic, uh these tv blocks into mimir, and it's we consider it experimental as it's still relatively immature it. uh It lacks. uh We like validation of blocks, for example, so it will not be detected if the client uh uploads invalid blocks.

D

So it's a feature to use with care for now and basically yeah. The future is also disabled by default, so one will one will have to enable it either globally or pertain it. Someone wants to use it and it's basically a combination of api in the in the compactor component, meaning http api and a command line user interface via me, mutual, so me mutual has it has a backfill also sort of command. Now, then, where you uh are allowed to upload one or more blocks to to linear.

D

Yeah, I guess that's about it.

A

Yep, thank you and last but not least, we have a minor release of the ham chart um with this release and, as you might know, the hem chart has an independent versioning from from mimir, so the version of the helm chart it's going to be 3.1 and it, like, I said, contains a couple of uh minor features listed in the changelog for the ham chart and just continuing of one.

A

I was talking about with the backfill and the history data that api on the compactor will be available through the venue install manual with the hem chart, because it's added to the nginx configuration we did update the minial subchart. So the minion support, because the sub chart that we used before was archived and deprecated, and it had some.

A

Alerts uh regarding uh fixes that need to be applied so now we uh we are using the latest and supported minion subchart and uh also a nice feature that.

A

When you install the mimir hem chart, you don't need to always supply a scope or guide the header in your requests, because the the ham chart configures nginx in a way that it will always inject a value and that values used to be the anonymous meaning that if you didn't supply the tenant, where you want to write it or it always ended up in anonymous.

A

But we have changed this to actually take the new tenant id from the mini configuration and not hard coded into animals, because some users were actually using a different username for for their non-authenticated or non-uh yeah non-specified um tenant. So now now, whatever you set in the configuration under uh no out tenant will be used in the in this injection and.

A

A

Yeah, I think more, most of the other improvements are quite minor, so I would encourage you to look into the change log. uh The changes are mostly backward compatible. There is one non-backward compatible change for minio, but since minio is we don't recommend the use of it in production uh that didn't really warrant a major change in our in our version, so yeah.

A

Let's see so that concludes the mimir 2.3 release and next topic is usage, stats and anyone care to liberate.

C

Yeah, I can speak about it since I built it, so we would like to learn more about. uh You know how the open source community used mimir um and the the main reason is. I think we need more data points on. You know some key configuration options. uh The communities is using, uh for example, which back-end storage, s3 gcs azure swift, or is the community uh deploying mimir in microservices mode or monolithic mode and stuff like this, in order to fly to drive better decisions um around around the project?

C

So you know where we should focus um our uh our work um when we we build stuff for for the community. um We didn't have any way to you, know automatically and anonymously collected this data, so um I've introduced the support to track some of this data in a anonymous way, um they're tracking. Well, some of this code is already included in the uh 2.3 release, but it's disabled by default.

C

uh However, uh I plan to enable this user statistics collection by default in the next next release, so 2.4, but there will be- or there actually already is um an option- a configuration option to opt out to to disable the user statistics.

C

Obviously we we kindly ask to keep it enabled. You know if, if it's possible, because I think this could help us driving better decisions over the time um in the agenda, you can find a link to a pr where I'm, which has not been merged. Yet.

D

C

I'm documenting every single piece of information we are collecting, um so you know you can have a better picture on which which information we are going to collect in a anonymous.

B

B

Will the results of this website be shared with users with community.

A

Any other topics we want to raise or give hints about upcoming releases teasers, whatever.

C

Peter uh I would like to answer the peter's question. First, yeah.

A

I got it so peter was asking.

C

If we, you know, if we can uh share back with the with the community, uh oh.

A

We can't hear you, marco, if you're talking.

B

You can't you can no, we can we.

C

D

So the question.

C

From peter was, can we share uh this data back with the community um yeah? um I think that would be great. uh You know to to to to share the aggregated statistics. um It's something you know it's a conversation. I already started internally um and I.

B

Think probably.

C

In the next release, we can have more more information to share about what we can do about it, but yeah.

D

That's something.

C

I also care about.

B

D

A

I sorry to say that I missed that, but uh if there's.

D

A

D

A

Can come to this meeting.

A

And uh yeah, we publish the agenda and the recording.

D

B

Thank you very much.

A

For attending foreign.