solo.io Gloo Platform, 18 Jul 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Enhance Observability with Grafana in Gloo Platform | Tutorial

Description

In this video, we explore how to take observability to the next level with the Gloo Platform and Grafana. The Gloo Platform already provides powerful observability capabilities through its UI, but with Grafana, we can unlock even more potential.

By collecting metrics using the OpenTelemetry collector on each cluster and forwarding them to the management cluster, we can effectively monitor the behavior of the Gloo Platform. But that's not all - we'll show you how to configure Grafana to build custom dashboards and set up alerts, ensuring the platform behaves at its best.

If you want to take your observability game to the next level and ensure smooth operation of the Gloo Platform, this tutorial is a must-watch. Learn how to use Grafana to gain deeper insights, set up alerts, and monitor the health of the Gloo Platform like never before. Don't forget to like, subscribe, and hit the notification bell to stay updated with more exciting tutorials and demos. Happy monitoring!

A

So we've seen a lot of capability in terms of observability that.

A

Are provided by the gloo platform ui,, but we can do so much more. as.

A

A reminder we collect all these metrics through the opentelemetry collector on.

A

Each cluster and it's sent to to the collector on the management cluster.

A

We can configure this to collect even more metrics and we can build some nice custom grafana.

A

Dashboards to help the operation of gloo platform itself and see if everything's behaving correctly.

A

So, let's deploy grafana...

A

So we can see the dashboard in action.

A

We're going to add the dashboard that we've already created for this demo...

A

So, let's wait for this whole stack to be deployed...

A

So, let's see what being where what we're deploying so in this monitoring namespace we're creating another prometheus stack.. We.

A

Don't expect people to use the prometheus, that is, that is being used by the gloo ui,.

A

But you can use your own prometheus stack that is probably already running in your environment.

A

And where you already have maybe grafana and so on,, so you can ship your metrics to that instance.

A

So that's why I deployed this stack here.. We create this config map for the dashboard.

A

And then what we can show is that we're able to create or define some alerts...

A

For example,, when one agent becomes disconnected, you can.

A

See that reflected on the dashboard and so on...

A

So, let's start by looking at that dashboard right, now.

A

So if I look at the services here and go to this ip address to go to the dashboard...

A

Cool, so now we can see here that I have some nice information about.

A

The way the management server is behaving...

A

In terms of like, the time it takes service to translate to different istio resources.

A

We can see the the number of agents that are connected.

A

For example, if one gets disconnected, we're going to see that here.

A

We can even see here some alerts that are predefined,.

A

But one of these alerts are going to be triggered. If, for example, an agent gets disconnected.

A

So, let's just simulate that...

A

If I disconnect by scaling down one of the agents... in cluster one...,.

A

Then we should have this alert. That's going to be shown when.

A

We look at that platform alert in the dashboard.

A

We also have some different alerts for like latency if it starts being too slow to translate.

A

Or if there's some translation errors or those kind of different metrics are all available, here.

A

So in this example, we just disconnected one of the agents so that should, you know, eventually when.

A

It passes the predefined threshold. It's going to change the state from normal to pending,.

A

And then, from from pending to to firing.

A

So if we wait...yeah...- let's see it's already moved to pending, but soon we'll go to firing.

A

So this shows you how you can use these metrics that are generated by the platform.

A

To get these alerts generated and so on., so let's scale it back to one...

A

We can also configure the agents that are already deployed with.

A

Some additional configuration,! So if you want to collect extra information.

A

So here, for example, we're going to tell the opentelemetry collector...

A

On cluster, one that we want to collect some istiod metrics...

A

And let's do a rolling restart.

A

Of the of the daemon sets that correspond to the collectors that are running on the clusters.

A

So restarting this daemon set will take into account the new configuration that we just applied,.

A

And let's also now import these other dashboards, so we can see the the new configuration.

A

So let's go back to the home and then look at.

A

The dashboards we can see this istio control, plane, dashboard...

A

And you can see all the various graphs that are available, here,.

A

But it's going to take a little while for this data to start populating and getting graphed.

A

So this usually takes a little while, but you can already start seeing some metrics.

A

Start trickling in., you can see the version of istio, the resource, usage,, translation, times.

A

But yeah this dashboard is showing you how like istio itself, is behaving but...

A

Alright, so let's move on past this.

A

There's also this gloo platform, metrics dashboard.

A

That are useful to see if all the agents are connected,.

A

All the metrics are flowing correctly,, the the translation, times,, the workspaces,, etc.

A

So you can like jump between the istio control, dashboard or the gloo platform.

A

Dashboard, depending on what you want to monitor.