solo.io Hoot Livestream, 5 Apr 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: HOW to increase application resiliency for Kubernetes services running on Spot VMs

Description

Spot VMs are excess Compute Engine capacity and cheap! Are you considering spot VMs for either cutting costs or test the resiliency of your application? In this hoot livestream, Eric will join Lin to discuss spot VMs and how to best test and increase your application resiliency when running on Kubernetes over spot VMs, along with live demos!

#resiliency #envoy #kubernetes #livestream #hoot #spotvm

00:02 Welcome + speaker intro
1:34 service mesh and Istio news
3:30 Discussion related to Spot VMs, increase application resiliency
7:43 two tier architecture with Envoy/K8s, best practice application resiliency
16:42 Review what will be demo-ed
19:40 Live DEMO
45:24 Answer live questions + Wrap up

To learn more about us, please visit https://www.solo.io

We are hiring! https://www.solo.io/company/careers/

Questions? https://slack.solo.io

Code Samples: https://github.com/solo-io/hoot

Suggest a topic to cover here: https://github.com/solo-io/hoot/issues/

A

Running your app in cloud is cost a concern, for you is building high resilience application a priority for you. This episode we're going to introduce methods to increase resilience for your application, while using spot vms to cut costs. Welcome to hoots live stream where we bring istio envoy, kubernetes, ebpf and graphql technologies to you. So you can be very well prepared at your job and be the best I'm your host for the hood live stream today and I'm so excited to have eric uh joining us today to talk about building application resilience on spot vms.

A

Eric. Do you want to give our audience a quick introduction.

B

Yeah sure, thank you, lynn, for having me very exciting to be on my first hoot episode.

B

uh So I lead the consulting practice here at solo, so I've been a solo for about 15 months or so, and I've seen a lot of new, exciting things happen here, including launching our service mesh product and we've had a couple solo, con events and a lot more interest in the company so um great to be at solo. Definitely.

A

Awesome, thank you eric uh before we get to the episode. Let's get to the news for this week. So first I would love to share. We've published the scripts for last week's ebpf and murr bridge. So if you want to try marbury jobs follow our script.

A

We also publish our next uh episode uh scheduled out in youtube. So we would. We will not have a hood uh next week, but the week after we will have denis and greg hansen join us on debug onward config and analyze access logs.

A

Bagger is the news for the past week: it's a new way to create cicd pipeline started by creators of daca, like solomon hikes, so check this project out. The other thing really also interesting is as soccer game momentum. It actually reaches 2 billion valuation. Congratulations to doc. I'm sure many of us are using docker. Today, istio kang has published the schedule for the second virtual. Is your con conference if you haven't registered for istio car. This is the conference where you learn everything about istio here about the it's your roadmap and listening on different customers.

A

Talking about is your adoption. Last not the least solo. I o is going to be at devops day in raleigh. I believe eric. You will be there right. Yes, so we are having uh drinks and small bites so join us on april 13th.

A

That's the news uh for this week uh with that. um Let me stop sharing with that um I'd like to ask eric what is spot vm.

B

Okay, spot vms are essentially virtual machines that you can run in any of the major cloud providers that have no availability guarantees. They have no slas, so these spot vms they could be shut down at any time right and so that that's the risk in using spot vms.

B

But the benefit is that you could save up to 91 of the cost of a virtual machine in the cloud right. That is a huge cost savings that really depends as well on what availability zone, what region, what cloud provider you're running, but the cost savings are too much to ignore right.

A

Totally yeah, I I know I'm a very cost sensitive person and that's exactly you know why I started paying attention to spot vms, yeah yeah.

A

So do you believe spot vm is perfect for development and testing of your services.

B

uh Yeah, it's definitely a great option for running dev and test clusters and, if you're really adventurous, potentially even production clusters to a limited degree, you could also look at it as sort of a built-in chaos testing right people like to do chaos, testing, but guess what? If your nodes fail or go offline at any unexpected time right, you should be able to recover from that right. So that's another benefit really when it comes to dev environments,.

A

Yeah, I'm convinced spot vms are very interesting. So can you talk about how to increase service resilience with smart vm? So certainly I don't want. My service goes down with smart vm dies right.

A

So how do I leverage the low cost from spot vm, where I can continue, keep my application running or continue to be able to perform resilience tests against my application.

B

Yeah so you'll see that in my slides in my demo today, but it really boils down to you know having good health checking having things like outlier detection having retries of request right- and these are all features of the envoy, proxy and you'll- see in the demo today. Actually, I'm actually utilizing two on-board proxies to provide additional resiliency for a kubernetes cluster. That's using completely uh spot vm nodes.

A

That's awesome um should I consider a mixed vm like regular, vm and spot vm, just kind of be able to leverage the cost, saving aspect of spot vm, but also kind of leverage, the availability of the regular vmware? What do you think.

B

Yeah, absolutely, I would say in most cases that's a almost a requirement because there are some workloads. You don't really want to run on spot vms. You know one would be like prometheus, for example,.

A

Right yeah, it's kind.

B

Of like a stateful workload- and it's um you know not not necessarily quick to start up. So if you were to have a note shut down, it's going to take a while for your prometheus to come back up again right. So that's one that you would want to run on a non-spot vm.

A

Yeah, that makes total sense, uh so you have to kind of pick which one carefully or maybe do a hybrid. um Are you ready to show us how this works.

B

Yeah yeah absolutely so I'm going to go through a few slides and then I have a demo that is very technical, um but I'll do the best. I can to explain how it works.

A

That's great, I think our audience are pretty technical too. So.

B

Right right, so can you see this see the screen.

A

B

Okay yeah, so you know we're going to talk about increasing application resiliency, you know like we like. We talked about um you know as we just talked about as well. um You can save a lot of money by using spot vms.

B

All the major cloud providers offer 90 91 off for for google cloud depending on what region you run in. You know it's not across the board. It's only for specific regions, so more popular regions, you might say 60 right, and it really depends on the underlying utilization of the region right and, if there's a region in, let's say, aws that it's not as heavily utilized you're going to get bigger discounts right so take that into consideration where you're actually deploying to what regions also there's other considerations around cost in running kubernetes clusters.

B

uh To be honest, you can kind of get nickeled and dimed when it comes to running kubernetes clusters, because you may or may not get charged for additional control planes and load, balancers and and logging monitoring etc. Right, so there's a lot of additional cost around running kubernetes. In the cloud and a few months ago, I actually created a github project and I just call it cheap gke cluster and what I wanted to do is maximize all the cost benefits of running a gke cluster in in google cloud.

B

There's some kind of some tricks and save you money around things like load, balancers, right and so for running a three node, six core 24 gigabyte of ram cluster. Using this solution you can actually run it for under 25 dollars a month which is incredible right. That's a credible price, and this is a very scalable uh deployment model as well, so you could potentially run a 30 node 120 core cluster, with 480 gigabytes of ram for under 500 a month.

B

I mean that is just crazy, cheap for to run a kubernetes cluster with those specifications, um and this is a little complicated. So I wanted to automate it, so I you know built in automation around terraform, so this is actually very easy to deploy. So if you want to, you can um go check out my github project and I'll just step out of this, so I actually have a github project out there. It's a murphy e, cheap, gke cluster, so you can go out and check out this project. Try it out.

B

I've had somebody alex contribute back to it already, and so it's really cool and I actually wrote a an article that was published on the new stack about this project. So we'll have we'll have links for that too. You can check out the the original article so really cool stuff. It makes it easy to actually deploy one of these clusters using spot bms.

B

But what is the catch here? Right, there's always a catch when you're saving money, you know especially a drastic amount of money right. So, as we mentioned, you know, there's no availability guarantees these spot. Vm knows they can be shut down at any time.

B

So, for example, um I ran a gke cluster that had three nodes and I had one node that ran for like 20 days, 20 23 days, something like that, but I also had two nodes that were shut down at the exact same time, so I lost two thirds of my cluster in that scenario right.

B

um So that's a great example of why, if you're, taking advantage of the spot dms, you actually want to run extra cluster nodes right and, to be honest, that's kind of a best practice. Anyways right, you want to have that extra capacity capacity for burst ability of your workloads right, but you also need that extra capacity simply for failover and resiliency of your cluster right.

B

um So if you can take advantage of extra capacity and get a huge discount up to 90 with spot vms, I mean that that's really good right, uh also like when asked you know having a non-spot vm cluster node pool is important for workloads like prometheus or any other stateful workloads, like you know, databases potentially or legacy apps that are running like java, ee, for example.

B

um Also, you could potentially run multiple of these clusters and have load balancing between multiple clusters and failover between multiple clusters. Potentially, you could utilize istio and glue mesh to have cross-cluster traffic that is secured right and have service level failovers between clusters as well.

B

So there's a lot of flexibility in how you could configure and deploy your clusters to take advantage of spot vms right, and you know, generally speaking, for best practices.

B

You would actually need to implement an api gateway or service mesh to to really take advantage of this as you'll see today,.

B

So here are some: you know very quickly some best practices and I'll go into this later, but you want to utilize anti-affinity rules for any services that you're deploying right. You want some guarantees that these services are not always running on the same node right. So if you had three replicas, ideally they would each run on different nodes right that way you have the resiliency. If one of those nodes goes that goes down, they still have two instances of your workload running right.

B

um Also, you know having horizontal auto scaling for those services is great because uh you can have burstability of uh your workloads inside your cluster.

B

Also, you want to implement things like health checks and outlier detection and load, balancing and retries and you'll see pretty much all this today in the demo, and as far as I know, envoy, is the best solution for implementing these types of things and do it in a performant way.

B

Also, when you're building microservices uh you in this type of environment, you want microservices that will start up quickly right, because if you have a node go offline and you need to spin up new instances of your microservices, you want them to start up quickly.

B

You know, ideally in under one second right, because you might actually have uh circuit breakers that are tripped and retries that are happening, and you want those services to be available as as soon as possible. Right.

A

Yeah, that's a good point. I guess if your application could not start fast enough, maybe it's not a good candidate for running on spot vm leveraging the technical you are sharing here.

B

Yeah yeah, but the good thing is: there's a lot of new technologies and frameworks that are available.

B

So if you're writing a java code, you know so, for example, there's the quarkus project um and grawvm that allows you to compile java to machine code and have a java microservice that will start up in milliseconds rather than seconds right. So I'm talking about literally like a half a second, and that would also uh include your like your your database connectivity and your orm and like everything right, which is really amazing, so that that's the type of uh frameworks that you you want to use in this situation.

B

Also uh for having these, you know, workloads that are not exactly um guaranteed right. You might have connectivity issues to things like databases and measures, message, brokers, right and that's where having your application reconnect to your database without having to restart the whole application itself is, is important right. So you want to kind of have health checks against your database connections as an example.

B

So that's another example of where you need to build resiliency into your applications, and these are best practices anyways for kubernetes. It doesn't necessarily apply only to running on spot pms, but uh from what I've seen working in consulting most organizations are not implementing all the best practices around fully resilient microservices.

B

uh So the demo today, like I said it's, it's a a little bit technical, um but it usually utilizes a lot of best practices around envoy proxy, and so what I'm gonna do is actually take a a three node cluster running igke and I'm actually going to cordon and drain an entire cluster node and do this while a load test is actually running right and so, as part of the deployment I'm deploying the glue edge api gateway, which uses the envoy proxy, I'm deploying the pet store application uh using anti-affinity rules.

B

So this pet store is running on multiple nodes and I'm applying resilient configurations for envoy, and this is both in glue edge and the google cloud load balancer, which also uses envoy right um and so essentially with the cube cuddle drain command. I'm I'm coordinating a node and I'm simulating a cluster going offline as a spot vm uh cluster node right. So this is the same process that would happen only in the case of the spot vm.

B

It would be google cloud triggering the node going offline, but I'm gonna do it manually here because I need you know, control on when.

A

B

Right, obviously, for a demo and the interesting thing apart about this demo, is I'm actually going to have the glue edge control plane on the node that is going down. So this is a very invasive node shutdown right, because you're losing a proxy you're, losing an instance of your service and you're, losing your control plane right and I'm going to, as I said, run this load test.

B

While all this is happening and examine the outputs of that load test, and hopefully there will not be any errors actually, as you as you'll as you'll, see from the demo and as I wrap up the demo, I want to dive a little bit deeper into envoy and look at statistics around the traffic that was sent uh through the proxy and see if we can find any evidence of things like http retries when this was happening.

B

Okay, so lynn, do you think that covers it pretty well, do you have any questions on what I'm going to do with the demo.

A

No, let's get to the demo, I I would love to see it.

B

Okay, um yeah and before I jump into it, I wanted to have a quick diagram as well, showing you know the three cluster nodes and in glue edge running here. If you can see my mouse and the control plane and the pet store service and outside of here is the google cloud load balancer.

B

So there's actually two envoy proxies involved here, there's one at the external load balancer and there's one at the api gateway right, I'm actually going to leverage both of those right and, as I mentioned, with the node shutdown, I'm actually going to destroy all of this all these all these services that are running, including the gateway right and then the new instances will spin up on the remaining cluster nodes.

B

So, as you can see, this is a very high impact uh situation here, with this demo, okay cool. So I'm going to go over to to vs code.

B

So the first thing I'm going to do is show you a few of the configurations that I have so you can have a little bit of upfront understanding on what's being deployed first before I actually do the demo so for the pet store application, it's just a very, very, very simple, microservice arrest, microservice right and I'm adding some anti-pod anti-affinity rules here right and so that is actually implemented through a label selector. So the the key is app and the value is pet store right.

B

So that allows you to select which pods are applied for this anti-affinity rule right, and you also need to have a topology key, and this is something that is for the actual cluster nodes and so there's kind of a kind of a default key which is kubernetes, hostname right, and so this allows you to distribute your uh pods across different cluster nodes right.

B

That's how the kubernetes scheduler will know where to put those pods in this situation right, so that's very important and also I was able to configure glue edge itself to leverage anti-affinity rules for the actual uh envoy proxy right. So that way, we have guarantees that the envoy proxy is also running on uh different nodes.

B

So that's that's one thing I did another thing I did. Is I used terraform to actually deploy the cluster and deploy all the configurations for the gke cluster, including the load balancer right, and so I have a a load, balancer terraform, and so, if you look in here, uh you can see that I have a compute region back in service.

B

So this is a configuration for a google load, balancer right and I'm setting some things in here like outlier detection, right and circuit breakers right, and so what happens is when I apply this terraform configuration it actually configures the managed version version of envoy that google will host for you as part of its load, balancer right, and so if there are two consecutive errors coming from your upstream and your in this case, your upstream is actually uh the glue edge proxy.

B

um It will uh basically eject this uh this uh upstream from its pool and it will not be able to rot traffic to it right.

B

So in the case of having a cluster node go offline, one of your upstream proxies will go offline and it will no longer route traffic for it to it, for, let's say, 30 seconds right, so that's very powerful when it comes to resiliency at the external uh load, balancer right and also there's things in here that are like health checks right and so there's like a health check, interval with three seconds and a timeout, and so this is useful for situations where you don't necessarily have any traffic going to your cluster.

B

But um you uh want to have the resiliency there, and so you need to have that health checking occur.

B

So it can, you know, behave properly, even though you don't have any incoming traffic, if that makes sense, also there's a configuration for your url map, and this also will apply envoy configurations under the covers as well, and so you can implement things like re, retry policies right and so here you have a number of retries, which would be five per retry timeout retry conditions.

B

And if you actually look at the envoy, documentation you'll see how this directly equates back to envoy. Right, but this is all configured through terraform for the google quad load, balancer right, so that that gives you a little little overview of what hap was happening on the google load balancer slide. But what about with the glue edge and the configurations there?

B

Well, we can set some of these same types of configurations for glue edge itself, and so what we're looking at here is an actual upstream configuration for for glue edge right, and so this is configuration that will define how traffic gets routed to an upstream kubernetes service. So here you can see cube, selector, pet store right, and so this is the actual upstream endpoint for for this. uh For this upstream right and then I have a health check set for it, and then I have outlier detection right.

B

So this is very similar to uh what we had set for google cloud load, balancer right and so you're checking for consecutive 500 errors and you're setting your max ejection percent, so that um basically uh will uh direct traffic to the other. Fifty percent of um the actual upstream, uh the f stream that is available at that point in time and then also set in the the circuit breakers.

B

So there's that one and then there's also the virtual service, which uh also defines where exactly the traffic is being routed to from your actual glue edge gateway right, which is envoy. So.

A

In this case, you have a question now so.

B

A

Can you talk about uh with the google load balancer? Why, in the architecture you don't have it directly go to the pet store application? Why do you have glue edge in the middle yeah.

B

Yeah, so that's a great question so to provide ingress from the internet into google cloud and gke they're they're, you know pretty much has to be a load balancer like an external load, balancer right, yeah, and so recently uh google cloud decided to switch their load balancer implementation to use envoy, basically, by default before it was not using onboard their load. Balancers right.

A

B

So because they switch your load balancers to use envoy, you can actually take advantage of the benefits of envoy directly at your external load balancer, and this is actually very important in demonstrating the resiliency of this architecture, because if your node goes off offline or maybe your whole cluster goes offline, you'll have this incoming traffic um from the internet coming through the load. Balancer and you'll have a layer of resiliency right at that external load, balancer level right.

A

B

And then that traffic will be routed to your upstream uh google or glue edge rather glue edge gateway and you'll have yet another layer see layer of resiliency with envoy. So there's two two onboard proxies involved with this architecture.

A

Okay, so it's the actual layer of resiliency provided by glue edge for the architecture, yeah.

B

So um so that's really cool. You know, and also you know, with a virtual service, you can implement retries. So basically, at both levels on both uh envoy proxies, we have things like retries and outlier detections and health checks right, so that provides a lot of resiliency to cover for many different scenarios where services are going offline or nodes are going offline, and it's it's really amazing what you can accomplish as you'll see in the demo.

B

Okay, any other questions on that lin before I.

A

No, this is great yeah. Let's see the demo.

B

Okay, yeah cool, um so there's a series of commands that I have to run here um and I'll try to explain as best as I can. You know, kind of what's deployed.

B

um If I can kind of widen this, so I can run a command here to actually show the pods that are running and what nodes they're running on right. And so this is where you do the dash oh wide right, and here you can see the pods that are running and what node they're running on right here right. So you can see, there's uh like the the gateway proxy, which is envoy, there's a like the glue, control, plane and there's actually two instances of the envoy proxy. So one is running on a cluster.

B

Let's start with vn here, and one starts with n and b right and there's also two instances of the pet store service right and those are running on also two different cluster nodes right and in the case of the nb 3q.

B

That is running on the same node as both the gateway proxy and the um like: the glue edge control, plane and the gateway and discovery right. And so there's a lot of things that that are running here on the same cluster, node right and so for the demo.

B

I actually want to shut down the nb 3q, because that will be the the most dramatic uh note shut down, and I I still want to show that everything can remain operational, even though this entire node is going offline right, but with all these different services, with with the gateway with the pet store, etc. Right.

A

Yeah, basically, you assimilate what, if that vm is a spot via well, you know it just dies.

B

Yep, exactly exactly okay, so so I picked which node I want to shut down, which is mb3q, so so what I'm going to do is I'm actually going to. I just set an environment variable, that's called well. First of all, let me show the nodes. Let me show that first, okay, so just to verify you know we have these these nodes that are running right.

B

And and there we go so so we have so we have these three nodes running and you can see there's that one one node running right there right so so I'm going to set this enviro environment variable I'm just calling it drain node and I'm going to set that value to to be that node. Okay,.

B

Okay, so what I'm going to do next before I kick off my load test? Is I'm actually going to drain that node? So what it's going to do is going to cordon the node, so no workloads can be provisioned to it anymore and it's going to uh shut down all the services that that are running on that node right. So it's simulate an entire node shutdown right.

A

B

But before I do that before I do that, I actually want to kick off a a a load test here, um so I'm actually just going to use the hey the hey load test tool right, so I'm just heading the actual gateway. So this is actually going through the google cloud load balancer, that's going through the google edge gateway. It's going to the pet store. I'm just running this on on my local machine here, right alrighty.

B

So so I'm going to go back and I'm actually going to oops going to drain this drain this node, so this effectively kind of kicks off the demo here right.

B

So I'm going to do that and, as you can see, it's evicting all these pods right here right from that node um and then, if I go to like a different terminal here and paste this in, you can see that there's all these pods that are being uh terminated and restarted in the glue system name space where glue edge, is running right.

B

So so there's a lot of things restarting, including the the pet store service.

B

um At this point in time- and if I run this again, um I mean you can see, things are running right, so so what I should be able to do now is exit out of my load test and we'll see what the result is, and you can see that I ran um during this load test.

B

It was like around 30, almost 38 000 requests and they're all 200. They were all successful, http response codes coming out of this right and without the resiliency that was built in you would get all sorts of uh 5xx and four xx airs coming back.

A

B

I don't know how much time we'll have lynn, but I can try to run this again a little bit later, to actually show that we'll see.

A

That's okay! I guess it would be for me it would be really interesting to use you know to find out how much resilience are provided by you know the google load balancer and glue, as you through envoy, in this case right how many retries and timeouts to be able to achieve 100 to uh 100 of the 7 000 response of all 200 response code.

B

Yeah, absolutely so, I'm actually going to show the envoy information um up next, so so um so what you can do actually is you can actually uh port forward into the the gateway proxy and access the envoy dashboard, and so I actually need to get the the pod name that I need to connect to right, and so what I needed is. I actually need the pod for the gateway proxy that was not shut down. That was running on the other node right.

B

So this would be the the one here that I need um yeah, so yep, that's the right one, so I'm gonna actually get this get this pod name for the other. The other envoy uh that was running and this one won't be valid anymore. So I'm just gonna paste this one in.

B

Okay, so now I can actually go back here and hopefully, if I hit refresh so I'll, be connected to the envoy dashboard. Basically, so this is envoy running inside the the uh glue, the glue gate gateway, proxy, pod right, and so um you know like in here. I can do like a config dump kind of thing, and this is the actual uh envoy configuration that is um generated from the glue control, plane and configured for the for the envoy proxy right.

B

And so, if I search in here, I should be able to find some configuration related to a retry policy right and- and so here you can see that I have a cluster which is essentially an upstream service.

B

That's running inside, uh and so this is the actual pet store right and you can see that there's a a retry policy on here right and so this this equates back to um like. If I go back to, I think it was the.

A

Stream yeah yep.

B

So if you go back to actually no, it was on the virtual service, rather.

A

Which was this yeah.

B

So there you can see the uh the retries, so that equates directly back to the actual uh envoy configuration right so so that that's really how google edge works. Right. It'll read this: this virtual service configuration right from from your cluster and it'll convert that configuration over to the actual envoy configuration and using xds that will configure the onboard proxy for you right, and so this the glue edge configuration is way more simple, obviously um than the envoy configuration.

B

um So that's that's how it works on that on the configuration side right yeah, but now we want to see you know what actually happened uh during the demo right um and so there's actually uh metrics that are uh that are produced.

B

uh You know by envoy right, and so, if you go back to uh the you know, uh the envoy, like the the console here, you'll, see a variety of different links, and so you can see a stats link right here, and so I have a tab open for that, and so you'll see a ton of statistics that are produced out of the uh envoy proxy right.

B

And so what I'm going to do here is I'm going to search for a specific value and what you're going to see here is you're going to see that cluster, which is the upstream, which is the default pet store, 8080 right and here you're, going to see that when I took down that cluster node 15 retry uh retries were made right, yep and so the situation this this happened.

B

If I go back to my diagram, the situation this happened was the traffic was routed from this gateway right here over to this one right and that one was no longer available right because the node was being shut down, and so, during this point in time, 15 retries were were made to the other service. That was still available to be running at that point in time right, so it went to one of the other pet stores.

A

Okay, so that's just the retry number for the glue edge gateway at the at the last virtual machine on your diagram. That makes sense um because the other one dies. So you don't actually know how many retry and then I assume, the the google load. Balance will probably also have some retry because it would have traffic goes to the the guide um like envoy, and then it would redirect to the one at the bottom.

B

Yeah, so I don't have the envoy statistics for the actual google called load balancer. um Maybe they offer, I don't know if they offer that, yet I'm sure they will in the future if they don't right yeah. But if you look at um the the actual terraform uh configuration uh that I that I have actually it was the other one you can see that there are. There is a retry policy for that google cloud load, balancer right and so.

A

B

There's five primary tries and one thing that I'll say that was really critical for this to work is having proper outlier detection configuration uh not only for glue edge, but also for the load balancer right, and you want to be very aggressive on checking for consecutive errors, and so in this case I set it to two right, and so that will actually remove that cluster. That upstream uh instance uh from the from the pool essentially and for a period of time.

B

You know, let's say 30 seconds, it will no longer send traffic to that particular endpoint, because it knows there's errors coming from it right and in the case of the google cloud loadout, it will actually rot traffic directly to the pods to the glitch gateway pods running in the cluster right. So this is like a a direct um connection between the load, balancer and glue edge. Essentially so.

A

B

Having these two layers of resiliency kind of just going going back to my uh my diagram so having these two layers, I've reasonably seen with envoy made it possible for me to actually run this. This load test and actually um not have any errors, come back right.

A

Yeah, that's very impressive, plus the configuration you are showing right from the google cloud load balancer to the upstream resource and virtual service resource from google ads. Those are like way more simple than the actual envoy configuration so back to what people are saying. Envoy are not really designed to be for human to write. The configuration.

B

Yeah, it's just thousands.

A

B

Yeah absolutely another.

A

B

Another thing I'll add too: um you can do a lot of this with istio as well, and so, if you have a complex microservice architecture um and you have services routed into each other across multiple cluster nodes, you're going to want to configure things like retries and outlier detection- and you know those types of configurations not only at the istio ingress but on the sdo sidecars, to help improve the resiliency of your microservice architecture.

B

A

B

Simple example, but it also applies to a service mesh.

A

Yeah totally, and that's so that's in fact, thinking of the diagram you showed the architecture made me think like in is your word from istio ingress gateway to the cycle proxy to maybe an app like pet store right. It's kind of a similar diagram as what you have here, where you go through to envoy proxy as well.

B

Right right, yep, yeah. In that case it might even be three because you'll have the ingress gateway and you would have sidecar.

A

Oh right yeah, if you have uh google cloud load, balancer or any cloud balance in front of you, so is spot vm. A concept also exists beyond google cloud.

B

Yeah, so all the major cloud providers offer spot vm uh nodes so for eks and aks um you know, but google is the only one that provides the envoy based load, balancer right and so, if you're running in another cloud environment, you would also want to have a load balancer that has resiliency built into it, or run your own instance of envoy as an external load, balancer, potentially, and so also there's an architectural model that some people you know, especially for our large organizations might consider, is you can actually run a separate cluster to host, um let's say glue edge as your external load balancer so like rather than you have your google cloud load balancer.

B

This could be a small kubernetes cluster and actually run glue edge on that separate cluster and then route traffic to your um essentially upstream uh clusters. Right. So that's a more advanced advanced architecture for a large organization.

B

But that's another way. You can add a layer of resiliency for traffic coming into your system.

A

Yeah very interesting, so anything else you want to share before we wrap up for today.

B

Okay, um are we at time uh at the.

A

Moment yeah, it's a it's good time to wrap up. Okay.

B

Unless you have.

A

Something major you want to show.

B

Yeah yeah, so I'll just explain just real quick. So if I had a little bit more time during the demo, um I could actually remove this like the outlier detection right and reapply it, and you would see uh probably around a couple hundred like five xx. Errors come back from the load test right and if I were to to remove like some of the resiliency from like the virtual service and the upstream from on the ledge side, you would see even more errors right.

B

So there is potentially several hundred errors that would come back without the the resiliency that we built into it and maybe even more, maybe even in into the thousands potentially. So so that's why you need to build in this types of resiliency, uh not only with with spa vm nodes, but I would say, in general, this is a best practice.

A

Yeah totally except I guess you could test your resilience with spot via much faster. Yes,.

B

Yeah absolutely absolutely.

A

Well, cutting the cost too. Yes,.

B

So really cool. uh You know, like I mentioned, uh check out the github project and I'm actually gonna check in the instructions that I used uh for this demo into this github repo I'll call like demo.md or something. So if you want to try this on your own, you could you could surely do it.

A

That's great yeah. I love to have uh the instruction also um the hood repo as well for today's episode. Well, I want to take a minute to thank you eric for preparing the demo. It's actually really interesting to showcase how you could, potentially you know, look into carcass and increase resilience with uh envoy-based architecture like glue as your instill ingress gateway or even google cloud load balancer. uh We do have a question now, um so thank you for joining us benja. It's the setup available to check.

A

I guess I guess that stopped his question. I thought he had something else to check um now. His next question is also is the new google cloud, no balancer tcp, um I guess layer, 4 or ingress layer, 7.

B

Right, so if you look at the google cloud docs for their new load, balancer type pretty much, they only allow for http, which would be layer 7 right for your traffic routing.

B

I would expect in the future they'll add the tcp as well, and so you know that way you would be able to route traffic more directly um into your system.

A

Okay, great and he also clarified he was asking about the code. It's the code available to check in I assume. Yes, we will make it available on the github repo as some of our past episodes as well. So thank you so much for the questions.

B

Thank you. Thank you for watching.

A

So much eric! um So how do folks reach out to you if you, if they have further questions.

B

Okay, uh you can reach me on twitter, my handle is just murphy e or you can reach me at eric.murphy at solo.io as well um or just reach out to atlanta and she'll. Get you in touch with me. So.

A

That works too. Yes, thank you so much eric and thanks everyone for watching. I want to give a shout out to everyone who have watched our past episode. You know give us a thumbs up or comment you do you know. I really appreciate that means a lot to us.

A

um So if you find this interesting, uh what other topics would you like to see related to application, networking comments and chat, and let us know- and please uh you know, subscribe to this channel, so you don't miss any of our future episodes thanks thanks for watching and see you in two weeks, bye now.

B