solo.io Hoot Livestream, 15 Mar 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Episode 20: 1-Click Upgrade to Istio 1.13 using Helm

Description

What is your Istio upgrade strategy? Helm, istioctl, Istio operator? How is IstioOperator API related?

In this livestream, Krisztian who has been upgrading to many Istio releases will join Lin to discuss Istio upgrade strategy and do a live demo of 1-click upgrade to Istio 1.13 using Helm.
#istio #upgrade #strategy #helm #livestream #hoot

A breakdown for the episode:
00:12-1:40 Welcome + some hiccups with livestream + speaker intro
1:40-6:00 service mesh and Istio news
6:00 -12:02 What is new with Istio 1.13?
12:04-29:40 Options & strategy for Istio upgrade + answer live questions!
29:40-33:40 Helm + Istio
33:40-44:50 Live DEMO
44:50-52:56 Monitor Istio upgrade + answer live questions!
52:56 wrap up + answer more live questions!

To learn more about us, please visit https://www.solo.io

We are hiring! https://www.solo.io/company/careers/

Questions? https://slack.solo.io

Code Samples: https://github.com/solo-io/hoot

Suggest a topic to cover here: https://github.com/solo-io/hoot/issues/new?title=episode+suggestion

A

Hello, I think we are live.

B

A

Hi welcome to today's hoot live stream. Yesterday I learned over 50 environments are not upgraded to the supported version.

A

Sorry, this echo, I need to disable.

A

Not sure, well, I'm getting the echo, I need to disable that.

A

Are you hearing, I call christian, maybe just no.

B

A

Okay, so sorry I'll just talk now sorry about that- um I I hope these environments are getting security patches. So today we will unveil some of the confusion around. Is your upgrades and dive into simple issue upgrades using helm?

A

I am so excited that I have christian joining us today. Christian do you want to introduce yourself quickly.

B

Yes, hi everyone, I'm christian faketer and I'm working at lastpass as a devops engineer, and today we will talk about vistio upgrades and embrace upgrade, particularly.

A

Very good: let's start with this week's uh the past few weeks, news so start with solo con. We just wrap up solo account, our second annual conference uh christian. Did you have time to attend solo car any perspective from you.

B

Yes, I was able to catch the keynote and the few sessions live, and after that I had a chance to catch up with the ultimate recordings, but I had not the time to to finish all that, so I have some homework on that part as well.

A

What's your favorite session.

B

My favorite session this far is this session on ebtf. I think the flow of that session was was really great and it was a great introduction to ebpf if you are not already familiar with it.

A

That's awesome yeah. I you know my true perspective with solo kind. Is the future of service mesh christian did a really interesting talk about. Well, the innovation of service messages happening in data planes so definitely check that out. um The other thing I want to mention is there's so many customer stories. If you like to hear customer adopt cloud native technology such as envoy and istio, you know definitely check them out on day two keynote. We have five customers, you know tell us their challenges, you know why did they decide to work with solo?

A

We also have chick-fil-a. We had t-mobile, we had constant contact, so definitely check those out.

A

Besides solo con, um one thing I want to also mention that's really exciting: uh is there it's a new blog um murbridge accelerates your service message with ebpf from the dow cloud team, so this blog was published about. I guess a week ago it was very interesting. Well, they use ebpf to improve the uh the ip table um performance. The networking performance instead of using ip tables and use ebpf to shortcut on the network communication for uh for for application running within istio so definitely was checking this out.

A

Also on last week, the isro community published the security releases. So if you are running in istio in production definitely check it out uh to see if your releases are patched properly, because these are important ones. um There's also one more thing I want to share from the issue community. Is the we've been working with the hpe team in upstream uh integration?

A

Istio with spell so this is something the community has been asking quite a lot, so um they've got interesting design and even demo in one of the most recent networking world group meetings. So the way they're doing it is really neat. Essentially, there's no code change in istio control, plane and then the istio agent would detect whether the the sds socket exists to see. If the spell agent runs on the node to determine you know what action would it take so very cool? I really like that design uh last night.

A

The needs on the news front I want to mention. There are some changes on the issue. Work group meetings so essentially we'll consolidate all the work groups um to have it on wednesday at I guess it's noon, eastern. So that's the combined work group meetings in upstream communities. This is the one you should go to. It works in your time zone, we're also in the process to schedule another one, that's more friendly to the asian time zone.

A

um The toc meeting also change frequency to be every month instead of every week before so that's kind of the news uh in upfront. Now I want you to pass to christian to talk about. What's new in istio 113.

B

Yes, uh in this slide, I'm going to showcase the three highlights of uh sto113 and the first is configure distillate, approximately the proxy config api earlier, the configurations of proxies could be changed through the mesh config api, but now it's promoted its own proxy config api, which is basically a new crd.

B

So, with this new crd, you can configure dedicated workloads and namespace related workloads uh more efficiently and similar to. Similarly to this crd, there's, the telemetry api, which was promoted to to its new to its own api in 111, and this 130 release is adding logging related improvements to this uh component and the last. But the most exciting er thing is that um there's a support for host name based, load balancers.

B

This was affecting uh kubernetes and these skill planner clusters running on amazon's eks. Basically, up until this point, you could have broken earbuds uh if they got a new ip address. But since this release it's not the case anymore, because it's you can auto discover how to resolve the hostname of of the load balancers. So you will.

B

You can get rid of your workaround, for example, chrome jobs too, to upgrade these components, and you can um follow this link and check all the other. Under the hood improvements of of this 113 rupees.

A

Yeah, thank you christian. I totally agree. These are my favorites as well, so um the proxy api, the proxy config api, is actually really interesting. We've been working in upstream for a very long time, so, instead of having you upgrade your proxy configuration in config map now there's an official api for that also the host name. I know a lot of our users are asking for that, especially people in aws clouds, especially people using multi-cluster or vm integration.

A

I believe that's like a game changer for for those users, um there's actually um also a bunch of feature update, which is also very cool, that some of our future like workload group, which is also help users to onboarding their workloads on the vm into the mesh. It actually uh first introduced as alpha in 1.8 and has been recently pred promoted to beta, I would say: 113- is more a stability release. If you watch it's, your strengths, you can probably tell like 13 is a stability release and 12 has more features.

A

So one thing interesting about 13. Is it's actually fixing a lot of things? So if you are looking at the release notes, you can see like fixed rows. It's like lots, lots lots of them, so really listen back to what customer is asking you know, making the product better is really what 113 is mainly after. In addition to the features that christian highlighted, um I guess one thing I would say uh this is something you know.

A

I work with elizabeth from sap to help this pr gets in, so that it generates a nice warning when you run instructor analytes to analyze your service, so really helpful to help you onboarding your services to the mesh, there's also a bunch of new um added configuration to it's still cuddle. I know my uh friend nick nelly's added this experimental onward stats feature to help our user debugging on week, onway stats.

A

So that was a really interesting feature added um also, if you're upgrading to 113 make sure check out the upgrade notes right, because these are the first thing. I would definitely recommend you to read, if not a new israel user. If your existing is your user check out the upgrade notes, for instance, there's a house probe no longer reuse.

A

The connection which should be transparent for you, um but thankfully we, if there's any issue you know we actually had a option to help you disable the reuse connection portion, so you can disable that uh multi-clusters secret authentication change. um So, if you're using multi-core multi-cluster today, if you're supplying your own cube, config was not using the istiocado crate. Remote secret command definitely check this out. I think um it's.

A

uh This is related to some of the security exposures and we remove some of the supported uh previously supported our syndication config cube config file, port 22 for ip table capture changes. This is important too. If you are using port 22, if not probably would not impact you um so so those are the major things I would highlight on 113..

A

So um christian, can you talk about what are the options for a user to upgrade istio.

B

Yes, I think I can see slides yes, so, basically, with this deal, you can approach two different strategies to upgrade your service mesh. The one and the original method is the in-place upgrade then there's canary upgrade, and if you are want to achieve these with ham, you can do them both. So, let's take a quick overview of the two aforementioned uh upgrade strategies.

B

First in place, as I mentioned, is the original one back in the days that that was the only way to upgrade your clusters, it's quite simple, but it's riskier than the newer canary upgrade strategy which is available since 1.6.

B

Currently, this is considered to be the best practice. It's safer. You can shift your workloads to the new control plane, which is running with the new version, and if you experience any issues, you can draw back and go with the earlier version that you were already using, and I will talk about the ham ham approach, because I think harm is a really convenient way to handle aggregates.

B

You are probably using ham to package your applications, for example your business applications or your tooling. So it's it can be a convenient way to handle your istio installation as well, and, as I mentioned, you can do both in place and can add upgrades with it, so it's a great fit for for every use cases.

B

First, let's take a closer look to the embrace upgrade itself. Basically, with this, uh you have a certain version of the control plane that the v1 is the od box at the bottom, and you have matching proxies then doing gameplay upgrades. There's, basically one predict with it. You can skip only one minor version.

B

Let's say you converted the upgrade notes. You've decided to do an upgrade.

B

You can do this with ham or with the steel ctl command line tool. With this dlctl you would, uh you would download the newer version of istio. That would help have the newer sdoc here. That's your client that you, you will use during the upgrade and if you issue the upgrade command and pass additionally, if you have some custom, sd operator or values file, you can pass that with the dash f option and basically uh that's it. That's the upgrade itself after you executed this uh command.

B

uh The control plane itself will be upgraded to the version of the ecoctl tool. After this, you will have to perform a rolling restart for all your workloads.

B

For example, if you are using english gateway first, you should upgrade rest of your english gateway, and, after that you should restart all your deployments in your business namespaces after the rolling restart is finished, uh the applications will get the newer version of the invoice sidecars and they will connect to the version 2, which is the version 2 seo control plane, which is your target version that you downloaded, and you use that istioctl.

B

But there's gonna recreate the cell yeah with canary upgrade. You can skip two minor versions, so, for example, you can go from one eight to one ten. I think. Maybe that was the that was the first version that uh that it was highlighted that you can skip to minor versions.

B

Basically, you are starting from a similar scenario. You have a single control plane running with v1. This is your current version and you have matching proxies um first with canary react, the canary approach.

B

You set your target revision, you are using this uctr install command and you can specify your target revision first then, after this, this step roll out a new sdo control plane, but at this point no proxies and no workloads will connect to the new control plane, the new sdod, which is illustrated with v2 on this slide.

B

After that you can canary test your new control plane. To do this. You first choose save namespace. In this example, it's called test ns, you choose this and you label this namespace with your target seo revision after you. Do you finish with this? You can perform the rollout restart for your target namespace and at this point, the workloads in this test namespace will communicate with the new target sdo version. So, as you can see, it's a more safer, much safer uh way to test the new control pane, the new proxies you can perform performance tests.

B

You can load test your application. You can validate it by end-to-end, test or or by hand, and if everything seems to be fine, then you can proceed to the next namespace and you follow this step uh for each of your uh in spaces.

B

After all, your workloads are migrated to the new version. You have no version, one proxies uh communicating with your previous still control plane. So at this point you can get rid of that and delete that installation.

B

So you are left with your new sql control plan and during process, because, after the restart all of the workloads are having the latest hto proxies.

A

Okay, great christian thanks for giving us an overview of in-place and the canary upgrade now there's a user uh krishna. Thank you so much for joining us today. uh I love your question with 113. What's the plan around is your operator. So folks, if you have a question, please uh enter through chat. You know we really love uh engage with with you. So let me take a step and answer that question.

A

So if you look at the seo operator install documentation today in istio, you can see. Unfortunately, we have use of the operator for new issue. Installation is this discouraged in favor of istio, cuddle and helm. While the operator continue to be supported, new feature requests will be not prioritized. Okay, so there's a reason for this statement right.

A

um So, first of all, we are liking maintainers willing to be able to step up to maintain the seo server side operator. um So that's the primary reason. Second of all, we know there are some known issues regarding the seo server side operator, particularly related to canary upgrade.

A

So uh so what should you do right? This is why we're having this live stream right kind of explain to you. What are the options out there um particularly related to helm and it's your cuddle, which is what is your upstream, recommends right? So the good thing is the: is your operator api, so the server-side operator and istiocado all uses the same issue operator api.

A

So from that perspective the issue operator api will stay so reuse is the operator yamo today, that's um consistent for you, regardless whether you are using istio cuddle or the server side is your operator, so that will remain and so and second, uh you know definitely check out the helm. Helm is a rising star in the community right now.

A

This uh you know it's just very friendly with the existing workflow that a lot of our users have so we we do expect to promote helm to beta really soon it was intended for 113 and we didn't meet it at that line of 113..

A

So christian, do you want to add anything.

B

uh Well, one thing that I could do is upgrading with ham is really convenient way to handle upgrades because the capabilities of ham itself. So, as I mentioned, you must probably already using it in your stack and there are benefits when you are using a simple tool to address different issues, because the team members don't need to.

B

Get familiar with the new method and use the already known methods of upgrading raw backing um managing your uh releases. So I think it's it's a great way to uh handle istio itself as well yeah. We will soon take a practical look at the ham upgrade in this session.

A

Yeah totally so christian, and let us know if the, if you like the answer or do you have any more questions. Okay, so there's one more question from you: uh is there a reason to use the istio cuddle worst helm? That's a great question! uh Honestly! I think it's also up to you know. Are you using the easier yamo using is your operator?

A

So if you're using is your operator, is your cuddle might be a simpler transition for you um and also it's your cuddle is a little bit more mature um that that's been more tested in upstream. So that's another reason. I guess. The third reason is even you use a helm. I would imagine you could still using istiocado some of the convenience command like it's. Your cuddle check if your environment is ready to be upgraded or it's your cuddle analyze command just to analyze, your yamos and all that christian. What's your thoughts.

B

Yes, exactly so, basically, you can use ham to manage your manage the state of your steel installation, but if you need to look into specific configurations or you or you are troubleshooting a live issue, then still ctrl is a great way to debug your your service service mesh. In my example, we will use, have to upgrade a service mesh and use the skill set here to check its progress, for example, using simple visuality or commands, like version or proxy status,.

A

Yeah very good looks like we should get into the demo really soon so before we get into the demo. I just wanted to mention a few things about upgrade consideration which we can dive into each of the topic in a following, live stream after today.

A

So when you do upgrade, you want to make sure you consider the ordering among your components, so your control plane has to be upgraded first before you upgrade your data plane, because if you upgrade your data plane first, that um your data plane is talking to out control plane, that's not a scenario with testing upstream so and we know sometimes it will break you. So definitely don't do that uh when you are doing gateway upgrade. We want to make sure to sync through right: are you going to have ip address changes for your gateway?

A

You know, how are you going to handle the incoming traffic to a different point, or are you going to reuse the same ip address or host name for your gateway? If so, you know, how do you do it safely and smoothly if you're using cli today, um I want to remind you that cli upgrades we don't support uh canary, so the only way to upgrade your istio ci is to do in-place upgrades and the final thing is about rollback.

A

You could roll back. Your is your control plan. You could roll back your data plan, but one thing uh interesting about rollback is we want to encourage you not using any of the new feature with the new issue release?

A

Otherwise, your rollback, you know, could you it could be very unexpected, so we definitely don't recommend you using and exploring any of the new istio features apis when you roll back with that, I think there's one more question just coming in I'm going to pin that question: hey thanks for the question, I'm not sure how to pronounce your name uh cassanche.

A

What's the advantage of using in-place upgrade, you said it's riskier and they can just jump one minor version. Christian. Do you want to take that question.

B

Yes, I can take that well in place, is uh more and more mature with each release and uh yes, it's riskier than doing a canary upgrade. But if you uh can afford that risk, it's a more simple way to handle upgrades and, uh for example, if it's it's not a production cluster, then you would be probably good with doing an impress upgrade and um doing only one one minor version at a time.

B

I think that's not that bad, because uh frequent upgrades are good and I would recommend going through each minor version and have a more updated version of your service mesh than jumping two versions, because that that could be a bit uh riskier, be scared and doing any press upgrade. I would say.

A

Yeah I I guess I would add it's really simplicity right because in place upgrade everybody understands it right. So I think it's good to use in place upgrade if you are okay with a little bit downtime of your control, plane and a lot of customers are okay with that, as long as they can schedule.

A

This is the time I'm going to have the downtime on my control plane. They are perfectly okay with that. So we've seen a lot of users here, stick with in place, and the other thing I would mention is remember. I talked about cli the issue ci. The only way we support it is in place right, so you still have to do in-place upgrade for cmi, even though you choose to do canary upgrades of your istio control plan.

A

So you can see the canary upgrade it's easy to get complicated right because different components, it may not all support canary and then you have to learn canary. You have to kind of gradually shifting it's just a little bit more complicated. But it's great: if you care about like zero downtime of your control plan, you want to be able to gradually shift your traffic. You are in total control, so you have more time. You have more resources. You understand canary upgrade. I think it's great okay with that are ready for the demo.

B

Yes, but before the demo, I think I have a few more slides, showcasing, simple, uh simple, even simpler method of performing have updates and it's called m-file, as I mentioned ham, is probably familiar to any of you, but, as you can see on this screenshot, there are multiple hem charts.

B

If you are you, if you want to upgrade um and upgrade this deal with ham, you would need at least um two or about maybe three uh ham charts to complete the upgrade process and there's a tool called hem file which can make it more simple, because, basically you can lock your charts. You can run ci cd pipelines through your service mesh pipeline.

B

You can handle dependencies between the charts, for example, you can say that I only want to upgrade chart x after chart y, so you can create these dependencies. uh You can. You can add these to the hand file itself, so you won't forget about those, and if you have a pipeline in place, you can critically sync uh the content of this file to avoid uh unwanted misconfigurations or unknowns in your environment.

B

So basically, that's it and the handful itself is, uh as I mentioned, it's basically a rock barrel and ham. It's interchangeable with ham. So if you have um original stock, vanilla ham releases, you can switch to using ham file at any time because only the names of the releases should match.

B

But if you declare all your repositories and releases and the names are the same, you can use the extended.

B

Extended options that have I can provide basically on this slide, you can see a simplified uh but complete steel setup uh in almost uh just 20 lines of code. At the upper section you can specify the repositories and before that and below that, you can list your releases and, as you can see, there's the needs close added, which is useful to handle dependencies between these so the dependencies in case of seo as the following. First, you have to install or upgrade the steel base chart with, which includes all the cluster-wide resources and the crds themselves.

B

After that, you should upgrade or install the control plane itself. And finally, if you are using ingress or rigorous gateways, you can add these and set up the dependencies so that you won't have the gateways installed before the actual crds.

B

And if you have a simple hand, file like this. Basically an upgrade is just about bumping. The versions of the charts from your old version to a new version and execute, have, I think, or apply to have the upgrade ready and in the demo we will take a look at the actual upgrade from the latest.

B

My better version of 112 to 113.

B

I think I'm ready to go to the terminal.

A

All right, okay,.

B

B

So in this terminal you can see a readme file, a ham file, still ctl binary itself and some additional components that I will talk about later. We will publish these uh code snippets after the session, so you can reproduce every step at home at your own convenience.

B

On the right side, we can see two watch comments running the first is watching istio ctr version. We will use this to track the status of the control, plane and data plane and at the bottom we can see the ports running in the default namespace.

B

Maybe you are already familiar with the spots. The these. This is the famous book info application. As you can see, I'm using the new the latest sdoctl.

B

Release so it's 113, the client is 113 because that's the latest still ctrl and the control plane of the existing stomach is 112.5 and the data pane is also 12.5. We have seven proxies. Basically, we have the applications themselves and the ingress gateway.

B

So let's take a look at the hem file. It's very similar that to the one that I showed you on the slide. As you can see, we have the repositories listed here for steel. We only need this histo repository, but I added these prometheus repository as well, because at the bottom we have a cube prometal stack installed, because at the end we can we might want to monitor the performance change for the new upgrade after the upgrade is is ready, so we can use this to have it installed automatically.

B

So, as I mentioned, the upgrade with hem file is basically about bumping the chart versions, let's jump to 13.2, which is the latest version.

B

As I mentioned, uh this is for the base, sql components, crds the service accounts and all the all the base components is still discoveries. Stod is the control plane itself. We should pump diversion here as well and at the bottom we should bump the version of the gateway.

B

Okay. We have 113.2 everywhere. So at this point, it's basically a simple command.

B

To upgrade the cluster hem file, sync.

B

We will see that ham is performing the upgrade for the components we can track these status uh in this window, and at this this window we will see that the control plane via soon uh have a matching version with the client.

A

Yeah, hey christian! Well, we are waiting for this. There was a one question from luis. Thank you for the question. Hi. How do you know what is your helm chats to upgrade in what order? Do you think it's a reasonable time to take the question or we can defer it to you.

B

We can take the question now. Basically, you can pass.

B

Dash debug to have all the changes changes uh listed during the upgrade, but I guess the question is more about having this before you are performing the actual update. uh If you are familiar with ham, you know that you can use ham template to get the changes listed. Basically, you can use the templating capabilities of ham itself to list all the changes, so you can generate pull requests based on the changed commands to change config, so you can uh review it by your own eyes before performing the actual uh update of the components.

B

I guess the question was related to this.

A

Yeah thanks for the question, let us know if that's the or just anything we should clarify for you.

B

Okay, so the upgrade should.

B

Be ready, but we haven't seen the control plane uh to be upgraded to the newer version.

B

Maybe the repository was not fully and today, let's upgrade.

A

These repositories.

B

And see if I made a mistake.

B

Okay, let's check the file.

B

Now there was an issue there, 13 13.2.

A

Maybe the 5 wasn't saved, I remember you changing it.

B

Yes, yes, I changed those and I remember saving it, but maybe I made a mistake but okay now now we have the the proper versions. So, let's.

B

Sync, the content of the hair file. Again and now, hopefully, we will see the control plane, upgraded.

B

We can switch to steal system, so we can. We would see the watch uh restarting after the update.

B

I guess base component is ready, so now we will have the control plane. Okay, so we have a new container, creating yeah.

A

That's great yeah.

B

That's the control plane itself after this is ready. The old control plane will terminate okay, we have this as well. He still ctr is listing uh the existing version and the new version as well. Oh- and it's called pilot- maybe there's a bug in istio city, because earlier this was called uh control plane. I think.

B

But uh okay, we have this ready. So we have the new sdod pod running and yes, okay now we have control pin. So if the oct is the binary, it was already on the latest version, and now we have the control plane on the latest version. Everything is working as expected. The upgrade is ready and the control plane is upgraded.

B

We can see that the steel, ingress gateway is still old and we can see that no data plane bots, are using the new version. So first we should perform a rollout restart on the ingress gateway itself. We will see a new pod coming up container, creating and after it's ready, we will have the old one decommissioned.

B

And yes, at this point, we can already see that uh we have a single proxy for with the new version and we have okay, it's jumping between the two positions. But basically we can see that uh we have a successful migration for one of the data plane uh workloads and that's the uh english gateway yeah.

A

Very cool, I think this explains why you know people would prefer in place for certain scenarios. It's just so simple yeah, especially. I really like the way you kind of have the helm file. You know to control. It's like a declarative configuration of your issue install right whenever you want to move to 113 instead of 112. All you need to do is update that yamo and deploy.

B

Yes, exactly so after we have the ingress gateway migrated to the new version. We can perform a rolling restart or we can perform the running restart for our business application, which is in the defaulting space and after I executed this command. Yes, I should switch to.

B

Switch to the default namespace and we can see all of our existing pots terminating and the news new ones will coming up and yes up here, we can already see that the uh the whole data plane is now running on the latest version, so the upgrade itself was successful.

B

We still have a few bots terminating, but after a few minutes, all of these would be gone and- and yes, basically, the upgrade- was this single command and the two additional commands of restarting the workloads.

A

Okay, that's really cool. uh Can you talk about how to monitoring the upgrades.

B

B

Istio is enhanced with the promotions compatible metrics. We have this uh covering all the uh all the components. So basically you have proxy level metrics, for example, for your invoice internal details. If you need to troubleshoot something there, then you have the service level matrix, which is the most useful when operating kubernetes and service meshes, because that's one of the key benefits of using this deal. Let's say you have applications running with multiple written in multiple languages.

B

You have go applications, you have java applications, you have php applications and if you want to have a consistent set of metrics, you will have your had to have your developers develop these metrics and instrument the applications one by one, but when using a service mesh, for example, still you can have a consistent set of metrics, uh for example, if your request total will be added to all of your workloads without a single line of code, so it's a really convenient way to have service level coverage for metrics, for example, if you want to track status, codes or latency, for example, for different services, and then you can have a control panel metrics as well.

B

For example, you can check the version of of the control plane or you can check other internal configurations or characteristics of your of the of the performance of your control plane.

B

I have a dashboard that I'm using to monitor the actual upgrade.

B

This dashboard will be added to the root repository. So when you are reproducing these steps- and you are performing your own, if you upgrade at home, you will be able to track the performance changes after the update was finished.

B

For example, you can see that when the orange line is starting, there's an inflection point and from that point you have the latest version and you can see you can already see a decrease in the memory footprint or the cpu or in the cpu usage for for your sidecars or for the control plane itself and additionally, other internal um metrics are exposed. For example, you can track the number of curtains running in these your components.

B

So if you, if you are running, if you're running into performance issues after an upgrade, you can have the educated guess regarding where it could what it could cause it. Basically.

B

So, yes, I think that about the in-place upgrade with helm and file and about monitoring the actual changes in terms of configs and performance.

A

Okay, that's great. I hope this really shows you guys that in place upgrade can be really easy, combined with health and the helm file. You know I just love the declarative way. um There was one question uh from krishna. I thought it was a very thoughtful question.

A

Is there a recommended order when upgrading multi-site installation and I've been asking for a clarification question from you? So you mentioned that it's it's your cluster on kubernetes cluster, for example, in u.s, east and the u.s west, with easter west gateway and discovery setup, it's a single mesh with common root of trust. You know I'm going to take a step uh and then ask christian. If you want to add anything, you know. I think it really depends on how you are configuring. Your multi-class iso configuration if you are using a multi-primary model of multi-cluster.

A

I would recommend you to upgrade your control plane first and then data plane right. You could do that on you. One cluster then move on to the next classroom, if you're using um the shelled control plane across multi-cluster, which, by the way, I don't really recommend that model. That model really says you know your istio control plane are managing data plane on remote cluster. In this model, you would want to make sure you update your control plane first before you upgrade any of your sidecars in each of the cluster.

A

So definitely the the answer really depends on your multi-cluster environment setup with issue so christian anything you want to share on that.

B

Yes, maybe one thing that I would add is: I would suggest you and basically everyone to have matching environments, uh that's not in production, so environmentality is. It can be sometimes because you are running uh mostly replica of your prototype environment, but having opportunity to test these upgrades and perform the testing and load testing with a similar with similar workloads and similar setups.

B

That are basically matching your collection as closely as you can get a cat can or can afford. You will have many benefits of removing the unknowns of of an upgrade, because basically we can say that canary upgrade is more is safer than than platform gameplays, but uh every upgrade is is risky in some way. So if you have a totally matching environments, then you can uh sleep better at night. Basically, if you can try this out without heating production.

A

Yeah, definitely we would recommend you using the same version. I would imagine you only deal with different versions during upgrade right as you move from 111 to 113, you may have some of your cluster on different version and krishna thanks for the clarification in its multi-primary setup. So I agree with your comments, so it will be. You could update the control plane.

A

um You know az1, and I think you should be able to update your data plan on ac one in that in in that scenario, then, you could also move then move on to your control plane on a z2 and then upgrade the data plane.

A

The reason is your control plane and the data plane are kind of the individual unit right in the multi-primary setup, where you don't do xds serving across different clusters, because each cluster has zone is your d, so it's much safer just to treat them the control, plane and data plan as the unit when you upgrade so you don't take all the cluster down right.

A

So, if there's an issue, then you only bring that easy one down, and if you really want to be sure, like christian mentioned, you want to do canary-based upgrade if you're, okay, with a little bit downtime you know in place, is definitely the easiest for you all right. I think we had a lot of really good questions. I'm super impressed with you guys. I want to take a minute to thank you christian, for you know doing such a wonderful demo and taking us through different upgrade strategies.

A

um I hope you guys find this session interesting. If it's interesting, I would really appreciate you hit the like button or the subscribe button, so you don't miss any of our future episodes. Also, if you have any interesting topic regarding istio or envoy or graphql or ebpf, you want to hear us talk about. Please enter in the comments below and let us know so. We will try to make it happen for you.

A

Thank you so much okay. So uh there was one last question uh from christian. However, as we were wrapping up, this is how azure we are. Are you, including the east west gateway? As part of the data plan, canary, unfortunately not possible due to cni, yes, absolutely, we are including east west gateway as part of the data plan, which is something you would really be very careful because that's part of your data plan right.

A

So, if that's done it's taking your traffic, so I would definitely recommend looking at canary version for it's your east west gateway. uh If, if the, if the downtime is a concern for you, you're, absolutely right canary is not available uh for cli.

A

Okay, there was one last question: well you guys just love us. Thank you. Are you? Are we going to upload the demo files in github? Yes, we are.

A

We absolutely will so give us a few days. Christian is actually on bank holiday today, so give him a few days to relax and we'll get it update this week. Thank you so much for joining us um yeah. Please hit the subscribe button, so you don't miss any of our future episodes and also let us know what other topic you would love to see, and you know we will make it happen for you. We have a lot of knowledge in this uh service manager application network domain. Thank you. So much thanks.

B

Christian, thank you.