Cloud Native Computing Foundation ServiceMeshCon EU 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Crawl, Walk, Run, Fly: Introduction to Adopting a Service Mesh in Your Organization- Christian Posta

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Crawl, Walk, Run, Fly: Introduction to Adopting a Service Mesh in Your Organization - Christian Posta, Solo.io

Deciding whether or not to use a service mesh and understanding the value/complexity tradeoff is step one when exploring service-mesh technology. Christian Posta has been helping organizations adopt this technology as a practitioner and architect for the last three (or more) years and shares practical guidance for how to adopt a service mesh for your organization. We explore top use cases like security posture, network debugging, regulatory and compliance, imperfect cloud-apps, and overall modernization efforts with microservices architectures. This is meant to be an intro-level talk with the audience better understand whether and how to adopt a service mesh.

A

Thanks for attending my session at servicemeshcon, eu 2021 in this talk we're going to take a look at adopting a service mesh in an enterprise organization and some of the challenges and things to look out for this is intended to be a 101, 201 level talk. This is not a deep dive or anything advanced for those. You know you can take a look at some of the other talks, I've done in the past or blogs or books that I've written and webinars that I've done.

A

So my name is christian posta, I'm a field cto here at solo.io.

A

We work with organizations around the world, large ones, small ones, adopting istio based technology and deploying that into production at high scale to build out their application. Networking architecture, I've written books on this and and have been discussing this and been involved in the seo community since uh the very beginning- and although this talk won't be an istio talk per se, it will pop up since that's what we we do here at uh at solo.

A

Our typical challenges are around or customer challenges that we work with are around modernization and going to a application architecture that better supports, moving faster and delivering features, delivering things more quickly, using public cloud private cloud to do that. Service mesh is a piece of that puzzle.

A

Service mesh solves the problems of how applications communicate with each other and in a typical organization. We see not just the modernization efforts around either building new services, using new tools, new architectures or bringing along older or more legacy systems, monolithic and trying to split and rewrite them and integrating with the rest of the organization all right.

A

So how do we connect services in a heterogeneous environment like this and solve the problems of service discovery, load, balancing timeouts, retry circuit, breaking the resilience aspects of application, networking things like security things like observability, and do this in a way that is cloud friendly and what I mean by that is api apis to programmatically control this at runtime, knowing that the underlying infrastructure is dynamic and is ephemeral right, so service mesh plays a role in helping to facilitate solving this type of problem.

A

The the problem that we're going to look at is deploying microservices into dynamic infrastructure and getting those things to communicate with each other. That itself is a complex problem using a service mesh to solve that simplifies some areas of that, but it's not a holistic jump in and adopt everything type proposition.

A

So the way we work with our organizations is to slowly adopt that first of all determine whether or not you need this type of technology, but then to slowly adopt that using a crawl, walk, run type methodology, but getting this sort of infrastructure into into place can really enable an organization to achieve its technical goals, which then leads to better business outcomes, and so that's where the fly part starts to come in. So the first thing you need to ask yourself is: do you need a service mesh?

A

Well, only you can answer that really. Are you building a micro services style architecture? Are you using multiple frameworks in multiple languages? Microservices themselves? That's a complicated field, so you might hear that service mesh is complicated. Kubernetes is complicated. Well we're dealing with a foundationally, complicated topic.

A

Now, if you're going to containers and you're leveraging some of this dynamic infrastructure that you know may necessitate a solution like service mesh, if you're doing things like rpc type interactions with your between your services using different frameworks, uh you know making updates to how these policies are are enforced or written.

A

You may need to look at some sort of automation that can do this service mesh can help and then, of course, if you're getting out to a large set of services, individual teams owning their own services, and you need a platform uh approach to solving these problems consistently.

A

So you might, you may look at using a service mesh for that things to keep out keep an eye out for if you're not there yet is you know the the number of services that you're trying to support, especially if they're heterogeneous different languages, different frameworks, um you know, being able to consistently understand, what's happening on the network by capturing golden signals, uh requests failing how long they're, taking uh whether circuit breaking breaking's, open opening up or how many retries are happening between services and trying to detect what's happening uh on the network in a consistent way, not leaving it up to each application developer to well, maybe they exposed the right telemetry signals, maybe they didn't and then, at the end of the day, just like any other technology, you want to pick a handful of high value, use cases and start to go down the path and iterate all right.

A

We like things like kubernetes things like istio. They provide a lot a lot of functionality and they touch a lot of different parts of the organization.

A

But you know the focus of start small and iterate pick a particular use case. Some of these are the the typical use cases that people start with. They want security, they have compliance issues. They uh data in flight needs to be encrypted uh services, communicating with each other need to know and validate that they are indeed the services they they believe they are.

A

They need to build things to be a little bit more resilient with timeouts and retries and circuit breaking, and so on do that in a consistent fashion. These are some of the top use cases.

A

So let's take a look at what this journey entails.

A

Let's start with, I don't want to say obvious, but you know it becomes very clear as soon as you start digging into these types of solutions, there will be a learning curve.

A

There will be a learning curve in terms of how your particular organization adopts this technology just like. Maybe you did and went through with kubernetes all right. So these types of tools where you're blending, infrastructure and developer concerns require different parts of the organization to come together and um and figure it out together.

A

Ideally, you will have a foundational platform in place. You will have automation in place things like your ci cd or any of the custom automation that you needed to build to get your platform to work.

A

You would ideally have a telemetry collection system or some some some place where you're storing time series based metrics and able to do dashboarding and and that kind of stuff.

A

Once you start to kick the tires on a particular service mesh or you choose one, then you will want to understand the underlying data plane technology, the proxies that are actually on the request path. It's a very important step in adopting a service mesh to some when they start exploring the the mesh. The data plane may seem like a black box in some cases, for example, istio or with envoy, or the I guess, onboard based service messages.

A

There's there's a lot to glean from envoy envoy is a is a white box, there's a lot of stuff that gets exposed and, uh and you can leverage to understand what exactly is happening on the network.

A

You know when we work with our customers and users in the community. We typically tend to see either people do this. On their own, or we recommend it anyway, starting with the edge first, so you want to build a service mesh where services are communicating with each other they're using these data plane proxies start with ingress start with one proxy, not 500 and un use that as an opportunity. First of all, it's a well understood pattern.

A

Use that as an opportunity, if you put it at the ingress or sort of an api gateway layer, use that as an opportunity to understand the data plane thoroughly how to debug it, how to troubleshoot it and so on how to integrate that particular piece and the control plane pieces with your automation with your observability systems, and that should be kind of the way you tiptoe into adopting a service mesh.

A

So starting at the edge again familiar uh ingress pattern. You don't if, for example, using istio, you don't need to use the sidecar proxies just to get the the traffic ingress routing at the edge. You can just start using edge and route out to any services.

A

Now, as you start to refine this, maybe you want to expand this out to different clusters or different infrastructure footprints that you may have. You may find that this, uh this proxy or ingress gateway pattern, starts to build up a layer or two, and uh you may isolate different boundaries, either different applications or different clusters or different organizational units.

A

Following this, the same ingress pattern and you start to see a mesh of edge gateways start to form, and you know from here now you can start to push to continue to push those proxies down into your infrastructure um and start to get the benefits of the service to service communication.

A

So let me pause here for a second and let's just come over, uh take a look at a quick demo. Hopefully everything is set up the way I expect. So what we're going to do here is we are going to maybe take a look at an existing set of services that we may have. We have a kubernetes cluster, a handful of services that are deployed in this case. The web api calls the recommendation service, which then calls the purchase history service. So you can see these pods running here in our kubernetes cluster.

A

Don't have any service mesh installed, we're going to install a service mesh istio in this case and we'll give it a second to go through the installation process and bootstrapping of the uh of the of the control plane. Let's see if we can do a cube, ctl.

A

Get on instead system and see whether it's coming up should be coming up. Let's also take a look at uh for this demo. We need to. We need our cloud provider to cooperate with us and trying to give us an external ip, so we can actually make calls so what we have installed.

A

If we come over here and look at the top pane, we've installed the control plane for our service mesh and then we've also installed the ingress gateway, and so this is an example of starting small right. We have our ingress gateway, we have our control plane and that's all we really need to get started. We finally do have our external ip.

A

So, let's, let's carry on so the first thing we're going to do is we're going to expose one of our services, the web api service onto the ingress gateway, so clients from outside the cluster can call in so let's apply these resources. How you do it in in istio we'll give that a second and then when we make a call using curl. In this case, we should be able to get routed to our service, and we see that that we do in this case. Web api calls recommendation.

A

There's some sample services here, which calls purchase history all right. So that's that's all good. We can also do things. Istio, for example, envoy, has out of the box things like jot validation, um some other. You know, tls termination, those types of things, and we can apply that to our our edge gateway and still get start to start to get value from it. If you're, if you're trying to go a full-blown api gateway, where you need transformation, you may need a complex uh usage policies based around rate.

A

Limiting you need things like uh oidc and ldap, and you know more more advanced security features, look at something like a glue edge, which is also based on envoy and delivers more api gateway functionality.

A

So all right, so that's that's the uh crawl part of our our journey, we're just stepping our are putting our toes into the service mesh world.

A

Now, once we've got a little bit past that we get to the the walk part where you want to start giving out or allowing different teams to use the functionality of the surface mesh, like I said: client-side load, balancing service discovery, timeouts retries circuit, breaking automated security policy enforcement and so on.

A

So you need to start identify what what teams and what roles make up the the different groups that we'll be using and consuming technology like this, and how are we going to expose it to them? Are we going to give them the istio apis directly or your respective service mesh apis directly in general? Those are kind of complicated.

A

um Are we going to build a an api, your own or tie in with your own, existing automation right? So there's a handful of things to explore there when you roll out the sidecar proxies to your applications, you're hoping to do this in a transparent way.

A

You should test that, because we've run into so many different uh use cases where this is not the case. Application uh you know comes up. First starts to try to make a connection to the outside world. The sidecar process is not ready, yet starts to fail. An application can't handle the failure it wasn't built for that. So pod recycles same thing happens right, you get this, uh it's crash looping or the opposite all right.

A

The service is being spun down for some reason and the proxy comes down first, but it was there's still live connections going. The applications can't handle that so explore exactly what transparent means to you and your application. These are typically when we're migrating or modernizing. These are typically imperfect. Cloud applications and you know figuring out how the service mesh will behave with those applications is, uh is part of the walking part of this journey.

A

Like I said, iteratively take advantage of the features start to roll out the service proxies enabling things like telemetry collection. So your prometheus, your data dog, scrape the the telemetry from these proxies and uh and start to build security policies and all the while build up. Your skills around debugging, the the configuration issues that you might run into or you have the proxies in place and things start to go wrong. Things are slow, things are failing.

A

You know, how do you actually debug the network in that case uh and then, of course, any day two operation of a service mesh includes being able to upgrade that service mesh with zero downtime all right. So, let's take a very quick look at how we might roll out.

A

A service mesh into our existing services right, so we have our existing services. We don't have any any service mesh deployed, no sidecar proxies deployed here, and what we're going to do is if we call from the sleep service to the web api service, we see that it works. The communication works here so now what we might do if we have automatic injection available for our our service mesh, we may start to slowly and iteratively notice.

A

This common theme bring workloads into the mesh, and so, in this case anytime, we make a change to our application. We want to treat it uh very carefully. We want to treat it as a canary or do a slow rollout in this case we're making a change to our application because we're injecting the sidecar.

A

Maybe we haven't, made any changes to the source code, but we've injected the sidecar. So now we have in this case two replicas of of the web api service, but only one of them has the sign card. So in this case we would do some tests. Do some smoke tests make sure that everything continues to function and look fine and it sure does so then we'll continue rolling out the the various services that we have in this case, we'll speed it up, um we'll speed it up so that all of the services end up slowly.

A

You know we're fast forwarding what would otherwise be a slow canary rollout of every service where we slowly introduce the service proxy right and we follow the same model for doing things like enabling mutual tls there's another thing that we run into with some of these organizations where they have services uh communicating with each other and they want to enable mutual tls and for the services in the mesh.

A

We can do that with things like a permissive mutual tls policy where, if both sides are, are able to do mutual tail loss, then we'll do mutual tls. Otherwise, if there is our legacy services that are plain text, then we'll still continue to accept plaintext. But then there are things you can do to slowly: roll that the plain text, services out and phase them out- and you can do things like monitoring and checking. Are there any services, communicating over plain text and then at some point say all right?

A

No more plain text everything's going to be mutual tls. Everything will be encrypted in the system, but that's again a slow phase. Rollout.

A

Let's come back here, so that's the walk right. So we we've laid some of the foundational pieces. We've started to roll out the proxies. Now we want to, we want to start moving. We want to get this out into the organization, it provides a lot of value to the organization, but we can't leave behind some of our existing policies.

A

We can't leave behind some of our existing infrastructure like vms, and we can't forget that out of the box, a service proxy might not do exactly everything that a particular organization needs and then there's other things right with that.

A

We have our regulatory reasons to build out multiple clusters or isolate certain groups from others, and uh you know doing this in a secure fashion.

A

Are all parts of you know now we're expanding now we're growing now we're bringing the rest of the organization into um into this platform and doing things like uh deploying sidecar proxies into vms, treating those as first-class citizens in the rest of the mesh when traffic leads leaves a mesh forcing it through a particular egress point where you can apply policies and uh security, extensions and, and these other things, and then, as I mentioned, extensibility in the proxy, is key and uh for envoy based service measures like istio.

A

You can do something uh you can write customizations to the proxy using webassembly. That's we're going to take a quick look at here. So if we come, if we do.

A

What we're going to do here is we're going to augment or enhance the data planes in a particular request flow with webassembly we've written a webassembly module that basically, it enhances the headers in a response to a particular service.

A

So if I call a service in our cluster from sleep to, let's say the review service review service gives us some json response. We can see the response. Headers look like this. What we want to do- let's say some team- expects a hello world header that needs to be returned in the response. Let's take a look at what that would be or how we would do that so with webassembly.

A

We can do that by packaging up by so we can write the extension in whatever language we want package that up into a webassembly module, deploy that into some oci registry and share it with everyone, and then you need a way of pulling it down and installing it into your service mesh.

A

Here's one example where we've already tagged a existing webassembly extension and now what we're going to do is we're going to install it into a particular workload. In this case, it's we're going to select the cluster and the workload where the the data plane might be running and we're going to install our um webassembly module into that. So, let's cross our fingers- and hopefully I set this part of the demo correctly. Now I don't remember all right, so we did that now, let's check the status of this descriptor uh all right.

A

I think it looks okay. If we look at the status, we can see that it has been deployed.

A

So now, if we try to call that same service again and show the headers, we should see that indeed, the um the response has been enriched with this new capability and so using webassembly to modify the request path or modify the message. The messages in the request, or do things like implement backwards, compatible security protocols and so on that your organization might have is extremely powerful and allows you to continue to scale out and and roll the mesh out to various parts of your organization.

A

The last part now we're running, but the last part is now you have this mesh now, it's you know, you're bringing in the vms you've got your security policies in place traffic is, you know, leaving the mesh through egress, gateway's, correct everything looks good now. What can you? What can you do? What does that give you that gives you a tremendous amount of uh power in your application network so how services communicate with each other is not tied. Now to some centralized api management system, everything is decentralized.

A

It's not tied to big expensive hardware, load, balancers and difficult to change, dns and and tied to ip addresses and so on. When services communicate with each other, they can communicate with a global name and across clusters of across vms across clouds. The mesh is smart enough to say all right. When service a is talking to surface b, prefer surface b locally or you know, service- be a little overloaded, so fail over to the next available uh zone or fail over to a different region without the client having to know anything about this all right.

A

So now, with con with federated with by treating this estate of uh of different service meshes as a federated unit, you can get extremely powerful networking and routing and then start to build things. On top of that, and of course you want to expose this and allow all of your teams to leverage this capability and so doing that in a way, that's isolated and secure for various tenants in your in your organization is extremely powerful.

A

So the use case that I'm describing is you make multiple clusters, multiple different types of infrastructure, but your services, they're simple, you focus on the business application or the business budget. What happens when you make a call on the network service discovery is happening, client-side load, balancing is happening, global service discovery, failover is happening, and you also get the timeouts retry circuit, breaking all the stuff that that you know augments and enhances an application and and makes it a better behaving citizen on the network, and so this is extremely powerful.

A

So this is what we work with our customers at uh here's, an example of what a deployment might look like to enable that- or you have multiple clusters- maybe vms and and you're, getting consistent configuration, consistent security policies, consistent service discovery and global load, balancing provided by by this this federation, and so this journey that I just described is exactly what we work with our customers, our open source users here at solo.I, o and, and so what I've just described, comes from experience.

A

Some pain as well- and uh you know this is uh this- is stuff that people are starting to learn themselves and we're seeing more and more successful deployments of service mesh at very large scales, uh hundreds, if not thousands, of clusters and control planes.

A

So that's all. I have thanks for joining my session. Come chat with us. um Go to the solo. Slack is a great place, a lot of resources about envoy and istio, and some of the open source projects that we're building and the tooling that we're building. On top of these things, to simplify the usage and the operation of this type of technology and help you be successful, so I appreciate again you're watching my talk. There's a lot of other good talks, so go ahead and see them and and have a good day.