Kubernetes IoT Edge Working Group, 28 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Which Edge Are You on? Service Affinity with Cilium Cluster Mesh - Raymond de Jong, Isovalent

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Which Edge Are You on? Service Affinity with Cilium Cluster Mesh - Raymond de Jong, Isovalent

Edge computing can require connecting hundreds of clusters across disparate locations and infrastructures. Without a networking solution to manage this scale and complexity, you will just have a bunch of computers talking to themselves rather than each other and your customers. In this session we will explore how you can leverage the new Service Affinity feature with Cilium Cluster Mesh for providing the correct connectivity for load-balancing, observability, and security between nodes across multiple clusters, enabling simple, high-performance cross-cluster connectivity at the edge. We’ll explore how Cluster Mesh allows endpoints in connected clusters to communicate while providing full security policy enforcement. The audience will walk away with an appreciation for how eBPF can ensure you communicate with the right edge.

A

All right, good morning, everyone, let's get started, um welcome on this session, about psyllium classic mesh and also a bit service mesh and how you could use them across your kubernetes clusters and on the edge.

A

um How many of you know about ebpf or psyllium, good cool, that's great for those who don't I will introduce the technology before we get started with psyllium plus the mesh. My name is Raymond Dion I'm, a Solutions architect, working at I surveillance and I surveillance, is the company behind psyllium and also in the foundation contributing to ebpf, so isofilence, founded, psyllium and open source psyllium donated it to the community um where the incubating project. So today we're focused on psyllium cluster mesh, but for before we get started um to explain about ebpf.

A

What ebpf is we like to say what JavaScript is to the browser? Ebpf is to the kernel. It makes the kernel programmable in a very efficient way without changing the actual kernel.

A

What it allows us to do is to based on kernel events attached ebpf programs to it. Today we focus on networking, but it can also happen on things like files processes and such so current event can be something like a process opening a socket or a network interfacing, sending a packet on The Wire. All those events trick could trigger an ebpf program and psyllium is built on that technology.

A

The good thing is is that you don't have to know everything about ebpf to work with psyllium psyllium abstracts, that technology for you and depending on the flags you set in your Helm values file, for example, it will trigger and mount the right eppf programs to work for you, so psyllium is built on ebpf and it provides Advanced networking capabilities for the cloud native H, especially for kubernetes clusters, but also Beyond.

A

Today, I will focus on the networking piece, which is mainly about load balancing across multiple clusters and a bit in combination of surface mesh provided with psyllium as well.

A

On top of silium. We also have observability platform, which is also useful for tracking workloads across clusters, and being able to observe flows between surfaces within your cluster and across clusters and being able to make informed decisions about network security, for example, for these workloads so overview of cluster mesh. First of all, cluster mesh provides for multiple clusters: the ability to connect clusters together to provide a unified data plane to be able to route and connect and load balance workloads across clusters.

A

On top of that, we provide observability for those clusters using our identity, aware solution. Instead of tracking IPS, we are able to create identities and attach identities, to the workloads and being able to follow and observe identities across clusters provide that we have released surface mesh. You can also use service mesh across clusters, and also encryption and pod IP routing are are available to get started. You have to have at the moment at least non-overlapping IP ranges in your clusters.

A

You need to be able to connect the Clusters together, using, for example, a VPN or or some kind of physical wire, depending on your topology, and it doesn't matter. If you run on clouds or hybrids or on-prem, it can run an open shift, but also on gke, and if you want, you can also run it on Bring, Your Own, let's say DB and release as long as you have a recent fairly recent kernel for the eppf programs to work a little bit more about the architecture.

A

It works in such a way that each cluster has a cluster-based API server which keeps track of its identities and exposes those identities to remote clusters, and each cluster has only read only access to another cluster, and that means that agents in each cluster are able to read and know about identities in other clusters, and that provides all the security and connectivity and observability across clusters.

A

So the best practice is to expose an API server through a load balancer because obviously that's high available. But if not available, you can also expose it through a noteboard. So let's go through some of the use cases. The main use case is obviously High availability, where you want to expose or low balance traffic to a given service across clusters.

A

So the cluster mesh technology works with the concept of a global Service, which means is that in each cluster you create a namespace with the same name.

A

You just create a service in each cluster, a simple cluster IP service and you annotate it with a special annotation which I will show later and that will trigger psyllium to learn about or advertise it and phones across clusters, and that will, depending on the configuration you will set, will load balance by default traffic across clusters, and that means that if you have endpoints failing in a given cluster traffic and low balance across cluster to the remaining endpoints in your clusters, another very useful use case for cluster mesh is that of shared surfaces.

A

We see more and more and also on edge clusters, smaller clusters or multi-tenant clusters, for which you don't want to necessarily set up shared services in each individual cluster using cluster mesh.

A

You can create a centralized surfaces cluster for things like monitoring things like Secrets things like DNS and then expose that surface through a global Service and connect that surface to remote clusters, for them to consume or to connect to those services in your Central Services cluster, and this obviously allows you to be more agile with your tenant clusters or Edge clusters in such a way.

A

Another use case is splitting services, and this is about stateful versus State, less so similar to shared services. In this case, you would have a centralized cluster, which you want to store data.

A

You need to know the state, but you want to keep your Edge clusters or your remote clusters, Nimble, agile and small, as you can, because you want to be able to life cycle them more easily as such, again, the same principle exposing a surface in this case a data store service, for example, as a global Service and exposing that service across clusters, for them to be able to connect and be routed to that cluster, then a bit of a little bit more about the actual core of this session.

A

That's the local and remote service affinity and we like to call it topology error routing. What this helps a lot with with the previous use case, is that you can engineer the preferred way of routing to your endpoints across a cluster mesh topology.

A

So in this simple example, we again have a backend service, and in this case we want to prefer obviously local endpoints above remote endpoints.

A

What this allows you to do is to obviously create high availability across clusters, but only when you need it only when all end points in a given cluster are failing, and this also obviously is very useful in multi-cluster uh topologies, you want to avoid cross-cluster traffic because that induces latency and such or cost using a local service Affinity effect makes step more effective and reduces latency.

A

The other way around is the remote service Affinity. The most useful use case here is to temporarily annotate a given service to prefer remote endpoints. Instead, what this will trigger is that all traffic by default will be forwarded to the remote endpoints, and this is very useful in Rolling, updates or upgrades or restarts of workloads in a given cluster.

A

And this is how it looks like so. This is a simple example of a surface of type cluster IP and you can see that it's a back-end surface and you can see two annotations, the first annotation uh that one of global servers equals true that triggers that given surface in that cluster, with its endpoints to be advertised across clusters. So you have to do this for each cluster. You want to expose endpoints for and the second annotation for service Affinity local triggers the local preferred, endpoints service affinity.

A

The other way around is the remote service. Affinity looks very simpler, simply very similar. Sorry, and in this case you would annotate it with service Affinity remote to prefer remote endpoints instead of local endpoints.

A

So if you don't have a service Affinity endpoint, it will load balance across close suspect, defaults and now a little bit more about how you can leverage psyllium cluster mesh in combination with psyllium surface mesh to do more advanced things. Celium service mesh is being released in psyllium, 1.12 and last spring early summer, and this allows you to create, for example, resources such as Ingress layer, 7 path based routing, percentage-based routing, Canary, rollouts things like retries and such so.

A

In this example, you can use, for example, an Ingress resource to attract traffic into your cluster, to do some kind of layer, 7 routing in this case percentage page routing, but you can also those surfaces which is being forward traffic being forwarded to can also be uh Global Services, and you can do the same traffic engineering with those Services as well.

A

This is another example where um we have the topology array, routing the service Affinity across clusters, where each front-end server service prefers. Obviously local endpoints and its back-end surface prefers local endpoints, but again in case of failures, um traffic is being forwarded to the remaining endpoints in other clusters.

A

Another example is that you can even use cluster mesh in combination with surface mesh, to spin up new clusters and to do, for example, Cannery rollouts to that new cluster. There may be reasons why you want to migrate to a new cluster. Perhaps it's a new cloud provider or you want to move from on-prem onto Cloud.

A

Your applications are already are already running and you might want may want to migrate them in a in a step-by-step way, slowly introducing more traffic or you even deploy a new version of your application on a new cluster and then slowly introducing traffic into that new cluster, using both psyllium service mesh and cluster mesh. So this this is just one example. It's super flexible in terms of how you architect, it and also without sidecar psyllium, doesn't introduce sidecars in the service mesh solution.

A

So that means that on node level, if endpoints are in the cluster already, we reduce latency like three to four times compared to a sidecar implementation.

A

This concludes my lightning talk about Gloucester mesh.

A

First of all, if you want to know more how it actually works, if you want to try it in a lab isofelion.com forward, slash Labs is a great resource for learning more also diving a bit on how you can, for example, use Network policies across clusters using celium cluster mesh on the open source side. We have the sodium.io website with also getting started. Guides of cluster mesh, also with surface mesh, and our slack channel is open where we are able to support the community. People are happy to answer questions as such.

A

We also have a booth, both silly and booth and isovalent Booth. If you want to visit us, please feel free to do so. We also have Liz rice with us who's signing uh the the ebpf book, so you can get a copy signed copy from us, a very good book to learn more about how eppf works and with that I'm happy to take any questions. Thank you.