Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2018 (Seattle), 16 Dec 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Connecting Kubernetes Clusters Across Cloud Providers - Thomas Graf, Covalent

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Connecting Kubernetes Clusters Across Cloud Providers - Thomas Graf, Covalent

It is well understood how to run individual Kubernetes clusters in all major clouds, managed and self-managed, but how do we connect individual clusters together? This talk will guide you how to interconnect multiple Kubernetes clusters running in different cloud providers. We will discuss how to establish basic pod to pod connectivity across clusters, add load balancing using standard Kubernetes services and finally add label based network policy to secure the communication both inside the cluster and across clusters. As a bonus, we will add Istio on top to establish mutual TLS across clusters for added security.

To learn more: https://sched.co/GrWc

A

All right good morning, my name is Thomas Groff I'm, the co-founder and teacher of I surveillant. That's the company behind solium, the Solon project I'm here to talk about how you can multi can connect multiple kubernetes clusters together across cloud providers.

A

Who am I, I've been a kernel developer for a long time spent about 10 years at Red, Hat, mostly focusing on working security and PPF, so I've been living low-level in the kernel for a long while and started the cilium project and founded a company about one and a half years ago to drive saloum forward.

A

The goal of this session is to run services across cloud providers. Specifically. What we will do is we will set up a cubanelles cluster in GCP, will deploy services will set up a second cluster in eks will deploy in the same service, and we will establish service load balancing between the cloud providers between the clusters and we will discuss and show how you can set up encryption between the clusters.

A

So that's the goal of the session and one potential use case we'll talk about others is that if a back-end in one of the cluster fails and the zombies are coming in, it automatically load balances and uses the backing of two across. So that's the HJ use case. That's one we'll discuss orders. So that's the goal of the session. If you're not interested in this, you can you can leave the room again.

A

What tools do we need? We need cuban artists. Of course, I think everybody is familiar. I hope if cuban artists will repeat a little bit about the basics that we need, we will need infrastructure, api's and I'm trying to be as generic as possible. We need some form of BPC with routing a virtual network context. All cloud providers support this. Your on-prem infrastructure supports this and we need an IPSec compatible, VPN solution, all clapping, all clapping call this VPN gateway, can't all of some sort and typically all cloud providers. Support iqe won some support.

A

I Cretu. So that's that's what we need from an infrastructure perspective if your cloud provider, if your own premium for structure, provides this, you can use the solution you can set this up. The last piece we need is psyllium. What is psyllium Selam is an open source project. Apache licensed it's a little bit over two years old. It's based on BPF, a new, exciting technology which we'll talk about just a little bit the next slide. We provide several things. First of all, we are CNI plugin.

A

We provide networking, we provide connectivity between pots, we implement kubernetes services, so we're replacing to proxy, where 95% they're to completely rip out q. Proxy BPF is an alternative, a better alternative for IP tables, which is the default implementation of Q proxy we're using the power of EPR, for example, to use to do multi cluster voting. We also provide network security so ability to define what pots what services can talk to each other. We use an identity based mechanism to do so.

A

We are DNS aware, so we don't only allow you to define what pots can talk to each other. We also allow you to define what DNS requests a pot can make. We allow you to specify policy that a pot can talk to this DNS name and solar will automatically track. The IP is returned by the DNS server and then the pot can only talk to that specific IP address which allows to, in simple terms, establish policy egress to services that are running outside of the cluster.

A

We are API and data protocol, aware that's where envoy is coming. It will talk a little bit about that as well, so we know whom they understand services talking talking to each other. We also understand the HTTP REST API calls that part are making. We understand Kafka Cassandra memcache T, so you can define security policies to not only allow services to talk to each other. But, for example, to say this part can talk to my Cassandra, but it can only do a select on that table and everything else will be denied.

A

So we allow you to do API where security in one-point-four will also establish transparent.

A

So you basically turn this on and all part-to-part communication inside your cluster will be encrypted now photo release will actually allow using this technology and integrate this with SEO and envoy to enforce em TLS using kernel acceleration. The last piece we have envoy initial integration. We do Sitecore acceleration. So as you deploy something like sto you can. We can accelerate the communication between your party or applications and the sidecar proxy, and we have a very exciting new feature coming up, which is called transparent. As a cell visibility.

A

We can actually provide envoy and sto with visibility into data. Even if your application is using SSL encryption, we will not have enough time to discuss. All of this. I will focus on the multi cluster, but I think a lot of people have heard about selling for the first times, I wanted to give a quick overview.

A

What is BPF? That's the official semi official logo BPF highly efficient, sandbox virtual machine in the linux kernel. That's the accurate academic origin. What it really means is we can it's making the Linux kernel programmable at native execution, speed and that's really really really powerful. This is driving a lot of stuff, that's happening inside of tunnels, Chrome kernel right now. The technology is jointly maintained by engineer and engineer from us and an engineer from Facebook, and then we have many our collaborators across the board.

A

We'll talk a bit about that who is using vpf right now, you will see a better view, will will be if I show you the list of contributors, but I think a very prominent one is Facebook. Facebook has just presented their latest developments in how they leverage PPF and they have actually been using vpf for all production traffic publicly exposed on the internet. So every packet that went into a facebook data center on the last one a half years, went through a DPF program.

A

There are many many other use cases Facebook is using is Google is using this for traffic, optimization QoS that just started on implementing network security. Red Hat has upstream developers working on using PPF to replace IP tables. I. Think many of you may have heard about Brandon Gregg talk BPF on profiling and tracing, if not go check it out. It's fantastic cool stuff. There's a large investment into vpf into this new kernel technology who contributed to BPF. You can see a lot of companies here.

A

A total of 190 contributors have contributed to the kernel portion of BPF over the last two years.

A

That's an intro like Sulley, maybe PF gives you a background on like where we come so now we're going to talk and dive into how we can use vpf and sodium to connect multiple cuban Etta's clusters. Together, I want to level set a bit we'll go over this quickly. What I want to make sure that we all have the same understanding difference between deployments and parts deployments? Is your desired state right? You will.

A

If you have two services running, you will have a deployment for the front end and a deployment for a back end parts or a collection or group of containers, as you scale deployments, as you define how many replicas you want, if a particular deployment that number of parts will be scheduled right so part is what's being used when you scale your deployments, if you do queue, control get deployment, you will see the desired state, the currently running parts and so on deployment in parts.

A

So, as you deployed this part, every part will get a unique IP address and they're just living on your cubanelles cluster. So you have, let's say three front ends, and you have four back ends. How does the front end talk to the back end? Does it need to know about a specific part IP of the back end? No, this is where the service is coming in a queue Boneta service, so you define a Cuban at a service.

A

It will get a so called cluster IP or a service IP and as the front end talks to this IP address kubernetes will automatically load plants to a healthy back end to a healthy part. If you do a DNS, lookup, cube DNS will respond if the cluster IP there's some special cases that I'm ignoring here headless services are planned so on. In simple terms, this is how Cuban Edit services work relatively simple and straightforward.

A

How does queue bananas know whether a part of a canned is actually still healthy, whether it's still alive kubernetes allows us to define a liveness probe and converse that then will probe that back in part, every end seconds ask why you're still alive? Are you still alive? Are you still alive? I are still alive and, if not, it will remove it from the list of parts that the service low balances to, and this is being represented at as the so called end points.

A

So as you define a Cuban a disservice, Cuban RS, the API server kubernetes, will automatically also create a end points and the endpoints resource or object will contain a part I piece of the healthy back ends of the healthy parts.

A

This is all you need to know about Cuban ideas in terms of multi cluster, we're not introducing anything on top we're using this exact construct to do multi cluster, if you're an advanced users you can manually or yes manually, maintain, endpoints and and put arbitrary IPS IP addresses in this and queue proxy, and other solutions will automatically local ants to those IPS as well. So this construct is not necessarily limit limited to parts.

A

So what we're going to do seeing is global services. It's a standard given at a service with one addition and a notation where every mark a service as global as we marked is for those that don't know an annotation is a way that allows Cuban edits for third-party plugins or users to define or add annotations to a standardized object and plugins or users can then interpret that annotation and do something with so we can annotate a standard object.

A

So with this annotation we're marking the backend service as global and Salim sees this and automatically does load. Balancing across clusters sounds good. Let's do a demo and I really hope that the Wi-Fi is working because I will need a connection to my cloud provider. So I like to do like theme based demos, some of whoever I've seen a demo for me before it's going to be it's a new demo.

A

It looks similar, but it's going to be a new demo a long time ago, in a container cluster far far away, it is a period of Civil War. The Empire has adopted kubernetes and continuous delivery. Despite this rebel spaceships striking from hidden cluster, have wonder. First, victory against the evil. Galactic Empire during the battle, while the Jedi have were busy studying the book of Sto sto co, DS the Empire managed to capture Leia and bring her to the Death Star. So we're here in the movie.

A

In this scene, Leia no Alderaan is peaceful. We have no weapons, you can't possibly then name. The system. Where is the rebel base? Looks at Aldrin for a moment then resigned, then tween they're on Dantooine. So that's the setting for our demo, so I'm switching tabs here and you will see a terminal split and then the upper half is my cluster one running on gke on the lower half is cluster two running on e KS and GK is actually in Europe and II. Chaos is in in the u.s.

A

so you're trying to make this as difficult as possible. So let's check whether we actually have connectivity to my cloud provider.

A

Yes, okay, it's a bit lag, but so I have three nodes running in GK and I have two nodes running in chaos: I have a bunch of parts running here: CVF love rebel base and we have ax wings and in my eks I have two rebel bases. I also have a Cuban ettus service rebel base. So let's look at this, but I was wondering case just my short elias for cube control.

A

So we have a rebel base service and it has a cluster IP, and you can see that this cluster IP is different from the pod IP. So these are the pod I piece right. Then. We also have this end. Point structure that we discussed before I can look at this, so we have a rebel base and you can see that this end structure contains the two-part eyepiece like this one of the tool rebel base parts right so kubernetes is actually only aware of the local parts.

A

Commanders does not know about the other cluster at all, as I saw when I had when I did cube, control get notes. I did not see any additional notes, so our global service routing does not actually connect to the control plane of cuban artists together to keep it as clusters are still I've still have exactly the same scope, so I can go into an x-wing. I will do control exact I won't just launch a bash, so I'm now piloting an x-wing, it's pretty cool and a yaks wing can curl the.

A

Rebel base and the rebel base will return and say I'm in galaxy, Alderaan and I'm on GK.

A

We can also curl the service name, and let's do this a couple of times, but all of a sudden the response was different and somebody from Dantooine in eks actually responded. So how did that work so Liam transparently did low, balancing and synchronized the services in in the in our cluster in eks and made it available in the first cluster and then those standard load, balancing based on Cuba's health check information. So if my, if my part dies or becomes unhealthy and cost it, it will be removed.

A

So let's try that let's do a for loop.

A

I'm really bad at typing: let's do one more, so it just doesn't add to exit. So we'll just do this a bunch of times and then we do coral rebel base, so we'll see that sometimes it returns GK. Sometimes it returns eks right. So what happens if we scale our deployments, so you met a scale that come. How can change the number of replicas risk? We scale our rebel base, let's scale it down to zero.

A

Now it's only Chi K. So this is very quick so, like we scaled it down, and now it's always going to the local cluster. I can scale it back up.

A

Now see that cube analysis is now creating the containers and as soon as the containers are up on, the health track is responding. We see eks coming back in.

A

Nice, so we have low planting now. What if I want to do policy right? I still need to do security in all of this. So let's look at how security works, so I have a Yambol file here that defines a in this case. It's Saleem that what policy, which is our cod4 policy, but this works with standard network policy as well. Look at this policy rule this policy rule. It has a name, it says: allow cross cluster. Has a description, allow x-wing to rebel base within cluster. One description is actually wrong.

A

It must be within 2 e, KS 1. So this policies us it matches on all parts with the name x-wing. If that part is in cluster 1, which is the chi ke cluster, and then we allow egress to end points 2 to 2 parts which level which layer which match the label, ek, s1, the cluster name and the rebel base.

A

So, let's go out here and actually apply this policy and I'll have a Salim Network policy. Loaded I will need the IP address here, so we can go back into our x-wing and I should be able to do. Let's see.

A

So this is the IP of the pod in the auto cluster, which is what we allowed that still works.

A

What if I try into a curl to the IP in the local part that will not work so all policy that Salim and all policy that network policy allows, will work in this model and you can define policy to either include the cluster name, in which case you can define what clusters can talk to each other or you can define a global policy and say I want all my front-end to talk all my backends across all clusters. So it's a flexible model and you saw we did not encode any IP addresses at all.

A

The reason why this is working is because psyllium does identity-based enforcement and not IP based enforcement, which does not require cluster two to be aware of the IP addresses in cluster 1 and vice-versa. That's a benefit of the identity based enforcement model.

A

So let's talk a little bit about design principles, the first of all- and this is I- think 80% simplicity, like we wanted to usability of this- to be as simple as possible right now. This is an annotation, but I think there will be interest in the community to actually standardize this in some form.

A

We also wanted to avoid a need for a networking degree. So you don't need to understand subnets or routing. We'll talk a little bit about what you need to understand in terms of setting up, VP, C's and so on, but that's pretty simple. All the very important is must be absolutely simple to troubleshoot and debug right. You don't want to be in a position where this is not working and you have to wait through thousands of IP tables rules or figure out with TCP dump.

A

What's going on so a design goal was also to make this simple. Then security, obviously like as we cross cloud providers encryption is, is elemental. We want security policy spanning multiple clusters. We saw that in the demo and we want compatibility with mutual TLS to run sto or other service measures. We want resiliency, we want to be able to not impact order clusters. If so, if something is wrong in cluster 1, that should never have an impact in cluster 2.

A

We do not want to actually run something globally that you have to manage and then all clusters connect to. So we preserve the fate of domains. We preserve the ability, availability zones make sure that clusters can live on their own and Salim. Does the connectivity on a networking layer efficiency? So we want this to be as efficient as possible, so we do not want to introduce additional termination proxies and so on. So we did not want to go across egress proxy ingress proxies and so on.

A

So we want to use standard networking principles which gives us the native performance. So we saw one annotation in the demo, which is Global Service, true or false. We support a couple of additional ones.

A

You can actually define whether a service should expose its endpoints to other clusters, so you can say: I have a service back-end, but I don't want to expose my own pots to other class I just want to use endpoints of other other clusters coming soon will be service affinity, so you want to be able to define a policy and says go to the local cluster first and if there is no healthy back and then go to the remote cluster, you want to do local node right always prefer a back end on the local node then go to a local cluster, then coded go to the remote node and so on not implemented.

A

Yet what's coming soon. If you want more talk to us on slack control plane so.

A

Actually, let me show you this always better right, so solium is running in a namespace called slip and you will see that we're running cilium itself. That's the agent that's running as a daemon set on all nodes. Then we have an operator which synchronizes cube.

A

Another service is true and HCD that EDD is actually managed by Salim itself, so we have a cilium @cd operator, which then manages the @cd operator and the actor the operator manages the HDD cluster, so suelen comes kind of bundled in one namespace deploys itself and synchronizes to the community services that you have annotated into at CD. We have done a service which is called Salam. This et Cie external. This is a type local answer service, with an annotation to make it V PC internal. So this is not exposed on internet.

A

This is exposed inside your your V PC and it is protected by TLS, so the operator automatically creates secrets, and then we have scripts to extract the secret import them into an order cluster to establish control, plane connectivity. So this is not just free for open. You need to share so in order for a class to be connected. Cluster one needs to share the secrets to cluster two in order for class or two to connect. So that's our security model.

A

We will then have all agents in the other cluster connect to that remote data, D and watch for service changes. So whenever something changes on its new service is added or the list of endpoints is changing, the HD key will change and the agents in the other clusters will receive a notification. It's important that all agents do this connectivity. It means that we don't have a single point of failure.

A

So if there is a problem, if one solemn agent in one node in one cluster, this will not have impact on other node, so we want to be as distributed as possible. I included the example here on how you can do the internal load, balancer type on on gke, there's I. Think all cloud providers support this I haven't checked, but at least Amazon and Microsoft support this as well. So let's look at the VPC or at a kind of setup, on the infrastructure side.

A

It's pretty simple: you you create two V pcs and you can create as many clusters in those mini pcs, but you need one V PC on each side. The cider block the IP range assigned to the V PC must be unique, so you cannot just use the same IP address and that's the requirement number one. We could support overlapping IPS. What I think for simplicity? You actually want to have unique IP addresses anyway. You will then establish a VPN gateway, and so that's kind of a single click or a single API call.

A

You then set up VPN tunnels. You want at least two you want. You want redundancy, and then you set up routes to more out the IP, the V PC 1 to V PC to across the the VPN gateway. You can do static routing. You can run BGP, you can do whatever you want. Follow your cloud provider guide. How to do this? All of the cloud providers have simple guides to follow and everybody supports IPSec. So you have compatibility to cloud providers or even on Prem as well.

A

You then set up kubernetes and you can set up cuban errors in any way. One to automate. You want self manage manage whatever. I don't think we even need a specific. You latest version. I think even one nine would work. Fine you deploy, psyllium Concilium will deploy, the demon psyllium will set up at CD will manage it. You will then extract the secret which trance access to the order at CD and deployed at it into the other cluster, and that's it at that point.

A

You're ready to go, you can deploy parts deployments, annotate them and you will have global services use cases. So I already mentioned one high high availability right running services across cloud providers or across clusters, and if something back in is failing in cluster one, it automatically fails over to cluster two but they're all the use cases. One use case is shared services.

A

You you're probably running common services and I'm using Walter, as example, but Prometheus will be another one. Dns will be another one services that are needed by your other services right so by running Walt or something similar in in a cluster. You can make it accessible from other clusters and then you can choose basically add more and more classes as you go without having to deploy Walt in every cluster, so you can bundle your shared services in one cluster and manage it there.

A

This may also have security benefits, because whoever is managing or operating Walt is not actually to have access to the other clusters all right, so you can do separation of concern and limit access. I think this is very interesting, very practical, so split, stateful and stateless right. You will have a bunch of stateless apps. If you limit a cluster to stateless, only that cluster can be thrown away. We created very, very simply right.

A

You probably want to have if you care well, you will care much less about your stateless services, but for the shared over the for the stateful stuff, cassandra order order order, databases. You will need to put a lot of more thought into when you upgrade that cluster when you move services and so on so splitting this will definitely simplify your operation, another one like cv right, so we all heard about the kubernetes bug that was fixed quickly bug in the api server.

A

Basically, all versions of cubanelles were affected. A lot of people thought okay. We actually have to upgrade our cuban edits class, so we can't get away with just running an ultra latest version, so everybody was like time to abandon a cluster. How does this help? Well, let's look at this simple version or simple example: you're running cuban s19, it's actually affected by the cv, you're running a front end, which is stateless when you're running a data store.

A

What, if you could evacuate your stateless services to a newer cluster version, 113 use global services to still have the front-end talked to the data store in the old cluster right. You can just move this over, as we saw in the demo like the the the handover is basically seamless, and then you can reach restrict access to your old cluster to your 1.9 cluster.

A

We actually have time to upgrade that cluster or do whatever you need to do to fix it, and you can actually have your developers work on the new, your cluster version, it's your integration, so we get. This question asked a lot. So what is the difference on what is do psyllium kubernetes? So it's geo primarily- and I will be simplifying this a lot, but SEO and kubernetes are basically control planes right. They do service management, identity, provider, integration, telemetry collection, a lot of other things. It's basically knows what to do.

A

Right knows what services don't run run knows what diplomacy run and so on. So liam envoy, data plane, actually handling or packets, all seven requests, doing, routing local and seeing security policy enforcement and so on they plane knows how to do it.

A

Both sto and Cuban areas right now have an an idea of how to do service definition and management. So we have like these do services and we have Cuban other services. We chose to implement Cuban Alice's first cumulated services first for simplicity, but we can implement SEO services as well. That's not exclusive at all, so Salem is completely compatible with SEO and enhances it to summarize so Salome based on BPF technology. It comes in the form of a C and ina or keep logging.

A

We implement kubernetes services primarily replacing to proxy there and can bring advantages such as global services, better scalability and so on. We implement network security, identity based very important for multi cluster routing, DNS server, IP, aware, HTTP arrest, your PC and so on. Data Pro data protocol aware Kafka, Cassandra, memcache, TMI sequel.

A

We just added Co extensions to envoy to extend envoy we've go and we're using this to add more and more data protocols like it literally took us a week to add Cassandra support, so it's becoming very, very simple: to make unroyal we're of additional data protocols, so we can do more finer, grained security, provide better visibility and so on and then super interesting. We do cite core acceleration, so if an application is talking to a site called proxy envoy locally, it's typically doing TCP TCP is invented or built force like lossy Network environments.

A

If you're on a single node, it's always lossless, so what Silla will do is it will see app talking to a sidecar socket the socket, take how TCP and just copy the data over the socket it's about three to four times faster thought about. The other element is SSL visibility, as your services talk to, let's say a cloud cloud provider, data store services, database, services, storage, solutions, all of that data. All of that communication is SSL encrypted. Why a technology called katie ellis kernel TLS.

A

We can split handshake and encryption and defer the encryption after envoy and provide envoy and is do a lot of service meshes with this without seven visibility into communication. That's leaving a cluster that is always encrypted with that I'm closing I. Thank you very much for coming. If you want to learn more, we have a slack channel which is very active. All of this is open source. That's our github repo! We have good Doc's. We you can follow us on Twitter to stay up to date.

A

If you want to hear more about Solem, there's another cute con talk in this room at 1:45, which will focus more on the security aspects. I think we have do. We have time for questions all right. We have time for questions. Yes, I will repeat the questions.

A

A

So the question is: do we sync anything other than services right now, such as secrets? The answer is, no, we don't would love to have a conversation on what makes sense, definitely careful in just randomly syncing secrets but I definitely interesting concept. The definitely love love to chat and follow-up. Yes, so, whatever.

A

Okay, the question is: how does this relate to Federation? You can use Federation. This does not depend on Federation at all. We are definitely integrated interested to kind of evolve together with our Federation. We did not want to take on all of the complexity at Federation brings so right. Now, it's completely separate but compatible.

A

A

Okay, so the question is: can we do low planting decisions on orders, other elements such as health checks such as latency and so on? Right now we do support weighted round robin, so you can say that certain backends should receive more traffic. We do not have latency tracking yet, but in the end, it's all implementable. Bpf gives us the flexibility to do all of that, but right now it does not exist. Yes,.

A

The questions can be replaced on what we don't even want to I. Think honor is great. We Salim has envoy filters on metadata listener, Network and HTTP level. We heavily use envoy and the accelerate envoy, so I think envoy and cilium are a great combination together. We we can do, for example, faster al for low cleansing. So if you don't want to use one way for just al for I think so, Liam low planting is a good fit if you're talking about offloading HTTP, maybe but there's no plan right now to fully do that.

A

So I think one more insulin master is a great combination to run. Yes.

A

The question is a coup proxy. You have iptables rules, you can you can manage those, you can look at them, you can use TCP dump, so we provide tooling. That is very similar, but we believe it's simpler because it's not rule-based its intent based. So if you look at, for example, the policy you will see about, you will see the security identities that are allowed. You're, not you know, you're not seeing IP addresses. You can still look at that. You can still use TCP dump. We think we actually provide better, truly I.

A

Don't have the time to demo. All of that, but have a look, run it and provide us feedback in case something is missing. Yes,.

A

So the end: what is the performance penalty of the encryption? It's using standard IPSec? There is clearly a performance penalty. If you are encrypting, IPSec has the advantage that it actually uses accelerated CPU instructions if available and modern CPUs do expose that and to allow for that, so it will be. It will be. This performance will be the same as if you use IPSec right, it will make use of Hardware if possible.

A

I don't want to quote the specific number, because it will really depend on what hardware you're running and what are you running, VM or bare metal and so on? Yes, so.

B

What about the case where you want to maybe such as stateful applications across clusters, where you want to split, speak to specific pod versus a random service? A random pod behind a service can.

A

You repeat, it I'm, not 100 percent sure whether I understood it sure.

B

So if you wanted to say, span and application across clusters, but you needed to address a specific pod such as you when you run stateful applications, so you don't want to go to a random pod behind a service, but you want to go to an exact pop. Yes, how would you do that here? Okay,.

A

Question I think everybody hold a question. So first of all, Salam is split. There is pod IP routing. You can talk to the part IP directly in your cluster. It will just work, so the global service is the lopa. Lensing is split from the palo p4. Well, you don't have to go through the service IP at all. You can talk directly to the other part I can in your cluster, but then you need services, call ray that tells you what that IP. Yes! Thank you.

A

A

Question is: can you use solely me phone Auto, CNI and just use the global services? Yes, and no, we just were just disclose to publish our initial innovation with flannel to run on top of flannel and cube net was a very basically leveraged at the forwarding decision of flannel and cube net and you new policy or low plans in whatever you want. We've cilium, we already run in combination with cube router, so you can run psyllium in a mode where it just passes. Information on QQ brother.

A

You can also run your own routing, daemon, BGP, daemon and just tell Salem's the desired routing and it will route according to the to the Linux routing table. So, yes, we want to split it right now. It only works for cube. Router, flannel and cube net will be next. Yes,.

A

Questions, how does the D yeah? So? How does the DNS in the resolution work is using standard cube, DNS, cubed NS will return the service IP and so Liam will do the lope Lansing from service IP to the to the endpoints, which means that in the cubanelles cluster cubanelles does not need to know about parties in your cluster right cube, DNS will return the service IP, the cluster IP.

A

A

Sorry knows: yes, yes, go ahead,.

A

Yes, yes, so I actually had three clusters connected about I just wanted to keep the demo a bit simple. If you can connect as many classes as you want.

A

I need a VP see we don't actually care about her. You have a VP C or now you need to be able to route traffic from notes in cluster 1 to cluster, so the VM or whatever, is running your node in the cluster. One needs to be able to reach the note you cluster. Does that make sense? Yes, yes, this is was a big cloud provider specific, but as long as you satisfy no, not connectivity which will work. Ok, yes,.

C

Since you're doing some some fairly deep packet magic here under the hood, have you run across workloads or add-ons that get confused by this, or that that this doesn't work well with or you need to do specific workarounds, or is this just completely transparent to everything you've seen this.

A

Is completely transparent, so this is on IP level right, so it will work for UDP TCP SCDP, whatever it works for ipv4 and ipv6 I think there are some specific applications that say Prometheus what actually looks at the endpoints resource of cuban areas and talks directly to the power IP. In that case it would not see the part IPS in your cluster just that clear so like well, networking level is completely transparent, but not everything is actually using the cuban at a service IP how it's in a standard way the.

C

Other thing I was wondering about specifically, is anything that tries to verify the IP of what it's talking to and compare it with where it is. Have you some run across any workloads like that that that get confused by this.

D

I'm clear on this, when you say, is transparent, is transparent as long as both ends are running cilium right and then you require both ends to be.

A

Yeah, well, you can totally define a service, define an endpoint structure and put arbitrary I piece. The Lopo Lansing will work, but if you want with automatic service synchronization, then you need to sell them in both clusters. Ok, we're at a time.

B

A

You thank you so much.