solo.io Hoot Livestream, 12 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cilium L7 Policies vs Istio’s

Description

Istio offers rich L7 traffic management and security policies. Cilium also offers L7 policies and we have gotten a lot of questions from users if they still need Istio's L7 policies. In this livestream, Yuval will join Lin to explain and dive into the difference of the two focusing on security perspective. Bring your questions!

#istio #cilium #l7 #policies

00:03 welcome + speakers intro
1:42 Cilium L7 docs
4:22 Live demo Cilium L4 + L7 policy
12:24 How Cilium L7 policy works?
16:32 summarize of Cilium L7 policy
23:09 demo of Istio mtls + auth policies
31:06 not just a certificate
32:27 Compare Cilium L7 vs Istio L7 policies
43:50 wrap up

To learn more about us, please visit https://www.solo.io, we are hiring! https://www.solo.io/company/careers/

Questions? https://solo-io.slack.com
Code Samples: https://github.com/solo-io/hoot
Suggest a topic to cover here: https://github.com/solo-io/hoot/issues/

A

Hello, everybody welcome to another hoot episode. It still offers very rich layer, 7 traffic management and security policies. So liam, to my surprise, I actually learned this recently also offers iso ci. They are seven policies and we've got a lot of questions from our user if they still need it still slayer 7 policies. If they're going to use sorry about my phone, if they we got a lot of questions from our user. If they still need issues, layer, 7 policy, if they use salience layer, 7 policy from silian ci.

A

So in this hood live stream, I am so happy to have. You join me to help me explain and dive into the differences of the two, um focusing particularly focusing on the security aspect. So you are, I think, most of our audience know you and many of them actually memorize your voice.

A

But for folks who doesn't know you, can you give a quick intro to them.

B

Definitely I'm yuval kojavi, I'm the chief architect in solo. I o, and I do a little bit of everything in terms of what solo does and I'm excited to be back. I had quite a few episodes on the hood, so it's nice to be back.

A

Thank you so much you are so excited to have you definitely love to have you back so uh what we're going to do first is we're going to talk about silence, layer, 7 policy, uh so let me add my screen to the stream.

A

If I'm doing it correctly, all right looks like I am so. um Like I explained to everybody, I was just I learned um as part of the psyllium cri. They do support layer, 7 policy right so to the layer. 7 policy redirect proxy traffic to a psyllium user space proxy instance. So apparently uh envoy is used as part of their layer. 7 policy support to help mediate the traffic, so there are even some examples on the layer, 7 policies.

A

So if we go to the cilian web website on layer, 7 policies, so you can configure layer, 7 rules on http kafka and dns so for this live stream, we're going to dive into particularly on http, because um http is the most common uh layer, 7 policies- and you can see it's allow you you can do from your endpoints, set up ingress rules as part of the cilium network policy. You can specify port number and most interestingly, you can specify the method and the paths as part of the layer, seven policy out of ceiling network.

A

uh One other thing I want to share. I find a very interesting eye service. Looking at this is uh not only based on label, you can also have a config service account in the layer, 7 or even layer for policy. I believe, are you. You are please correct me if I'm learning things, because I want to invite.

C

A

To validate my learning so feel free anytime right. So this is an interesting example right, not only you specify the endpoint by labels, but you can actually specify service account as part of your from and also the target endpoint. So with that, we are going to do a quick demo on this. Let me go ahead, remove this from string and let me go ahead and share my other screen. If I'm doing this properly, um oh great, so this is my kubernetes cluster. As you can see, I have kind uh installed.

A

I have a two-note cluster uh using, um so I have a worker one and work worker two on my kind cluster and I have a bunch of parts installed. I have psyllium already installed I'll, follow the default ceiling installation I use cillian to replace the cube proxy. I also have a couple of sample installed in the default name space, so I have a sleep pod, um sleep service and also a lot not sleeve and the hollow wood version one and version two. So the only thing on version, one and version two is version.

A

One is on a different note uh than the sleep part. Well version two is on the same note as the sneak pod, um so for this demo we're going to explore a little bit on the cilium policies.

A

So the first thing I'm going to do is I'm going to do a status command, just making sure uh you have visibility to my ceiling environment. So, as you can see, I have um ceiling and it's operator installed. I have hubble and cluster mesh disabled, so we already have the parts installed.

A

So the next thing we're doing is uh from the sleep pod. We can call the hollow uh word and also from the not sleep pod. We can also call the hollow wood right because there's no policy at the moment, um so I do have a civilian layer for policies so which we're going to apply next. So what this policy does is leverage the kubernetes service account construct in the selectors.

A

So it essentially says um I'm going to allow from sleep to hollow wood on the port 5000 for any of the tcp traffic. So let's go ahead, apply this policy right. So so, with this policy applied. um What's interesting, is um syllan, generates the endpoint object and also identity objects. That can be used for this policy. So if you're looking into a little bit information of the endpoint object, you can see a little bit more information about any of the endpoints, for example.

A

The first end point is the hollow word, so uh it it captures the endpoint from silence perspective, along with the identity information. As you can see, the identity is pretty much based on labels at this point.

A

So now that we apply our policy, let's continue to try to reach from sleep to hello. Right so remember what policy is we're going to allow from sleep to hollow on put 5 000, so this continue work which is expected, um but let's see if we, what if we access from not sleep right.

A

So in this case you know it's um a different service account by the way I forgot to mention the um the not sleep part was using a different service account, which is also called not sleep where the sleep part was using, the service account sleeve.

A

So this is what you expect on the layer for network policy which essentially blocking this traffic right. So this is not surprising, not a great experience. Well, you know your requests just hang there, but it's kind of expected.

A

So let's take a look um of a layer, 7 policy I have in my environment, I'm going to apply to my environment. Excuse me so in this example, it's pretty much same as before. Right, the only thing is: I'm adding a rules, that's http rules- and I'm saying you know on this particular port.

A

I'm going to also allow I'm only going to allow a get method, call to slash hello and nothing else right. So, let's go ahead, apply this layer, 7 policy, so um we got our policy applied and next thing we're going to do is to do a couple of commands just to check you know. Are we allowed to access to this hello world farm sleep and are we allowed to access from sleep to hollywood, but on slash, hallows 3, which is not a past? That's supposed to be allowed. Also.

A

Are we allowed to access from not sleep? So, let's hit enter on this so, as you can see, the first one runs through, which is from sleep to hollywood, on slash hollow the second one. uh We actually get a really nice message access denied. So I love that message.

A

um So essentially it's telling us we're not accessing a path, that's being allowed. The third one. There was no feedback to us same as the layer, 4 policy, um so you know we just didn't get anything so with that, I'm going to cancel this out- and uh um oh one last thing I want to show is: I do have my pod inflammation captured.

A

So in this case I have this pod config to a cilium pod, which is this particular one. So what we're going to do is uh dive a little bit deeper into you know. What are the process running on this cilium pod right? How is the layer 7 works right in this case, so how does celia knows to say you know I'm going to allow a get request on slash hello um for from the from the from the sleep part, um the service account associated with any part uh that associated with sleep.

A

So, as you can see, there is a psyllium envoy actually running. In fact, we can actually dump the configurations on this envoy. So if we do a config dump, we can you know we can get all the configurations on this envoy.

A

One thing interesting here I mean you, while you are the only expert feel free to correct me if I'm uh or append anything so one thing interesting is: there is a h layer, 7 policy, which is the http filter as part of the envoy configuration for trs traffic. I believe there's also another one for the plane traffic, the lower buffer.

B

Yeah, the tls is for a different feature that we didn't cover.

A

Yeah, do you want to elaborate a little bit on that so.

C

A

Layer, 7 policy from ceiling.

B

Yeah sure um I can elaborate on that so the way, maybe I'll start by describing kind of how it works, and then I we can go into a discussion of kind of pros and cons of this approach versus its use approach, and does that sound like a good structure.

A

Yeah definitely perfect.

B

And obviously, if there's any questions from the audience, you know I love answering them so feel free to send them at any time. So the way psyllium achieves layer 7 is whenever it detects. You have an l7 policy in your network policy. It spins up an android proxy and transparently proxies traffic to it right. So you have one android proxy per node.

B

That uh interprets and applies the l7 policy. Android uh uses xds to not only get uh its regular configuration. It also gets these cdml. Seven policies from the serium control plane and the serum filter evaluates the policies and applies them to traffic.

B

uh So basically traffic, that's meant to go into a pod, goes into android stand and then it leaves envoy it will go into a pod. That's kind of the general structure of how this works versus the initial, where the policy is evaluated in the sidecar using a filter that was upstream to envoy the arbuck filter that apply our back rules and and uses that to authorize requests based on the spiffy identity of a request, or it could be other properties as well.

A

So, are you saying this particular filter is not upstream to angular, yet.

A

Okay, that's interesting yeah! I didn't know about that yeah, um so one other command. Actually, denis taught me to run. This is if you, if you want to try to understand the content of the layer, 7 policy from psyllium, so there is this command. That's a little bit more complicated.

A

It's essentially sent this request to the psyllium xds service right and it's trying to you know trying to find a little bit more information about the the network policies. The layer 7 policy is trying to use.

B

Yeah, so what you're doing right now is kind of emulating what amboy does and contacting the control plane asking it to deliver the network policies to you. This is essentially a very simple implementation of the xds protocol.

A

Yeah interesting, unfortunately, this command rounds are pretty slow, but I do have the output captured, I believe, uh from an early run. I had. Let me see if I have it somewhere. I.

C

Think I have it okay, so I have the config dom. um Maybe.

A

Oh actually, no it's in the temp directory yeah! That's why? Oh, maybe not! I guess it's being overrated yeah. So um I guess we'll show this uh when this is rallying now after this finish is running okay, anything else, I think that's the demo. I have uh from a ceiling perspective on the layer, 7 policy.

A

As you can see, it's uh it's relative, flexible right. You can it's pretty simple to use, um but from what I can tell it's not using mutual tiers. It's not to do the the tls handshake from parts from. I guess. My sleep till the hello world.

B

Yeah and I I can expand on that, if you like- yes, please so the way sodium works by default, and I'm not talking about ipsec or wire guide modes, I'm just talking about the default mode without encryption.

B

uh Basically, the pods a go into an ebpf map, naturally, because it's an evpf based path and and the policies are executed in ebpf right, but in order to make it kind of manageable, you don't want a policy pair pod. That's why they allocate identities to pods based on you know, whatever properties, it could be labeled service account. Why not? And in fact you when you, if you can scroll up to that um earlier, where you show the identities and the pods, you can see that identity is basically an integer right.

B

So the way it works is that when serum receives a connection from a pod, it looks up its identity in the map right, it has what's called the uh it caches the pos, as we said so it looks up the pod uh ip and converts it to the identity and then uses this identity to run policy right, and that means the source of the identity is not a cryptographic primitive right. It's actually the network identity, the iep of the part.

C

B

So it's has some issues around eventual consistency has less strong guarantees than a cryptographic identity, so that's kind of where you have to weigh the pros and cons for yourself.

B

So, for example, if you're concerned with someone in the cluster spoofing a pod ip address or if you're concerned, with eventual consistency where, if you have hygiene environment with pods, can go up and down very quickly, there might be something to think about when you implement a network based approach uh to uh to identity, not to say that you shouldn't do it. I think defense in depth is great, but you should also consider the stronger guarantees the cryptographic identity provides.

B

So let's compare that with the way it's done in izio initio, a service account token that is provisioned by kubernetes, is exchange for a client certificate, and now the kind certificate is used for identity. So, even if the pod ip it does, it doesn't change, but even if it will, even if something happens, you cannot mistake, mistake the identity for anything else, because it's embedded in the network connection itself.

B

The connection would not be made with the wrong identity.

B

If that makes sense, I hope there was a clear explanations, are very complex topics and I try to kind of uh simplify them, but basically the the way is psyllium gets its identities using the pod ap, which it translates uh to the identity that we saw above and then use that to run policies.

A

I guess the one thing I found interesting as you were talking about this is with cilium right, because it's a ip based right as the ip changes. I assume your identity would actually change with istio. um You know your past could have the same identity right for the same service. uh Even maybe it's deployed into different cluster or virtual machine right, because it's it could be the same identity uh for for that particular service.

B

So here the identity is kind of something that similarly manages. You create network policies, and uh you know or serum network policies for that matter, and then serum kind of infers the identity for you, which is nice um in izu, the identity is always the service account yeah.

C

B

So uh the way it works with cdm system kind of allocates because they need to store everything in ebpf maps, you can't just have arbitrary constructs it basically distills the properties of the identity into an integer. It needs to obviously save it on the cluster to make sure every you know all the nodes agree on it and then maps pod ips into these identities.

B

uh There are edge cases like I I said before, if you're concerned about spoofing or a high churn rate, where the map might not update in time, uh but for the most part like for the vast majority of network flows, will work uh fine.

A

Yeah, in fact, uh yeah uh by the way my output is back now. So I assume these are the the network policies that the the layer, cilium layer, 7 policy that we saw early in the envoy config. That's uh this is what's it's consumed.

B

Exactly so, the filter you showed earlier in the config dump inside of android there's a separate cache that caches these objects and that's what the filter uses to run the policy, and this is that's kind of equivalent to how the ebbf work right.

C

A

B

A

Yeah, that is the l7 part all right. Let's, let's show a quick demo of istio. I know you are you talk about istio with the mutual trs crypto graphics? So let me show a quick demo of the sdo. uh If I can get my seo environment running all right.

A

So this is my istio environment. I deployed a kubernetes cluster with istio inside in the cloud.

A

So on my environment, I have um okay, so I have a bunch of pods deployed, but um I didn't have the psychiat injection enabled. So the first thing I'm going to do is enable is your injection, my default namespace, and then I want to deploy all the samples again right. The same exact samples with sleeve, not sleep and hollow word version. One version two.

A

The only difference here is my cluster is pretty small, so I only have one node, so you're gonna see hello version, one and version two running on the same node, but otherwise the sample is exactly the same. It's just a capacity um issue, so my parts are running now. So the first thing I want to do is um so in istio. You know everything is allow you access by default because it still has permissive um enabled by default.

A

So the first thing we're going to do is enable mutual tls, which is one of the, is your security policy and will enable me to tell us on this particular namespace, which is the default one. And then the next thing we're going to do is uh we're going to get to the sleep pod. So we're going to do uh if I can remember the correct uh the command correctly, so we're going to exact into the sleep part and and we're going to run a command to open, ssl command.

A

Let me see if it has openness as a command. Okay, so I think I need to get to the instaproxy.

A

It should have open ssl yeah, so okay, so so the first thing we're going to do is we're going to use the open, ssl command. Sorry, it's running pretty fast, because I did a copy um so to connect to the hello world uh on port 5000 and ask for the search right. So when we, when the client connect to the server it's going to ask the server, you know what is your uh certificate right, because you know this is how mutual tiers work right.

A

I asked for the service certificate and server also asked for the client certificate. When, when mutually the certificate is coming from the trusted uh chain, then they can establish the connection. So if you do a little bit uh digging into the certificate uh which, uh let's do that, so let me uh for my machine what I'm going to do is copy the certificate.

A

um Sorry, let me see um so if you um so, let me see if I can get this right. So if I can copy the certificate.

C

And and if I.

A

Can copy this certificate over? I'm not sure copy paste works really well on this environment, but the the idea is. I want to kind of show you uh what the certificate looks like, because the last time I do it, I actually did it on my machine. um Actually, this is probably not going to work yeah. The reason is this machine doesn't have a step installed, so what I'm going to do is I'm going to change to my other window so that you can see my terminal.

A

um So let me bring up my terminal here, sorry about that hiccup, okay. So what we're going to do here is uh we're going to copy the certificate over uh and the certificate is from the the show search command right. So what we're going to do is we're going to copy um the certificate over, because I have step on my machine.

A

I should have installed on the other terminal too, but I don't at the moment um so once the certificate is um copied over we're going to use step into the certificate and inspect the certificate and and then just check the identity of the certificate right. What is the speedy identity? You know how the certificate is generated and all that sorry, it's pasted very slow.

A

I need to modify my shield to be a little bit faster, all right, hopefully, that's it so step certificate inspect, and this is uh my certificate uh for for the server certificate right. So you can see it's actually expiring in a day right. This is today and tomorrow- and you can tell the spf id, which is the hollow word right. So that's how mutual trs work in the istio environment.

A

That you have a very strong identity between your client and your server side. So going back to my other screen now, uh let's go back to my other screen yep. So so we walk through the certificate right. So we were able to confirm that server side is the hollow word certificate and the server side is also going to do similar and confirm. The client side is also the certificate from the sleep one thing istio does um is um is still have authorization policy which we recommend.

A

People start with now, nothing in a particular mesh or namespace. Well, you then gradually enable things as needed. So, for instance, I denied allow nothing in my default namespace and you can see from sleep to hollow now it's not accessed with cco. You can config authorization policy to allow specific um source right very similar to what we will review from psyllium.

A

You can based on the principle based on service accounts and you can match to whichever label that has uh hello world and you can say I'm only going to allow a get method so with that supplied.

A

If you do a kirk man to hello, you can find out. It's continued to working, but if you do it from not sleep from not sleep to hello, you can find out that we got your abec access uh denied immediately right. So this is how istio works uh with the cryptographic information between your client and your server and allows you to build authorization policies on top of the the identities built into sdo. That's service based.

B

C

I'll emphasize this.

B

It's not just the the fact that there is a certificate by itself; it's also how they got the certificate, so it's starting with the service account. So you know it really represents a service account. A pod cannot just get a certificate for any service account it wants. It must present a valid service account token to ico, then that gets exchanged to the certificate.

A

That's a great point: yeah in fact: yeah the service account. I believe it's mounted to the israel proxy. So it does all these work to send the request to the israel control plan. uh You know, get the certificate and it also take over rotation right because every 12 24 hours it's going to be expiring, so it continues making sure you have the right certificates. That's valid.

B

Exactly and in this case, the pod that requests a certificate for mitsubishi actually sent something called the csr and the main distinction here that the phone never sends the private key to its certificate.

B

Yeah, they always stays in their pod.

A

Yeah, that makes sense, um so I guess we've gone through a review, a a quick demo of silence layer, 7 policy we've also gone through a quick demo of istio slayer 7 policy. So you I'd like to kind of get into the comparison phase. I mean we talk about the comparison along the way, but it might be good to summarize a little bit and you know dive into particular area, as you think makes sense.

B

Yeah- and I see we also got a question.

A

Oh yeah, yes, I think.

B

It's it's a great timing. It's related to the exactly discussion, we're about to have.

A

Yes, so thank you faye for that question, so the question is, would it be safe to say, uh because israel have a stronger identity based on service account and speedy? It will be better to use as layer 7 policy will leave celium to enforce at layer, 3.

B

The way I said, it's kind of two distinct things: uh once you got an identity, the issue is not necessarily l7 or not. The issue is that you need to know that the identity is really what it is right and with a cryptographic identity.

B

You cannot, you know in any reasonable means, uh go around it where, with a network-based identity depending on the implementation, there is a chance of confusion right and I'm not saying it's the common sense and I'm not saying that it's likely, but it's not.

C

B

Either right versus a cryptographic identity, where you know it's cryptographically proven that the pod, the connection you're processing is the correct identity, and so that's kind of the first point of comparison uh right. So the identity is kind of and again I'm talking about siloam in the in the basic mode without encryption, which is how you know the default and where you deploy it, identity is based on network properties. Ips. How do you derive?

B

You know the identity of a connection so on on one side, it's kind of ip based network based on the other side is on it's. Your side is cryptographic based and the other point, and now I'll talk a little bit about l7, is that with ezio l7 policy is evaluated on every part uh which and in silicon it evaluated on a node right. So you have a single envoy on a node that processes all the l7 policies of pods. On that node. Now again, there's trade-offs right.

B

So uh on its use approach, you need a you know, an envoy on every on every pod right. You might incur more ram costs versus the per node case, but now you have a silly, an envoy on a node, that's doing l7 processing for multiple identities. And if you look at the envoy, cve list you'll see that the most high severity cves are in l7 right.

C

So again, I'm not saying absolutely.

B

Yeah and I'm not saying, there's a security issue right now, I'm not saying it's definitely not secure, and it's about probabilities right, the probability of an issue with l7 that does multi-tenancy in terms of uh policy, because it evaluates policies for multiple identities, all the identities. On the known to having an issue here, there is higher than an an issue, impacting you know a single android. That only does the identity, for one only does policy for the one identity you know assumes.

A

Yeah, definitely I mean we work in the institute of product security. What google and probably 95 of envoy security vulnerability are related to layer, 7, processing, yeah, and I think that's that's one perspective. The other thing, if you think about wasm right, the extension of running wasn't for maybe for one of your purposes, but the fact that you are doing a multi-tenancy envoy wouldn't that impact anyone else on the note 2.

B

Yeah so, for example, if you want to use wasan to implement a custom policy and it's not something that psyllium currently supports, but just for the sake of argument, you probably wouldn't want to do it on a per node basis on l7, right and the reason.

C

B

Because the reason you use that wasn't to begin with is to have flexibility and autonomy to your teams right, so you can have each team kind of have their own rules that are accustomed to their business use case.

B

The problem is that wasm filters are touring complete and they, you know if you have an infinite loop in a wasm filter.

B

Now envoy is stuck right and you don't want a situation where one team has a bug in the code and another team is impacted because you have a single angle in the node that does all the l7 processing.

B

So you want to be very careful when you're doing multi-tenancy in l7. On the note.

C

B

You want to be very careful in what you do there, because it's it's easy to kind of in l7, especially because it's a lot more complex than l4. It's very easy to uh you know do things you don't want to do.

A

Yeah definitely the other thing I was talking to you uh um before the before we chat is so. I think why you covered about the encryption right between the nodes uh in salem's environment right. So essentially you don't get encryption by default, which is what I showed you in my environment and you could potentially enable wire guard insulin to do encryption or you can enable ipsec.

A

But if you do wire guide, for instance, is one of the common way. If you go to the limitations, 7 is actually not supported right. So, if you want encryption with uh wild guide, you can't really use layer. 7 policy, at least based on the latest documentation.

B

Yeah, that's a great point.

A

Yeah, the other thing I would say you know I've also heard people talking about like fibs compliance for wire guide. I think the crypto module used by wild guard there's no plan to be fixed compliant. So if you you are a user faye, I think you probably are. That cares about uh phipps compliance, and you know that's something you know you want to take into consideration as well.

B

Yeah and mtls is used everywhere um right from cloud it's kind of ubiquitous, so it's more compliant and not just with flips, with a bunch of other standards that I'm not even sure if ipsec is fixed compliant. But but don't quote me on that.

A

Yeah, but I think you made an interesting point about mutual tras being everywhere right, so that's important right because you're not just going to have your services running in kubernetes. uh In fact, you may have your services running on vm, or maybe some other mixed hybrid environment right using something that's everywhere. You know it's so just so much easier to enable your communication among your services.

B

Yeah yeah definitely.

A

Okay, uh anything else uh you want to add as fast. How do they compel? I remember you mentioned about eventual consistency before.

B

Yes, so that was related, one of the edge cases where a network cache based identity approach might fail. So imagine you have two nodes, a one then, and they need to maintain a cache that maps network ips to identities right they get that cache by or they update the cache by observing by watching the kubernetes api server and that as soon as you have more than two components- kind of working in the network, you get into eventual consistency, problems right. So it could be that one endpoint will process the data faster than the other.

B

And now you have inconsistent caches and.

C

B

In normal deployments, this is reconcile very fast, but when you build your thread model when you think about this stuff, you kind of have to take these edge cases into account.

A

So this could happen when pods are deployed rapidly and.

B

So if you have pots coming and going in kubernetes pods can reuse ip addresses, so you can have a podine and a new pot being created that gets the ip of the old pod, but has a different identity.

B

Now again, usually this whole thing propagates very fast to the different controllers, but there's no guarantee that it will right and again talking about edge case. You gotta take to account when you build a thread model. This is one of them. It could be that due to the slow propagation of the new pod information to a the serum agent, that it will kind of update the cache at a later point- and now you can have a case of mistaken identity.

B

So it's not something that normally would happen, and I don't think it should prevent you from using uh network policies, uh but it's something you do need take to account and it's something that we definitely encourage in solo is to do defense in depth.

A

Yeah, I'm thinking that too right. So you have uh ci layer, provide you, the layer, 3 and layer 4 policy and then have the mutual gs cryptographic provided by surface match, like istio.

B

Exactly and I think that way, you kind of it's a win-win right, you have, if one of them fails, the other will will catch it.

A

Yeah totally and it's best practice for zero trust network too yeah.

A

All right um is there anything else. You want to add. uh There's been great discussion about how they compel from the angle for interoperability, encryption, mutual tls, security and multi-tenancy and eventual consistency.

B

A

I think we covered a good.

C

B

If happy to answer any questions, if there's any from our audience, but otherwise I don't have more to add here, at least for today. I think it's a lot of material.

A

Yeah, definitely it's a lot of material. I think I may have to re-watch it myself too. So thank you all for joining us. If you have any questions, please send them a line. If not you know we are available on solo slack. You can always hit me or you are, you know, feel free to start a discussion with us.

A

So you are. I want to thank you for so much. You know for coming back to the hood where you started, and you know and also answer a bunch of questions and validate my thoughts around uh psyllium layer, 7 policy with istio's layer, 7 policy.

A

So folks, if you guys find this interesting, please give us a thumbs up and also subscribe to our channel. So you don't miss any of our future uh episodes. So we can learn together on these cloud native technologies related to istio, envoy, psyllium, evpf and graphql, and hope to see you guys next tuesday. Thank you. You are thanks. Everybody for joining bye.