solo.io Hoot Livestream, 9 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Optional Sidecar? A Tour of Cilium Service Mesh

Description

Cilium 1.12 reached GA, which introduced Cilium Service Mesh with optional sidecars. In this livestream, Denis, creator of the envoy UI tool, who has been debugging Envoy and Istio across many releases will share his most recent experience in exploring Cilium Service Mesh with live demos!

Check out this blog: https://www.solo.io/blog/cilium-service-mesh-in-action/ for demo steps!

#istio #envoy #cilium

A

Hi everybody good morning good afternoon, wherever you are welcome to our hood episode, 34, where we are going to discuss optional cyca, a tour of psyllium surface mesh, pseudium 1.12 reached ga. So congratulations to the entire australian team. One of the interesting feature introduced in this release is psyllium service mesh with optional cycas.

A

So in this live stream I am so happy to have denis join me. He is the creator of the envoy ui tool. So if you recall, I think, denis you were back at 24. I think of hood, where you and greg right were giving us a presentation and demo about your envoy, ui, 2 and greg's ingot tool, so you've been debugging envoy and seo across many releases, so he's going to show us a live demo and share his experience of exploring selling service mesh.

A

So denis do you want to quickly introduce yourself for the audience who who don't remember you.

B

So hi everyone, so I am donating. I am based in paris and uh I am the director of field engineering team in emea, at solo, dot, io.

A

Awesome, I am so excited to have you and paris, that's like a dream city for many of us to go to to vacation or even live there, so that's totally cool so before we get to the episode. I just want to briefly share two of the news I heard recently so the first one is the team that started cloud.com cloud stack and rancher. They are back with a new company that produced kubernetes tools, uh so this company called aqua maps so check them out. It's uh it's pretty interesting.

A

um The next news I want to share is about my blog. So for some of you know, I had a blog to compare layer, 7 policy between psyllium and the istio, so that blog unfortunately was taken down from cncf, so we did publish that blog on solo.I o for folks who are looking for their blog, and I also publish a follow on blog just trying to explain which you are uh how network cash-based identity could potentially be mistaken. So there are steps in the blog if you're interested to try them.

A

If you prefer to watch me to do it in my environment check out the recording I have so that's the two news I would like to share with the team with that. I would like to go through a discussion uh with denis service manager of ceiling.

A

So the first thing um I want to ask you dini is that you know, I think you and I both have a very strong service mesh background right. So to me, service mesh provides three key functionalities, connect, secure and observe right. So what other functions? Have you tried uh with celium service mesh.

B

So what I did is like I agree with you like it's. um You know composed of many components right like if you look at cdm, for example, herbal provides observability, and you could put that you know in that bucket right. What I did is really focus on what the cdm documentation is putting in the service mesh category right and.

A

B

And what's new right? Basically, that's also, you know uh what's interesting here right. So if you look at the cdm documentation- and I will show you that uh in a minute um basically, the service mesh is composed of two main features like the ingress and the uh traffic management like l7 traffic management right. So these are the two topics I focused on.

A

Okay, yeah, that makes sense, I'm actually at the docs uh looking at now. So if you want to share, feel free to share your screen.

B

A

Audience easy for our audience to follow.

B

Can you see my screen.

A

B

A

See here basically.

B

If you go into getting started, you know things are in different and you have an observability. You know chapter right that could be potentially part of the mesh right, but what is called service mesh? You incident is really these two components right: the ingress support and the l7 traffic management. So that's that's where I focused my okay experimentation.

A

Yeah makes sense, I guess so that may be one way to interpret. This is maybe the initial thing that's introduced as part of their service mesh function. They focus on the connect. A piece of you know, connect, secure and observe.

B

Yeah, because the observer and secure are are already there right, I would say the observe with uber and the secure with the network policies, not with probably the granularity.

B

You would need in a mesh right, but uh but there are some, you know some components there as well right.

A

Yeah they have uh party encryption if you enable wild guard or ipsec. um Certainly, it's network cache based identity right. So it's not mature tls, like what istio has.

B

Yeah and you have some network policies as well, where you can, you know, define some security rules about which pod can talk to which pod, so you can do basically that with layer 4 based on ebpf or you can do that with 7.

B

In that case, it's uh leveraging envoy behind the scene yeah, but there are, there are some limitations in the l7 in the in the rules. You can create. Let's say.

A

Yep, okay, so let's focus on psyllium service mesh and particularly on the connect feature, since that's uh the documentation, particularly about so I guess I you know, I haven't personally played with it myself, so I want to ask you, you know how's, your experience. Can you share with us your learning journey uh if you're open to share.

B

Yeah sure I mean I I think I will go. You know, step by step so that I think everyone kind of get the same experience I got, but uh without the pain of you know, setting up everything, so that's uh should be a way for people to very quickly.

B

You know understand: what's what you can do with it, so I already created my um gk cluster and you have to use this node tense here when you want to um install serium later and then you have the celium cli. That is quite nice because it detects basically the environment and will automatically you know, deploy serium based on this environment, so here it will detect that this is gke and it needs to do things in a particular way and you can see here.

B

uh What I did is that I I set this value the hand value like ingress controller, enabled right. So that means that now you know when you I use this command, I should be able to use this new english functionality that is described here, basically and and feel free to interrupt me. If you have questions, but the first thing I did when I uh I deployed it right, I was like okay. Let me check what's happening there right.

B

What's um you know what's going on and especially on the serium components right, so I can see here I have my serium. You know, demon set, that's create like a serium pod on each node right yeah. I have signals, and I have the operator component, which is having multiple holes in the life cycle of cdm itself. You know watching some objects in kubernetes and you know doing some maintenance behind the scene and so on right and when I did that I was you know when I ran this command.

B

I was kind of expecting like a new pod like an ingress controller pod here right and facts, not really what's going on here right. So what I can do is that I can just let me just move this.

A

So can I ask a quick question so I know so: I've been still selling myself. uh I've played with layer, 7 policy, so one thing I noticed new is the psyllium node in need. Is that related to the ingress? You are enabled no.

B

No, no, I think it's just related to gke. You have something.

A

Oh okay, so yeah! Okay, that's why thank.

B

You, um and uh so so, if now I go to you know tube system, and I go to this. You know one of the part of the one of the psyllium pod right, the thing that you can notice here that there is this serium android that is started yeah, because I enabled the english controller option.

B

um I don't know if some people have already joined the cdm workshop that we are delivering regularly, but in this workshop we explained that when you, when you create like a cdm network policy at the l7, it automatically start this cdma envoy to enforce it right because everything l7 in serium is enforced on envoy.

B

So here what you see is that as soon as I enable the um ingress controller option, it starts envoy in all these demon set pods right. So all the all the syrian pods have the envoy running from the beginning. It doesn't come yet, but it's there right. Okay,.

A

So I could have two ways to kind of activate this component inside of the ceiling pod. One way is um through a layer, seven policy office land like the lab. You were mentioning. The other way is through that helm uh parameter. You just show to enable the ingress. Is that correct.

B

Yeah, I mean you, don't really want to in to in to enable this stuff manually right yeah, you want to apply networks, policies like at l7 or you want to use ingress, or you know, yeah.

A

B

It does is that it will kind of deploy this component because it needs it right, but uh I I like that because I know that for a lot of people, it's a little bit confusing because envoy is there, but you don't really see it here. You don't really see.

A

B

It's really just like a process that is started inside the container right, which is kind of an anti-pattern to what you looking you find in kubernetes in general, where you try not to have multiple. You know, processes in in the same container right. So that's why I I like that, because it's not natural for user to find where envoy is running right. People are looking for. Oh perhaps there is a pod or like an envoy, pod or or at least a container in the cdm pod. But that's not the case right.

B

It's really like a process here makes sense.

A

Yeah, definitely, if harder for people with me for my istio background, because in istio or what is your ingress gateway has its own part right so yeah, it's just! You can find it basically on the name of the part, without peeking into the processes.

B

Yeah exactly and then the other thing that I was expecting a little bit as well was to find like a cubanity service that represents my ingress rate. um So let's say you deploy like even if it's not involved right. Let's say you deploy like the.

A

B

Controller, for example, right yeah, then what will happen is that you will have a an nginx instance that will run right and you will have a service that will be created and this service will be by default, like a service type load balancer with an external ip to access it right and even before you start to generate an english code. You already have this service created and you can already know how you will be able to access your object later right, but here there is no such service.

B

It's not the only case right, I think gke default controller is also kind of you know. Using the same behavior you can see.

B

I have no service for the gk default ingress right and if I would create an ingress, it would create a service right, um but you will see later that then there are some issues with this um with this approach, but in any case, what I'm going to do right now is just like um create an application uh deploy an application, and after I deploy this application, uh we can create an ingress and we can see what's going on right, but I just share.

B

You know in the same order, while I discovered and what was my you know, um yeah.

A

So you deploy the famous booking application now.

B

Which is you know this application for people who don't know right, which is basically composed of like the product page uh front-end, that calls the reviews back-end and the details back-end and the reviews back-end some of the versions called the ratings back-ends, so you'll see it will be good because I will be able to use it to expose different services and play between the versions like uh you will see a little bit a little bit later.

A

B

This one thing.

A

Here was highlighting, is uh you are deployed without sidecar right? So it's just.

B

The application.

A

B

Exactly like, if you can see it here right, you you have like just one slash. One means like one container.

A

B

um So then that's an ingress right. So an ingress is, like um you know, a standout cuban subject. You can see here. It's you know, part of the kubernetes api.

B

A lot of people have heard about this initiative to have like uh the new gateway api. That is, you know already there, but still you know evolving, and basically the idea of this ingress object is it's very limited about what you can do with it and the gateway api is supposed to or is it is adding some more functionalities that you generally find in a api gateway, but still you see most of the api gateway.

B

Vendors like us, for example, right, have their own api because it's very difficult to find an agreement between all the vendors about how this api should look like, and there are so many different features that you can get in uh in an api gateway that it's difficult to standardize.

B

That right, but at least I would say, the ingress allows you to expose easily your services to outside, and if you want to do things a little bit more complicated, then you have the gateway api, and I know that the cdm community wants to support that gateway api as well in the in the future. Yeah.

A

B

Then, when you want to do even more generally, you you come with your own api so that you can cover like uh more complex uh use cases.

A

B

A

Summarize well so. The ingress api is very good for simple scenario, but users do find out, there's a need for the advanced, uh a more advanced api and that covers different edge cases, which is the um the kubernetes gateway api, which.

B

A

Derived from the is your networking api.

B

Yeah yeah exactly so here you see, I do something very simple. I want to expose my detailed service when a request comes from. You know, start by slash details right, yeah,.

A

B

Now, if I look at my service again, you see that I have like this new uh service that is created for this detailed ingress right and it was the name of my english right, detailed ingress. So it's just like a suffix to this right. Cdm ingress, dash, detail, ingress and uh you can see right it's by default, like a load balancer.

B

um I don't think you can change that in any case, but so you you need, like other prerequisites, to use it. You need to have like a load balancer integration like if you are on premise, you could use something like metal lb. If you are in the cloud you you don't need to really care about it because it will just like deploy your cloud load balancer for you, but this is quite important to understand.

B

What's going on right, so, basically, here I get now an external ip to access this ingress, but this external ip is not just like an ip that is allocated and send the request to your cluster. It's a really uh cloud load, balancer that you have to pay for right. So, every time you create a service type load branch in kubernetes, you end up with a new load. Balancer um I mean like a google code load balancer in google cloud or you you would have like a um you know like an elb.

B

You know in aws and so on right. So that means that here I get this, um this load balancer created and now I can access my ingress through this ip.

B

So if I set this environment variable uh which basically you will see, it's just like uh this ipe all right yeah now I can send a curl and get my details right. So that's fine! Okay,.

A

Yeah, so you can now you, through the external ip you can access to your detail service in the cluster, from your laptop, which is outside of the cluster. So I have a question so when the traffic arrives on this ip external ip, so how does the route happens? That so somehow does it go from this dot 100 to the envoy on that particular node or running in the ceiling pod? And then that always knows to route the traffic, because this goes to slash details and then it knows to drop the traffic to the detail service.

B

So yeah exactly basically the way like this cardboard balancer works is that the you know, service type load balancer is is is, is also a service type node port right. So that means that there is a port. You can see here like this part here, like 3258, that is open on each kubernetes node and the load balancer that is provisioned in the cloud is going to spread the request across those three nodes in my cuban's cluster on this port.

B

So it will come in any of these um nodes and then, when it's arrived there then there is like serium is like using ebpf to direct the traffic to the envoy proxy that is running in the syrian pods, and then this envoy proxy is sending the request to the to the pod at the end right. So the role of ebpf here is only to direct traffic from that comes in this port to the uh android that is running in uh the serial port. Everything else is employed.

A

Okay, got it yeah, so especially this particular service that doesn't really having like a deployment serving like in the traditional kubernetes sense right. Even when you have a service, you have a deployment corresponding to it. This service uh doesn't have any concrete deployments, but somehow, through ebpf, the traffic is redirect to the envoy running in the ceiling. Part.

B

Exactly exactly- and you can see it here right, so there is no nothing targeted by this service directly right. This kind of service.

A

B

uh Then what you can do is that you can also see that there is like a android configuration that is, you know, generated it's what we call a serium envoy, config, it's a new crd and this basically is created automatically from your ingress. So basically the what serium does it takes your ingress yaml and it generates this uh envoy configuration from it right.

B

So if I look at it and if you are a little bit familiar with android, then you'll see that basically here right, I have this cm android config and you define what are the backend services. So you can see here right the details and default and whatever right and then you can. um Basically, um you know you have some default values there and the most importantly here right, you have this route that says like for any domain.

B

uh If it start by slash details, then I want to send it to this cluster here and this cluster is defined just there right. You see. This is a cluster definition in envoy right, and this is the name of it, and you see there is no information about the ip right now right. So, to do that, what happened is that there are a few things that are going on there right.

B

So the first thing that you can take a look at is all this configuration is basically then passed to um is then passed to the uh envoy configuration.

B

So if I look at the normally, if you are familiar with envoy, you have this way to get a config dump by doing a port forward on the port, where the admin interface of android is running yeah. But when I started to look at it right- and I did that for the workshop- you know that we have so. I just use the same way, but when I did the workshop, I struggled a little bit to find a way to get this config dump and by the guy.

B

I found out that it was not listening to any port, but is just like a unique socket. So basically I I I get. You know this configure from the uh unique socket here, and that gives me the full config dump of envoy right now, and I can see you know the same things that's coming from uh this um serium crd right yeah.

B

The other thing that I can do is, as I said, there is no. um So where did I put that? I think I put that in.

B

There is no ip right there, so what happen is that in envoy you have this, you can get some data from the control plane using xds, and one of this, the data you can get is the endpoint information coming from eds right, which is.

A

B

Components- and you can, I found a way with a little bit of weird snippets, to get this data sent by you know the serium eds to envoy.

B

um You know using jr picker, and what I can see here is that, because I created this ingress, it created this android config and what it did as well is that it's basically um provides through this eds. It provides the information about okay, this is the cluster I want to use, and these are like the you know, end points right. So, for example, here this is the ip of the pod right. So I have only one for the details component, so this is the ip of the board.

B

So all these things together are making this ingress functionality working.

A

Did you write that to the eds tool, do you pull out the eds computer? I mean.

B

It's just like a little bit of weird stupid.

A

B

You know I need to you know I I've, I kind of you know found you know what was this uh grpc endpoint you know, and then you know, because I know how that works right. This endpoint, you know I was able to you, know, get the nod information of my envoy through the config dump and then injecting that error. So it's a little bit weird. I don't want to go too much in the details there, because.

A

B

It's not easy to understand if you people are not like expert on on android.

A

Yeah makes sense.

B

um But let me do another thing right now. Let's say I want to explore the second service right like the review service, then I create another ingredient object and I think it's quite important because that's I like what I was explaining before, which is that here, every time like someone would use a would create an ingress object that would basically generate the creation of another balancer in the cloud right.

B

So, if you have like, I don't know, 10 different ingress objects, then you will have 10 load balancer, which will be obviously at at the end, will start to be expensive because you pay for each these load balancers, but also it can become a little bit trickier about how to share domain names right, because if you, if you don't want to use ips right, you want to use like a domain name.

B

So you would have to configure your dns in one way or another to send the traffic to one of this load balancer and not all of them right so right. You know you would have to find a way to split it right and uh yeah. That can be. That can be tricky. That's what I said like normally.

B

What the english controller do is that they create a default service when you deploy them and you have a unique load, balancer unique ip, then you can, you know automate, you know the configuration of your dns or you just do it yourself, but then all the requests for your domain names. It goes through this um ip and it's shared by everyone right.

A

Yeah, that's what a glue is your ingress gateway does right. So there is one mapping between that particular ip to a particular host name, and then the traffic can just go through that particular host name on maybe different paths yeah.

A

So I find that it's interesting. It seems like it. Each service has its own dedicated ip, but it doesn't have the dedicated envoy so because in istio we recommend user to have dedicated ingress gateway. If you really need to be like your application is very critical, you don't want to share your ingress gateway with any other application, but in here, even though you have your dedicated ip, you still share the envoy with other people.

B

Yeah exactly so, that's an interesting thing right. We can talk about that later, but it's basically means that you can generate conflicts. um The chances to generate conflict by creating ingress object is probably lower than the chances to do that in uh in writing. The serium and config directly like we will do a little bit later, but still it can be.

B

It can be a little bit uh complicated right and the only way you can share a service here would be that you create only one large. You know ingress object with everything in it. The basically.

A

Is really difficult? Oh yeah yeah makes sense.

B

So that's the ingress, um but let's jump into the traffic management, because it's just an ingress controller right as we discussed before right. There are a lot of things that, in the documentation you could put in the service mesh that are in other places.

B

You could also argue that these ingress things could be outside of the service mesh, because it's an english controller right. It's it's not like. You know purely service mesh right. So you know it's just a question of uh the way you you interpret things um so the second thing we want to do is- uh and let me just like close my whatsapp.

B

uh So the only thing, the other thing we want to do is like- and this for me is really more like what I call service meshed right, which is uh having this layer, seven traffic management rate, and here uh what I want to do is like, when a service send a request to the reviews service right, you remember here. We said that there are v1, v2 and v3.

B

Then um I want to define to this to send 90 to v1 and 10 to v2 right this kind of a typical use case. You have in service mesh, you you want to go, for. You have a current v1 in pollution.

B

You want to, you know, have some requests now going to v2, but only like a small percentage of it just checking. If everything is going well and then you can start to send more traffic right, you could even have like uh you know, use you know some nice projects like flagger to even automate you know, and if the latency is good and there are rate is low, then I want to automatically increase the number of requests going to v2 and and so on, right yeah. So that's kind of a difficulty.

B

One thing: perhaps some people could say hey why you put details here on that product page right, because this is normally the product page. uh The only reason why I did that that way is because details is the the container, the image that has the curl installed, which is not the case of the product page and these services. You cannot add new packages on it and I didn't want to rebuild the images just to have curl. So I said: okay, let's do it in a in a simpler way.

B

So it's just details are used, but it could be whatever right. Yeah.

A

So it's just a client exactly.

B

A client trying to send a request to reviews right yeah, so I looked at the example that is in the documentation which basically deploys two-service echo service one and echo service two and does a 50-50 on this two one right. So in my case, I just want like reviews here right and reviews v1 and reviews v2 right. So what I did uh at the first attempt right because you'll see I will do different items so that you can see.

B

You know how I I got it working at the end is that I you know created this uh cdm uh envoy config here, where I just say I want to you know when a request goes to reviews right, then I want to send to default previous v1 90 percent and default usb to 10 percent, and I you know I define these services there right.

A

Wow, so this is, I mean this is, I guess, people coming from istio background. This is the equivalent of writing. It's your envoy filters right.

B

A

Know exactly what you're doing.

B

Yeah in istio you have this abstraction. That is a virtual service right and then it's yes to android configuration here. It's really raw and void configuration, so obviously android is not for human right is for like being programmed by a machine right, and uh basically that means that um yeah. Definitely it's you know you. You will create conflicts very quickly, right like, for example, even this root name here you see, the lb roots has to be unique. This listener as well right.

B

You know this root name, you know all those things like. If people use the same things, then everything will be broken because someone will configure. Let's say I have like my route right and the other guy called high roads right.

B

Then there will be a conflict, because there will be my road created twice right and then you will start to inject the traffic in the wrong place, right and and even like, when you have an abstraction layer on top of it like istio, is you know, having the virtual service and then is translated by istio control plane into anybody configuration the most difficult part is not to translate. The most difficult part is to enter.

B

You know all these name spacing and all this conflict resolution right and it took like years for seo to handle all of the possible cases that you can get to break it right. So definitely even with the control plane is difficult. So writing the raw configuration uh could be.

A

Yeah, basically, a user is trying to do. What is your control plane? That's for them right, because they need to come up with the rule, configuration and then deploy it to their kubernetes cluster. That runs salem service mesh.

B

Exactly so, if I try to curl now from ratings to to reviews- and in fact I think I put details here- I need to update it. I don't know why I said details, it's writing. uh That has the curl command and uh if I do that, I get this no lc upstream right. So I could uh obviously you know before I've shown you like the config dump right. um So I could I could write this config dump in a file and you you mentioned you know the android ui that you can yeah config dump.

B

It doesn't work with this config them because it's not in the format. I exactly expect. So I have.

A

B

Version on my laptop and in this patch version you can see here right a little bit. What's going on now in my invoice configuration I have these details for the ingress details right. I have the reviews for the ingress reviews and I have these lb roots and, as I said, you know you it has to be unique right and for any request here. Then I will send the request. You know 90 to v1 and 10 to v2, but then what we don't in show here is the endpoint information right.

B

So if I look at the eds right like I did before.

A

B

What we can see, we can have a look at what information is provided by serium to envoy through eds, because it tells us that we have no healthy upstream right. So here I can see that I have the details from my ingress. I have the reviews.

B

Cluster name review star right, but I have nothing for reviews, v1 right and that's the problem. I have right now and you can see that as well. If I would just go to uh instead of like looking at the config dump, you have something called clusters in.

B

Envoy and if I would do a graph reviews, I can see that the the default reviews here as like um end points because you see I have like one ip of the v1 or v2 v3. I don't know right uh one ear and one ear right, but the reviews v1, I don't have any end points and delivers v2. I don't have any endpoint either right and.

A

When you run the cur command, were you calling reviews or review version, one of version two.

B

I was just putting reviews because my service in kubernetes, I need to use a service name right, so I usually because that's my review service name, but normally the way it works like if you look in istio, for example, right when you say I want to use v1 or v2 or v3, I want to do like you know some.

B

I want to do the same thing basically right with this here yeah and I will create what we call a destination rule to define what is v1, what is v2, what is v3 and it will create a subset which is normally coming in between these two columns here. So.

A

B

Would have default, slash reviews, column, 9080, column, the subset name, let's call it v1, right and and so on right, but here there is no way I can configure the subset here.

B

So what I found out in fact, is that there is like uh in the documentation um it says uh somewhere when I was looking in uh now it was in the troubleshooting guides. So where do I have it here? I have it somewhere. I think.

B

Where was this troubleshooting guy.

A

I think I put a link here, in fact,.

B

So if I go down yeah it's here, so if you go to the troubleshooting guide, then you find somewhere that there is here. You see the android discovery service. So yes use this convention rate, which is name service, name and port right.

A

B

That to be able to make my scenario working, I need to create some extra kubernetes services right so now is just like creating this reviews: v1 service and reviews, v2 service, right and now, if I do this that way, I can, I won't be able to see more here right. I still have no endpoint configure, but what I found out is that you need to basically add these services there before they can. You know be reached with this um endpoint uh discovery service right so before the endpoint discovery service can configure them right.

B

But um while we were discussing about the ingress before, I think I perhaps have an idea about another way to do it. So let's try. It live right, I'm curious about it, so I want to check with with you guys right now all right. So you remember here.

B

This is the serium and void config that has been created uh by the ingress right, and there is this services here right.

A

B

So that's interesting because I just realized when I was doing it before that this is different from the services here right. So let's try something.

B

So let's try to take this envoy configuration here, because that could be another interesting option and instead of setting the services, you know v1 v2, that I created right. I want to try with these back-end services, okay, okay,.

A

Is this new you're trying.

B

Yeah, I'd like to try stuff live just to see.

A

So hoping that would save you from listing version, one and version two yeah.

B

Let's see, let's see, perhaps it will pick one of the problem I was still having. So let me so it's like namespace and then there is this number so.

A

Yeah port number.

B

Let's put this part number as well just.

B

Okay, so now I can go here and see.

A

So you're hoping the eds information would show up yeah well on version 2.

B

And basically, you can also so the fun is that, because I put this in by configuration, you can have some wrong syntax or whatever right. So the only way you would be able to see it. Let me check again here it looks really bad, but the only the only way you can see it is that you have to basically go into the uh um the the the logs of the serium operator.

A

B

And when something it doesn't like, it will tell you yeah right, it will tell you, like uh you know, rejected by envoy or whatever, like you have like a little bit of details there right, but in any case, unfortunately, I think it didn't fix my problem right. No, if I return again.

A

Oh it does do you want to go back check the end point. Maybe it's yeah.

B

I would check again.

A

That's pretty nice.

B

You see it still send me to red, so it didn't.

A

Oh, it didn't do the traffic shifting yeah.

B

There's another side effect that I was not expecting so so that's not what I want so basically yeah. So that's that's why, when I do here, I don't see vip, because what it's configured now is just like going through that right. We could probably even find it in the config dump.

B

Try to open this one. Perhaps we will have a better idea, but how it has configured this thing. So here it's just yeah. It's still sent to this default reviews, v1 and v2, but these clusters have no endpoint. So basically, I think it just bypasses, probably just like android that refuses it, but it doesn't tell me anything about it. So, basically it just doesn't go through and void right. It's just the bypassing.

A

Yeah round robin so basically the fact that it went to the red review, which is version three. That means our traffic uh rules are not in effect right exactly.

B

So the way I was walk around it last time, so now, let's go back to sorry. Let's go back to my initial way was like to list those three services directly there right yeah and when I was doing that, then I was able to get the end point correctly set for v1 and v2, um but uh and then and then I'm able to make it work correctly here right. So if I go there now I have the either okay, no.

A

You still have red.

B

Yeah, I think it is broken. Basically it probably broke the android configuration because last time I tried this one was okay right, so it was basically able to so probably yeah. Let me try to, I think we broke it.

B

This is not going to serious or also envoy. Now, let's try to apply.

A

B

And this is this one right just check again.

B

Now, if I try oh.

A

B

I was not on the okay, I I know why it was working with black, because I was not calling the right thing. I was calling the external one. No, oh,.

A

Yeah you're going through the ingress. That's why.

B

A

Should have catched that that's.

B

Why I was surprised that I was getting black and red before right, but now.

A

B

Now let me try the right one, which is.

B

B

So now we can see that most of the time we not we get to v1. So there is no stars at all.

A

B

90 of the time we go to it and 10 percent of the time we go through the black one right, so we should go to black soon at very low number. It's not like exactly mathematically.

A

B

Things right, yeah, you see, I get the black right.

A

So that sounds good.

B

But there is a side effect of this, which is that if now I want to call explicitly the version v2 right I want to. Let's say I have this preview, that is for production, and I want to shift my traffic right, but before I do that, probably I want to send a request to v2 directly or first I want to do some other testing.

A

Yeah test exclusively on my version too, I assume.

B

You cannot anymore right because.

A

Oh, it goes to version one now.

B

Because it's part of these services, so that means that it will be, you know, also redirecting the traffic the same way for all these services right.

A

Oh okay, that's.

B

Why you were trying.

A

Back-End services, hoping it would not change.

B

The review version one.

A

B

With the back-end services right yeah, perhaps there is a way right, but it's just like even me, who knows really wealthy and voice stuff. I I didn't find right and uh the last one I want to try is just like. Okay, I can keep the sim three services here.

B

I can keep that, but here I just put the domain here. We've used 98 here right and if I do that, then you see, I would basically get like a four or four.

B

When you know I go to reviews, I was. I was hoping that if I do that, then it will not impact the other ones, but the thing is that it's still attached my route to the it's still redirect traffic for these guys as well, and then there is no listener or there is no matcher right for this domain and then you know you just reject it.

B

Basically, so it's not as well the right way, and I also wanted to point out that if you want to do it properly, you would also do like reviews, then reviews dot default.

B

Then reviews.default.svc.cluster.com, you know all the possible options like again like istio is doing that on the control plane and generate automatically all the possible names and so on. Right.

A

Oh, so you need to add these to all the cluster configuration.

B

No, I mean like it's on the matcher right here, like.

A

On the matcher, okay on the domain, yeah.

B

A

That makes sense.

B

Not the match of the domain, so yeah I mean like that's, that's pretty it right.

B

I mean the obviously you can make it work uh for some cases, um but even if it works right, we discussed about the the main issue right, which is like users, cannot really write this code right, and I think you know I I watched like uh um a webinar from either valentine recently and, and they were saying the same that basically, this is not like meant for users right, it's more mean from being the foundation to be used by an iron abstraction layer like it's the case for the ingress right now,.

A

B

So I think the idea is to have in the future like uh either using the gateway api or whatever using you know, having like a layer that allows the user to express what they want and having that you know higher level abstraction translated into the cd or main voice configuration.

B

The challenge is that on on the ingress side, right on the north south, it it took years for api gateway communities or vendors. You know to to expose all these features and being able to generate the right configuration and to manage. You know the all the potential issues and conflicts- and you know all this stuff right. So that's a lot of work right to just even ourselves and then on the east west. It's another set of challenges right.

B

It took again years, for you know, istio or other communities, to build this abstraction right in the scalable way being able to manage. You know all the multi-tenancy and conflicts and manage you know telling you which envoy is in sync and not and plus the fact that there is only one invoice right. That means that the consequence of a bad configuration will impact bad configuration will impact everyone basically right.

A

Yeah yeah yeah.

B

A

So that's a recap: what we just learned! So essentially you did a demo of using kubernetes ingress resource to expose two of your services out to um outside of the cluster, and you were able to gather work, but each were assigned their own load balancer ip, challenging with host name resolution to these eyepiece and they both were sharing the same uh envoy from the syrian pods.

A

You also showed a demo of trying to get a real basic scenario. I guess that I would say in the industrial world right to do 90 on version 1 and 10 on version 2., that's a very common scenario and to the end, I think you try different things and you get it partially working, but it's not fully working because you can't fully test in review version two now, uh when you go there, one hundred percent, if it gets to ten percent. So it's it's.

A

You know, I guess, uh and you with a lot of your expertise with amboy. uh You know you you still haven't been able to get it fully working. I guess.

B

Yeah- and I mean it's- I would say it's not even the main point, because you know I've just found this back end service perhaps could help and I'm pretty sure that there is a way to find a way to make it work right. If I would look at the code of the way you know eds works in psyllium, probably I will find a way you know a walk around. I would make it work right, but, but even if I could right the it would be really challenging anyway right so yeah.

A

The complexity of the configuration for something- I guess it still- it would be like five out of ten lines yeah. This is um very, very hard to digest and create this configuration. Unless you really know our boy and been working with angry, probably for over a year or so.

B

Yeah yeah no, definitely and.

A

B

Like we've seen, also people who started with building their own mesh before histo was really mature right and invested a lot of time and energy to learn enjoy and to build their own control plane on top of envoy and investing a lot of money in that right and at the end, these guys most of them decided to go to istio right. Even if you're.

A

Watching this yeah yeah, because.

B

Of you know the complexity of maintaining all these different configurations for because we speak about like just basic routing, but then you have like to implement all the retries and time out and circuit breaking and- and you know like a lot of other features and and the problem is always the same is like. If you will have only one team, doing everything you could still make it work right, but the problem that you want to have different users um having their configuration.

B

You know that provide their configuration and then managing conflicts is is very complicated. You you really need a lot of experience to to make it properly having a right convention for all the roots and all the listener and all those things- and you know keeping the the previous configuration that worked in case. A new configuration comes and is not good so that at least the all configuration is still enforced and not break broke by the new configuration.

B

Then you need to report why the new computer is not working and then the user needs to figure out. You know. So it's like it's it's it's it's a full project on its own right and but but yeah I mean like I started, you know I was playing with the cdm service mesh beta in december, so I was really curious about it. When I I heard about this project and the reason why I was really interested by it, because I thought okay, let's see, how is it different right like?

B

uh Can it be like ebpf doing all these things right, but then, as soon as I discovered that 99 of it is like envoy, right and ebpf is just used to redirect the traffic to envoy, then I felt like that.

B

The value is like a lot lower, because there is already there are already a lot of invest controllers that are really good. There are already a lot of service mesh options based on android that are mature right. So yeah I mean it's. I think it will be a very, very long journey if you you want to reinvent or recreate all these things in serium itself,.

A

Yeah that's uh wrap up. We are almost out of the time, uh so I want to take a minute to thank you, denis for sharing your journey and put out a interesting demo and show your struggle to get things working. So I'm sure our audience would really appreciate your experience with that. So thank you.

A

So much and folks, if you guys find this episode interesting, please give us a thumbs up and also subscribe to our channel, so you don't miss any of our future educations uh uh psyllium on ebpf on istio envoy and all the other clown native technologies. So you can be educated at your job. uh Thank you so much everyone for joining us, happy learning and see you at our next episode. Thank you. So much deeney bye.

B