VMware TGI Kubernetes, 27 Aug 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: TGI Kubernetes Episode 164: Deep Dive into L7 tech for Kubernetes: Contour, Nginx, HAProxy

Description

Join Ricardo Katz, Steve Sloka, and Jay Vyas as we dig into the internals of how Layer 7 is implemented in Kubernetes. Specifically, how Contour, Nginx, and HAProxy route external and internal HTTP/S/TCP/UDP traffic into pods. Also, we'll look at the emerging Gateway APIs.

0:00 - Introduction
3:00 - Week in Review
8:25 - SIG Docs is looking for help!
11:10 - The high level architecture of Ingress
15:40 - Getting Started with Contour
24:40 - Shoutout to Curiefense WAF
28:10 - Coming back to Contour and understanding its architecture
34:00 - Going deeper with complex Ingress Objects
38:30 - Watching what happens in Envoy
57:00 - Trying Gateway API
1:19:30 - About SCTP (going back to L4)

A

A

All right, so this is ricardo's first tgik. Thank you. Ricardo.

B

A

You got everything set up. Did you turn your screens? Can you get your screen going.

B

Yeah, let me share my screen here and we can go to the notes.

B

B

Yeah, so now we are live now. uh Thank you. Thank you. Jay, for the invite uh we have steve here with us as well. Steve is one of the maintainers of contour, say hi, steve, hi, steve hi,.

C

Thank you. This is thank you dgik as well, so as he says, uh yeah come on as a guest.

B

Cool yeah, so so for today uh it's uh it's gonna. I hope it's gonna be a fun episode. It's my my first tjk as as host as well. I've been in past here with jay.

B

uh We are gonna, make some deep dive into ingress layer, seven stuff, and I think this is going to be fun actually, because I am one of the ingredients. Maintainers steve is one of the contour maintainers. I have never messed up with uh with contour, so it's going to be fun because I'm gonna, I really wanna try to deploy contour before trying to do things that I actually know right and hope that if we have time hey jay how.

A

Are you doing? I just wanted to see whether you were able to get there. Okay, cool yeah.

B

Hey hi, steve yeah, cool hello, everyone uh so hey good evening good evening, philip good evening, k, k, r k, r f.

A

How do you say hugo my huge meyer.

B

Hugo meyer hugo.

D

B

I don't know I don't know from germany, yeah cool, that's that's late in germany! So so thank you for being with us today and then I I hope that we we've got some some some time and some to to take a look into gateway api as well, and another layer, 7 stuff right so uh before uh starting. Let me just make things bigger um if someone thinks that we need some, some more, some more phones, some increasing the size. Just just let me know being ping j on youtube and he can yell at me.

B

Okay, cool! So, first of all the weekend review we have seek windows. Devtools now has working cni providers so jay. Do you want to talk a bit about that.

A

um Yeah we were we getting things networking fully functional in our, so we have the sig windows dev tools repository. I think I've announced it before in tgik and it allows you to spin up a kubernetes development environment from source, including the windows, kubler and windows coupe proxy and we've got now.

A

We had andrea support, but there was a minor bug which was fixed by the entry of folks and it's going to be in an upcoming release related to double mix, because virtualbox makes multiple knicks. But then we also had a similar issue with calico and my good friend ameem nabin, uh just uh fixed that up and actually has an upstream pr into calico so that you can run windows. uh So you can run calico's agent um on an arbitrary nic which is kind of cool um from an installation perspective.

A

So we've got that so if anybody wants to try windows out from source um with either calico or andrea, uh now is really a good time to grab that repo and start banging away at it and testing. It's a really good way to learn about windows on kubernetes.

B

A

B

Jay, hey good evening, rykel, uh the next one uh I can. I can tell a little bit so uh increasing ginx richie has reached version, one zero, zero, uh ga and we are dropping right now. The ingress api version with better ones. So we are gonna, keep support for the next six months of the the legacy ingress. If some bug or something like that happen, but then we are planning to drop support for legacy, ingress and gen x and just keep uh version one zero zero.

B

So uh please make sure that you, you all update your your kubernetes cluster, to a version that supports ingredients in genetics, version one and and also, uh and also uh to update uh your ingredients as well, and the next one is about capping.

B

That now has dual stack implementation for iptables, so we've made we've made an episode on best about kaping. If you want to take a look, I guess was the 160 1 59, I guess, but capping is an effort that sig network, which we are part uh actually uh started with our friend mikhail and that's an effort that that we we we are getting to to uh improve or evolve the q proxy implementation.

B

In kubernetes, so instead of relying in in the implementation from kubernetes you can, you can rely on on just a control plane and develop your own data. Plane, right and and now kaping got got support for dual stack in ip tables back end. So do you wanna? Do you wanna talk a bit about that as well jay or I just.

A

That's it that's all there is. It works yeah.

B

All right cool, say, hello, everyone, so uh so jay we've got a question for you from robert. How is windows kubernetes support stating in the community right now? Is it very behind.

A

No, it's it's working! It's like you know. um You know you you. You can pick from multiple different cni providers. We have csi support through csi proxy. um It supports container d. Now network policies work. um You know we, I layer, seven stuff. You know uh you know I have never tested on it, but I I know other people have have used various layer, seven technology, obviously with it. So like you know, those are usually the things that you would you'd.

A

Think of as advanced support right and all that stuff works, so so yeah it works. Definitely try it out. That's why we made the dev tools so that people could really jump in there and just run vagrant up and have a running windows kubernetes cluster on their local machine, so so yeah.

A

If, if you, if you get some time- and you want to play with it- that that's a really easy way to just get involved and start learning about how it all works. Thanks for thanks for asking, though,.

B

Cool: hey hello, alex from from smoking, not not northern california, so yeah and yeah. So next one is uh a pull request that that got into kubernetes website which improved the the kubernetes dock site generation generation time in more than 80 percent. So it's a cool pr for folks that are not actually related to goal coding, but with using hugo and static site sites generation, and it's it's actually cool to to try to understand that just one line uh improves so so much the the the website generation.

B

So uh if you want to take a look, just just take a look and also seekdocs is looking for people to help.

B

So if you are interested in documentation, if you are interested in tech writing, if you are interested in in in helping users and and thinking actually that documentation is only the one thing that is allowed to to be used in kubernetes certification, uh please jump in sick dogs and kubernetes slack because they are looking for for some help. All right, hey, hello, hello, rajas is here jay all right.

B

So then the last one uh for the weekend review is uh a medium that I've put here. uh If someone want to take a look, so, let's flashy from from microsoft, uh he wrote a bit about the new gatekeeper for those that don't know what what gate gatekeeper is. So gatekeeper is a a validation web hook that you can write rules in open policy agent for uh for your kubernetes cluster and the gatekeeper now supports mutating as well. So uh this is.

B

This is a a pretty common scenario when you have, for example, uh an image and you wanna you wanna change that image for is, instead of always using latest using latest blah right or you wanna, take any any of the you wanna take any an image which have uh like run as user root and instead of just block you wanna mutate to run as you run as user a thousand and one. So this is what mutation web hooks.

B

They do and that's a a really interesting uh article about how how to use gatekeeper and to do that and a a curious thing about that is that uh gatekeeper actually choose not to use open policy agent for the mutations. So uh the gatekeeper started just to be like an easier and to make it easier to to use open policy agent in kubernetes, but they are not using uh open policy agent for the mutation part.

B

So it's it's pretty cool and I I highly recommend anyone that want to take a look and and understand how mutation and how validation works. It's a gatekeepers may makes stuff much more easier right and I guess that that's it for the week in review for my side. uh So let's start breaking things.

B

Are you all up to break things.

A

Yeah, let's show them something.

B

Yeah so jay do you want to draw something here yeah? I.

A

Could we started this because ricardo doesn't have an ipad, so all right, we started doing a um sort of like a little high level thingy. So all right, where does in here, I'm a user and I'm accessing um a pod and my pods downstream of here over here? So what's the? What happens? How do I get through contour um to here like normally right?

A

If I didn't have an ingress controller, I would use something like a cloud to provision um to provision an external load, balancer right and then I'd get an ip address through my cloud and then that would get routed through my cloud to a node port and that you know normally that would go through a node port here, I'll draw one: here's a node port right and then that would go through my node port and then that goes to my container port right.

A

That's the nor the sort of primitive way of thinking about how to direct traffic from an ends from an end user to a to a pod. So what happens when I start using contour? How does that change.

C

Yes, that's going to be your service type load balancer! So that's like the classic way of doing this um ingress controllers. Let you do this at layer, seven right so, instead of having um so what you described is you're going to have that over and over for every application in your cluster right.

B

C

If you had, you know, 100 applications you're going to put 100 different load, balancers 100 different node ports. You know a lot of holes in your cluster, so an ingress controller lets you- and this is generic. This isn't contour specific, but it runs at the layer, seven of the stack right. So you can route application traffic right. So now you can inspect the the host name and the path and then take action on that.

C

So now you can have one load, balancer route traffic to that thing and that thing will do your routing and decide where should this go within your cluster? So what.

A

I'm going to do so, I'm running contour. Am I running it on the host network, or am I running it in the pod network? I'm running on the host network right.

C

uh It depends it depends on what you need to do, how.

A

Would you do it if you were me right now.

C

Yeah, so our examples out of the gate show you using them yeah with with a host port, so I'll, be buying 80 and 443 to the host and that sometimes helps you get rid of different hops through the network stack of kubernetes.

B

A

But you can, you can.

B

You can still use like noteboard or for that or or even right, yeah.

C

Yeah a couple of things yeah when you want to get traffic to you know to your your proxy. In our case it's envoy for contour, um but there's you know, there's ways you can route traffic to that local instance, and that's that external traffic policy and there's ways you can use the the the cluster to route across horizontally across different nodes um and again depends on where you have your your um proxies running and how often you have this those configured. That's why we call them examples in contour, because there's so many ways you could.

C

A

This you know the canonical way is host host network right. That's like the usual first hello world for contour, right, yeah,.

C

That's the game kind of thing.

A

And this is a damon set right. Okay, so for those that don't know, that's how you normally do. This you've got a daemon set and it's running on all your nodes, all right, so all right cool. um Can I just so real quick. So it's it's sitting there. So now what I'm an end user over here and what do I do do I do I do I make an external load balancer and have that load balance everything to contour or does some do? I do something else.

C

So so contour, specifically we use envoy as our data path component. So all traffic is going to route through envoy, so contour itself is essentially just a configuration server and its job is to send configuration to envoy so you're gonna run. All traffic through envoy on contour itself does not route any traffic or do any kind of data.

A

Okay, so where, if I'm trying to get this running- and I don't know anything about any of this- what do I do? I don't do I download envoy and install it, or what do I do.

C

So our examples will will help set all this up for you.

A

B

A

Let's try it yeah. Why don't we try it? Okay, we.

B

Should try? Okay, cool okay, so just to be clear, uh I I I I never use it on volume contour as well, so I will probably make something wrong but yeah. So I am here on the side and I want to go to the get started so why contour quick start yeah? So quick start, I like quick start so, but I I I don't have kind installed. I have this. Where is my cluster here? I can use it here. Let me increase my screen.

A

Oh para is here: what's that pear error, it's my friend he's like this. What do you pair the pairs from kaloon?

A

It's like the cto of ibrahim.

B

So I'm gonna skip everything here, I'm just gonna deploy.

B

Should I do that.

C

Yeah, that's a good starting place. So so we give you a couple ways: you can spin up your cluster if you need to um they're, not by many, you know by any means, you're not required to do it. Those way, they're, just you, know easy ways to get started. If you don't have anything but uh ricardo, you already have what you said. Yours is a cube admin cluster.

B

I I have created, you mean I have created with cube idm yeah yeah.

C

Okay, yeah, so you can start from here, so you can just run that quick start so um in the contour repo there's a bunch of examples that we have in terms of our back permissions and service accounts and crds and services, and all that stuff. What you just saw all got blasted through into your cluster.

B

Let's, let's see if it's creating, oh, I have a crash loop back here. Look at me, jay.

A

I wanna it's, I'm actually trying to get this running too, so I would use the can. I just use the vanilla, contour examples. Also I'm trying to get this running on a windows cluster. While you do that, and I wouldn't do anything different, would I what was the yaml you? You used ricardo.

B

uh Quick start, this was quick start.

A

B

Steve told that that we have some other manifest. So let me take a look.

C

B

Project contour.

C

Is the is the org.

B

A

I'm gonna see if I can get it working.

C

And then contour is a repo yeah.

B

So if I come to docs or examples good.

C

Examples examples just has the ammo and go to contour and it's not straightforward. The first, the first directory there yeah so.

B

C

Essentially, is the quick start, so the quick start is just we smash all together into one yaml file to make it easy but jay. This is what you can use if you're looking to to have each piece and part uh customer.

A

Are you on the vmware internal network, ricardo?

A

B

Was going to say because.

A

You know docker hub. We can't pull from docker hub because of the rate limiting if we're on the internal. No.

B

No, I am not I'm not.

A

B

So here, okay, so we have the deployment here jd. If you want to try to to.

A

Download yeah yeah, okay.

A

Project contour dot, io, slash quick start. So what's your crash.

B

So why are you doing that in windows? I'm gonna, I'm gonna break some stuff here. Okay, so since we.

C

B

All right, so, where is contour running so I have this container creating here, I'm gonna wait a bit more. My internet is pretty slow.

A

Mine's I'm running mine too. At the same time,.

C

Yeah, so what you got with that quick start was envoy runs as a daemon set, so every node in your cluster is going to run envoy envoy scales, typically very well with cpu threads. So if you ran like 100 envoys in one node you're not going to get much better performance versus running just one, so that's why you get one per node and then, if you're running, host ports like we are in the example, um this ensures that only one pod is going to have. uh You know those ports.

A

Allocated so I'll uh fret per is happy because it's friday allah says um so. Where is where would a waff sit in this system?.

C

So we don't have, I think, there's an open issue in contour to support laugh. uh You need, like a different envoy, build to get that working.

A

I don't even know what it is web application firewall.

C

B

Yeah yeah, oh my my my cluster is broken, so I'm gonna just uh create.

A

A kind cluster here, let me see if mine's, maybe mine, will come up hold on oh yeah, no mine's, not up yet either coupe ctl describe.

B

Pod kind create cluster. This is going to happen. Come on yeah.

A

Set up, I I my cluster's, broken too ricardo, I'm getting a chone error and I've never seen that before, like what's going on today,.

B

It's friday, this could be.

A

Is coob dash, api-access, qk glw? Is that some kind of contour thing?

A

Why would this my kublet is not able to start this contour pod and, and it is running other pods.

C

A

Running on windows, it's running on linux.

C

Oh okay, that's exactly.

A

This is going to be an interesting day for.

B

Us ricardo, yeah.

A

Okay, you want to see. Can I share my screen while you look at your while you try to work yourself.

B

Yeah you're happy. Let me add you: why not.

A

Poor steve this is like okay. Here we go coupe ctl. Let me see see, look at this. I've never seen this error before I've. Never seen this. It's like cube api access. It's like a something is happening when it's coming up and it's trying to the kubelet. Oh wait. No, this might be a windows specific thing.

C

Contour cert yeah, so we so the other thing is we. We auto-generate self-signed certs, um because contour and envoy don't run in the same pod. They run separate so to secure that traffic between them. We create our own self-signed certs folks can override those if they want, but it looks like that error is that contours. Third error is what you're getting.

C

A

Hope it's not when it could be windows related, but it shouldn't be secret. Oh I see so so the way so that the problem is that the cert isn't coming. So my problem was windows specific because so because my contour search gen tried to come up on a windows, node right and so coupe ctl delete, because I don't have my windows nodes painted. So I'm going to delete my contour.

C

The other thing that might be it might be. We recently just did a patch to fix some security things in envoy, and I wonder if that might cause an issue as well. So we run, we run the uh the envoy admin page as a unix domain socket because there is an exploit with external name services, so folks use external name services and an ingress controller. It allows you to you, know reference any dns, name and contour supported that, and someone actually found an issue where, if you made your external name localhost, you could.

A

C

The admin web page running on localhost in the pod so then.

A

You could do.

C

You could do nasty things so envoy. Has this admin webpage built into it, which lets you like see its configuration like clusters and things it also lets you like you, know, quit envoy and shut it down through that web page, so it could be so we just we just actually re-engineered that so that you can't get to those now. So you can't talk over the socket now so.

B

We've got: we've got a vulnerability on on ingress in china in best where we we, we program, uh the lua back ends with uh with a socket as well right, so we have uh to to to not rely on on dynamic reloads uh in genex saying actually the go program sends sends the end point to a lua, socket saying hey. When someone tries to reach you it's to be here and some folks, they they, they could use external external names from services to reach there right.

B

So if we we've got to disable that but yeah cool that there is a cool hold on jay, why? Why do you do that? That there is a cool discussion being happening in in in the chat about web application firewall uh and that I I won't? I won't go into the discussion of why or like how it it helps with pci or something like that, because I don't know- I even don't know about that. But uh there is a cool project of web application file, a new one which is called curia fence.

B

I don't know: have you seen that steve curie offense.

C

B

The link to it it's uh it's incubating in cmcf. Let me let me open here and I'm gonna put in in in this in the notes as well. So uh this is. This is a a a a sandbox project which is actually based in in envoy as well, but that's uh that's our application firewall. So, oh, that's! uh Sorry! That's j! I'm gonna remove j screen. You.

A

Can take it away, I can't figure out why this cert gen refuses to not run. It refuses to not run, except on the windows note on my cluster, and I have no idea why.

B

I I'm not gonna make any jokes about windows with you, because I know that you get mad at me. Yeah yeah, cool, just.

A

B

So so so uh so uh we we we get. We we have this uh offense, which is uh uh uh a cncf sandbox project uh for uh web application file for distributed web application file, and it's based on on on envoy right. So uh that's a bunch of lua scripts. As far as I remember that's based on on invoice. So if you want to take a look folks, we we've been discussing about using this in ingress and jynx.

B

I know that it already supports aj proxy and we can actually take a look into contour as well. If this is something that someone wanna wanna take a look right uh but yeah. This is mostly what I I have to to tell you about about this. This web application fire in the discussion, and also I I I I I might be old school, but I I really like still like uh mod security, how it, how it, how it works.

B

Strange you can't see my screen.

C

No, we just see black right now, because I I might I'm like.

B

Yeah yeah! Okay, let me oh sorry, folks! Let me see again. Does this works right now.

C

B

That's good now.

C

Yeah yeah sorry.

B

Folks, yeah, uh uh okay, so uh if, if you want to take a look into the the web, application file thing uh just take a look into into a curia fence. Okay- and I'm gonna- put the link for this one on the meeting notes as well. uh So uh folks we have some some folks actually asking us to go back to the angry to the ingress thing.

B

So let me add this right here, so I won't forget about kyrie offense and let's go back to uh to the contour stuff. So I I have deployed my. I have the applied contour here right, so uh I I now I have a kind cluster. I I have just dropped it off my qbdm cluster, because I I have broken it and I have contour here. So what should I do? Next, maybe create an ingress object and see what happens inside it.

C

Yeah yeah I wanna yeah.

B

Yeah go ahead; no, no.

C

It's at this point, contour's up and running and argo is running, so it's contour. It won't start doing anything until you actually give it something to serve yeah. So.

B

Okay, why do I have two counters and just one envoy, because I have one node and like two control planes? Is this the reason.

C

Yeah, you only have one node, so you only get one envoy, so the name set only runs once again. By default. We run two contours. um Each instance of contour can serve traffic to any any envoy, but they'll do leader elections so that only one contour will actually write status back to the api server. um So, in an event that.

B

C

Contour would go down then you'd have another one. That's ready with you know, with a hot cache of envoy config to then serve the other envoys.

B

Okay got it so if I do something like cubectl create, because I'm too lazy, okay, so I'm gonna do cubectl create ingress.

B

uh I can't remember the syntax, uh my ingress one should I pass some ingress class or it listens to any ingress class, or should I try.

C

Right now the contour is going to listen to any ingress object, so you don't have to have a class okay.

B

But we can do.

C

That yeah so ingress class for folks, it's a way to have multiple controllers run on a cluster, so you can have you know nginx and hd proxy and contour all run on the same cluster at the same time, but separate them out with this class, and it just gives it an id that you can run against ingress. V1 now has that baked into the into the spec which, before we had to use annotations cool um some.

D

C

Here about folks from getting started, I'm not sure if you want to keep going or.

B

Okay, can I can I go forward, then yeah.

C

B

Ahead full equals to.

A

By the way, I'm filing an issue on contour right now to move off of docker.io for your default container places.

C

Pr's welcome.

B

Okay, so I have created an ingress object, but I forgot actually to run something else. I guess quarter or some services right. So maybe I should use this thing just.

C

To have something something about traffic too yeah.

B

Okay, so now I have this service required and the deployment and the object. Oh, it creates the object, cool, so cube, ctrl get ingress dash, oh yeah, all right. So uh what does this do? Okay, so this is a catch-all for anything.

C

B

So if ctf gets service.

A

And the contour.

B

Is running in in something some service or something like that.

C

Yeah so you've got us, you've got to send traffic to envoy. Somehow so I was just going to say, we should have set up your kind cluster to expose uh your port 80 on your host machine to port 80 in the container, but um so you have to port forward to get to it.

A

Yeah, like in my picture you weren't trying to take, I drew that picture for a reason- ricardo. Sorry, I thought you were watching me draw that picture.

B

I was giving you something to do other than yell at me and steve, but if, if okay, so if.

C

I do you could just put it forward for now or we could destroy it on recreate it, but I don't know what you want to do. Yeah.

B

Let's just port forward yeah port forward dash n, so I should port forward to envoy right. Yes, okay,.

B

uh I don't remember how to do that. I'm really lazy.

C

I just put 80 at the end yeah.

B

80: okay, okay: ah I need to to put the local part as something like 8080.

C

Oh there you go yeah.

B

Yeah cool all right. So if I do something now, like one two: seven zero zero one.

C

That's probably not gonna work come.

B

C

B

I'm on a different machine.

C

Oh, that's really not gonna work, then yeah you're gonna have to pass in the host name right. So so I'm always going to route based on the host name. You gave it, which I forget what the example is. Oh, it doesn't have anything. It's it's a wild card, so yeah I mean that will work.

B

C

B

It's working so I can. I can try. Oh it's forwarding this one. Let me let me do this a little bit better, so I can pass my local address. Local port remote part address. Okay,.

B

Hold on folks, because.

A

All right so now.

B

We we can, we can have something like this.

B

No oh, come on.

A

We have to do the ip of the machine right, yeah.

B

Yeah yeah, okay, cool, we'll get it so I wanna I wanna, I wanna, I wanna improve this ingress object and maybe add some some more. uh Some more things like uh this server may expose be careful. Oh I'm sharing my screen with you with something that that exposes sensitive information. Thank you, folks, yeah! So so I, when I wanna, I wanna, uh maybe add a virtual host or something like that and and then start looking into uh the envoy configuration.

B

So let me I I want to really try to do that. So keep ctl get service right. So if I do something like create, create ingress- and I want to say that this ingredient is like ricardo ingress really close to the kind of.

A

You wrote that feature. Are you like trying to show off your feature that you wrote in groups.

B

I can't remember how it how it works, really, I'm not kidding.

A

B

Wrote that, but I can't remember.

A

And you don't know how to use it either: okay, yeah.

B

Sorry and I want to delete the other ingress ctrl delete ingress.

B

Okay, so I have deleted this one. What happens right now? It gives me like a 404, something like that.

C

Yeah, I think envoy's, saying yeah, there's nothing. There's no config.

B

Okay, so I should do something like curl resolve. I can remember, what's the I am help grab resolve.

B

Dissolve come on, don't I have.

C

What do you want to do just pass the host.

B

A

I don't know what what happened, what I was just sending a screenshot.

B

C

My wife, because she's.

A

Like texting me and I was like I'm doing tj, I can't stop telling me about your flight. What okay, what do you? What what what's going on? What are you doing.

B

Don't need your help anymore jay. Thank you.

A

It's too late: okay,.

B

Parts refuse it: oh 80, 80.

A

It's not listening on there yeah it's 80 80 on the other.

B

Side, cool yeah: it works right now.

A

Okay, so for folks that don't know what's going on, he he's running card, which is like some kind of demo kubernetes app and then he's got 80 port 80 he's got his por he's exposed to port 80 inside of his cluster to the to the host, and then he's accessing contour, which is running inside of his cl inside of his nodes inside of his cluster on port 80 and so 8080 is getting forwarded to 80 and then he's just used.

A

Coupe ctl, which is something most people probably don't know, is that you can actually use a command line to do to create a simple ingress rule. um uh So he's coop ctl create ingress to create an ingress rule so that when he tries to go to show me the curl command again.

A

Yeah, so he set up a um whatever. What do you call it? What do you call that a c name or a name? What do you call that.

B

That's the uh you mean the the v host the host name here.

A

Yeah ricardo.foo.com, so you set up that. That's what you call that a v host a v host record inside of contour. Is that what you call that a virtual host? What's the contour word for that.

C

Yeah we call it a v host, yeah yeah.

A

C

A

C

Pass the host header and and onward would route that in terms because we're just playing.

B

And if you are using, if you are using tls, it's called sni, which is server name, I don't know. What's the I means.

C

B

Yeah, it yeah it extracts from the from the tls header as well cool. So I wanna right, so I wanna I wanna. I wanna take a look into how how actually this is how actually this is inside contour.

C

B

I know robert has a question earlier.

C

Like hey, what's the what's the relationship between contour and envoy, um let's just dig into this right here yeah. So this is good.

B

C

Oh yeah, so so the example that ricardo applied that yaml file deployed these things. So right so you had an envoy daemon set and you have contour running as a deployment contour. What it does is it's watching for things in the cluster things like secrets, ingress objects, services, endpoints, all that kind of stuff, that's interesting, and then what it does is. It builds a configuration for envoy and then it passes that configuration down the envoy now envoy. We could dig into it too.

C

If you want to see it, but onward has this thing called a bootstrap config and essentially what that does? Is that tells envoy to go? Look for contour, because all contour does is. It provides um configuration for envoy dynamically and we we call that xds.

C

So contour is the xds or like control plane for it for envoy. So when onboard spins up it says, hey, go, find contour and then contour's job is to send down. You know updates for envoy configuration, so what happened was is when, when ricardo created, that ingress object and he created that the um the qwerty service, all of those things fired, events to contour contours saw those happen, and then it um it generated config and pass that down to envoy and that's how it got routing.

C

So, if you want, we could dig into the envoy routing. If you want to see how that looks.

B

I wanna do yeah.

C

So I like I.

B

Like looking at those stuff.

C

Yeah every every envoy has that admin page, so I think we're running on 118.1, I'm assuming so this has the patch so we're going to have to. um I can send you the docs for it, but if you go ahead and port forward to envoy on port 9001, I.

A

Mean so what is the thing that envoy does? That makes it special.

C

It's routing all the traffic for you so when they saw that request come in for was it ricardo.fu.com, it looked in its routing table and it was able to route that to the right thing in terms of.

A

How is that any different than like what you know you already have things like iptables and netsh on windows, ipvs, and all that it's like. What's the difference between envoy and those things.

C

It's just how it's routing, I guess you can do it at a user level. Users can manage this themselves now.

A

C

B

Yeah, so I ip tables iptables deals with the layer layer for rj, so you have a a source and a destination on a search part and a destination part. So iptables does does the routing, but it doesn't know what what you have inside the packet so envoy in china, xha proxy. uh They actually when when we did that uh uh to when when when we did, that call to ricardo.com iptables would wouldn't be aware of that, because you could be pointing to the same machine to the same to the same ip.

B

But you want to route to different places, for example right and so in envoy that packet it it gets open and you you know, for example, uh for which part to send so what what was the header which photo to send? Or maybe uh if there was some cookie saying that you should you should direct to server a or to server b or to end point a or to end point b.

B

So this is the main difference between uh the envoy uh uh in china, x, uh h, a proxy and and like using ip tables or ipvs, or something like that you are. You are working on different layers of the next.

A

Okay, it's just like yeah. I got it. Okay,.

B

Yeah so uh el alex sent to us that sni server naming indication, so thank you alex for for this information cool uh all right. So oh a meme's! Here, hey, I mean okay. So let me open the documentation that you've sent to me.

A

I always felt the reason I asked is, I always feel like engine x feels like layer seven, but I feel like envoy is like lower level than that. It's just that's just some fiction that I made up in my head, like envoy, is not capable of doing lower level load balancing I I thought it was, but I could be wrong.

C

Yeah yeah envoy can do like l4 and udp routing and stuff, um but you're not gonna, get all the fancy things you would get with like a full load, bouncer like managing ip addresses and all that sort of stuff. um But you.

A

Could do that, could you so you could replace something like iptables with envoy well for certain use cases like, for example, in the coop proxy.

A

I think so so do we need to make an envoy back into kaping now, ricardo.

B

We can do rha proxy, we can do it with both yeah. How does that sound to you jay? I think that's.

A

Right actually.

B

Envoy is is used by by ecio as well right, and I don't know if the if the like, if the the the lower path of stio just uses like envoy as ip tables or something like that,.

A

I don't think pear is not happy with this idea.

B

Okay, so let's, let's take a look into this one uh keep ctl blah so.

C

Essentially, what we're gonna do here is just need to port forward to envoy on 9001, and then I have the list of endpoints that we expose so before. I mention that, like cv that came out, um the actual envoy web page has a lot more than just these, but these are the read-only ones, so they send. Essentially, if you now deploy contour and envoy in your cluster, we'll still have these exposed over localhost, and if someone would exploit it with external names, then they still couldn't do anything malicious short of seeing stuff.

C

In your cloud, your your in your configuration.

B

And we have received that vulnerability. It was on a friday right.

C

There's been a couple things all over the place: yeah.

A

C

Know envoy just patched a couple cvs as well. That came out last week.

B

So jay we've got a question for you and pair how about sctp routing? Oh.

A

B

Because we I'm kidding but but hey alex, we we were discussing about that earlier today, and this is why I made the joke to jay. But uh I actually I don't know I. I am not sure, because one thing that we've been discussing is that we, we are not sure about the sctp use case right so uh feel free to join us, also in copying and seek network, and maybe.

A

You should have come to kaping today and you should have heard rant about sctp and how horrible of an idea it is to do on kubernetes. I didn't fully understand it, but pair can talk to you about this, and so can so can lars and sign network on slack evidently sctp makes basically no sense on kubernetes. It seems to be what everybody's saying, and I guess nobody even knows why it's ga, because nobody uses it, and I I don't fully understand why.

A

But I guess it's because of the fact that, like there's not really much value in having multiple sctp back-ends for the same service.

B

But this is something that we are curious about because it's a it's a really specific use case right: okay, yeah.

A

I guess telco companies use it, but the question is: why do they use it on kubernetes, I think, is the question.

A

We can give.

D

A

Let let you if you want to join now, we we should. We should bring a into the stream in in a little bit ricardo. Once you finish this in pair, if you're around, then I can give you a stream link and you can join us.

B

Yeah, so uh so, with that comment, I've got these axes. What what is at least is here steve. So it's uh like the there is a mighty, an id here and a random id and the priorities. So is this a configuration or is this like a status or something like that.

C

Yeah, so so these are clusters in envoy and clusters, kind of equate to services and kubernetes at least that's what contour does with them. So you'll see here. You've got the default, that's a namespace and then qwerty, that's the name of the service. It runs on port 80 and then like a hash.

C

So essentially you could deploy the same app a bunch of times, um so the hash makes them unique within our configuration because there's different things we could apply in terms of like health checks and waiting and that sort of stuff and across different ingress objects. So what you'll end up having here is every service. That's referenced from an ingress object is going to end up in this configuration.

C

So this is what contour stream down to envoy through that xds connection and you'll see things about this, like the active connections and total connections and failures and that sort of stuff. These are just clusters. If you go to like listeners, you'll see the listeners that we opened up in envoy um yeah. So there's the there's. The three listeners so by default you've got your 8080, which is your insecure one. There's a stats one and then you can see the envoy one we're listening to now on 9001..

C

B

C

See you know so that page I sent you gives you all of the debug endpoints, which are interesting. There's a slash, um go to config underscore dump, that's the more interesting one.

C

So if you take off listeners yeah that fourth one.

B

C

Do the config dump that one's probably the most interesting one.

B

Which one configure.

C

Them yeah, I do that one okay, so this one will show you like all the actual routes and bits that that we programmed to envoy with so, if you're, if you're having trouble or we're having trouble, you know adding a feature to contour or something. This is what we'll dig into. So, if you scroll past a lot of this stuff, you'll see all the it shows you metadata about what what version you're running of envoy and what filters and stuff are installed.

B

So umvi works already with quick with http http3.

C

I believe yeah they added that yeah.

B

Okay, but contour doesn't yet right. No.

C

It doesn't nope.

B

Okay, prs are always welcome right I can I can I can. I can try that.

C

Yeah, I need some.

B

Discussion about how.

C

We integrate that. Yes, if you scroll past a lot of this stuff. This is just all of the extensions that exist in or that are enabled in envoy. You can you can build these in and turn these off dynamically sort of, um and then we get down to some things.

C

So there's two things in envoy that are interesting, so there's static things, there's dynamic things so static things come from that bootstrap config we talked about so you'll see here we have a static entry for contour right, and this was the when we told envoy to go look for contour for its configuration.

C

We essentially created a cluster in envoy, which said: hey, go, find contour at this address called contour, which is the dns entry for the service and that's how envoy goes and finds contour for its configuration and then below that you'll see dynamic resources, so you'll see lds and cds, so there's listeners and clusters and then from there everything else gets streamed down dynamically. So there should be a route for your. um Your ricardo.foo.com.

C

Yeah, it's probably down a lot further yeah. Usually it's towards the end.

B

C

Yeah there you there, you so virtual host, so sub-zero is required at food.com, you'll, see, there's a route and the route should point to uh a match and a route, so that so slash and then there's the cluster we talked about so that default, qwerty 80 we saw on the previous screen with all of the clusters and.

B

Where is this here in the.

A

Yeah, what is that.

C

That's the name of the cluster in envoy that contour program to envoy with so basically envoy's, looking at or contour, looks at all the services in your cluster right and then it goes and finds all of the endpoints. And then it adds all the endpoints to a thing called eds, which is like the endpoint discovery service, the eds, and then we feed those into envoy, and then we say hey route to this cluster. So essentially, then we tell envoy here's a set of endpoints.

C

You can use the route to which are sitting behind that default, slash, quality, slash and that's how we get traffic to to endpoints. So english controllers simply don't route to services they route to endpoints directly and that's how we can do different kinds of load, balancing strategies to get traffic to these different endpoints.

A

So when, when envoy manages its quorum and its cluster and all that, when it manages metadata for its cluster, is it kubernetes native or does it have its own underlying like consensus, implementation.

C

um Envoy, there's no idea it's running in kubernetes. It's.

A

Not it's completely. Okay,.

C

Yeah contour is doing all the work for the actual integration. um So if you want to yeah so it's it's so we we programmed the cluster name to have kubernetes specifics in it just to make things easier to like debug and realize, um because those those names.

A

There's shared state between the contour between the envoy routers right: isn't there don't they have to like, have some kind of? How do they talk to each other or how do they share data with each other? They.

C

Don't they all run shared? No, they they all. They do. Is they spin up and they talk to their xds server, which in this case is envoy or I'm sorry contour, so contour will stream it's config down and then it just routes.

A

So contour it's a push model or it's a pull. I I kind of blanked out during the xds part. That's.

C

Fine, so there's a grpc connection between contour and envoy um and that.

A

So contour pushes config directly to envoy instances. One.

B

A

It's not a pull thing envoy, isn't watching contour on an endpoint.

C

uh No they're connected yeah, I mean they're on voice spins up and creates that grpc connection, and then everything gets gets, pushed down to it, yeah. So, okay, yeah.

B

But what what happens if like, uh but.

A

Envoy doesn't have to tell anything to the server it's the server that talks to envoy right. The connection is this way: not that way.

C

Yeah, I think so yeah, okay, I'm trying to think in.

A

My head, I mean, there's always some weird rule that gets broken right, like even in kubernetes. There's that rule that nothing talks to anything except sometimes the api server talks to the kubelet when it needs to get logs, otherwise, never right so, but in general, that's probably what it is: okay, yeah.

C

Contour builds its configuration and then when that changes it streams that down to envoy yeah.

D

Yeah right now, contour.

C

Uses what they call unbecause the state of the world, so it streams the whole entire set. Every time we're looking at adding the there's a delta api as well, which just would do the diff.

A

Meme is asking you know you can make bubbles on these ricardo. You can click on the questions so that it's the people on youtube can see it. um Anyways amim is asking. Does contour support, envoy filter editions like istio on voip, filter, crds.

C

No, we don't do it dynamically now.

C

A

There were some discussions.

C

About about about doing filters, that's it! No! We don't do it, we just we don't right now, yeah, um but open an issue, we'll chat about it. um I know some folks who compile envoy to remove filters because, like if you there's a bunch of filters that are installed that could you know create a security risk for you in a different kind of environment, so you can actually compile envoy and disable a bunch of things. So at the top of that screen recorder, there was a whole list of filters.

C

You know like the quick filter was installed and rate limiting and um stuff that you may not even care about, like mongodb is in there and wise and things zookeeper.

B

C

Can actually recompile envoy and and remove them um and then run so there's a page we have in contour, which shows you the things we actually need um need to be there to actually have contour work. So.

A

Why do they have these just to make it easy for people to run popular applications.

C

Yeah they're, like extension points, so if you want to you, know, send traffic to or mysql or doing rate limiting that sort of things you can enable these filters and they help. You know, extend envoy.

A

I mean what is it it's just a port. What do you, why is it? Why do you need an extension just a thing to.

B

You can you can use the information from the header. Remember that you we are talking about layer, seven.

A

So what if you.

B

Want what, if you want to like block a mongodb query, a specific mobility bigquery player? Seven can understand that right. So you can, for example, want to talk steve.

C

Yeah yeah, so some of these will give you, like the connection manager. There gives you l7 right, the http connection manager, so you onboard's going to receive you know an l3 l4 connection, and then you apply this filter which turns that into an l7 thing and then filters below that can then use that, as as an l7 connection.

A

So, okay, so there's my sql specific, so that's like compiled into contour or is that a.

C

This is envoy here. This is only on I'm.

A

Sorry envoy is that compiled into envoy, so like yeah, my sql contributors go in there and they make a patch and they say I want to add this logic that.

C

A

C

They're all thing you can write your own too, you can add your own in there, but you'd have to compile them in yeah. You can also extend it through um through lua, and then you can also do some wise and things that.

A

Was the lewis stuff? Okay, I got it yeah.

C

So you can write custom, lua things and add that in there contour doesn't do that today again, there's an open issue for that. We can chat about.

A

I don't think pear likes that idea either.

C

Yeah you, you can do very, very bad things, and- and you can, you know, break through.

B

I, like I like doing I like breaking things yeah, I'm not paid for that, but I like breaking things.

B

Okay, uh I I wanna I wanna. So we've got this working and I I could see that. I think that it would be interesting for us instead of jumping into uh ingress in gynex or hp proxy. I can. I can do that as a as a parallel, but I wanna really take a look into maybe gateway api working because, as this is really something really new for for for latest layer, seven routine, maybe should be it's. It's gonna be interesting for me as well.

B

I need to implement that on on on ingress in ginx, so instead of reading all of the documentations, I just want to go through and see how steve didn't copy, probably to english in generic, so I'm just kidding, but I I I'm really curious about that. Yeah.

C

B

C

Api, I don't want to go to the root of that.

B

C

B

This site here.

C

Yeah, so this is the root, so gate api started out as a thing called service api and the idea was to kind of replace um it started out as how to do you know, ingress v2 and it turned into like let's replace services and think about how you can uh manage services.

C

So if you, if you go to this to the the sick group for this they'll, have a better explanation of some of what this stuff is, but um essentially there's this idea of of gateways now and that's why it's called game api now, and so this is sort of a way now that you can route some route traffic into your cluster using l4 and l3 in different ways. You can describe right, so this is going to cover all of it, not just ingress.

C

um So I I I yeah go ahead.

B

C

There's a so this: this is the v1 beta 1. literally last night, they released the v1 alpha, 2, which you can see a little warning down there, which is a breaking change for folks. So right now, contour supports v1, alpha 1.. Now that v1 alpha 2 is in a release. Candidate we'll go, try and support that, but in terms of contour and how this works is, and we look at code, but contour supports ingress and it supports our own crd, which we call http proxy and we support gateway api as well.

C

So what happens is when traffic comes in? We said: contour watches the cluster, for you know, services and endpoints that sort of stuff. It also watches for these gateway objects. If they exist in the cluster and if they're there it'll, you know, build ingress information off of that. So contour builds. This thing in memory called a direct, a dag or a directed acyclic graph. So we build this graph in memory and then we walk that graph and generate envoy config out of it.

C

So essentially, any ingress configuration item whether the ingress or our cd or or gateway api gets shoved into this like middle layer. And then we walk that graph to then output on voice config and that's how we can support all these different types without having to rewrite you know contour over and over and over and over um cool. So we just add a new processor to the list and go yeah.

B

And how can I deploy this just like deploy the crts.

C

Yeah, so there's a couple ways you can do it. We have a quick start for this. If that helps.

B

Quick start, I, like quick, starts yeah.

C

So go back to the quick start plans um at the top. I think there's a quick start or getting started. I'm sorry! It's a guide, go to the guide guide and at the bottom there there's a using gateway yeah.

C

So there's a couple ways to do this: you can do this with contour or you can use an operator. The operator is an alpha thing, so I would just stick with this. For now this is probably easier, but um we have another quick start which has all the bits in there for you and we can dig.

B

Into that too,.

C

So essentially, this quick start is going to give you what you have running now. You know namespace with envoy and contour and stuff, but it's also going to deploy those gateway. Api crds, as you're familiar with ingress before is baked into the api server. So an upgrade to a new kubernetes version would upgrade your ingress now because of crds, you can kind of make those more independent, which is nice cool,.

B

uh So if I, if I may, I just apply this over my my latest contour deploy or I should okay, there.

C

A

A question for.

B

You about plans for contour to support udp.

C

Yeah right now there aren't um there's an issue again, there's always an issue for everything. Right, um it's come up. I know it's quick. It's come up as well, because you need that udp to do that to template quick, so um I'll, see if we can go find that issue- and you can comment on that.

C

But right now today, contours focus on a layer, seven type proxy um getting in the you know, tcp and udp adds some things that you need more around the project and right now we're just focused on layer seven, but there is an issue: it's not a no uh what's what's what is it no is forever, but right now. Yes, it's forever and knows not right now, how's that saying I forget, yeah.

B

No, no! No! No! It's temporary! Yes! It's forever right.

C

There you go yeah, so yeah, that's kind of where we are with that. um So go ahead and run that again yeah. So you had to hit the thing where the crd wasn't ready in the api server yet yeah. So what you have now is you have contour's still there you have now you have the gateway, pi crts installed. You can see those there in the middle and then you also got a gateway class and a gateway.

C

um So right, git api is based on this idea of you have a gateway class which describes kind of what you're trying to deploy and then a gateway which implements that class.

B

So gateway class is not it's not name, spaced right. It's just a it's.

C

Cluster scope, yeah: it's.

B

It's it's the same as ingress class. So if I do something like that, I'm gonna see the oh okay. So the the structure is almost the same of the ingress class right. Yeah.

C

uh You can apply it in here. There's a there's, a it's in the spec, but there's a way you can apply configuration to this. So you could say: hey. You know this ingress cloud. If you actually go go to those docs, do you have that that website open um it's very persona based is the idea behind it.

B

C

The api webpage.

B

Where is it, I keep losing things here.

C

B

C

Okay, so there's these different, these different, um you know personas, so the infrastructure providers. So in your gateway class, you could say hey. This is a you know, an external type gateway which has certain specifics about you know aws or google or azure or whatever you're deploying it into, and then the gateway actually implements that. So actually it's the thing that routes the traffic and then from there. You actually have routes. So you can have you know http routes.

C

You can have tls routes, you can have tcp routes, that sort of thing yeah, and then you have services behind that which are where that you know so.

B

That deployment created the gator class and the great gateway right so I've seen- and I want to see the gateway right now, because because I am just like curious about what's what's inside this yeah, so the gate, the gate gateway, one is: is cluster scope as well.

C

No gateway is our name space code, so.

B

I should do something like.

C

It's probably in project contour, yeah, cool.

B

C

uh It's called contour, I think, yeah, not an example.

C

B

Why is false this status, waiting for the controller.

C

uh I think we have to restart contour and explain why but yeah um so so gateway api. Now has these statuses, which are great right and ingress as you as you never recorded like, like you, don't have any status feedback to users so if something's wrong, you're, typically tailing logs of your controller, now there's the status condition which lets you get feedback so right now. This is saying: hey someone's generated this, this gateway, but no one's picked it up. No one said: hey, I'm going to own this and and manage it.

C

Okay is where we are, and the reason for that is right now. Contour needs restarted because we we apply the new configuration behind the scenes and contour needs to get restarted to like apply that config.

B

Okay, so let me restart it because.

C

Yeah I just like delete contours.

C

You can do dash, do dash l and then that's fine too.

B

ah I didn't want to delete the one. I don't want my users to lose the traffic.

C

It's like right that one says: come up um so there's two things today that this uses there's there's a config in our file, which tells us hey what gateway class to begin. Is this contour going to own, so it's kind of like using an ingress class and then the second thing is: is contour's not going to enable gateway apis unless the the crds exist in the cluster? So when it started originally, those crds did not exist.

B

So it didn't start.

C

Any of the controllers, because it would error out looking for those theords so now that they're there it'll start watching for those as well spun up the informers for those those crds.

B

So now yay cool.

C

Yep, so now that yeah so contrast hey, I can process this stuff. Now it took ownership of it. So you'll see here these these reason, codes and stuff. So it's valid um status is true and type is ready, so um contour owns it now.

B

So, let's create an http route, so the the specification here- okay, that so that's persona, related and and in this case, when, when that persona, because this is namespaced- it says that the cluster operator, but but it can be, for example like the namespace owner, which is not like the application developer. For example, like.

D

B

Okay yeah, so I wouldn't allow like an application developer to deploy a a specific application without like being being passed like the security folks or like going through a pipeline right.

C

Yeah, I think everyone's going to manage this differently. We've heard from different folks in terms of you know the different ways they want to use this, um but yeah and the v1 alpha 2. Some of the bindings are different and what we can see that here of how how you bind routes to gateways um and and the new version it sort of flips on its head and it's it's different um for person, different reasons, because your gateway now doesn't have to live in the same name space as your ingress objects.

B

C

Yeah they can be yeah.

B

C

So, like contour has our own our own cd, which does the same kind of idea, uh but this this can work the same way. So you could have your gateways live in, like some admin name space and your routes live somewhere else.

B

C

B

What happens if I create something like uh okay, okay, okay got it so here you are not specifying which uh wait you are. You are specifying like match. Labels app equals to quark, but this is from the deployment. This is from the example right. This is not from.

C

B

Specifically, if I wanna, if I wanna, create like a a gateway and say hey whatever http route you put here, you are gonna watch this works. If I, if I remove the selector or no.

C

Yeah, so so this is that this bit, that's like different now, I think um so in in the v1 alpha 2, which just came out last night. um Okay,.

B

C

This is the binding. This is saying: hey. This gateway is only going to bind to http routes in that group from the same name, space matching those selectors um but yeah. We could change that. We could make it all name spaces or now, in the new version. There's there's a two-way binding where you've got it, the gateway, I'm sorry, the route is actually going to say: hey, I want to bind to a specific gateway.

C

So it's going to be the reverse yeah.

B

I don't know I I've sent to the english enginex developers about this tgik. I don't know if someone is watching, but please take notes, because I don't wanna read the documentation again folks. I just want you to help me implement this all right. So.

C

There's an example yeah you.

B

Could use, but this needs to be in the project counter namespace, because the selector here says that the namespace is from the same right.

C

It's interesting that you still have the app quality on there for the labels. I thought I took that.

B

Out, I think this is from. I think this is from the example.

C

It is, I thought I took.

B

C

Updated these last night, so I was wondering it's interesting. You still have those.

B

C

We can just create it and then figure it out: yeah, yeah, okay,.

B

Forgot the name space yeah.

A

Wait so I have a question about the api server on this. So how do the crds for the new gateway get installed.

C

You deploy them, someone, someone or something deploys them: yeah, okay,.

A

You don't get them for free, no.

C

No they're, not they're, not part of the api server yeah.

A

So right now, when contour bootstraps, it creates those crds for you.

C

uh With the examples they do yeah so when.

A

C

Did that apply that that gateway example.

A

If somebody wanted to install those on their own, they would what just pull them down from kubernetes sig, slash, ingress, v2 or something.

C

Yeah it's in the it's on the on the the game api webpage. They.

A

Have examples of how to do.

C

B

Let me make this first.

C

B

Because I want to contour gateway, so this is like it generates for something here and you have the crds right here.

B

I hope I expect yeah here.

B

That's x, net working yeah. I can't remember.

B

B

So they they have generated for us in the deployment.

C

Cool yep and then I put in the chat recorder, there's the link to that to the docs and how you do that. Well, let.

B

Me, let me send.

C

I can put in the notes, I guess there. I don't know.

B

Yeah yeah and I'm gonna put here: where is the notes.

B

It's right here, all right, cool amazing. So so now we have this, and this is the configuration that I've grabbed from here, which says that I am adding a name called local.project.contor.io.

B

Which matches the prefix to the service name quad on part 80, but I don't have a card running on this namespace. Do I it's running on the default? One.

C

I don't remember.

B

Let's see so quartz is running on the default yeah. So if I try to do something like.

B

Dash resolve what was the name of that theme.

C

That was local.projectcontour.io, so so we set up in our our contour dns. That's just results in localhost that that dns name so when you're using kind. If you expose port 80 from your kind cluster, you can use now a dns name which points to localhost to help just make things easier. Yeah.

B

But I don't want to destroy my kind closer.

C

Right right, yeah, just that's what that does.

B

Yeah, it doesn't work, probably because of the.

C

Parts, no because you don't have any service pointing to it. That's my assumption. Okay you're getting.

B

uh Okay, because maybe maybe this is because of the because of like uh the it's it's pointing to the quad, but this gateway is running on project contour, name space and not on not on on the default name, space right and we have in the gateway. uh uh Where is it.

C

B

Yeah namespace from scene.

C

Yeah we could change that, we could, you could deploy the the route and the default namespace. If you want.

B

Let's do this yeah.

B

So, hey what I did come on gateway. What was the name.

C

uh It's contour, I think, in project contour.

B

So I want to deploy this instead in default.

B

And gateway class name is example, part 80, part 80. Is this right, even if I pass part 8080.

C

Or not yeah, because unvoice is sitting on 80, but your your port forward is change, the from same to be from all.

C

Yeah, so this will let you this will let you bind a route from a different namespace.

C

B

Oh so, should I should I edit this instead of just because I'm not editing, I'm creating a new gateway, but in default name, space.

C

But you can't have another gateway, you can only have one.

B

See I was almost doing something wrong. You should. Let me do something wrong.

C

B

Can leave you can leave.

C

Something you leave. The gate. Excuse me leave the game. Oh you're, just going to edit it okay, I see what you mean.

B

C

Thought you're just going to apply that back again, but no, I see what you're doing yeah sorry I'll yep yeah and then just um create your route. Now.

B

It's already created.

C

But it's in the project contour name space. I think.

B

C

B

Okay, just change the default right.

C

B

Should I change the host name as well? Maybe right.

C

Yeah, if you want.

B

Because we have the other one.

C

That's true: let's.

B

Do let's, let's not try to.

C

So we haven't implemented that yet, but that's a conflict right, so that was the classic ingress example of two routes with the same host the same path. What do you do? um The canonical way to solve that and give api is whoever got there. First wins, so the oldest created version would always win um hunter hasn't implemented that yet, but that's how you solve that um in our crd, we solve it a different way, but yeah.

B

But I got that not found again.

B

C

Take the edit the gateway again and just take off that selector.

C

That um I'm on the bet, the actually you could status, do this real, quick status, the the route that will tell us what's going on. If it's waiting, then it didn't get picked up.

B

A

B

A

Yeah, what does that mean?

A

That means that whatever's ratcheting watching the route hasn't seen it yet.

B

Even in the the other one, which is probably in the right place,.

C

Yeah change that.

A

What, if you delete the what, if you delete the contour server and let it come back up, will it fix it.

C

No, it shouldn't do it delete edit your gateway again and trash the um that selector. I think that selector shouldn't be there, because it's doing both it's saying, hey can only bind to routes from all namespaces that also match this selector leave the one you had on there.

A

Status, though,.

C

I want to leave what you had but delete the selector.

C

Can you undo, oh okay,.

A

You use all my vim ricardo.

C

A

What is that plug in with the underliner.

B

I don't know I downloaded it from someone.

B

Okay, now we have status, but this is on the the project counter. Let me open on the default.

C

B

C

There you go now it's about, but the other one used. If you read that it said it was head, resolved, resolved refs issue, meaning it couldn't find the service that told you what was wrong, which is cool if you status the old one here now. This should work yeah.

B

Okay yeah now that's working cool, yeah, yeah cool great, so uh we are almost uh one hour and twenty and we could see contour and doing some some gateway api stuff right. So jay do you wanna have where you succeeded with windows.

A

I it looks like it, I I think it um it never got it. I mean it, it didn't work because it uh it looks like it was trying to it, couldn't pull images and then I'm not sure why it couldn't pull the images, but I think it was trying to there's definitely a windows, specific configuration that you have to set contour up with that I didn't do.

A

um It looks like out of the box, it doesn't know how to schedule the windows and linux nodes, so I I I maybe I'll file an issue or, if there's somebody already working on that. Just let me know steve. You know.

C

A

C

I think apparent issue to talk about windows support but yeah.

A

C

That we fully bait that out yet to see that, even if that even works so.

A

Contour on windows, I see that yeah yeah. We need to add some details. There, yeah, okay, cool.

B

Yeah, so today was pretty intensive right, so I don't think we are gonna. Do you wanna make take a look into hp proxy in janax? I I think it's uh probably folks. They are like a bit tired and we have per here waiting to run about sctp hyper.

A

Pair, why don't you just give us your rant about sctp.

D

No, I don't want to give you a rant. Oh come on different level. Oh I have that. I just don't think I have to fit the um the model with the micro services, because it's normally more used like tunnels between two nodes and uh and then there is protocol signaling.

D

To to figure out what happens if you lose one end point and so on, and if you try to build a prox for that, it doesn't really fit well.

A

Can you pull up.

D

A

Of the whole sctp thing for folks ricardo, while we talk through this so.

D

I'm all with lars there so that there's applications that need it then sort of.

B

A

B

A

B

A

C

Have a diagram of.

B

A

That he's talking about in there, I think, maybe not yeah, I think no go.

D

Back yeah, um it's a complex, complex protocol, it's doing more than just normal um or what you get with tcp, for instance, where you have a connection- and you know it's between two end points is easy to load, balance and they're easy to scroll to scale.

D

And if you look at the cube proxy to me, it's a scaling element.

A

So the first pair, what's the difference between sctp and tcp in at the most stupid level, for we have a pretty large audience here. So, let's start from the basics right, okay, backup.

D

Intel code was, or there is something called signaling protocol 7. That's how all pod uh I mean. All telephone networks were built using ss7 and they're still widely used and it's sort of packet oriented, but with the with the resend, you can say so 3dpp, which is the standardization organization for mobile telephony, so 3d, 4g, 5g and 60, and so on. It's being worked on there. They wanted to have a protocol that had sort of a mix of what they had in ss7 and udp, because tcp is a stream right.

D

You have to read it and then you have to frame everything. That's in there, whereas with the packet oriented you get, you get the message and it has a format.

D

So that's what I wanted to do and instead of doing something like quick sort of or something simple udp would retransmit they. They came up with this protocol. That's what they're trying to do what was done with ss7, and that is by having dedicated networks and dedicated paths. So it typically gives each and end points to ip addresses on two different interfaces, and then it's assumed that these will then guarantee delivery over different infrastructure.

D

So if someone picks up a fiber, you will still have a dedicated path, which would ip is very hard to guarantee, because at some point these things will go to the same routing, same routing, stack or routing context. You can say, and the packets get treated the same way and then you have at application levels means to steer sort of which, which path things should should take on.

D

So it's a very specific usage normally of the protocol when you use these dual end points and there's semantics that comes with it and, like I said it has typically been used with the fixed ports on both sides. You set up connection between two nodes and then you do can always call it application specific load, balancing when low, balancing or not low, passing but distribute the streams inside this connection.

A

So when you say I have two things like, I got two nicks.

D

Yeah, so you would, you would be.

A

D

I got it you would, you would tie your process to use both nicks and then you would expect them so from for that. You that you had inserted specific routes into your application, so it would choose the right path based on which address you're using for the other side, so it doesn't just have a default route that sends it down that very specific routing or connected to a kernel or.

A

Can we draw ricardo? How about we draw try to draw this? So, okay, I'm gonna, I'm gonna delete the sticky session thing. Look I'm getting rid of it! All! Let's see if you can, let's see if we can sesame street this okay, I got this over here and I got this one over here and okay.

A

When I have a tcp connection, I usually have one nick- and I have a nick over here and this neck has ip 1.2.3 and four, and this nick has 1.2.3.5 and I go here and I have a dac and I have a sin and I have an ack and these they just talk to each other and they just talk over the same interface. That's what I'm used to at least.

D

Yeah, absolutely you could add a second interface if you wanted and have another connection specific right so.

A

Yeah I can have another one yeah, so let's, let's take the sctp example. So now I'm going to do in purple, I'm going to have two nicks yeah all right now. What do I do.

D

Another box two nicks and then you have one one association: that's using both these interfaces.

A

Okay and it's the same ip, no.

D

They have different ips.

A

Okay, so different ip.

A

I think I asked ricardo on twitter, whether it's possible to have the same ip for two interfaces the other day and he didn't.

D

And then you route through the specific interfaces, that's.

A

That's everything through the same thing, yeah well.

D

You can take it in either way.

A

D

Then all right.

A

Yeah, so so now you have so now. What what do I do so I've got tcp. So how so this is tcp up here. So what is my sctp.

D

Can you just draw a big? uh What's the color you used over all these interfaces.

A

Okay, let's make it different color and, let's ricardo, what color you want me to use green for sctp. Take the green now andrew you'll use puke green for sctp, so draw.

D

One green line between each sort of pair of uh mix.

A

Okay, so I go from here to here and then what do I do do the same at the bottom? Okay,.

D

And then sort of you draw a box around all that, because the same association or a session.

A

Uses box around these green lines: yeah exactly: okay, okay,.

D

Cool, so that's sort of what defines your connection so to speak.

A

D

This is, you can do similar things with tcp with multi-path tcp, but let's leave that for another discussion. So so now you have two sort of the ideas, then, that you have two paths that these two endnotes can talk to each other. But in reality you have a packet network here in the middle. So there's nothing that guarantees that uh they're going to separate paths. I mean separate routers and separate switches and so on at some point. Typically, it goes together.

A

So this can go now so that so one packet.

D

A

D

A

Of uh what so, what one packet gets like the first time it gets transmitted, it might go this route the next time it might go. This way.

D

They tried to just do this straight pipe. Actually, so they do this to have redundance to have redundancy and resiliency sort of that. You you're trying to assume that you have that think of these two logical uh wires and that's how ss7 works, but with packet networks, it's very hard to guarantee that your logical wires doesn't pass through the.

A

Same infrastructure yeah, so it winds up being yeah, okay and.

D

The sort of you can use things then like new tools like srv6, which is segment routed for v6 uh to rather specify in the packet the parts this has to go through. So I could sort of take a packet and say it has to go through this router, this router and this router, so that you you traffic, engineer the past is going to take through the network. Then you can actually build something like this, but the problem we have with kubernetes is that this is used as a tunnel.

D

So so, if you think that when you look at when you in order to scale a little balancing something you want to have something that can vary. So if you look at the typical well five top pulse, so you will have.

A

I mean, I guess what we didn't put in here is we didn't put hops, so this doesn't really make sense without hops right.

D

But you know when your little balance, you have your destination address, you have a destination port or you can say, destination address and source address yeah and you have a protocol and then you have destination port and source port. Okay, and in order for something to to load balance, you want you need to have that the session or a connection is unique. So this five tuple needs to be unique, but if all of them are fixed, you can only have one so so.

C

D

You lock down all these uh variables, then you can only have one connection so so for for something like the service proxy to to work. You only lock down four of them. The source destination, address protocol destination port and then you typically have a fixed source address from the node you're coming.

D

But then the source port will change the ip stack on the machine where you set down up the connection for will look and ensure that from this machine this is unique and then then, on the other side, you see sort of you check again in the stack. Is this unique? Yes, it is, then you can load balance that you.

A

D

Or whatever, and that's what we do, the whole idea.

B

No, I was going to to say that jay made us go from layer, seven to layer, two, I'm just.

D

Kidding this is sort of maybe three and then did it.

D

So the cube- oh, this is a bad explanation on top, but the cube proxy is not the right place to do it. I think that's what lars is trying to say as well sort of that we need to have a way, and that's why caffeine is is good right, because it's you can do the sort of thing.

A

So basically, what you're saying is that we should just put a big sad face over this whole picture.

D

Yeah, so so that you should someone that really needs this to write his own backend. So you should.

A

D

The the specifications using capping, but then someone like ericsson or uh whoever you are write, your own sort of service proxy for for uh okay. Thank.

B

You just how you feel.

C

Screenshot this right now, okay,.

B

D

But uh I mean, if we're going to talk about protocols, that's interesting. I mean we had in what steve was talking about. Enboy envoy can handle quick. He can handle it pretty well. But if you look on kubernetes, there is no support for quick at sort of the q proxy level.

D

Should there be or should it not, and in that case how and how will it tie up to the things you can do in envoy, okay, the back side of every load, balancer sort of is that it also works as a blocking firewall for ports that are not defined. So should there be a a path between things, you define an envoy and network policies and vice versa,.

A

I I feel like sctp was some kind of like like appendix or something like, or what do you call it like a it's like a it feels like it was something that might have been useful at some time and then I'm.

D

Not gonna, this is recorded, so I'm not gonna, be yeah.

B

Let's, let's be careful about.

A

That I don't understand it well enough to have an opinion. I'm just saying from.

D

The way you describe it feel the clear requirement for for for telco companies when they went from circuit-based networks to try to to run the sort of the phone calls and all this over a packet based uh network, so that that's so what sat become came from so in one way, I would say it's a it's a something that's used for technology shift uh as it was used in that technology.

D

If that and is still used, will will uh telco be able to use other protocols when walking uh when moving in the future, like quick, uh so that we can just have one protocol that we use both for for web and telco signaling. Perhaps.

A

Okay, now I can get rid of my little sad man.

B

Oh yeah, I took a screenshot of that. Okay, okay, I guess we are. We are here like one hour and a half I I don't. I don't know alex is saying that sctp is more than just that, so I I would invite alex maybe to to to join us on on the copying meetings and and discuss.

A

B

With cube proxy right, alex.

A

You should definitely come. We meet every friday and you know like the context for this is that we're kind of working on like porting some code from the original coupe proxy, the existing one into kaping, which is a little more plugable or a lot more plugable and so pair came today, and he was talking about sctp and and someone else who was working on sctp and a lot of like concern and questions about like what's the actual use case in kubernetes for sctp.

A

And that's like we're joking around here, but like that's kind of the serious sort of sort of like roots of this conversation. So every friday we we meet and if we're on the kubernetes, cigs or come join us and hang out yeah. And I mean.

D

Yes, to be clear, there's no one that says that as a tp is not needed. The question is: how is it best supported and integrated into kubernetes sort of based solution? It's not about that. I mean there's there's a lot of applications that use it right, but I just don't think that the support is where it needs to be, and I don't think sort of the the traditional way of just adding in another protocol. You think is just another.

D

The same thing uh I mean satp, tcp udp, uh are very different uh protocols and sap is hard to manage the same way as we did with sntp as tcp, especially the use case has been used for you can.

B

D

Use cases that works, but.

A

I'll give you the link now alex, are you in? um Are you in the.

B

A

B

Can join us on on on kubernetes leg and being you or ping me, how does that sound, join.

A

Us on kubernetes slack and then our food proxy, clean I'll I'll, put the link to our google doc in here. I'm.

D

Gonna see I'm gonna check linkedin. I think that's also a good way to connect.

A

Yeah, um where did we go here? We are stream yard there. It is okay, cool, I'm putting the link in youtube right now alex, and so it's called the coop proxy working group. So you can go to kubernetes network. You can just google for kubernetes sig network on github, and you can see all of the meetings that we have, and one of them is specifically this this group and we we meet every friday morning but yeah come hang out.

B

Yeah morning, depending on where you are it's lunchtime when they they meet yeah- and I.

D

B

To keep my lunch to join them, yeah, okay, cool! Thank you folks, so I think that we can. We can end for today. Right is everybody. Okay, with that, okay.

C

B

So, thank you. Thank you, steve for the contribution and helping me and jay not going too deep on the wrong path of stuffs. Yeah. The.

C

Wrist breaking.

B

C

A

A lot thanks, steve and para for joining us and obviously ricardo for doing this whole thing. This is really cool. Yeah.

B

Yeah and and hope hope you we can see you all on the next dgik see ya. Folks, that's good! Take.

A

Care everyone all right.

B

See y'all bye-bye.

D

A