Antrea LIVE !!!, 27 Oct 2021

Previous Meeting

⏯

youtube image

►

From YouTube: Antrea LIVE: Episode 1 (Antrea 1.3.0, FQDN, and K8sNetlook)

Description

!!! Our first antrea-LIVE show !!!

Focusing on upstream K8s network diagnostics with k8snetlook and FQDN policies (evolution in upstream as well as the antrea specific implementation).

- Antrea 1.3.0
- FQDN Policies
- k8snetlook

Hosts
- jayunit100 (Vmware)
- yashbhutwala (staked)
- sarun87 (Vmware)

thanks to @Arun Sriraman @Jianjun @abhiraut @Yang Ding @vrabbi @Amim Knabben @Luther (monson) for coming !

Todays Topics:
- ClusterNetworkPolicies
- Kubernetes Security
- AntreaProxy
- KubeProxy
- K8sNetLook
- Troubleshooting Kubernetes Services (ClusterIP, NodePort)
- Securing L7 traffic on Kubernetes with FQDN NetworkPolicies

Come say hi in #antrea in Kubernetes slack!

A

Do you want to introduce yourself it's our first andrea, live stream, so kind of excited for everybody to be here. Why don't you all introduce yourself, then I'll introduce myself? Then we can uh get started.

B

Sure yeah I'll I'll go uh yeah. uh My name is yesh butwella, I'm currently working at a company called state. um We run. uh You know a bunch of different blockchains for different cryptocurrencies and provide staking services for customers, uh but I've been you know, sort of in the kubernetes space for for a while and um j, and I used to work in the past that synopsis.

B

That's me, cool and arun.

C

Yeah sure, hey everybody, um I'm arun, sri raman. I work in the vmware sd van space, so the product essentially allows you to connect multiple locations, edge locations uh to your data center and cloud, um and I've been doing or been involved with kubernetes for quite some time now and uh worked with jay in the past and currently in in some far away. So.

A

C

Working at vmware too so.

A

Yeah me and me and roone over here at vmware. We work on I'm so yeah, I'm jay most y'all probably know me that are watching this stream anyways, because I'm pretty active in upstream networking, sig network related stuff and yeah me and arun are here at vmware. We work on a whole bunch, a whole bunch of networking related stuff. So um so hopefully this is our new stream. So in case we we're getting logistics and everything set up, but folks can type comments in the um inside. The little comment thing and we'll answer them.

A

This is uh partially office hours and it's like, if you have android questions about releases and stuff um depending on who all is here, we may be able to give you some really interesting answers. It's also partially a hack session where we're going to be able to go through things and look at stuff together, um and so it's kind of a little bit of both- um and I see yang, is here so it's great to see yang and.

A

So yeah folks want to jump in we're in the k8's upstream, uh andrea channel, um so yang. What's up yang.

D

How's it going jay.

A

Great to see you cool um so yeah good, uh so we just got started. We just did intros so um yeah, I'm jay and my friend here is uh arun and yash is here. Gosh is also my my my buddy, um so so.

A

Like the purpose of this stream is for us to all get together, look at stuff going on in upstream networking in the sig network area in the network policy area. So it's great that yang is here. He can maybe give us an update on that and do a little bit of hacking and hanging around and learning about how andrea works and abhishek is here. Abhishek with your telescope is here: okay, okay, so and and abhishek is used to be friends with the roon before I was friends with the women, so we're still friends, you're still.

C

Friends, yeah yeah they're still.

A

Cool abhishek, you want to do a quick intro for I'm yang. You want to do a quick intro for yourselves.

E

Okay, maybe I'll quickly start uh hi. This is abhishek and I have been part of the nk team since the beginning of andrea, so I've done some of the network policy part and the antenatal policy for design along with young and yeah, we'll probably talk more about it.

A

D

Hi, uh my name is young and uh I've. Also I've worked uh alongside with a attack on the network policy sites in south ventura and I've been, you know, a part of the project uh I think uh over a year now and uh yeah we're we're glad to be here.

A

Cool okay, so um I was thinking what I would do so me and aaron were just playing around with clusters. We had like we had an infrastructure panic for a little bit, but I think I figured it out arun, I'm going to give you an ip address and you can ssh into it unless you have a running cluster right now and it's an internal ip address. So um I'm just going to give it to you in the stream yard, chat, okay um and you know, get.

C

Get everything up and running by the time you give me so we'll.

A

C

A

It's in there and you can you know the username and password to those machines right, okay, so so arun is going to show us how k8s netlook works in a little bit, but first well, I don't know uh abhishek or yang. Do you either? One of you want to talk me through what's new in 1.3.0, which is the newest, andrea release,.

E

uh We might need to pull the release notes because I was out of office last week because.

F

E

Has just come back from his potential, you.

A

Just got back, I know, okay, I know what's new f2dn, so I'm gonna show I'm going to show folks ask you to nyx.

E

Oh you're, talking about the 1.3.

A

One yeah, so um I was just playing with this and showing a rune. So this is kind of cool, because I remember when the first fqdn we were trying for a long time to convince the upstream sig network folks to get fqdn working, and I don't know where that landed, because um I I kind of started focusing on other things. But as far as upstream, where are fqdn policies.

E

uh You know there are a couple of guys from google who were driving the fqdn policy effort.

B

E

And they, I think, have merged some of the controller part um in in their gke repository and that's.

B

E

That they want to donate to you know upstream, but in terms of the the cap itself or uh how we want to achieve fqdn policies with network policies that is still not agreed upon so yeah and I think the upstream uh community is looking for. Volunteers to you know, drive this effort uh into the community, so uh you know anyone who's listening or who's on. The call here wants to work towards that uh feel free to join us on the sig network policy, api uh meetings on mondays, um so yeah, but yeah.

E

So as part of that, you know, we wanted to uh fast track that in entry, so yang had a significant uh uh uh work that he did for the ftd and along with grayson from the anchor team.

A

Yeah, so that's a young! You got anything to add to that before we dig into uh some some live stuff.

D

Yeah, so I think it's been a while since, uh um since I was uh you know developing this, I think uh um the fqdn policy sort of like code was ready uh around the two months ago and then they undergo some improvements. But uh at the time that you know we implement this fpdm policy. I think the challenge for this to be upstream is that you know uh there are two flavors of this right. So if you think about fdm policies, people apply fpga and policies into flavors.

D

One is that they will have a specific ffd and they wanted to sort of like allow a vlog, for example, google.com or facebook.com right, and the other very common use case is a wildcard use case where people wanted to say. I don't want anything coming from facebook.com to access my workloads, so people will put something like star.facebook.com, and this is an uh on the upstream side.

D

This is a you know more difficult uh use case to solve, because you know if it's a static f2dn, uh you can always sort of like do a dns query and resolve that fdm to a specific address. But for you know wild card cases, um usually, you know, rely on some mechanism in the cni itself or the implementation itself to do some sort of like proxy.

D

uh For example, we actually wanted to look into the dns response and to see if that it actually matches one of the wild cards that users specify and then after that, do some sort of like uh data pass, rule, realization and decide. You know whether the package should be allowed or not and yeah. I feel like this. Is you know the major thing that's uh um also blocking the fpdn to to be you know, moving forward and upstream side. Is that there's a lot of uh implementation, specifics uh in the ftd and policy itself.

A

So what is the proposal from the entrance side on fqdns? Is there a proposal doc that you all made? I remember the google one we demoed. It live in the upstream network policy meetings and then I know here's how they're implementing it.

D

Oh yeah, those are, I guess, those are just the uh just the uh manual on how you can use an fpdm policy, but.

A

D

A

An example and see and see if we can go from there what's up luther, it's good to see you. I muted you because there are some background noise, but now I'll unmute you. How do I meet you? I know luther and scott rosenberg just joined us. So that's that's cool. I appreciate you all coming um so uh let me see here so I'm in a cluster coupe ctl get and if I get my cluster network policies, network policies- I don't know if I have yeah, I actually have one.

A

So let me delete this coupe ctl delete uh cluster. Is there a short term for this? I have to do cmp every time.

D

Cmp, if I remember.

A

I think I tried cnp and it didn't work, uh acnp, acnp, okay, okay, cool, so there's nothing in there. So so andrea has this concept of cluster network policies. Now abhishek also can tell you all about there's um this whole cluster network policy thing going on well, young and both abhishek have worked hard on that and there's a kept for that and abhishek.

A

If you want to put the link to that cap uh in here, I can link folks to that, but we're looking to get all the interest cluster network policy stuff as much as possible, at least into upstream uh over time, at least if we can at least whatever makes sense.

A

um So, let's see uh so, if I go in here and I do- and I do, uh I've got I've got. I've got an fqdn policy here right. So folks, look here, you can see. I've got a policy, it's got a priority of one. Does that mean it's high or low.

E

uh That means it's high, so any uh so it's basically think about it in terms of like how would you prioritize your work items right when you think of certain certain work items are being like highest priority. You would say it's a p0 item, so lower the number higher the priority okay. So this will.

A

Override any allowable, so if I had an action, allow right! So if this was allowed, this would get overwritten by this policy right.

E

So if you have this uh so we're looking at the one on the ac and pfqdn1.

A

E

A

Me know if this is too small if it's.

E

If it's the priority, one then you're overriding any other policy instances that you have in your cluster, because uh you know as.

B

E

They map to the same as long as they apply to the same set of pods this priority. This higher priority ecnp will take higher more precedence. So in this case, your your drop rule for for the foobar.com uh we'll have the is priority.

A

Okay, cool the drop rule for foobar will have the highest credit, okay, cool.

E

That's the only rule that you have in your cluster right now, so yeah.

A

Obviously yeah so um so, if folks want to go and try this out and play at home, I have this directory here and it's got the policy. So if you go to kj unit, 100 k8s prototypes and entry live 1027 2022, you can go in here and you can copy paste this and um this same directory.

A

You can go into this same repo and you can go into kind and then you can go to run this kind, local upscript and you can just run it with the andrea flag and if you run it with the entry flag like this.

G

A

A

Cluster, like that, you can just literally like run this command and it will generate a multi-node kind cluster for you, so group cpl get nodes, um each of which is running android 1.3.0.

A

So if you want to play at home, it's not hard so and then um there's a coupe ctl create f one other thing that I'm creating inside of the same repo, which is um there's a smoke test directory, and you can like see this nginx pod service. So if I do cat, oh sorry, cat got that slash.

A

Cat dot, dot, slash smoke, tests, engine x, pod service, that's just basic smoke, pods right, like a busy box container and I'm labeling it with app entry and you'll, see why that's important here, because it's a match label for that. So I want to block stuff like young was talking about right. I want to block things so coup ctl get pods. I've got this running and I go down here and I want to make this network called. So let me exec into this pod. Okay, so I'm in here and let me wget www.foobar.com.

A

Okay, so it's giving me an error 403., that's not a problem right! It means it's able to get there. It's just that. I I'm going to a random website, so I'm not returning anything. So I'm able to reach foobar.com I'll be able to resolve the ip to one 173.2312, one. Nine three four make a tcp connection to it: connect to it connect to the web server.

A

The web server says you can't access this resource, but I see you and I can hear you okay cool so now I can coupe ctl create this fqdn policy right, oh no. What did I do? I forgot to put the file here. We go okay. So now I made this policy and this pods policy is matching the label of that pod that I created right.

A

So um this match label is saying: I want to apply this policy to this specific pod so that this pod drops all egress traffic so that the cni andrea drops all egress traffic to this fqdn right. So then I'll go over here and now I can do it curl our w get www.fubar.com, let's see if it works.

A

Wait. I didn't. Oh, that's weird: it worked it shouldn't have worked. What did I do wrong? Then you get foodbar.com?

A

Oh maybe I did it in the wrong name. Space could ctl get um what is it a cnp?

A

Okay, it's there I'm getting a forbidden here, but I have this policy. So why do you think that would be the case?

A

I wonder: gtl delete customer network policy, try to delete it and see.

D

Okay, could you could you double check the pods again that you you exactly into yeah? I might not be in the right pod.

A

Oh, I know why I'm in it I'm in a I did a docker exec into one of the nodes, because I was trying to show around something. So let's try it again. I think it'll work this time.

A

Yeah, so let's yeah, let's, let's: okay, let's watch it work; first: okay, cool www.fubar!

A

Thank you for that yeah you're right. I was going into the wrong thing. Okay,.

B

So I can get in.

A

And the policy's gone now, let me create it: let's try one more time: okay, boom, all right now run the same command again.

A

Okay and now you can see it hangs okay, so thank you young for realizing, I totally exact into something that was not a container, so um all right, so we're in so so we're able to block traffic through fqdn. So this is cool. Just so, people understand the difference here. The normal network policy api doesn't. So. If I go to network policy api kubernetes, if I go to the docs, if I go to the existing docs, you can read about these.

A

If you're new to this stuff and like it, you know we support, matching, pods and then blocking on pods and blocking on name spaces, and you can do some blocking on ciders and stuff, but you can't block fqdns using the kubernetes api. So this is something you need a custom cni provider to do like andrea is doing for us, so really excited about that in 1.3.0.

A

If folks want us to show anything else, feel free to like add questions in the chat um thanks josh for taking notes. Josh has taken a bunch of notes over here so um and the cluster network policy, and it looks like abhishek um so abhishek. You can add them, you all can add the public stuff in these uh in the youtube comments in the thread, so folks can see them, um and then I can do this see. I can pop up the little thingy on the bottom and it looks cool um so yeah feel free.

A

If you all want to add comments and links in there go for it so yeah um and then um hi everybody. Let's see if this works. Okay, cool next up is a roon, so arun we have a couple of options.

A

You can use a cluster that I've, given you um we're going to look at this tool called k8s netlook, so kns netlook is like a network diagnostics tool that arun was hacking on for a while that allows you to sort of anytime you're like in a customer environment or whatever, and you want to see like why networking isn't working. Usually the first thing you do. Is you try to curl from a node port?

A

Then you try to call curl from like a cluster ip to see if basic stuff's working, then you try to see if dns is working with the api server. There's this list of things we all do that are kind of intuitive he's kind of automated all that over here at kns netlog. So he's going to show us that, working, I guess, um and now how do we want to do it a roon? What's the what's the plan here, do you want to get into my cluster or do you have something I think you're on mute?

A

C

Yeah, I think we all speak on mute all the time.

C

So um I did take a look at the cluster that you gave me, but I might not be able to get net look on there, I'm still figuring out how to get into your cluster completely. So let me share what I have okay, we need to see if things are running as they are just give me a sec.

A

Because you can ssh into the nodes I can. I could. If you start a, we can association.

C

That's an easy.

A

C

Get files onto the nodes double get because I know it's put on linux, so.

A

Oh yeah, is it photon um there should be, but you should be able to yum install. What do you get? um Let me see so I'll do coop ctf. Let me start a t-much session, so t-marks new dash sg, okay. Now let me do um I will I can share this screen as well. While we want something you can see.

A

Here we go so if I share.

A

uh Window share my terminal.

A

Okay cool, so now I'm sharing my terminal. So if I do so for folks interested in vmware tons of this is a vmware tonzu cluster, that's running and we run andrea by default in vmware timezone right. So if I do ctl cluster info, uh you could see um ctl get nodes, okay, so here's I'm in a management cluster and uh so here's here's my three nodes. So a roon. I can ssh into one of these nodes by doing ssh cap v and if you do tmoxa you can join me in here.

A

I don't know if you use too much, but so if I get wget uh okay, so if I do young, can I do young install cash? Why w get? I don't know?

A

I assume I can, but I don't know.

G

And then um let's see did that actually finish? Oh no.

A

I think it's just hanging out, so we can try other ideas, so this curl work yeah. We do have curls. So if I do curl dash, oh, that should work right. Oh.

C

H

C

Be able to do a dash dash output and give it a file name.

A

Okay, so you want to just curl yeah, I mean that works right. So do you want to just curl the k8s network onto this cluster and run it yeah? Let's try, okay, so just do tmux space a and you can get in there.

H

Man you're asking me to get on a tmx session. Oh wait! We got w again. It already did w again.

A

All right here we go all right, so we're able to get all right so now, how do I get the king? That's.

G

Not okay, yes, that should be a single command.

A

Yeah, that's not hard! W get so arun! You presented this at kubecon. How.

C

Did it yeah it went well, um I think there were about 500 plus people who are yeah both both live and virtual, okay and I'm happy- and I heard good feedback that it's going to help people figure out networking issues without losing all of the hair that they have so so.

A

So, let's okay, so you're set up so you've got k8s netlook on here. So what do you want to do now.

C

uh Is there a coupe config on the node that you can export to yeah? Are you? Are you in this session with me? No, I am not so let me yeah.

A

Into that that jumper it's 10.92 to 118.96 and then I can meanwhile get a koop config for you. So um actually we should have a coupe config on there. I guess this is the first time I've ever tried to do this. Fine slash name.

A

I don't know, I mean it's a it's a it, so we might have a queen config on here. You know okay, but I can get one explaining it exit out of here and then cat home.com.

A

Okay, so there's my concrete now I guess I.

B

A

That right, so I can scp.

A

Who's, let's see who's better at like typing scp commands by.

C

Hand so I finished doing that on tkg mgmt, vcmd0 node, so, okay, so we're in the wrong node here. um So I guess you're in a different, not wrong, but in different words.

A

G

Which one are you on?

G

I could just copy it. What is that.

C

What is the ip ip? Not just just do sap sap that file to yeah to.

A

One of the nodes yeah I'm on okay, scp- I'm going to do cap v at here, colon slash, temp right. So I'm going to copy the file. Okay! So there's your coop config right. So your file is now in cat, slash, temp, config!

A

Oh wait! I got ssh back in and then then I'm gonna, let you go then you're free, okay, you've got everything you need. You got a car, you got a coupe config.

C

So mgmt vc control plane. So let me log into that node! Next, you don't need to log in anything you just log in you can execute commands on it. Jumper, okay, just.

A

Log into the jumper and do tmax a you're free you're in the node.

A

And I can do uh config yeah, you still do like you. Do like those old-school single dash command line options.

C

Yeah, okay, try host first and see if at least the host ones pass.

G

C

Well host and then and then the config part after.

A

Okay, here we go, it worked.

C

There is a problem: oh.

A

C

Let's try use it.

A

Why don't you explain this to us really quickly and then then we can try the next thing. So what does this yeah.

C

So what k does netlog does? Is it? It essentially has a bunch of uh a bunch of checks built into it. It goes in and tries to execute a bunch of not just commands but also network traffic.

C

If it's running on the host side, it's going to just be a process running in the host, and it's going to run some icmp packets, dhcp dns packets around here and there and figure out how the overall network connectivity looks like from the system uh in relation to kubernetes, of course, okay, so.

A

This is a healthy cluster. I've got android agents running on every node, I've got a controller, um and so how do I.

C

What do I do now right so now we need to see what's going on right and uh the tool itself does need some permissions on this level to be able to create packets and send them out on sockets.

C

So it does need a net raw socket uh permission uh when it's running, so it might be it probably just a pseudo problem. So if you're on a debug at the very end, it'll also tell you what's going on when, when, uh when each of the checks are being run,.

A

Okay, let's find out so this dash debug.

C

Okay right, so you can see that it's actually able to talk to the api server and run live z, check and health checks for all of the api components. Okay, it was able to what happened that it's not able to open an icmp socket to run icmp or ping tests, and that's because it's operation is not permitted. So you you need to run this tool in sudo or oh.

A

That's the only problem is, I didn't run.

C

A

Oh, that's it. You should have just told me run. It is root. Okay, so we run it. So it's working so, okay, okay! So we're able to do it. So this is something.

C

The host site checks look good right, so it's able to talk to the default gateway from the host. It's able to talk to the kubernetes api server yeah. It's able to also run uh service checks right, so this kubernetes api server, endpoint ip that's going to be the cluster ip within google play. So that's what is backed by either proxy or cube router or one of the other implementations out there.

C

Maybe in future caping right, yeah,.

G

A

Thanks for the campaign shout out yup, I didn't know that was coming.

A

G

A

Episode, that's why you all should come back because we're going to talk about that thing. Next time.

C

Okay yeah, so uh it is going to run um the ip icmp check on the kubernetes cluster ip so, which means it's also checking service, ip pathway which is host to service ip to part. um Okay, some formula.

A

So this is confirming that it. How is it, how is it checking this like? What is it? What are you doing to check this? Are you making a pod go further down? Yeah go further.

C

So this is all running from the host okay, so it is it is. uh It has got the service ip from from kubernetes, because it has a cube config. uh It's able to so it does a http call to see what his, uh what what the api server returns and it returns a 401, because it's not there. It's forbidden we're not using a an authentication, token or a barrel token, but that's.

A

C

It's it's a past test from a greatest network perspective, yeah and then what it does is. It tries to get all of the end points that are backed by that service ip. So you have three uh control plane, endpoints, three api servers running somewhere.

A

Okay, so it's going to the cluster ip that forwards me to my api server and I've got a three node cluster right. So the first test, if you look at it, yeah.

G

A

So, just for folks that are playing at home they're, maybe because we're going a little fast. I just want to make sure people get to see this. So if I do coop ctl get notes, you can see, we've got three control plane nodes right. So what arun is talking about here is that these three nodes here correspond to those three control. Point nodes right, exactly exactly.

G

So if you scroll a little further up, jake.

C

And yeah, so the first one that you see there uh from host running cube service ip connectivity check. So that's going to the service ip and the next one that you see, which says running cube api server, endpoint ip connectivity check, which has those three ips underneath it. And so what.

B

C

Doing is first, it's trying the service ip and then it's trying each end point individually to see if all of them are healthy or not. Okay and.

H

C

Of those control plane nodes, don't respond you're going to see the pass fail.

A

G

So this is the cluster api. Okay,.

A

C

Right and then it's going to talk to the api server and actually get its its status that it knows.

A

Okay, so each one of these arrows is like a different check: yep, okay, cool great. So what is all this stuff.

C

uh That's the api servers livesee endpoint right that the healthy endpoint for api server gives runs a bunch of api server checks, that's already in upstream kubernetes. So you can do this today. If you want to hit the kubernetes api servers, services.

B

Okay, unhealthy.

C

Yeah and then it tells you if all of the plugins are loaded and the api server is healthy in some form.

A

Why do they all say post start hook after them? What is that? That's the name of the endpoint.

C

That's probably the name of the the method. That's returning the check, so kubernetes has a bunch of hooks that that tie in to tell you if each of these controllers and the feature gates are enabled or not right, so crd, informer, bootstrap controller, our back stuff, everything looks.

G

C

Cool uh you can do a dash or s uh at the end of the command they remove debug and say dash. Yes, oh sorry, not dash s. Yeah silent. I forgot to use the short form as well. So just a second yeah.

A

Did you how come you did you use? Go flags like you didn't get cobra, you just use, go flights? Yes, I usually just want to add that extra dependency. You know why.

C

One is dependency and two to keep the size of the file as small as possible. So right now, this binary is about 8 mb and I'm going to process to get it down to 2 or 3 mb. If that's possible a.

A

Meme is here he said: kaping, okay, so you're just trying to make the smallest binary ever so small that you can't even use fancy cobra command line options; okay, right.

C

Right, you see I'm working on the sd-wan side now, so it's all packet level, processing and oh yeah, bandwidth and line rate. So we've got to keep keep things small, oh yeah, okay, okay,.

A

Yeah, if people aren't allowing all.

C

A

Code and see nowadays, okay.

C

The system's cool but yeah. So if people are interested to automate using this tool, they can use that as silent option and then get get a json.

A

Block so this is do whatever with it, so we need to build a sauna boy plug-in next right. So next thing you need to do is we need to put this in the sauna brewery and then have like a something that's network plug-in.

C

Yeah, we could do that um so so cadence network itself can be run in multiple ways. It can be run as a binary on the node, where there's a problem or are on the node, where a pod is exhibiting an issue. You can also run it through a docker container.

C

There is a docker container, that's uploaded with the binary, so you can run that similar size uh or you could just use, uh run it as a kubernetes job. uh Okay, there is a yaml spec.

C

You can and give it a node selector so that it lands on the right node, where the problem.

A

You want to give me the you want to give me that example of that, and I can try to run it for folks.

G

uh Sure actually.

A

You can see it on.

G

Github, okay,.

A

So you don't get help all right get up. Okay, I'm googling right now: okay, let's not look hard all right! So, okay, that's network problem detector and then I go okay and then look can.

C

I just do parts examples. Can you just do examples yeah? You can do part here through the binary.

A

C

A

Smart, okay, I see the raw file I'll, get it okay. Let me get back to it so so.

C

That idea yeah the idea why uh I don't uh recommend running it as a part within the kubernetes environment is because you're debugging a kubernetes problem, and now we want to deploy another new deployment into kubernetes.

H

C

It sounds weird, so I I mean.

B

The monitoring.

C

And the debugging uh tool running in the prop in the environment that you're trying to debug. So so then what do you do you exactly.

A

Into it, dash t.

C

F

It already deploy it.

C

So it's not going to run right, so you need to change a couple of things in that card in that spec, so delete it. Okay, cube cuddle, delete minus f, okay,.

G

So what you need to change is if.

C

You go at the very bottom yeah yeah, the the command is there right, so that needs to be edited to whatever command you want to give it, and- and you need to change the hostname- to represent the host that where you want this, to run.

G

Okay, that's the north selector.

C

A

Okay, so I got to put the ip address uh all.

G

Right, it's not the hostname label.

C

Yeah, it's those names, ctl get nodes.

C

Do you want to fire a couple of uh parts? First, do we have parts in that cluster.

A

Yeah we have you know every cluster always has coordinates right.

C

uh Let's deploy a couple of uh applications: yeah yeah, because codiness will anyway get checked by kds network.

A

Okay, but wait a second. What do I do? What's the hosting I have to edit this note right to get this so ctl edit.

C

You can just say get notes.

C

A

Should have that right.

C

Isn't there one it should be there yeah the annotator.

A

Oh, it's just the name, so the host name is.

G

A

The same as the regular note name: okay,.

C

I didn't know that I don't know well, it depends. It is the name that cubelet uh uh registers the node with. So if you have a hostname with cubelet, then yeah yeah. So this this looks good. So your.

G

A

G

C

Registers on your cluster looks good. Okay,.

A

So I'm going to delete k8s dash afk. ah Where is that everyone run kate? Here it is okay, I'm deleting it like. You told me to delete it, I'm deleting it now. You want me to not run coordinates.

A

No, you want me to not. You want me to run something else. You know what I.

C

Already have um and a couple of um host names there's a host names. Example on kubernetes documentation right it gives you the host name of the part.

G

For debugging purposes,.

C

Or nginx, whichever I have my own.

A

What do you here, hello? Where is it smoke test hold on? Let me get him all right here. We go, I'm going to make this I'm going to run here. We go all right, I'm gonna try this one and then, if you don't like this one, I'm gonna do the one you wanted me to do.

A

I think this is fine, because we use this in the other example right. So this gives you a busy box um and an nginx. So this should give you everything and it's got a damon set on it. So.

C

Sounds good and how many nginx parts, let's have multiple.

A

Okay, multiple engine x, pods, multiple to make.

C

A

C

A little more realistic you, you do have uh normal.

A

I think this is a damon set, because I I think I usually do a demons. Oh it's a deployment. Here we go there. We go.

C

Wait is that a docker rate limit issue.

A

That's a yeah, it's a docker hublot rate limit issue because we're inside of a vmware data center right now. So so, let's see if this works and then maybe we'll get lucky, maybe one of them will pull down. um If not, I can change to using our hardware proxy over at vmware. We have a hardware proxy that we use. um So can we see if the pod basically works, though? First, how do I test it? So.

C

You need it, you need, you need to give it a source part under destination. Part right which, which are running on the running on the container network.

C

If we have to, we can we can give them, we can try that source and a destination, and what is it going to do with.

A

The source in the destination.

C

So the assumption is that you have a you: have a problem um uh at the source or the destination, but but but app a is not able to talk to app b.

C

So what k does net look uh and everything looks running right from a kubernetes perspective, so what data will do there is try to run a bunch of checks as if it's running within the part, so it uses the network namespace of the part on the host and then think of it like running a sidecar container, but not really through kubernetes, just just outside of it. So.

A

Okay, so select name spaces, so I switched over to this, so we could draw this out for folks, so I've got say a node here and I've got a node over here, okay and if I'm running k8s network in a pod. Maybe this is running on my third node.

C

So yeah, so let's talk about how you want to deploy kds network first right. So are you talking about it uh in terms of running it as a binary on a host or as a likewise.

A

Let's think about this, this example that we're doing right now, where we're talking about running it on a pod, okay, just.

C

A

Only because, since we're kind of at that 45 minute mark, I want to make sure people get to learn something other than watching me, edit yeah, so that's cool. We got it. We gave them the basics and now, let's show them some something about how it actually works at a higher level. Yes,.

C

So when you deploy the scadas api, yeah you're going to specify a node selector and the idea is when you choose that node selector label, you choose the one where the source part is running. Yep.

A

So I selected n3 okay, let's say.

C

Right so I selected and three you you'd normally want to select an and one okay, so I did the wrong thing you wanted to. You want to test part one yeah.

A

Okay, so if I want to test this thing, this is good. I didn't realize that so so, if I wanna, I could drag the letters one at a time, I'm not good at myro.

A

Oh here we go okay, so normally! This is how you do this.

A

So I would normally do this right cool and then um so I'm running k8s network on here and then what does it do? Does it exec into this pod.

C

It does not exact per se. uh What it does is that kts network first runs as a process within the host network name space on that host, and it does a bunch of host checks that that we saw oh okay, so so kubernetes networking the way I see it you you need to look at it in three three terms: part-to-part connectivity.

B

C

Part-To-Host or part-to-service and then service to part and external right, a bunch of these combinations and permutations yeah. So that's.

A

Certainly, if you talk about.

C

It yeah, if you talk about it in the networking terms, it's east west versus not south versus inbound right, okay, yeah, so k does network, runs on the node and and does a bunch of checks, yeah and then what it does is given a source part. It's going to use linux, network namespaces and linux, okay, okay, to go into, or rather switch the process into running into that particular process. The parts network namespace.

A

Okay, so this is.

C

A

Net ns lookup thing over here uh and a center.

C

A

C

A

Okay, so your ns entering into the network namespace and then how do you you look up like the pod name from the container d information.

C

Exactly so yeah, so you you give the uh you give netcatus netlook cubeconfig, so it can get information from kubernetes, so it gets the no the node name, the container id and the runtime. And then it goes to the container runtime down there and tries to get the exact proc id or process id to use to figure out which network name space it it's running in okay, okay, cool and then it logs in there and runs a bunch of these checks.

C

So when it's in there and it's running it's as if the pod is sending out information or receiving information. So it's like a side. Car, okay, container.

A

So then, this is essentially a side car to the pod and it's on the host. It's like a host.

A

It's not really a side car, because it's in a different name space- I guess- or maybe I don't know I get.

C

It it's running it's able to run in both namespaces, it's running some of them and the host network name space. It switches itself into the pods networking say that runs it. Oh.

B

C

The advantage is that it doesn't uh it's: it's not known to kubernetes, so it's like a side car. If you think about.

A

It so it's actually running processes inside of this name space and in the host name: space. Okay, exactly that's cool! That's really cool yeah! I think we're close to time maroon any three we got. Does anybody have? I know june jen you're here on youtube if you want to leave any andrea announcements or abhishek or any any other folks that are on this call. um Now is the time otherwise, um and you of course, do a room so jay.

C

The uh cubecon talk has a good demo that goes through all of the checks all of the parts and how the tool is built, the architecture and and what all checks it does. How does it how it does, as well as how to debug a kubernetes problem in general and then what what to do so do take a look at that.

A

Yeah I mean: let's do, let's do another.

H

A

In a few weeks or something to ruin like you know, this is our first entry of life. So it's like you know we're just playing around, but if folks are interested- or even you know, whatever people are still learning, so maybe nobody understands what to ask yet, but maybe we do a deeper dive and you could show us how it actually works under the hood.

E

Yeah definitely.

C

Cool it can work with any any backend entry or other way, so so.

A

Can I google this kubecon kansnet? Look? It should come up yeah, okay, so it's over here, but it's not on the internet. Yet right.

C

The video is not on the internet, but the slides are on it, so you can get a lot of commands and diagrams and pictures on the on the powerpoint slide.

A

Okay, so the slides are here: yeah, whatever you drew there.

G

I think there are some pictures.

A

There are some pictures that are but they're not as good right.

G

Okay, I'll leave that to you.

A

All right, okay, here we go, this is beautiful. Look at this awesome here we go cool, so this is cool. Thank you. Thanks for helping me out everybody thanks for coming josh, um thanks for showing up josh, I don't know, I know luther you showed up. I don't know if you have anything going on on the cni end that you're excited about.

I

Well, you just want to talk about cni's or do I have to talk? Do I have to talk about andrea.

A

I mean well, it is the android live stream, and you did tell me that you were going to get it up and running on rancher pretty soon right. I.

I

Am I'm going to give it up? I just need you guys to support privileged containers and support fpv6, get it going.

A

All right we're on it: okay,.

A

He's talking about he's talking about entry on windows, and uh so by the way. Speaking of entry on windows, we have a couple of bug: fixes I'm gonna bring those uh there's a cut. There's a windows. There's a couple of windows bug fixes one is related to reboots. That's on its way um and host process. Containers are coming along and antriaproxy, which is a big thing for windows.

A

Has we have prs now for andrea proxy, which is a replacement for andrea proxy, which is a replacement for the coupe proxy and there's uh node port cluster ip and whatever there's like the third pr, there's like three or four seminal pr's and those are all in wait.

I

Hold on elevator pitch.

A

uh Well, we've got some andrea folks in here abhishek. What's the most exciting thing or young, what's jin jin is here jiu jin um yeah thanks uh so uh jin jin. Do you have a elevator pitch or abhishek about um andrea proxy and why it's such an exciting new feature.

A

I can tell you, god.

E

I think if ginger is here, I think he's probably the right person to talk about it. But essentially I you know we. What we want to do is like get rid of q proxy and do everything with obs in with entry proxy, and that will also help us. You know, do network policies uh efficiently and in addition to that, it's portable to windows as well, and I believe there are other few topics which is related to notepad services that can be solved by using anterior proxy. uh Very well.

E

uh But I believe again, junior might be a better point of contact on that. uh But but I think you know once once we start handling uh you know. Things is with obs with entire proxy there's more flexibility for us to do a lot more and we can do more features as opposed to trying to get it done using cube proxy and have just just having to run one single proxy is easier. um So but yeah again, I think I'm more on the net policy.

A

Is breaking up, but he likes network policies, yeah june jen. What's your yeah, so I yeah for me from a windows perspective, it's huge because um normally like with windows, right like if you think about it like. If I have a multiple. If I have a.

A

If I have a cluster and I'm running, I have a windows node over here and I have a linux node over here right and I'm running coupe proxy on on windows, as is right, I have to run coupeproxy without privileged containers. I have to run that as like a well unless I'm using rancher wins, but even then right.

A

I have to run it as a as its own host process and manages that as its own as its own piece of hardware right and on linux, um I'm running antrip, I'm running the coug proxy and that's also that's running in a pod and that's running in a privileged pod right. So here it's in a pod. Sorry for my horrible, my role, but here it's in a pod here it's on the host right.

A

If I run entry a proxy, then I'm basically can I can run every andrea. Can manage both my cni right and can manage my cni for both linux and windows, and it can also manage all of my um all of my service routing and everything else all through the cni. So it's one less thing. I have to worry about um to me. Like that's the biggest thing. The other big thing is that ovs open vswitch, the the thing that enters based on doesn't like to run that on windows.

A

I have to assign binary and all that so once we move over to running entry proxy, I have that all that work to set up obs, but it can do everything right, so ovs can kind of like handle all that stuff from me, and now I get all these benefits from it, and I don't have that extra cost of having to like having to also run the coupe proxy which forces me when I do user space stuff, because I'm running an obs, I have to run the user space proxy, which then I can't run the kernel space proxy and the kernel space proxy for windows has a bunch of features that are, you know, sort of better than what you have in the user space proxy, because it's faster so part of the big thing is part of the big reasons for me.

A

Is I like the fact that ovs now can be like a first-class citizen in my cluster, managing all the networking, as opposed to half of it, being windows and half of it being non-ovs um so for windows. People here that might make sense for the rest of you. I don't think you care about that explanation, but um here's june jen he's got a couple of points, so the major point is: keep the whole forwarding pipeline in a single data plane. Oh yes, good for management, troubleshooting maintenance.

A

um It gives a lot of also like for us at vmware. It gives us a lot of native integrations that we're interested in there are performance advantages compared to ip tables as well. And yes, windows is a single data. Plane implementation across windows links so yeah um yeah. So it's going to make windows networking fast, faster, uh faster. It's going to be it's going to be great.

A

Now one interesting thing: I compared linux, windows and windows and linux, networking bandwidth over the user space proxy on android and the performance is about the same, which is interesting for a single pod or is pretty close like. So it's not like it's really. You know things are pretty good right now, but yeah um all right anything else. Anybody else wants to share we're kind of at that hour mark. So I think I'm going to close up otherwise and thanks everybody for coming to the first andrea, live stream.

I

The andrea proxy will be open first, just like if you're right, I assume you gave it the same name. For that reason, right.

A

Yeah, hopefully, yeah definitely yeah and hopefully we can convince people to put it into kaping soon. So maybe I'll bother you indian about that, but we have all these back ends. We as and me me and me, we're gonna. Do it me and me we're gonna get the entry of folks to put it into company.

I

um I get your point, though, because there's also the cni right there as well, so we do the same thing before having both in one pretty slick, pretty easy.

A

Yeah we'll do a big demo of android proxy soon, maybe next time so. Okay, thanks luther for coming luther, is over at rancher by the way folks. So um thanks for coming, it's good to see people from different companies coming and hanging out to learn more about andrea and um everybody keep in touch with us on the andrea channel in upstream slack on kubernetes or come find us um well.

A

You can find me in luther in the sig windows room and you can find a rune in upstream slack and everybody else here, thanks again josh for showing up and helping me run these from stakes and um we'll catch up later, I'm going to end the broadcast bye. Everybody bye, abhishek yang, thanks for showing up june jen, take care.

D

Thanks jay bye.

B

Thanks guys, thank you.