Cloud Native Computing Foundation Kubernetes Forum Seoul 2019, 10 Dec 2019

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Advanced Network Capability for NFV on Kubernetes - Ashish Billore & 박종한

Description

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Advanced Network Capability for NFV on Kubernetes - Ashish Billore, Samsung Electronics & 박종한, Samsung

Kubernetes is becoming the choice of platform for containerized applications.
Applications benefit from the built in capabilities of managing and orchestrating container lifecycle at scale.
Telco and NFV usecases, especially for 5G, too are looking to benefit from these capability of kubernetes.

However, usecases for Telco and NFV containerized applications require advanced networking capabilities for user-plane or the data path.
In this session we’ll cover some of these advanced networking requirements and how some of these are fulfilled using various kubernetes capabilities while others are being adopted from some of the existing options such as Neutron (OpenStack networking project).
Session also goes over the challenges and future directions.

A

Hello, everyone welcome to our session so today we'll be talking about advanced networking, networking capabilities for nfe on kubernetes.

A

So this is our session and briefly I'm Ashish hashish, Glory working with Samsung Electronics in South Korea here, and we with me mr. Jang man. Park knows.

B

An impact on your decision in behad she sometimes under the talk suppressor era with Smita.

A

Thank you so moving forward, so this is what we have for the is in that today we'll talk about why telco care about kubernetes and when we go about that, what are the telco specific requirements which we feel are very specific Cornish for any cluster and especially how we can do it in kubernetes kind of environment. So, basically, we have tried to group.

A

Our experience in findings are related to working with telco or nfe requirements on a kubernetes cluster, so the cluster topology, which we have to customize based on telco requirements, the networking capabilities required to support telco use cases and, lastly, the load balancer.

A

So these are the three broad categories under which we have clubbed, but I will go over some of the things which we faced and how we try to build it or overcome it, using whatever is available in kubernetes, community or the other communities, and then we'll also give up give some of the pointers where, if you are interested, you can look for more details.

A

So this is the brief agenda about our session today, with that, let's see why telco cares about kubernetes or why the needed, if you see you could do like you have been playing around in the initial days where you have like individual machines, you can do a lot of things with docker, but then the problem becomes manifold when you try to make it across a cluster.

A

So if you would have realized when you start with one, not two, not three, not fine, and now you start thinking of ten nodes, 100 nodes and apply a specific customized configuration. Then that is where you start running into issues. But kubernetes gives you a way to manage everything as a unified cluster or unified resource of schedulable instances, and then you get out of box workload resiliency, so you have Auto star, I, call it so auto scaling, auto recovery, auto healing and blahblahblah more coming with CRD and operators.

A

So you can it's it's up to your imagination, what you want to do with your workloads and have kubernetes, do it or manage it for yourself, and then you get the lifecycle management for your workloads as well as the infra. So we saw in the previous session with the machine API and the cluster API. You can even manage your infra using kubernetes now and then you have the possibility to scale from edge where you have one node to node and go all the way to the scale of public clouds.

A

With this uniform set of API is and constructs and resources with which you can build things from as small as one node and go all the way up to a public cloud scale with thousands of nodes and workloads running on it. This gives you a possibility to efficiently manage your resources, and your infrastructure now becomes more much more responsive, dynamic and a gel based on your requirements and which is much more important for tailcone. Now, because of 5g and the new use cases it has opened up for telco.

A

And another thing: is you: don't have vendor lock-in, so it's 100% open source. You have a warm welcoming community, which is always there to back you up or support when you run into issues which often you will while trying to use it. And lastly, that's the cool thing or the in thing, don't believe me, try telling to your users or customers that you don't use kubernetes and see the reaction for yourself. So with that, let's see what we have going forward with our requirement so I'll hand over to my colleague, mr. Park, okay,.

B

B

Aaron kitsune delivered Sangeeta Haga principle: you want, we implemented actuator co2 gas, oven, approved VM, given royalty-free handle me on the wooden co-op thought nipped ago tongue, since I saw talking about the role the wood no charge here: sweet Catherine and a sister of VM ravenica, a delta T Canasta, your journey free Emily us up huh coop and Audrey chugga, the song, the Murshid, a young man in a give years.

B

Oh did oh well, then infra f-feel, my children and the free area became four quadrants at tango, jewelry, so the tongues in size alone free, you know some toys and a guitar is a cloud native heritage, paso de tribunal or closed et vous human and ape who ve got yoga every week. We, oh sorry, God, oh yeah yeah, we know hi, wouldn't tongues inside wouldn't miss a turn. Is a closed, a table, some Dutch chick tipped over by some demon.

B

It's silly human and if we root forget all our giveaways, little research Oh forgive at K or quality situated telco, and if we kill another quick, we are.

B

Google Jung Hyung, Joong, Ki, Joo NATO, you got open stable us all. Free, VM, Cabernet or tumbled had a wound, his son and there freedom, given s, can be close to a or talked a poetess Hungama I'm movin. Mr. isayama sister cachinga you've enjoyed in English Mita that taboo Boone, a vmg, Bonnie Chang, didn't even tell you at the cleaner Pro from there. We any food ruin or underrated.

B

Talk trophy dinner with Sunita between the IT Claudius alone, who scarab it to the years old LCD excellent article Yamaha death functions is an educational unit, so I'm gonna hug, hello, its me that the quality still important alliance and calico we didn't I, you can show so you don't take that she held in Google. Could you go 200 G orchids on a VM given roti pretendin their tongues in size alone, I do cut kubernetes to rural. You hollandaise me up. Sunita coumadin parameter a sick joke. Retro result a yun, jeongsu guru, services, hotel car.

B

It unleash me no Joe not yet it will banjica IT service about India shank of enjoy saying given would they were told it was obvious how much room candlestick with current a hello, couch, Otunga enduring listen turn over designs and language Smitha. So no can't even finish, indicate didn't go to the kitchen cool take Chinese. How long has still not have me alone? Jed I love it. We don't in their intention. Okay, sorry but inevitable.

B

Savior gave our children do Chrissa to Jesus alone, parameter luggage on a VM on a container Palazzo come on a New Orleans gentleman Google, her metal mesh panel Quran yoga Sangre de coeur de baño, gay William Anita continually give ESO Bowden Contino ha Gyung Priam rope, Sharonda, managing a girl geometry um built a container together on Chrome window vessel hijito, kilometers I gotta, see mr. Davis mr. Cairo Metro commonest needa.

B

Kohanga G it'll all turn fearful of chase alone or Borden was a container of endogenous Anita, Ivanova, a merman delgo. If we, my continuous in mrs. Wickham Iran, was Runanga temper, chocolate, hibiscus mr. Putin ultimate, the container of an Eloqua hajiman as a cardinal freedom and we'll cook who INF we database engine to Lhasa, injunction or services panel who ran ops at least we're on track to Calcutta grazie. Mille your journey could run fungus.

B

The monitoring logging on micro service, one came out here in known to imply a low angle mode from ATO stingy, will go frontier on what radio code crown wind agenda. Kohanga G his own fvm, a sudden help, couch, ok, so Chillicothe ten crowns handle well, ok, hunting, misogyny I think she noticed Inca iran-contra Sunita president OpenSocial yep well. Tom now ascertain on kubernetes, open stadium, reso Kuya Manny, no continue, no toga toga boom ha ha and we drank realized.

B

Mira technical resolve, NJ qualities that pure metal container, gorlovka, Dorado, Kalyani, raga italiana, see who will elected a melon Hannah a control logic, OpenStack dude thrown a saw Budhan the focus here, cicada neutron or it's a matter of degree. Sunita is academic achievement or ban orange again, including Mohammed area, potentially permanent or free M window eco, who free M, inevitable urban Iran got three Mugen I drove like a continual closed network consisting seasonal divisions, ruined a Iran container, closet or garage opens tech, neutral tone, Esteban de la cabeza, Budhan tangent at heterozygous, journalist, simcha, Jacobovici.

B

Girl Hezekiah a piece of our signature on John, J, Malan, honking, siren, open, stable us or free, my Bonnie in Africa. The fluid is really particularly similar. Young a saw, Conti nagi, Bonnie Venapro Yokohama on Obon again go to a Paraguayan internship Joe. You know when to could appear on an equality suitable. Oh hey, whoa, ho GAE container, triple lumen, tanida, Kurtzman, bandagi emil, austro-hungarian gusta, an objection gusta to freedom want another iguana, you cappuccino degree. So sorry gotta run, but yes, I could urinate. We all situation is a thank you.

A

Yeah, so thank you, mr. Park, so now I'll talk about some of the networking related requirements from telco and how we were about. We met them. So when it comes to networking, we realize that more is less when we are trying to educator for telco use cases. So most of us are familiar with normal kubernetes networking, which is like a CNI and a single network interface, and things go wonderful and fine when you provide that, but then beyond that, there is a telco networking so where we are supposed to provide multiple network interfaces per pod.

A

Now, why is it needed is because of various reasons like in telco. There is a strict requirement to isolate the user traffic or data plane completely from the control, plane, monitoring or your bookkeeping for your pods, and things like that. So there is no there's, absolutely no room to tolerate any packet interference between them. Some of the protocols which are specific for telco, like SCTP, requires or it becomes much easier if you have multiple interfaces per pod.

A

Now this is one area which have been raised in many kubernetes SIG's and there is an NP WG network, plumbing work group in kubernetes which looked at these and proposed some options going forward, but still in when it comes to concrete implementation, it is still lacking and there are options available like multiple Sienna Jeannie and things like that. So people have gone ahead and try to provide whichever way it works for them.

A

So this is one area and which we also faced while working for telco requirements and we'll see how we were able to meet them. The second one is: we want much more control over network, so the normal kubernetes CNI, abstract out a lot of things. You don't care about what IP your part gets unless you want to go specifically into it from some SSH and it is running some message service.

A

But when it comes to telco, you want to be able to apply a fixed IP for a fixed interface of that pod, even control the MAC address on that, because of various reasons that this becomes easily discoverable by other telco services, and you want to be able to control it using its MAC address if it is like a pure l2 kind of environment.

A

Also, there are requirements where you want to use reuse. Your IPS for various protocols in telco, so in the same IP, can be used in multiple isolated Network for your instances to support that, but your normalcy and I wouldn't be able to provide that you're out of luck. When you try to do that, normal kubernetes, simple networking and then there is a choice needed. So you don't talk about like l2 or l3.

A

Only or things like that, even in cab, dekap is not tolerated and netting doesn't work with lot of telco protocols. So those are all off the table. Some cases you require pure l2, networking, VLAN networks, and things like that. So that is where another challenge comes.

A

When you try to use some of the common sea and ice, so these are some of the networking challenges we faced while trying to make our workloads or on especially with user plane or the data plane on kubernetes kind of, and so let's see what we did about that oh yeah and there are, as I said, more is less so there are still more requirement we should be able to now, as mr.

A

park explained like we have valid use case, where we should be able to talk to many of the install base or the brownfield kind of deployment where it has to talk to the VM environment, which often happens to be OpenStack kind of environment.

A

So we should be able to have some kind of a seamless interoperability with Neutron kind of networking from kubernetes clusters, especially for the data plane, where the traffic need to go seamlessly between the pods and the VMS and some of the typical requirements we faced while working on it is some telco workloads have the requirement to create multicast groups over l2 networking. So now that becomes pretty challenging when you try to think of it.

A

In kubernetes kind of networking you can still do that if you have like in calico where you use l3, so where you can define some network policies, the network policy, the framework which have come in for kubernetes but you're out of luck again when you want to do it for l2 network for the pods. So that was another challenge for us and then, of course, the telco network is all about latency and throughput.

A

So there comes requirement for SR, IV and T PDK, and you use a plain packet processing and things like that and again, these have to be like not just one or two but like multiple interfaces on across different networks on on your cluster for the part, so it becomes pretty complex and challenging when it comes to networking for telco and kubernetes.

A

Ok, so let's see what we were able to do so in our case, what we did is considering these requirements, so we relied on mainly two or three projects of one of them is multi source, as you might be aware, to provide multiple or chaining of Siena's to get multiple interfaces and courier kubernetes, which gives ability to be able to provide neutral networking for your parts from OpenStack cluster, all the way to the kubernetes pods running on kubernetes clusters and again here we had a choice.

A

So if some clusters are created where we had the luxury of using l3 networking or some encapsulation, especially for the kubernetes control network or kubernetes primary network or the CC net, as mr. park told so we had Malta and that we chained calico as the primary CNI and the IPS and ports which are given by calico were available in kubernetes as the resource endpoint IPS. And then we under that Malta's we chained courier kuben.

A

It is for OpenStack kind of networking or where we required some sr IV or DP DP capability, because often it was needed for talking to VMs over a fast path or the later network, in some other deployments, where we didn't have that luxury or to use pure ultra networking like it was already set up. Imagine like OpenStack cluster, which is already running, and now we are supposed to integrate or bring in containers in that environment. So all the networking, the switches and everything are configured with VLAN and all the networking is villain.

A

We can't make changes to that and bring some l3 or ask customers or the set up to not rip that off and put some l3 routing and things in place. So there we didn't, have the choice of bringing in calico or some of these in capita kept kind of like flannel, and things like that. So what we did is include your kubernetes itself. Now there is a option available to create or request multiple networks, and we use that as the primary CNI and that could cater like it can give the primary network.

A

Now it is the neutral network. This container cluster network is the neutral Network, and additional networks are also provisioned by courier kubernetes on the neutral networking. So that is how we were able to achieve that and, while working on that, we figured out that in career kubernetes, some of the capabilities were missing which we needed.

A

So we worked on with the community and developed capability to support multi a V and D PDK, both in bare metal as well as nested using career communities, and all these enhancements have been upstream to the courier kubernetes community, so they're available. Anybody is interested, they can look at it or use it freely, and this is a sample networking which looked in our case. So, as you can see, Malta's is the primary C&I, now visible to kubernetes a control plane and that provisions calico. So that is the first case.

A

What we had calico has the default CNI provider or the networking provider, and that IP or the port is now exposed to the kubernetes control plane as the resource endpoints primary reachable target and then any additional ports, especially the data ports and the user plane. Ports were provisioned through courier kubernetes and using that we were able to create, like multicast groups on across different different VLANs. So as in the diagram, what you see will end one will end.

A

These are actually two multicast groups which were created for these pods and then you can also see courier. There is another network which goes out which is for external networking or talking to anything in VMs or OpenStack networking, and things like that. So that is how we achieve the networking part of it, and then next is performance.

A

So here we needed capability so again, telco networks when they run packet, processing applications, so no amount of jitter or noisy neighbor behavior is tolerated because you have packet drops if you start running into those issues and then to avoid that you need a dedicated course and isolated resources guaranteed resources for your ports for running such applications.

A

And again, you may be aware that the pneuma placement affects a lot of performance that you don't have to go through the qpi interface in in your host across the socket to prevent those overheads based on your PCI device placement and memory available. So here now in kubernetes version 1.16, there is alpha feature no topology manager, which had been very helpful for our use case there. You can create these things through policies.

A

Previous to that we were using cmk Intel cmk, which gives you these constructs, but it like the capability to be aware of the PCA or the other devices or the nodes, but with no topology manager. That is available and all these things you can do that and it's still under development and extensions are possible. So if you're interested, you can give your requirements or discuss with the community on those yeah so moving forward, the last challenge was load balancer, so again, I hand it over to mr. Park to go over those.

B

In the balance of symmetry, wouldn't there get five budget of I mean thought sukham, the orange ovn increment, which make if I wish he could have been there good. So I turn green in there. If we do gain by lying Budokan HTP to Rotom still ahead in that I was spec hojin speaker than now, I seem that that Tom took in the trend right still ahead and go to the ramble. Iran would head Junoon rue de belem Sabah obstinate Rosanna in dinero.

B

Could you go arrest, okay, which Miren to or access to an America, Quattro canti healthy and she assisted Piron on even the Thames Valley run trot welcome to Anita Yoshi Protoculture woman on Lu, the balance of a high up, Santa Croce, absolute balance, a german-jewish degree Puyo dead or eternal rubella salmon do wh a pisser. No one had done cobalt, it interface, really you mean so queer has Jesus Anita. Is that Emerson dream on Louisville result and are not specific and harem and Rubio iPad yoga, wouldn't pick it up?

B

Hardened I may be pardoned, assisted TP, my speed to be wondering tcp/udp. Naturally, resistant dog came up Hadassah. Silly walk: walk crude, the Rosario had the adrenalin Quran Shapiro aqueous Mira. What could mean a woman watch, Haram and boy do I should be a yarmulke and Suchman and directed opioid Tara autumn protocol. She won't have hidden a terrazzo, Hulu double result and a monopod yoga container taro Hodja Balaji Damita, even own research, I did or kitsune those East wish to do so you got zero and then we'll do the balancer row header.

B

Ok, we owe a curio I think in the open, so she okay, no constitutional Union. They load. Balancer logic tell us about our sim de origen, the and Robert Hamme on the OpenSocial gave had she contribution how cool Hancock every level K we get. Smitten.

B

With linguist Anita Cholewa, Simran joy it at all so again, Sugarman oxygen guy.

B

A

So we will be available in case you have any questions or discussions after the session. Also Thank You Sandra from Sumida.