Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2017 (Austin), 15 Dec 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Building an Edge Computing Platform for Network Services Using Cloud Native Technology [I]

Description

Building an Edge Computing Platform for Network Services Using Cloud Native Technology [I] - Stephen Wong & Vikram Dham, Huawei Technologies, Inc.

Edge computing have become increasingly important due to the demands of latency sensitive applications and explosion of data from end user devices in cases such as Internet of Things (IoT). One common intelligent edge deployment is the buildout of mini data centers on network edge that are centrally managed and operated by the cloud. Unlike traditional data centers, these mini data centers are constrained by limited resources and minimal operational supervision, and as such they impose challenges on traditional data center infrastructure including network services, here defined as L3-7 network services such as packet gateway and application firewall. These services usually are implemented with the need for heavy manual configurations and complex provisioning, which are particularly ill-fitted to deploy at the edge.

In this session we will discuss how we built a new edge computing platform for network services that can achieve auto provisioning, dynamic service deployments and updates, and high resiliency. By running componentized network services in containers orchestrated by Kubernetes, and utilizing projects such as gRPC, linkerd, and fluentd, as well as making use of cloud native related projects including etcd and IOvisor, this platform essentially treats network services as cloud native applications, and thereby able to achieve the associated benefits. We will show a demo of the platform as part of the presentation.

About Stephen Wong
Stephen Wong has had 20 years of software development experience in the networking industry. Currently he is a software architect at FutureWei Technologies, the US Research Center of Huawei Technologies. His focus at FutureWei is to advance the field of Network Function Virtualization (NFV) via contributing to the OPNFV project. He is also a long time contributor for various projects under both OpenStack and OpenDaylight —- having co-founded the OpenStack Tacker project (network service orchestration), and was core reviewer for OpenStack networking-sfc (Neutron service function chaining), OpenStack Tacker, and both the OpenStack and OpenDaylight Group-based Policy (network policy framework) projects.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Welcome to building an etch computing platform for network services using cloud native technologies with a title itself being such a mouthful, I guess we'll just get right to it, so these are genders. So we will first look at the problems that we actually looked at first. The framework that we came up with to address that problem, the demo, which is actually based a description of the doubt, the framework also- and what are we going to do after that?

A

So who are we? We are a team of OPI Navi contributors from Huawei USA, actually future way. What what is OPI Navy? Who PNV is a open source project specializing in integrating a bunch of upstream projects and to address various different energy use cases? What's nav nav is network function.

A

Virtualization is basically the idea that the majority of your network functions that are currently deployed as a vertically integrated network appliances can be provided instead by software running on commodity servers and each of these network functions, we would like to call it virtual network functions like a firewall, for example, and that is a term that I'll use well, so that's why I? Actually kind of put it out there for that team that just mentioned the OPI nificant reverse team in wall way.

A

For a few of us for the last several months we've been focusing on one out, we can build vnf, cisco native applications and the obvious benefits. Are there I, don't want to insult a bill? How native applications given the venue and if you are able to build a car natively in half I, would we would actually we actually think that is actually very suitable for the edge network use cases.

A

The nature of elasticity for the cloud native applications allows you to optimally utilize this resources, so that's actually for an edge case. That's actually kind of good, because the constraint on resources are much greater. The telecommunication industry, particularly edge networks, comes in all shapes and sizes. It goes from something like tens or even hundreds of servers, all the way down to maybe your home gateway that your Comcast give you. Needless to say, if you go further and further towards the edge, the maintenance cost is actually higher.

A

So the the nature of Wistar millions in fault, tolerance is a great benefit for the edge case, usually for for etch notes there, the imagers have been pushed by central management system and for and all the Matson before, I read it somewhere that if you push in last patches or something depends on the action or itself, you can have anywhere from 5% to maybe even 30% ferry rates.

A

On the push in this particular case, if you're, how native apps, you can definitely create something like a sandbox after you run your CI on the cloud, you can still run a validation on on the box itself to see what actually runs, and if you really don't like it, you can have intelligence on the edge notes to actually roll back, and on the other hand, if you really like it on one site, you can replicate and repeat on various different sites moving forward. So with all this great benefits on building vnfs cloud native applications.

A

Why heavenly? Well? Actually the big reason is because there actually a significant amount of gaps to get there. There are some that would take a very long time for a lot of people to do, which are things like security issues, performance requirements, latency requirements, throughput requirements or compliance requirements. All those things would take a lot of efforts for a lot of people to get it going for this team, particularly we're basically looking at two functional gaps that are somewhat obvious.

A

Actually, the cloud native application most common, most of the common cloud native applications, which are mostly web back-end services, work, web service, backends, usually expose an endpoint, allows the clients to send request to an end points and then from there they basically just go through the application flow. There is a very large subset of vnfs that are not exposing any end points to the clients, so we call them transparent proxies. They basically intercept the packets and start actually processing it.

A

So we need so commonly refer to that as a service insertion problem, and we need some mechanisms to basically redirect traffic's and that's only one one type of things.

A

You also need to allow them to listen to to traffic and if they listen and analyze for a while, you want to give them a programmable that they can basically say all I want, deny it or you want to rate limits, that traffic flow and finally, and most actually kind of challenging me is you have to allow vnfs that reported to your cloud native applications to be able to bring their own data path?

A

You know I'll actually talk about that later, and the second obvious gap that we're looking at is what we call VN FFG is an N. Every thing for F G stands for forwarding graft is an August raishin function where the operators can specify a chain of vnfs where the traffic would actually traverse through, and then the realization of that desired is actually being carried over by the data path.

A

If you converted the vnf into a cloud natively in half most likely instead of having I, would imagine, I would hope anyway, that you don't do a lot of data path stitching, but instead is actually creating a service graph by just stitching all the, but by making the API it's calling each other between micro-services.

A

Now, the really big difference here is, instead of the applications defining what the service is you're going to subscribe to, it's going to be operator defined, so those two are the gaps that we are going to address and the framework is supposed to address both gaps before we go into the framework. These are the terminologies that I'll be using describing frameworks, cloud native network functions, which is basically what I've been saying: the cloud native vnfs.

A

But then someone told me that if you're, if you're, networking guy you're a chrome goes beyond beyond four letters, you you panic so I go into three just CNF. This is something that is defined by the operator and it's made of a list of micro services. So what is an operator? An operator is the guide that actually defines it deploys.

A

That's those CNF for the clients to subscribe to, willingly or unwillingly, which is in one case, maybe for enterprise case- maybe the IT administrators for you for some your mobile case, maybe AT&T and Verizon, or it could be cloud providers on some side to micro services are basically just micro services. It's not it's nothing! Special! It's a is something that consumes some computer resources and expose a set of API and in in CNF.

A

We kind of look at that as like a LEGO pieces that we constructus II enough and the clients are just the people that would initiate traffic's that go through node. So this is the diagram of the architecture, the operator thinking the mouse is hearing or not the operator. It goes into the central management and defines the CNF. That's the rest, call that goes into the edge, so they manage many of these edge notes. Supposedly, the rest call goes down to the agent. The edge agents would request the kubernetes api server to create deployments.

A

So, let's say in this case you define CNF one as making up of micro, surface 1, 2 & 3, even 4. They would create those paths and then the diagram here for what what extractor in the pod I'll describe that later so kind of remember this, the pod at least contains two containers.

A

One is the micro service, the other one is an event handler and to prowl traffic's MS one in this case with all the arrows MS 1 is the one that actually is were the the the traffic beginning point is there's a data path manager that will be listening to kubernetes api server and they would program a data path and that really good-looking little unicorn icon. There is actually BPF for those who don't know, I am NOT going to describe a lot of EPF.

A

Probably basically just tell you why we choose the EBP FS the data path engine, but I'll talk about that later. So this is a completely event driven model. Basically, what we do is we are creating an event hand or containers as a sidecar.

A

Every single pot that runs to micro servers, the micro service themselves would expose a set of events, and the operators on on the on a cloud side would increment the handlers. For those events- and they don't have to with that's the default, they will be a default that that's nothing, but then, if they do, what they would do actually is. We are the framework will give them a set of deep SDKs and the sta calls for them are stubs internally.

A

They're gonna be translated into G RPC calls and that's how you actually fetch things together on the program over there. A path front. Look at number two. The rules format is something like this: it's the optional IP port numbers. You can choose the actions redirect or copy to a list of labels and, as I said before, we actually utilize IPPF, more specifically our Vice at BCC to low vpf code.

A

One of the there's two major reasons you that the first one, the first major reason is, as I said, we want to allow people to bring your own data path and literally I can actually allow you to bring your own data path.

A

If you know how to write BPF code, not the bike code, the C code, so you can write on C codes based on what PPF requirements are and I can load it inside your your parts, ingress and egress port, so that allows vnfs the micro services to bring in their own data path, because obviously, I cannot design a network data path that would actually satisfy all the VNS, that's just impossible.

A

So to better describe. This is actually going through a demo. So in this demo we are going to have the operator using the portal to create a CNF, so this particular CNF I would call it at security.

A

Even when you just do a prototype, you kind of realize that for each security system, you're more, protecting the clients rather than the applications and servers, so you actually see us processing a lot more on the HTTP responses than the requests.

A

So when you define a CNF by including a multiple microservices which could be in this case, there will be actually proxy a policy manager and a content in spec engine, and each of this microservices actually has a set of events and the operators would have to implement or actually they can choose to implement these events and customize them through the SDKs that we provide them.

A

So in this case, our SDKs all in Python, and that there is a DPM code also expose for you to the dev health manager code API for you to start routing the initial traffic to the extra proxy and then you deploy it on the edge node and then those nodes get spawned into the introduce those pots get spawned in the notes and then clients later on cents.

A

It should be requests, and then you see that they actually go through the call graph that we activ the operator defined and then at some point during the demo, we're gonna, add and others micro-service into the CNF antivirus, which actually is a very interesting use case for edge.

A

Because if you think about IOT use cases, perhaps the end-user devices may not have enough power to run endpoint Security's, so you may actually want to run em for securities directly on to the afternoon, but we add antivirus and then we are chaining that, with an anti-virus specific call to create a scene to make the scientific standard to add services to CNF, basically and then we'll see that it actually goes through antivirus and the operators now.

A

So this is third step wants to narrow what needs to be scanned, so for them they have the ability to see because this actually comes through scanned, xtml files. So, instead of that, they may want to scan it only when you have you embedded inside executable. So in this case you can actually do a logic to check whether or not you have an executable embedded and then you only run antivirus when that happens. So here's the demo I.

A

Am this video recorded not running, live, not testing a demo guard here, so initially you're creating a CNF. So we have a really nice UI here. So for a guy one of my team mates actually build this UI. So thanks to him and from the from there, you picked a set of events. So the first event you actually handle is the deploy event. The deploy events, because of time reasons we already populate a code, the defaults, but we'll change it.

A

Actually as we go so the important thing is here so in here you specify the set of micro services that you're going to add into this CNF, and then each of them is basically a CO, hopefully is readable here so you're doing proxy start policy manager that start and content inspector start that basically creates three pots into this CNF and then later on. Do you specify using a DPM call? What is the rules that you're going to reroute? What traffic to this particular CNF, so actions redirect their labels to label?

A

And then the port number is twenty, eight thousand one they just made up a port and- and then you have to tell the deployment agent that this is done with deployment and then later on. As you can see, as I said earlier, we are doing more of a use of protection, so the request actually does nothing. They should be requests as an event. It does nothing. The response is the beginning of the chain. So so, after response is received from the server side back to refine, the the first thing we'll go through.

A

Is the policy managers to do a policy check and then and then we'll change that to a Content inspect and then content inspect as the end of the change. So we do nothing so and then we deploy it. So this is a recruit demo. The way we show you that it works this by actually showing you trace logs. So hopefully you guys trust me that I'm not just sending a bunch of prin apps to a UH to a web page. This is actually something happening underneath.

A

So now you see everything get deployed. Those three microservices get deployed, and you can see here there's a deploy agent that we dynamically load the pot that we line reload and then any when you set done they're actually queue itself. So they were the one responsible to execute a deploy, handle code and.

A

So all the parts are up so now we can use traffic. We generate traffic using curl, not because we are giving more crude demos is that curl actually works better than the browser in this case, because curl only do a one actually request and run response back, so you'll be able to see the trace much better than using a browser which is continuously fetching.

A

So in this case you are seeing we're going to girl cool calm and you see that goes through actually proxy and when a response happens, it goes through policy manager and then content inspector on line nine and then the response are all coming back to to the to the HTV to the proxy. So they send it back to the client. So now let's say we want to add antivirus, so you you, you go back to the same CNF and modify your deploy hand or by adding this SDK.

A

So this is a good simulation of how you write something based on an SDK. We do it on web, and so now you want antivirus to start, and you want to change it at the end of the current service chain. So that would be content inspect and in there you make a SDK call equals to basically an SDK that antivirus that scan and voila you clear the trace first and then you deploy it.

A

From there we actually do a complete update of deployments so those things that has image changes, who actually get terminated and respond as pods and now we see the parts coming up, antivirus is up and running, so we are using trafficking and we're sending to google.com again and now you can see that it goes through antivirus, embedded inside content, inspect and policy manager all the way down to sending a response back. So now, let's say you, you figure, you don't want to scan. It indiscriminately scan all the HTML pages.

A

You only want to scan them for if you embed a java script executable inside your HTML, so you can change that logic by first removing the scan the entire scan antivirus can call from content inspect and, let's say, added into policy manager. So you can wait until you get the content inspect to show you what the content actually contains. So in this case, javascript is a javascript as a as a key to dictionary and you say, get javascript. This is greater than 0. Then we only when that happens. We do a count.

A

We do a anti IRIScan. I mean I I type this, which is very slow, as you can see,.

A

And then deploy.

A

So now the co logic, the new order, all the parts are now restarted with the same there's a new sidecar, so they build tack and push to a registry, and then kubernetes will fetch the images from from the latest build and now they're up, and if you do Google comm, which actually contains no no embedded JavaScript zero in the line ten there, it actually doesn't go through antivirus, as you can see the line, content inspected and in policy manager right away, and we are fortunate to find that there's still some website that has embedded Java instead of continuously fetching Java scripts so and that would be cnn.com in line thirteen.

A

We fetch cnn.com and you as you can see and line eighteen. We are doing antivirus, so the logic get executed.

A

These are actually two call floats of what happens on the three scenarios that I'll show you I, don't think I would actually get into that through too much. But the important thing is actually, on the left hand, side, you can see the cat scan number of calls and that's actually what the operators will define when they define those calls there. We allow them to actually change things together, so in essence, we allow people to define the service graft. I will blow the slides after their presentation.

A

So if you want to read that go ahead on schedule or condition paper read it to PDF format of it, so so this is what we have right now and then moving forward. There are a couple things that we actually want to do. Also one of these is actually the the data collection aspects. This has been in use at the edge case. We want to definitely allow them to collect a rich set of data.

A

That's actually the second major reason why we choose a EBP, f e vp f is actually a tracing tools by nature, particularly for iovis or BCC. It comes with a bunch of sample EPF code that you can use for Colonel, tracing and I think because we're building this for any generic edge cases. So we we are not assuming you're going to run some analytic engines under the edge, but the very minimum we would want data correlations to happen on the edge.

A

So we use the cookie field, which is the in disk in current case, we're using the five tuples, the source, IP source port and the protocol ID, something that both the data path and the use of space can actually generate, and they would all key that as one of their their their major keys on the tables. So if we want to fetch data once there once they become available all the trace data and we have Rattus working terms, which also use that as a key, you can actually fetch all data using the same cookie key.

A

It's a string actually in the containers and it's in this binary and no indeed EPF you can use the same. You can use the key to actually fetch all the data corresponding data and send it to cloud so. Data collection is something we have considered.

A

Someone would probably realize that I talked about your usability on the first slide. We actually kind of violated that policies by sending the event handlers to bcnf specific, the event, handlers actually loads, the CFC NF specific code that defined by operators and and then run a society are next to a micro service. So in essence, your micro service cannot be reused, which is a pretty bad thing so moving forward.

A

We are thinking of using a multiple sigh car type architecture, so in essence the each of them would be running, CNS, CNF, specific event, handlers and how to distinguish how to go to each, which particular sidecar is actually based on the cookie again and and how we map that most likely I would imagine is the client. So you have to identify the clients to client maps to a particular CNF, and then you map the sessions back to the client.

A

So now you're, actually TB requests have to do something not not by the operators but by us inside a framework and then that information we get propagated throughout all the event handlers for all the parts, and we may actually make that programmable, not sure what the use case is yet for vnf cases, we will be bad if we can support legacy vnf. The biggest reason is there are many many decades of vnf that are not well NF anyway.

A

Now, where services that are battle tested fully loaded and deployed for a very long time, so you can't just tell people to write everything, just because it's how native so to really think about that. How do you support legacy? Bnf? Our demands for a rien F micro service is two things.

A

The framework asks for them to generate events when something happens, and it has a JRPG request, handling interface, so obviously legacy vnf doesn't have that, but luckily, for us actually kubernetes has a really well-known design patterns for our containers, pod relationships that would actually choice off this problem. It's called the adapter pattern and I actually did that for the antivirus pile micro service, the antivirus, our user cry, maybe the clan, maybe daemon actually runs in a container. The I use, Python soap, I crime, tea runs STI once on the adapter has an interface to cry.

A

Maybe demon and the adapter itself is the G RPC server that actually runs. I use your mq for event handling, so I, actually invention that sorry, there's a once a 0 mq event loop, so you connect I hope to make that into a framework, so people can easily just port as long as your legacy, vnfs runs is in one or more Linux processes.

A

You can pretty much use similar framework to support the legacy vnfs into this particular framework, this cloud native framework and finally, if you did all this, this is relatively it's not how you present cloud native. It's definitely no stainless or no streaming locks all that stuff, but it is actually good enough to take advantage of a lot of cognitive computing ecosystem tool sets. One of this is, if you want to scale up, you can the surface match.

A

Obviously, because now I'm converting the entire interface to everything to to auto micro services through G opposite, so it naturally works with envoi, nginx or linker D and obviously, whatever this deal and I particularly I love the service graph tools on this deal. So you can, you can run traffic for a while, and then it actually generates a graphical look of what the what the service graph looks like, given that this is operator defined service graft, it's kind of good to use that as a visibility and validation tool.

A

As you see from the demo, I basically show you a demo by showing your results of tracing, so tracing is important and right now we are pretty ad hoc or now tracing just kind of rush. The demo for today in the future, we obviously want to use open tracing API and probably Jaeger has a tracer. And finally, if you are cloud native applications running on kubernetes, you will be able to leverage all this CI see. The solutions like spinnaker should have tons of them actually really and maybe try. Barry eight I think that's another one.

A

So for informations, if you like what I just said, we have a shameless plug, we have a project called clover under Opie nav there, the charter of the project was approved about a month and I have to go, and the Charter is really specifically investigating how to build vnf Escala native applications. So this is one way of filling the gaps, there'll be more gaps found and then there'll be more prototypes being built as part of this project. So you can go into this flag channel and you can also email me. That's the last resort.

A

Thank you. I said.

A

Let me see well, I actually have about five ten minutes, any questions or.

A

A

All right, thank you.