Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2018 (Seattle), 15 Dec 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: Optimizing Kubernetes Networking at Datadog - Laurent Bernaille, Datadog

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Lightning Talk: Optimizing Kubernetes Networking at Datadog - Laurent Bernaille, Datadog

As your applications become composed of more and more microservices, network traffic becomes challenging both in terms of latency and throughput. Datadog applications are network-intensive, so we invested a lot of time in optimizing the network when we started to migrate to Kubernetes to minimize the induced overhead. This talk will start by giving a quick overview of our challenges. It will then explain how we manage the different types of traffic: pod-to-pod, pod-to-service and ingress. For all of those, I will show how our setup evolved and why and what we are looking into for the future. The talk will detail the CNI and ingress plugins we explored and the issues we faced along the way, as well as some contributions we made to these projects.

To learn more: https://sched.co/GrVD

A

Hello, everyone so yeah I'm going to talk about how we optimize it working a dead dog in our communities in Berryman's. So we faced several challenges and Edinburgh. The first one is we're migrating to kubernetes and we need to be able to have a natural that as efficient as the one we had in the past and when we're using VMs those in terms of throughput and latency, and also we're running pretty big clusters so like in thousand two two thousand nodes. So we need solutions to scale through this number of nodes.

A

In addition, we have to several clusters, so we need to make sure that communication between cluster is going to work and we also need to be able to access to the test workloads from central VMs, because this will simplify migrations. So, in order to address this in communities, we do several things. The first one is: we use IP vs4 to proxy and server IP tables. So IPPs is a native load, balancing solution in the kernel and it's much more efficient than IP tables.

A

When you have a lot of services and other endpoints and they move fast, it's still a bit young, but it's been working great for us. In addition, we don't do any bridging on the host, so we only do routing, and this allows us to avoid the cost of using bridging in terms of latency and if your usage, so, of course it requires specific C&I plug-in, but it's also pretty efficient.

A

And finally, we do native pod writing, which means we give every bird an IP, that's rattleball on the network, and this way we have, we don't pay the cost for the overlay network and it also allows for cross cursor communication between. Since all ip's are audible, they can work between clusters. In addition, they can also work from our standard, VMs accessing xx, workouts and finally, to make for much better invested solution. So how did this work? So, if you want to do give all Padre double IP on your network, you have several solutions on premise.

A

What you can do is you can do BGP. But, of course, if you're on a cloud provider, you can do BGP because there's no way to interact with that network controller with with BGP. So the way you can do it is you static routing inside the network of the cloud provider, but this doesn't scale very well, for instance, on AWS you're limited to 50 routes, 50 static routes on a network. So of course you have more than 50 nodes. It's not going to work, so you have solutions for that.

A

Simple plugins solution, a plugin are going to give you a solution for it. So the way that work is, they add additional IPS to interfaces and give this IP to pods, and so your instances of several IPS and some of them are given to pods. So this works pretty fine and you have a single solution and GCP when you can alias whole blocks. To instance, it's an interface instead of eyepiece, but it's basically the same idea.

A

So if you work with cuteness and pressure you're familiar with the challenge of getting traffic inside the cluster, so the width usually works. When you have standard services, you make it a load. Balancer service in what's going to happen, is the controller, is going to drive a load balancer and create it and attach all the instances in your crostata to load, balancer and then traveling is going to be sent to this instance on note balls and not balance to appropriate codes using your proxy.

A

So, as you can imagine, if you have thousands of nodes, the I mean all the thousands of nodes are going to be attached to load, balancer and, and so it's going to be, it's going to be pretty big and complicated, and also what's going to happen is traffic is going to be sound like randomly across the cluster and you're, going to end up having like HTTP queries, for instance, ending up in a very sensitive and a very sensitive node like I know where calf commercials are running, for instance, so you have kind of workaround for this, which is setting the service to external traffic policy.

A

Local, in which case what's going to happen, is your proxy is going to fail health checks for nodes where there is no bud with the actual service running? So, as you can see in on this slide, it's much more efficient because the data path is only going to go to nodes where the service is actually running. However, you still need to manage very big number of instances and health checks, and it cannot scale with thousands of nodes.

A

So in terms of HTTP ingress is it's very similar to load balancer services in terms of design instead of having the ingress control is also going to drive, load balancers, but HTTP one in that case, but the design is an issue that basically the same ones. If you use Nelson controller like nginx, HT proxy routing from proxy to bird is also going to be, is going to be IP based. So it's fine, but you to get traffic to these proxies and some issues as before. So the good thing with is native code.

A

Writing is what you can do, is actually drive a little balancer to route direct traffic to varieties and it's much more efficient. So this solution of their reasons by the word pretty well on those Edda great and GCP, and this is what we're planning to do like everywhere.

A

There's only a few challenges remaining, which are like it's limited in terms of the type of load balance were supported, and it's today limited to HTTP ingresses, so you can do TCP or UDP and hopefully it's going to change in the future, and that is that we're sure I mean. If you have question you can ping me on Twitter or you can come to the data taboos and can have we discuss it together. Thank you.