Cloud Native Computing Foundation EnvoyCon NA 2022, 28 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: Envoy Upstream Enhancements - Alyssa Wilk, Google

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Lightning Talk: Envoy Upstream Enhancements - Alyssa Wilk, Google

Over the last year, Envoy's upstream feature set has grown dramatically due to the productionization of Envoy Mobile. New features such as automatic upstream protocol selection using ALPN, HTTP/3 upstream with seamless TCP fail-over, and Happy Eyeballs support have enriched not just Envoy Mobile, but Envoy as a dynamic forward proxy. This talk will run through these new features, and how they can improve your Envoy deployments.

A

I'm one of the unvoyed senior maintainers I've been working on the envoy project data plane uh for ages. Now, in the last two years, I've been focused on Upstream enhancements, uh largely targeted towards unwind mobile, so I'm leaving the team at Google, a unified client networking team. That's been working heavily with JP and the folks over at Lyft, getting Envoy mobile up and working.

A

So when we joined this project, it was kind of clear to me that the envoy Network stack really had been written for use in data centers, so you had things like having to hard code, your Upstream protocol right and in your data center. You know if your endpoints are speaking hb1 or h2b2, but when you're working on the open internet, that's that's generally, not how things work. Also.

A

You roughly had to hard code the address family, so we did have things like V4 and V6 preferred, but essentially Envoy would resolve at one address and try to use it and again on mobile networks. If that address family wasn't supported you're out of luck, it didn't have connectivity.

A

um This is obviously sub-optimal for Envoy mobile, but this also isn't great again. The unlimable um Upstream code is the same code that Envoy uses. So if you're, using Envoy as a dynamic, Ford proxy say it's an egress Gateway, you may have these same problems and the fixes that we did work both for Upstream Envoy with Dynamic Ford proxy, as well as on y mobile.

A

The first thing we did is ADD alpn support um and that basically gives you we added a new Upstream configuration that lets. You negotiate, whether you use HTTP, 2 or HTTP, one as part of the TLs handshake. So there's no latency penalty right. This is done in line. You know a couple extra bytes on The, Wire um and it'll automatically use the best available protocol.

A

If you're talking to an older endpoint on the internet that doesn't support alpn, it just fails over to using HP one one so again, you're pretty much guaranteed to get your traffic through onward defaults to assuming your endpoint is going to use HTTP 2.. So what does this mean if you're um started NPR Envoy with a cold start? It hasn't talked to this endpoint ever before, and you have a flux of incoming requests. It will try to establish one TCP connection, assuming that those various uh streams can be served over one connection.

A

However, we didn't want a latency penalty if that endpoint supported hb1 right, where you can only serve one request and one connection at a time. So we added what we call. This alternate service cache and the else service cat caches that protocol, so subsequent streams as they come in, will remember that hb11 was used and prefetch one connection per incoming request.

A

This alt service cache is cluster local by default. So again, if you're a cloud provider- and you don't want information shared between different customer clusters- um it's completely isolated, but if you're running this in-house- and you want to share this information so that if you have for some reason, two Dynamic forward proxy clusters- and you want uh to not have to you- know- look these up every time on cold start. You can really easily configure different clusters to share this cache for config, um it's pretty simple.

A

So if you're using modern Envoy cluster configuration, you'd have a section explaining your HP protocol options and you would have had something like explicit, HTTP config, here's, my H2 config, you just replace that with auto config. Optionally. Again, you can add in this alternate protocol cache if you want to share caches across different clusters.

A

The next thing we added was HTTP 3 support as Matt mentioned. This is one of the features that we're most excited for Envoy having going forward and while Envoy has been able to term it h33 speaking, hb3 Upstream is a little bit different. So we added two modes here. One is explicit h33. This is for in data center use when you know that UDP 443 or whatever Port you configure, is going to work on the internet. That's not a good idea to use.

A

So we added Auto HTTP 3 uh support, so that um if your ISP that you're on is blocking UDP 443 right, then Envoy will just do the right thing. So explicit hv3 will just fail if UDP is black hole, but it will guarantee that you use H3 and you don't have to look at your stats and metrics to make sure it's being used when you're on the internet, um Auto HTTP 3 relies on the the standards uh alternate service header, so it won't use HTTP 3 unless your endpoint advertises and supports it.

A

Because at this point it is not the case that most servers on the internet do. However, if H3 is advertised, Envoy will try to use it, um but it will also attempt TCP after a short timeout. So again, if your ISP is blocking it if you're behind a corporate firewall or something else you'll end up using whichever connection is established more quickly for the best available latency.

A

Shortly after we added HTTP 3 support, we added support for zero Roundup handshakes. This is one of the significant signature features of HTTP 3.. Basically, if you've communicated with an endpoint before you cache your credentials and then the next time you talk to that endpoint, you send those credentials and then it can immediately start sending uh get and head requests without waiting for a TCP handshake or a TLS handshake. So this is awesome for latency it'll be used by default for safe methods.

A

But again, if for some reason you don't want to use it, you can turn it off via a per root. Early data policy configuration so hard-coded, H3 config, it's as simple as taking your H2 config Crossing out the 2 and adding three. There is one other difference elsewhere in your cluster. Can big, though, because um we basically have to repair the crypto to do HTTP 3 um logic. Instead of wrapping your crypto config in a TLS transport socket like you would have normally done for encryption, you wrap it in a quick transport socket.

A

But if you look at these two Yellow Boxes you're literally just copy pasting, the config you had into a different wrapper Proto. So it is a pretty easy transition for auto config again it looks a lot like Auto config for ht2.

A

um You just say your hp3 protocol options, but again this time you have to include your alternate protocols cache it's required because Envoy won't work without it. It won't remember that that endpoint supported hb3, so we just failed config on Startup, so you can't make that mistake same thing in your filter chain. You can figure that service cache. So as you get response headers we insert again into that cache so that Envoy will remember what protocol was used.

A

The last school feature we added was happy eyeball, so happy eyeballs is the de facto way to use IPv6 on the internet. It basically attempts connections to Preferred, address families, while trying connection both to ipv4 and ibv6. So from my desktop I did a DNS like over google.com I got a V6 address and a V4 address. The implementation will try the V6 first and then, if no connections established again, my ISP is blocking it for some reason. After a 300 millisecond pause, we'll attempt a connection to ipv4 um and again this is fantastic.

A

It resolved a lot of issues on various carriers and networks um where you get to resolve both addresses, but only one of them works and it's a surprise as to which one will um and then again to use this. All you do is configure Envoy to use DNS lookup family, all so it'll resolve not just one but often multiple, V4 and V6 addresses so also when you're migrating. You know across networks and whatnot. It can be very helpful to try different endpoints, because again, the route to one of them may be blocked.

A

Happy eyeballs configuration again super easy where you would have your DNS lookup family for, like V4, only or V4 preferred you just stick in all and then all the logic gets done under the hood for you.

A

uh In conclusion, we've added a bunch of new, really powerful features to Envoy, and we've done our best to make sure that they have very simple configuration. I want to call out that the team that did all this work is the team that launched hb3 for Google.

A

So we have you know, years and years of experience, tuning HTTP 3 tuning timeouts tuning these failovers to be optimal for both Google's web search, which we really deeply care about, and it's very, very latency sensitive along with YouTube right this, this long-haul streaming type traffic and making sure that we have kind of optimal throughput and bandwidth and quality experience for both. You know kind of your fast traffic and your long-haul traffic.

A

So we have really high confidence that kind of the default configuration is going to work well on almost all networks and almost all conditions and again you know we have knobs. So if you're in some weird special Corner case, um you know you can tweak it accordingly.

A

uh As usual, there's a bunch of examples of this and documentation on Envoy proxy. I o, and if you have questions and concerns we didn't get to today, we're we're all on slack uh so feel free to pick us.

A

Do we have any questions.

B

There any more low hanging fruit. We look forward to tackling.

A

Yeah, so we've got a list of other things, um Envoy. What we don't yet support, and we want to is connection migration for HTTP 3., so essentially uh JP mentioned earlier on that, especially for Lyft they're, doing a lot of like walking out of houses or buildings. You know and you're switching from Wi-Fi, um so we we've had good luck in Google for actually migrating. Those connections like from your Wi-Fi to your cellular interface. We haven't yet done that plumbing for on my mobile and it is on a roadmap.

A

The other one we want to do is better handling for hb11. It hasn't been a high priority for us or Lyft, because we all use H2 and H3, but it is really important right to to handle like Legacy cases on the internet, and so there's things that um kind of best in practice for Network stack like when you get one request to give an endpoint, and you know you're using HTTP one.

A

It makes sense to prefetch a number where you can hard code, that number of connections so like browsers, if you talk to google.com, will immediately establish six connections because you're like if you have one request, you're likely to have more following on so there are a couple things like that, and we've got a list of improvements that we're hoping to do in Q4 and then going forward any other questions cool enough to the next speaker.

A