Cloud Native Computing Foundation EnvoyCon 2020 - Virtual, 12 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Multiplex tcp requests through Envoy HTTP/2 stack - Yuchen Dai

Description

Multiplex tcp requests through Envoy HTTP/2 stack - Yuchen Dai

This talk will go over the recent update of HTTP/2 CONNECT support in Envoy. Envoy not only can terminate or proxy an H2 CONNECT, but also proxy the raw tcp plain text request in establish H2 CONNECT. In this talk, Yuchen will also go through the on going efforts to optimize the CONNECT dispatch. With these efforts, Yuchen will demonstrate istio sidecar proxy(which is literally Envoy) tunnels http requests and raw tcp requests into HTTP2, multiplexed in one TLS/TCP connection as if the request is directly established by application.

A

Hi everyone uh today, I'm going to talk about multiplex tcp over and over hdb2 stack a little bit about me. uh I'm my name is I'm a software engineer at google. Since 2019 uh I mean the usq networking team almost work on that data play, and this is outline of today's topic. uh I've introduced the background, the problems as a solutions and the real-world usage.

A

This is the traditional uh scenario of the service mesh. So, on the left side we will have a tcp, client and away as this sidecar of the client would run the tcp proxy network, plugin released by a stream to from tcp client to the other uh um way as the server-side sidecar, and that's how I would also use tcp proxy to relate bytes to the tcp server and uh the variation today is about hdb2 uh in the stack.

A

So the changes I marked here use the red color instead of relaying the by stream from this reclined to the upstream uh onward server side arm way, the live stream would be translated into http2 client, a connect method with the data framing encapsulating the bat stream that beef, of course, at the server side, down way. uh The http connection manager has a network network filter, would terminate the connect request and extract the byte stream and relay to the ttp server, so the problem uh or what we can benefit from the variated structure.

A

I will explain the further slides so uh in this slide on the top corner. uh There's a thumbnail of the whole structure and on this side the sun weighs the tcp client side and way which is responsible to relay the black string to the upstream h2 request. So this is done by our tcp proxy, but with the h2 extension at the tcp connection pro, which is recently developed by alisa.

A

Thank you alisa, and this h2 codec in the tcp connection pool would custom magic translate from by stream to http to connect three, and this slide is about the server side and way so this server side, I would use uh http connection manager, uh live in the tunnel listener on port 80, which is a common http port and the specialized configuration is that in the route configuration you can use the connect config field to declare that, uh instead of relay the http to connect method, please use extract data from the connect stream and relay to upstream, and in my specialized design I would introduce another tcp proxy listener, which is similar to the traditional architecture.

A

This tcp proxy uh network filter would do byte two byte uh trans uh relay. uh You may wonder why we are why I'm introducing a duplicate listener. uh The idea is uh maybe quite naive, because in service mesh, especially in skill at the server side, we already invest a lot, including error for rbac network user, including the access log, the monitoring pipeline, which is a promise to the developer and these two users.

A

So we don't want to mutate the structure too huge to break the existing structure and this side give an introduction on the necessary config or the component introduced. In this scenario, I can explain in the further slide so what we can gain from this complex structure, so we can obtain the risk we can get the functionality of metadata exchange between the two and voice so because the to one wise is using is connected with hdb2 connect stream.

A

So then we can use the h2 header uh to encode our metadata in this page uh at demonstrate as x, full client id, which is my fake client, node id and server, would respond uh whatever you like, but in this example, is a server id and what we can obtain.

A

Beyond the traditional tcp proxy connected scenario, we can use the hdb2 uh http filter, which is far more powerful than the tcp proxy routing, so we can match the headers we provided in the metadata to decide which upstream endpoint we are we the server side hd uh upstream we would redirect to and uh we can obtain the low cost handshake uh in the service.

A

Mesh scenario are the clients and wei and server and well mostly, would be connected with trs hand, connection, and everybody knows that drs handshake is inexpensive in terms of the latency and the cpu cycle. What is even worse is that this traditional tcp proxy uses tcp connection pro, but the connection itself is not reused. So for each incoming connection, the connection pool would establish a new connection, tcp connection to the upstream and introduce another handshake, but with http tools, stacked to incoming tcp connection can be encapsulated in the same upstream tcp connection and the boundaries.

A

The data frames are the hv2 streams uh between the two away, so you handshake once and you use the trs connection for many many tcp uh connections between the client and server, and you may wonder uh with this uh actual layers, would it be expensive?

A

Yes, it is uh without optimization. uh There are many copies I introduced uh between the two listeners at the server side.

A

I'm way so each we basically create two extra connections and uh the kernel space would to maintain two socket buffers and connection user space connection copy to circuit buffer in the kernel kernel to copy between two socket buffer and socket buffer would copy to the user connection again, but remember the scenario that that's two listeners sit in the same onward process, so I introduce a concept of internal client connection internal listener and a specialized io socket implementation to eliminate the two socket buffer with the two connections extra connections. So the data is not copied.

A

Instead, we use a way building buffer to move chunk of data uh in the pipeline in runway components. So not many data are copied, so the real world usage will be in ucl one night which will be released in november 2020, and I see you don't have to introduce the whole stack in your system.

A

You can use the connection internal connection uh with very little config change uh and gain the change uh listener. Regardless is chained, tcp, proxy to http connection manager. I also use connection tcp to tcp or hdb2 to hdb2 or other protocols, so the code is still up streaming. You will see that in.

A

Along with my upstream so uh this page, I provide some links uh in that to the rfc of hdb or and the life of amway request and the building component in our way to support the full picture.

A

And thank you for your time.