Rust Programming Language RustFest Paris 2018, 28 May 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RustFest Paris 2018 - Scalable Networking with Rust by Tigran Bayburtsyan

Description

Tigran Bayburtsyan knows how to scale
a network service without fail
using #rustlang, MIO,
and a threadpool to go
so fast all competitors pale

Real-time networking applications becoming more popular, but building backend systems is challenging in terms of memory and cpu efficiency. This is a story about how at TreeScale (github.com/treescale) we got 10X+ memory and cpu efficiency using Rust MIO as a main network TCP/UNIX handling system with thread pools.

(Limerick by @llogiq)

https://paris.rustfest.eu/sessions/scalable-networking-with-rust

A

Teagan by Bucheon knows how to scale a network service without fail using rust, mio and a threat pool to go so fast. All competitors pale.

B

Thank thanks for great introduction, actually I'm Tigran for the past, like eight years I've been doing system engineering, mostly helping companies to optimize their cloud environments and especially Network heavy applications, and you will be amazed how much you can save as a company. Just optimizing, your network stack and it's it's just saving a lot of your resources for cloud computing, so I'm typing many programming languages per day. It's four or five.

B

That's my daily work and rust is not the major one, but it's something that like I, enjoy even doing like after work, and this project actually started the project like to feed my interest and I'm doing a lot of motorcycling adventure, writing and also skiing. That's a lot yeah, so few words about rescale. We not going to dive in too deep how it actually behaved.

B

So in a few words, it's just a scalable pop subsystem where you have the entire event distribution system without any central point and without any failure it just routing for the events. First implementation was on goal, obviously, because it's it's just easy, it's easy to code, but after running it on a production with very heavy scale, it turns out that goes. Garbage collection messing a lot with like memory, deallocation and heavy traffic, so second implementation was on C++.

B

But after having a week working example on the production, we got a segfault error and we even didn't made any debugging. So we just started to write on rust because the yeah.

B

At that time we had some experiments, but we thought okay, maybe rust is too early, but then we actually saw that it's a pretty mature language to use and for our specific needs we made on C++ the plays loop library called base loop, which is actually almost the same thing as a my own, but with less features. The specific need features that we need it for that time and after we saw that mio like completes all the features that we need. We just starting writing on that and actually writing to rust.

B

We canvassed from mio, because, if rust community didn't had that library, we wouldn't start doing the rust and the base usage of mio here is actually the example code. It's very simplified, basically what it does. It just makes event loop around the existing operating system, a pole or K event based on the operating system, and it just registers specific sockets to receive events and make some data processing with them each single event, as you can see, there's a like infinite loop, which contains your entire logic and it's operates.

B

Anterior, your application is life, so it based principle of event loop and it's single threaded. So if you can imagine the application which works with the event loop, it's something like this. You have the infinite loop, which produced specific actions based on kernel events, then using a thread pool just to optimize your processes, you, the threat to land, picking up some pull some thread inside that tool to perform some action and return back to your event loop and continue doing your process.

B

This is like the base for almost any kind of single threaded application, but we actually faced some issues with this especially performance issues, because, like first point of three scale was to scale, then it should be like super heavy network application so having working it on all CPUs without like a single thread. That's like pretty important part and rust helped us to develop a technique with mio, which is looking something like this. Let me this works. Okay, so which looks something like this.

B

So basically, we have instead of thread pool we have threads, which is the same amount of threads as you have a CPU cores and each thread running single threaded loop, which we called IO loop, which means that we have a main thread which receives the connection, makes some authentication stuff the first initial handshake and then passes that socket to IO loop, which then performs the specific IO operation with its own action and tasks receiver mechanism inside that single thread.

B

So this helps to handle a lot more connections that we could in a previous example, if, when we have only one event loop- and in this use case, we specifically could interact through event aisle loop, if someone stuck on one task, the main loop detects that and catches that action and just performing that later on when other threads are free.

B

So basically we have some control system over multiple I/o loops, and this is mainly we got this performance because of the rust thread. Saving thread safe fitting model- and this is like the entire process- works completely non-blocking. So everything is written with the thread channels, which is pretty awesome performance in terms of real code execution. So here is like the example of the main main handler loop.

B

Basically it whenever you got some TCP socket to accept or if it's a client socket you're, basically making some validation around I know, maybe certificate checking or the data validation. Whenever you can perform that and then basically, what you are doing is using Mio principle. You are the registering that TCP socket from current loop and passing that using the rust channels to one of the threads that operating another loop. So that's the main principle for transferring and after this transfer operation, this main handler loop, don't know anything about that.

B

Tcp socket it goes away from him and the other processing and input/output operations is just on that thread.

B

So this is a little bit like more code, but the concept is that the ill handler loop receives that and just registers that inside his Pole inside his event loop. So that way we can just transfer connections between multiple event loops and have operational like completely I, think principle without any blocking data. So that's the main benefits and the optimizations we had customer, which is operating like few petabytes of networked data transfer, especially images per day, and they have got like from 6 to 10 times optimization in terms of memory.

B

After deploying this, this principle versus go so, and the main benefit from us is that using this technique, we are able to like scale the code, because rust itself is checking the safety and if you are, for example, hiring a new developer. He don't know this trading model and he writes some component around that it just rust, prevents some memory leak between passing some data between channels and threads and, of course, using multiple cores as a multiple event loops, not only tasks.

B

We got really huge benefits of network bandwidth, because now we are able to make input/output more aggressively with using multiple threads and yeah. That's it the main benefit. So in terms of closures, this is the are something that we we have a lot in our code base.

B

So the principle is that when we need to implement or execute some tasks, we are not passing data and we are passing the closer which contains the logic itself, with the data in it, which helps just to make some kind of a generic task, executional threads or inside the thread tool which is like generic, and you can pass any kind of operation which makes CPU execution or other stuff with the data itself in it.

B

And it's just helped us a lot to scale the code base from like having multiple traits into multiple traits, even without like changing anything inside the base code. So this is the main feature that we liked a lot after like moving away to rust in terms of infrastructure, so we partially open source so based technology itself. Three scaled is open source, but it's some kind of a demo.

B

We mainly showing that to customers or they're trying out on their own, but the base code is written in rust, but supportive technologies, I, will say the API endpoint is not rust because mainly the lack of hiring mainly so that's the main issue of getting started with rust.

B

Yeah I guess that's it so I prepared a very short talk, so yeah and if you have any questions, please.

C

So we have time for questions, but we need to figure out how to start these.

C

Tester, oh, it's the red puppy. Okay. Thank you. Thank.

D

You hello, thank you. I was wondering if I played with a 0m q and a is it possible to do kind of the same thing of connecting a Rose application with a Python application, not GS yeah.

B

Yeah so the communication protocol itself, the custom binary protocol, but we have like API, integrations with high-level application, including the WebSockets, so one of our clients is using inside the mobile lab. So basically we compiled our rust SDK inside the mobile app and providing them this real-time networking feature for their mobile app, so basically it just integratable and thanks to mio we can just integrate that to any any kind of platform.

C

E

Other questions, thanks for the talk actually I, have a question on the data. How much copying of the data is actually happening? When you have a movin' closures, I mean: do you have like multiple actions which should be executed on the same data? Oh yeah,.

B

We we have multiple actions, but data itself is not copying. We have basically the byte array and every time when we are doing something we are actually doing by reference to that array and in terms of protocol. We are just appending, like 60 byte to that original data and we're not doing any data manipulation to like customers original data. Whenever you have some API endpoint and transferring data through three scale.

B

We are not touching that we are only working with our sixty byte header, which is our main protocol stuff, and that header is all all the time first and they code it using bite reference without caulking.

B

So we are using some little pieces of unsafe code just to give that like manipulation more easily, because we have some big endian and little engine Indian conversed conversation between just to figure out the lengths of the bytes some in some places. But it's some that protocol came from C++ code and we didn't change that. We just made the unsafe rest code. Okay,.

E

F

Hello, you said you're using mio yeah.

B

F

It make sense in your case, or is it possible to use Tokyo so.

B

During that time, when we started writing the Tokyo, wasn't that stable I mean it's not to release released yet, but then, when it's released with started to think about moving, but it was too much change because the principle of features inside the Tokyo is not relevant for our case, because basically we're transferring TCP sockets between multiple threads and actually inside the github issues for mio I raise the question: is it really thread safe to pass TCP connection between multiple threads and Alex actually replied that I generally?

B

No, but if you, if you are using in linux, based environment or unix, so basically it's not. It wouldn't work on windows, but for windows we did don't making like thread TCP socket thread passing. We are using different technique, but we all have only one customer which requires a Windows. So it's not to deal.

C

There was another question over there: yeah.

G

So you mentioned that you are taking the existing or.

F

G

Existing messages that you receive and and our prepending some stuff at the beginning. How do you make sure that you have room to do that or are these like fixed-length header for the dynamic? And how do you make sure that when you actually read the data in there's enough room before the data that you're getting in order to place your headers yeah.

B

So if you can imagine the white flow, we're basically getting your data and make making sure that we have a proper length because of the like nature of TCP. We know that if the airflow ends with some specific point, then your data- that's how it is. We basically measuring the length of bytes and putting that with the begin. The end that you have this amount of length, it's a four byte integer for us.

B

So basically, whenever our another node reading your data, it tries to find first first four bytes to decode and understand how many lengths length he needs to accept from another node. So that's how we transferring data and making sure that there is no data loss, we're basically transferring the length as a first four bytes.

A

And do you play any you integration from stock before so for weather,.

B

No, so we have one customer with web integration, but we provided for them. Websockets, I, guess we'd webassembly, it's really complicated, because not all production browsers right now supporting and not all customers wants to see Hecky website web assembly right now because generally it's not in production, so most of the like companies don't want to see that on their environment. That's from my experience because we tried also experimental on that way.

B

Now, thank you and enjoy your lunch.