Rust Programming Language Rust Zürichsee, 3 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Stefan Schindler: Parallel Programming with Thread pools and iterators

Description

Concurrent programming used to be hard. Stefan Schindler is talking about techniques to structure code using ThreadPool and Iterator with rayon.

This talk will teach you the basic building blocks to not get lost in a forest of threads and tasks.

Slides: https://estada.ch/2019/3/20/parallel-programming-in-amsterdam/
Examples: https://github.com/dns2utf8/thread-pools-and-iterators/tree/master/beispiele
ThreadPool: https://crates.io/crates/threadpool
Rayon: https://crates.io/crates/rayon

Questions start at 41:06

A

Is going to talk about Travis pair problem? Give us.

A

So I will be walking over.

A

And try to record this mostly because I want to know if I have any bad habits.

B

A

If the recording actually works as your laugh, you might be on it and I hope.

A

This is later by the way old stuff never dies.

A

So let's go through our index or table of contents. Oh yeah, it has this floating bar here. So over time there orange growing yeah.

B

A

We had that already important part is this? Can if you use Ross paralyze a lot, and sometimes it's already done for you just a little bit about me. Oh I forgot to update the parrots line. Okay,.

C

So this line is wrong. This.

A

Is rust-eze for t-shirts from asbestos? The european conference for frost.

A

This line describes Paris, which was in spring roll was in the next film will be also in four, because we have to I, take two steps back and look at our project and say: okay, we are doing the same things wrong every time we actually need the fixed time. So that's what we're currently doing and by any chance there will be rust, fest in the Barcelona in Spain, probably in September world yeah, you organize some talks. This is an alternative festival.

A

I try to translate this I'm, not sure if it's correct makes any sense all right. So if you want to go to a very alternative festival in Switzerland check this out.

A

Okay side projects- some people might know me from there. It's the rock track cool.

A

So it's also the oldest one, so it doesn't have or anything it has started to redesign it over a year ago, but I don't want to break it too much stuff.

A

You see how that goes. I also did some cluster work so as GE some of its a little cluster manager. Originally they look tempo has some for Solaris, but we use your own infrastructure when you sound to each other, it's decentralized and it stays that way if we use it, but.

C

Today, I was in.

A

A small coffee house and they had ipv6 only right- it was 16 million 40 megabits fast was great, but.

B

A

Can do without.

A

Also, you can't complain about it, because Twitter is so.

A

Naturally, the next step you go to the forum of the Ross language, the.

C

First hour can complain, locking Oh 50 heavy guitar yeah.

A

A

A

A

Program: okay, love your hand. If you know what function is still good: okay, iterators, okay,.

A

Okay, Atomics still, you all know what that comics are. Okay, you know what the memory fences.

A

B

Know that there.

A

Are memory fences for memory fetches?

A

Okay, you also know that there are reordering barriers for CPU instructions as well as memory instructions.

A

A

All right, let's start reduce, so who can guess the language of this program.

A

D

A

Notice there is go to but still see. Yes, this is valid in C. Oh I think it.

C

A

A bunch of problems- oh and I, use these names here from a book and I'm, not sure if it's common around here, it's a very funny book in German mark will be clean. It's about digital age, it's called quality level which is Germany after they renamed themselves. After some political scandal, okay see also improved a bit over time.

A

There is this for int. This came with c98 I. Think, plus you can. You could get a speed-up by having pre increment operation instead of post increment operation. Today's compiler usually optimized this available. Still, oh here's our protagonist list. These are the main protagonists. They.

E

Translate into Peter.

A

Unemployed, send system administrator and Peter Cook closing that both the last name, the job, the parent of the same-sex had when they created the child. So.

E

Nine months.

A

A

Okay, so how do we do that? The Raskob? We can't go back to go to also.

B

The length is in the type already.

A

If we have constant data, don't have this length trick.

A

Yeah I will be using this I copied in this flag here so that you see you should probably shouldn't use it, but if you ever get into the case or it generates cold, and it gives you this ugly warning, then you can allow ugly code. There used to be a great link which was allowed bad code. But then some people know I mean I'm going to attack something as bad coldest.

A

Okay, so how to be iterate over a bow, easy peasy to see start right, there's a problem with this actually too. So one of the problems is here. This semicolon is optional and currently the return value is. This expression returns something the index increased and it gets lost for now, but in the future, there's a good chance that this book will return the last of these values, so that might come in as a problem. At some point, Plus this index access might give out of bounds index. So it's problem.

A

Good news is the compiler will probably in line the whole thing, because it's so short this time, so we can make that safe using the for each before in it's already the reference to reference here, if we forget to add ampersand and have multiple for loops of the same thing, you can something like data was already moved or excess off to move access, which is okay,.

C

B

Can also use the iterator explicitly.

A

So we create iterator object here and then we consume it with the construct or we can also.

E

Rewrite it and functional.

A

I'll just say for each if you have to be backwards compatible with all the rust like 14 or something like that, then for each isn't. Therefore, for each is think 122 or something like that that there are like two breaking points you actually have to remember. So one is 113 the question mark operator and 120 ish 92.

A

That is for each because before that you have to use forward which does the same okay, open.

C

A

Have questions just like we shout so: what's the iterator in rust, it's a trade and that's all the stuff. We need to know it's. The absolute minimum information iterator needs to fulfill has to have an item this time here makes it generic, although it does the half day, turbo fish, syntax and then the next big option and option is our.

A

Faster all right so.

A

In it we take the example from before, so why would you do that? Why would you complicate the syntax? Well, I usually say you filter maps and all the technical stuff, but one other thing is maintainability. If you have short functions that you one thing in one line can help.

A

It depends a bit on people. If you have beginners in the whole project, it doesn't help. But if you have people that come from.

B

House, like I.

A

C

At it, in a combo that means I.

A

Can enable and disable filters with like two packs like 405 all right- it's also nice for the compiler, because it can optimize. You have this functional style. That compiler knows you touch one element at a time and then it can do some hardware, optimization and process for element plus there's also a great talk from Pascal headlight. He gave it in row, yeah.

C

A

The over here, the fact.

C

That programming.

A

Is this more or less function, programming, the trust so when he did his night, he showed us some Monday morning code that asked if you were hey, what's wrong with this code and they found so many errors. Some of them he didn't plant.

A

Plus you get the speed up if you use these kinds of function because, especially you just take or skip and the compiler can insert early returns. So you don't move over a structure, links.

B

A

Here I will upload the slides afterwards, all right.

A

The other thing most of us do is if we have these little tiny languages that we write to parse, unstructured data or config files or old code, something that's really handy is split because split is an iterator and we couldn't collect split into a vector if you'd like then we know how long it is, but we risk whoops.

A

We risk out of bounds annex so what I've seen people do. Is they collect into the vector to know how long it is and then take another iterator to loop over it again? That's when iterators our entity return, none and anything so much easier, it's so much easier to check to return okay. So what are we going to do here? We assume that every person has just the first name and the surname.

C

A

That they are separated by space, so we go through our data data from before and split it, and here we say: okay split up into the thing that comes first and the thing that comes second, so we create that everybody.

A

Crazy, perfect, okay deconstructed already over here and then you have to see. Is it there if either one of this them is not there, then we.

B

A

That didn't work, so we we have an early return here, exit the function and if we don't, then we say: okay, let's I,.

A

Think I was this right there, okay and last but not least, here we unwrap it because now we know that they are there so I'm red and into because, in fact we didn't handle strings here. We have a string here in the name symbol, but the split gives us so they are used. So we carry create an object. These are smart pointers yeah.

A

We have to copy them again because we want to have a life string, because after this line the name gets given away for something else and in the end we say collect in the collect. We have some little magic and the.

B

A

Great collect has a great magic try.

B

To find the source code.

A

C

What he can do a collect is when.

A

You have a sequence of results or options. You can extract that outside of your fear, sequence, and this.

E

Works with vectors.

B

A

With lists laughs and half stairs so with this line, we can say okay process until there is an.

C

A

If there is an error, give with the error, if it's so, this gives us either a correct list or the first error all right. So this is Hardy code right. So let's improve it a little bit so first, you smash, so we can get rid of the other atoms right can match on this tuple create person if it's if it worked or we can create the error for everything else. So this.

E

D

Over here it's.

A

The default case, we don't do that we have to write all the possible cases, so it may take a while.

C

Well, here is the common magic.

A

Case I forgot: okay, here's the output.

A

This pretty print comes from this hashtag here. So if you put this hashtag between the corner of a question mark people in line everything nicely, this.

E

A

When you create your own native structure and you'll, implement the debug trade yourself, don't write by it to the field it gives. You take.

C

A

Iterator and say: hey constructing my name, constructing a block and then it will attend one more and then here is a field without a and then depending on how the format is specified, it will either take multiple lines or one by four prepended with zeros or spaces or tabs or whatever. You want to say minute. This way, all right, so so much for the base. I hope you weren't bored. If so, wake up.

A

Okay, what what are we going to do? It's usually one of these things, some things, because you have to generate lots of time or maybe on the fly, messages or the or all that stuff. That's real time, because all the stuff, that's slow is usually solved good enough with all the languages that nobody cares.

A

C

A

Use positive processes.

B

A

Not if we program some embedded board because may be looking over humans that touch my maybe for 10 buttons on some remote use of cells. Now imagine years if you has like 80 mega hours or something like that or hundred still so much faster than.

A

If you're going too hard and there's interrupts, if you go there, there is a pretty cool operating system. The tacos it's rust, based it only works on certain are microcontrollers, but then you don't need locks anymore. So you can have locks with the interrupt table. Manipulate, interrupts, table the type system and then get very cool.

A

Cindy is the thing that I think that look forward. So there are some languages that already do that, like C++, if you use for each or any other algorithms loop thingy, it will create two nested loops, one for start from zero to the end and inside that start from here to four or and then loop for inside and then jump outside again and what this tells the compiler is hey.

A

Why don't you try to process four elements at once and do that with simply this integration same accounting, not sure if it's already stable is it does anyone know could be there's a credit for that.

A

Okay next week we actually have stuff, that's not to say all the time that we have to wait for stuff. Maybe.

E

Some of you have done CUDA.

A

To the most expensive part to see if I have a branch that one arm of the primes is like wait longer for you. If, if statement has one long part and one short part, that's very expensive for the graphics card of the CPU, we don't have this problem because we have SMP simultaneous multi-threading where you can have stuff from independently. But now we have to synchronize that the old approach is the pool. We have a list of jobs or fixed size of workers, and then you go out in requests at the Apache web service.

A

There are other programs that we might use is OpenSSH, OpenSSH doesn't use Freddy, that's a fork program. So when you create open an SSH connection, your connection gets forked away from the server in that smile. You can restart OpenSSH server. We have losing the connection because your process is independent.

C

A

Let's implement something you remember: the old take free from rusty the safety speed concurrency, so.

B

They can't be enabled.

A

By these through market rates, but the compiler assumes, when you create your own data structure, is that they are there and then it looks for conflicts. So.

E

Whenever you play.

A

Data structures with that say just use diocese in it is it send yes.

B

Because all the you sizes are saying, is it sync.

A

D

Create a data structure: yes,.

E

Presente merci, okay,.

A

So send Queens, you can.

E

A

Across boundaries of affair, friends, without having big problems, you can't do that with, for instance, some file descriptors, some file, descriptor start a thread lock or socket. Sometimes.

A

Sync means you can access them from multiple threads. At the same time, this does not solve the problem that might one fret might start allocating. We get one, the other one is abusing it, but at least you and the thing is around- you can call methods on it right for me to.

A

D

Don't have sync.

A

And we share something, it must be read-only or.

A

If you have static unsafe, actually trying to measure the difference between those and became really really hard to, if.

B

We have time I have another presentation.

A

C

A

C

The standard library we have st.

A

Fred's spawn and then we have to join that again so when we spawn a flag to get the dragon handling, if you just have one task that we want to get rid of and do something else at the maintenance, sometimes good enough, we can also use pools when we have fixed size of workers. This is a huge advantage when we have something asynchronous happening like an event loop, and sometimes you have work, that's uploaded to the workers that over our machine.

A

So if the work is you really want to have pool, because just 500 drops to you and your CPU is a force.

A

There are two widely used tools: a flagpole and future spectacle.

A

Fred pool is the old one and the future spread pool is the new one inside Tokyo, sometimes the futures rifle outperforms, plus it can do runtime optimizations, but.

A

F

E

Is the implicit.

A

Rayon work-stealing rayon is an iterator approach. Where you spawn work with either divide and conquer like recursion, then you can have recursive function that calls itself twice. So it can have one task for your own friend and one task for the others to take, or we have some indexable access to data data. Parallelism can.

C

A

My whole thing is a million entries long. It can split it up into 50 joints and keep it away to my workers, maybe four of them work at the same time, but it's still faster than doing other stuff. If you are into academics and simulation there is time.

B

A

Frank did this project, while he was in the States now he's in Switzerland continuously prove it round-faced, tuple simulator. It's really fast. We have performed some smaller clusters with a laptop. That's really cool.

A

Okay solve the problem of handling work over and over here, but now we have still to synchronize.

E

And communicate.

A

Thus over-communicating is also performance, so how do we synchronize? That here is a small example. Sorry for this compressed Ness, oh.

A

This is actually working now, so with Russ 2018. You don't need the external crate anymore, so this actually compiles all right. So.

B

We use spread, pool.

A

From the crate, and with this statement we have a compiler to link it and we use channel from the standard library which is always used, and they say: ok for Merkers, eight jobs can increment. This number, like crate, is pool the old way, there's a new way. The other way where you can select stuff like stack size of the workers or say hey, just make all the detail. I don't want to care. How many to create?

A

Ok, let me create a channel Channel. Is it's in this acronym? It's multi producer single consumer. So it's more or less.

A

Okay, for underscore again, we ignore the index. We create a range. This gives the compiler I'm going to optimize this. However, it wants- and let me say: okay, we clone some assigning and for everything shove it into the pool with move move statement here, it's important because we want this TX symbol to month to have it gone, must be copied into the fret and then no longer accessible to our anything else. Area is very simple: send.

B

The number one and.

A

This one, you know unwrapped, this is it's it's bigger brother, except it's a really really nice, because if you have release mode code- and it just says, hey something panicked, then you look at it like sure, a panic, an option line for hundreds, something in standard library. Thank you that doesn't help at all, but with Excel you can have a message and I usually start them with unable something something something so I must have Ricky unable to write to channel on this I brought it's there for.

A

Then, when you add little details between these messages, even if they're from the same problem, if you search.

B

For the exact wording in the source.

A

Code, you get it down to one time that helps tremendously with debugging stuff other side. Note I try to use a debugger once or twice with rust. Yeah I stopped using them because usually the type system catches it or I'm.

B

Good enough fifties expect statements.

A

Or just use paint still works or encounters. Counters are also very handy. If you have stuff that's like statistically behaving and then you have in the range of millions, you have some atomic you size counters and then you keep on for lower bounds or upper bounds. And then you look at the.

C

Numbers- and here it only runs ten thousand times, that's.

A

Not possible you try not to be it, gives you feeling or averages okay, so I guess up till here is, is everything clear drop? Everybody knows drop so just an empty function, but why do we need to use it here.

A

D

A

There used to be this recommendation that you should use take, but we take. You can end a traitor after and elements, but this gives you a problem if your pool yields multiple things like I wrote, the front, end generator ones and a what wanted to have no redundant files, because I wasn't ready thousands of times so.

B

E

A

I only create two family if the image is bigger than the current size, so I have three sizes, I think on 2300, something and it fails it something pixels, that's the white there's a longer end, so some pictures would yield three families, some just one or two, and then this take behind forever.

A

Assert is also clear.

A

Okay, I wrote a low generator once to test out different configurations and pinnings.

A

What are you take? Cpus.

B

So everything that has more than.

A

16 CPUs have so-called new my units in them, and local and foreign memory, but near.

E

And far, memories.

A

Sometimes I think Amy says near and far this means and a Fred Ronson CP of zero and request some memory CPU serial. You might not have memory attached to it, but on trees, CPU 27 men might have memory attached to it. So CPU zero has to communicate over the bus into another.

B

C

A

The other CPU and tell.

C

It hey I, want to have some memory over there and then.

A

This has to travel back and forth. That's why if you ever have a big system and you're still doing harder by yourself and you have beard memory, latency issues try to have more memory, but not too much, because then it gets low and in this way, whether it happen because amy has more memories. Okay, anyhow, so what this code does is it gives every job here? Is your input from the last one? And here is your output to the next one, so you chain them together and the.

C

A

Of this test is to not put too much memory pressure on the whole system because it doesn't store all the intermediate results all the time, because it keeps adding them up, and then it calculates PI step over here.

A

Stuff, you can do.

A

Questions or just that.

A

Should we go on our season confused.

A

So, from from a hardware point of view became imagine, this is fiddled your CPU with one thread per physical unit and then just let them run for 20 million iterations and then, after that, they will start to collect in, like a wave over the whole silicon down to the last one, all the results, but at the same time, on the top, they start running again. So you get still waves over.

E

C

Or the data just.

A

A

Let's memory, communication, okay,.

A

There's a receipt I'm to be honest, I I held a similar presentation in front of students once and then they were like. But how can we actually do that, so we.

C

A

Imagine you start with regular code. You have some vector some pictures. Maybe they are not. Actually the pictures were passed to the pictures of just metadata that you have on them and you process them by getting the length and then I don't know, maybe get something back from there or if you have. The pictures are the receiving ends of channels.

A

And at some point maybe return so we are. We want to push the information here. It yeah everything for now so.

D

A

For every picture and push it into this database, it's just.

C

A list of all the.

A

Metadata all pictures that works it fails when some pictures don't get generated yeah. So we can use the iterator instead solve some problems. Then this is still pretty ugly. So, let's get rid of that, we can say for each okay, then at some point you might say maybe I want to fill there. Sometimes the pictures are not wanted in this collection. So.

A

For that transition actually could have filter system at before, and then here here we are back at collect again. So have this error or the return thing in the other? Okay.

A

We know that we get this result. First, if it ever occurred, and then we say.

B

A

Collect into think or expect unable to transmit pictures gives us two messages right. If there's one message here that becomes searched for and then when we know that we can turn on. You know this environment wherever stacktrace equals one.

A

C

B

It called stack, trace, yeah yeah.

A

B

Bad with names.

A

So it's probably wrong so enable that then you get this message all right. Okay, now crayon did you see it?

A

That's it! That's all of it. Importer library, add four characters.

A

Just like that sinking.

A

Maybe so maybe you have all your CPUs maxed out, maybe you decided I have multiple pools, maybe I use on your fourth for this task and fourth for another task, but it doesn't really matter because what you get here is an potentially infinite sequence. So it doesn't make much sense here to paralyze this, because we only like look at it have an error in the processing before. But if we did something else here, we could.

E

A

Some colors or calculate average brightness or whatever. If we did.

B

A

We could have a series streaming in, have them automatically distribute over our pool and, at the end, still get our normal connection yeah, but with lots of type system working for you and giving that away. Because you don't need to you don't need to know right, you don't care, because it's an infinite sequence anyway. So.

A

We were to collect the whole thing into a vector, then we could set up the length of the thing and I could split it up it. Maybe chunks of 50.

A

A

A

F

A

It's still hard, the.

E

Best thing you can do is.

A

Look at the system tools if you have like the load, average or CPU usage of top of Linux or h-dr or whatever you use. They.

B

Actually, have performance.

A

Problems that level you need to go to the vendor of the CPU and tell them to keep debug tool like Intel, has a good one in the cold AMD.

B

Has found that.

A

B

One for all.

A

A

The very biggest when you enable, when you have performance problems, usually the first thing you do- is enable release mode and play they're gone.

C

Release mode so.

A

The standard answer to any performance problem before paralyzing anything should be release interval. It gives you speed up from 20 to 400 times, depending on what you're doing. That's just insane. You do nothing else. You can't have the serial code as before. Sometimes human code is still faster because of all the branch prediction and stuff like that, when you want to know what was going on counters, selenium I used to.

E

B

A

Per task and timers as well, what didn't work very well was having my custom timer, so I read about the system clock and how it might change our antique. You may slowly shift it over the day, but for for my purposes they never shifted so much that it affected me and at the end it was so horrible to get it right.

A

Loading, the timestamp all the time that it just said. Okay, forget it little princes yeah.

A

There is an inspect, filter filter, it's a function that takes T and gives you t there can insert that e to the iterator and they may be printed or, if you don't want to print it, to standard out ready to a file.

A

It's a problem that hasn't been solved.

A

More questions they have to back up side, so I can.

D

A

D

If back to results.

D

30 of what's the whole situation with in Iowa happens. So what we do to this product.

D

D

Stop all threats meet me or wait for a while.

A

A year ago this was undefined behavior, so I don't know if they decided on all the behavior, but.

A

C

The discussion back.

A

Then was something like the overhead of communicating that this kind of talk has to be aborted, is so much higher that it's just not worth it in in the general case. So when you want to have control over that, you have to have some abort signal. I mean you're at the point where you're like okay do I still use rayon the frat cool for my experiments. I was using the fretboard already so I have just added a single yeah skip parts that have a function with like four parts in it so skip.

A

Some of them will just abort the whole thing. Then you get the downside of checking every time.

A

But Tommy relax.

A

A

Yeah, it's 40 cycles on AMD and this something between 20 and 120 endo, because it fellas magic, no employed, spiritual connection. So when AMD released in 64, the architecture Intel was like, oh they had attained. You remember died of 10 years ago yeah, so they.

B

Were like I mean we are fast abused.

A

But this instruction set it's complicated, so they implemented.

C

It in software.

A

Designed today, every instruction you run into a CPU, it gets analyzed traffic in some internal language and it's virtual. The instructions get routed on the CPU and the hacker fighting and all the fancy stuff they're doing. Oh, there are all the chip so, but there.

C

Was this discussion ones with these embedded people that said I know exactly which instructions that CPU executes.

A

Image organ yeah sure you can do that, but the CPU doesn't care it just tasted thing and looks at it.

B

A

Figure out what's happening.

E

B

E

Of these repetitions are resolved, though, and others are fast, it would be a priority in which I can get the fastest computational, whatever I'm working is that it seems to me that I have to wait until everything finishes: people ready for like the grind. Yes, these are some patterns with great. You know something that I can get fresh reputation and go as fast as possible than me. Maybe you know, but.

A

C

A

E

A

And in the workshop on the current program in Rome, and then we we have to learn something we can see. They.

B

Actually measured.

A

So we used the bench tools of rust nightly and then just iterated over it. It has some some clever logic to figure out how many million times it has to run. Then we realize the funnel generation process inside the image lab. They already use the trail, so they paralyze the image resizing already eternally, so that task is happening, but your task of analyzing, the image for metadata or artifacts, or something like that. It's also running concurrently on that pool, so some tasks for out of the pool and some others are cute back and forth.

A

So yes, that works, but how does that work at RPC? I guess is question.

E

Also you mentioned a bit of our work is stealing is all mining if this is going in the same direction or a reason to be separated.

A

There is one global pool per application for rail, it's also the hidden global default pool.

A

So yes, if you have an event loop dispatching, although HTTP requests into the pool there will be only one pool executing all the queries might be a problem, because your new short request might stand in queue behind four or five longer ones, but after day, if you have like most of your jobs are short and they come in asynchronously and some of them are long and the long months are not too close together. Then.

F

A

But then you can't use them anymore, then you have to use or futures sure, but it's not possible in in contrast, this yield. So when you have a long job and then you know it's a long job, when you have Java, then it just yield right and then it reschedule it's another job that doesn't work.

A

What you could do is implement your drop in multiple steps, any multiple futures and then, when it spawns the new future, it will implicitly yield sometimes and therefore to free the coop again thanks.

B

A

And pools and channels, you would have to resubmit your next step onto yourself. It's a bit ugly in the code, but it's cool.

A

Generally speaking, there's my list.

A

For there's the list you receive this, this list.

A

The job queue thingies are yeah, that's the difference of queuing and work-stealing and it's futures. We might have the less of this problem, especially and then yeah I, Oh bounce stuff with the futures that will be coming in this year. This will automatically insert the gaps TP response time.

A

A

One get away for the break: you can either to elaborate scheme or I will just probably.

A

Okay, so let me close my eyes and then you have to tell me if I'm, looking more or less at the wall.

A

Yes, yes, okay,.

C

All right, thank you very much. What is interesting, let's.

A

Take a break for ten minutes.