Rust Programming Language Rust Linz, 6 Sep 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Rust Linz, August 2021 - Rainer Stropek - Rust iterators

Description

This year, important things have been improved in Rust regarding iterators and arrays. Finally, we can iterate over arrays by value using a "for"-loop. It is kind of strange that this seemingly simple language feature took so long to arrive, isn't it?

In his talk about iterators, Rainer quickly demonstrates the new iterator features that came with Rust 1.53. Then he tries to shed some light on what problems the Rust team had to solve to deliver this important language enhancement.

For beginners, Rainer will also recap what iterators are and how they work behind the scenes in Rust. So if you are new or you have limited Rust experience, don't hesitate to join.

A

So my session for today is about iterators. I would like to spend an hour or so depending on the number of questions that you have an hour or so on this topic and tell you what's new in rust, 153, which came out a few weeks ago, and I would like to take this opportunity to dive deeper into iterators.

A

Let me quickly shut down this one here. So now it's good iterators are interesting. Iterators are very interesting. I know them from various languages that I use, typescript, c, sharp and so on, and when I learned rust, iterators were one of the first things that I was really interested in. So I take, I took a close look and for me it was rather surprising that it took rust so long to be able to just iterate over a bunch of, for instance, numbers in an array, and this will be the main part of my talk today.

A

I would like to talk about iterators, especially in conjunction with arrays, and I would like to dive in some very important rust properties. That are the reason why it took relatively long for us to learn these tricks for iterators and arrays. If you want to ask any questions, if uh something is new to you, if I'm too fast or too slow give me feedback, ask questions by using the chat window in youtube, or you can also ask questions in discord.

A

I will take a look at questions on the discord server after my talk so now, let me share my screen with you here. um You see uh the the storybook, it's just a quick gathering of notes. What I would like to walk you through today. So if you find some code snippets interesting, you can take a look at this url and copy them out. Everything else will be done, live okay with that, let's dive right into it.

A

The first thing that I would like to show you is: I would like to show you the old version of rust, 1.52 and show you what we had, what problems we had there. So let me quickly oops sorry wrong one. Now we are ready. uh Let me quickly run a darker container with rust, 152, okay, okay, here it is: let's go into visual studio code and open up visual studio code within this docker container.

A

So now I am in visual studio code connected to this uh to this docker container, and let me quickly here uh install the rust analyzer here, yeah. Okay, it's already installed in this docker container. That's fine! That's good!

A

ah I just broke that one. So, okay, let's go here, create a directory rust demo.

A

Demo here we are and open this folder up here in my docker container, so open a folder, and here we are that's good.

A

ah This this demo broke. So let's reopen this one here. Maybe it works at the second. Try again start ah now, it looks much better. I think yes much better. Here we are, and now I can. I think I can also yes check the language service installed. Everything is fine. We are good to go sorry for the quick glitch. I had some problem opening connecting visual studio to my docker container, but now it seems to work.

A

Let me quickly start a new demo here with cargo new here we are and open up the uh the code here looks good. Everything is fine, okay, so if we quickly type in some code here, let me do that. uh For instance, let's define some numbers here and let's do them something like this. This is just a numbers array, nothing special. So in the good old times prior to 153, I had a problem when trying to do the following.

A

We are and if I want to say, for instance, print line something like this and something like this print the numbers, something like this. I had a problem. This doesn't work, or this didn't work, because an array with a fixed size of 5 did not implement the proper traits up to rust. 152.

A

This changed in 153. What you could do in rust 152 is. I could change here that one to a reference and that brought me- you see it here, a reference to the number, so I could already do an iteration through an array by reference.

A

This gives me a reference, and I can prove to you that this is really a reference by, for instance, doing a print line here and, for instance, here saying: let's print. Oh sorry, let's print the pointer, let's take the address of numbers, zero, something like this and then here, let's print the address off n, which is just a reference here, and if I run this guy cargo run run cargo, sorry, cd and now to cargo run.

A

Again, you will see that the reference of the first element in numbers and the reference which the iterator gave me here are exactly the same. They are referencing to the same address in memory, so we are really iterating over uh over. We are really iterating by reference that this is really a reference. You could also do a mutable numbers here and, for instance, add another for loop, something like n in uh this time.

A

Let's take a mutable numbers, something like this and then we can also say star n multiplied by two, for instance, something like this, and if I run it again, you will see that my values change, so I can iterate by reference or by mutable reference over an array of numbers. This has been possible for quite a while in rust, and it has been possible in rust, 152 also and in rust. 153 a few months ago, things changed a lot. We can now iterate by value over a race.

A

So let me quickly demonstrate that by quitting my rust darker container 152 and go to my machine, which runs a newer version of rust, where I can do nearly exactly the demo as before, I will copy the code, and now a big difference occurs iterating by value.

A

So here I have uh opened my here. It is. I have opened my environment, let me quickly exit from here and do a cargo uh new iter, something like this go into the iter folder and open that one in visual studio code. As I said, this is a newer version of rust. This is on my local machine and I will quickly paste in the code that we just wrote, but this time the big difference is, let me get rid of the mutables here. We don't need that one.

A

I can now change the ampersand here and that in 153 gives me an iteration by value. I can now simply iterate over the values in the fixed size array, and if I cargo run this guy, you will see that I get the proper values.

A

I don't even have to store that one in a variable. I can put that one directly in the for loop, nothing special here and it will still work. So this is what really changed in 153.

A

Well, you could ask yourself: hmm are we done? Is that all is the talk now over? No, it's not! Oh! It's just the beginning. I wanted to demonstrate to you what's new and now based on that, we will dive deeper into what's going on behind the scenes, so we will take a look what these iterators really are. We will take a look at the source code in rust.

A

We will try to re-implement some aspects of iterators on our own to learn more about the inner workings of iterators, and then we will take a look at why it was so difficult for us to offer this seemingly simple functionality, so we will check out so-called const generics and we will check out some breaking changes which the which the rust language team has to uh has to had to solve, uh so that this great feature became possible in 153 okay. So with that, I think we can uh get started. Okay.

A

I see some first um questions coming in in um in discord. I'm very sorry I have to discord quit this chord for now. Oh, I just saw that you asked me to um increase the font size. Of course. Of course, I will do that so here yep. Thank you very much for uh for pointing that out. Please give me feedback if the font is still too small. uh The big chunks of code will start from now. So I hope it is not too late.

A

Okay, there we go nice, so, let's dive into the the most core abstractions, oh so good. Thank you very much. The most core abstractions, which are behind iterators. The next part, might already be known by those in the audience who already have a very good noah or a good know-how in rest and who have already written a decent amount of rust code. But maybe we have some beginners in the audience who for whom this might be new.

A

So in every programming language, if you want to build something like iterators and helper functions on top of iterators, you need a basic abstraction. You need an abstraction for an iterator. What is an iterator? An iterator allows your code to iterate over a collection of values read only forward only so you cannot change the content of the underlying data structure and you can only go forward.

A

We will see that read only is not 100 correct here in rust, but let's say for the sake for for for now that this is what we understand of an iterator. We iterate we go through. We walk through in our code, the elements of a data structure and by having an abstraction, we can build helper functions, like sum filter, mapping, minimum values, maximum values and things like that over all data structures which fulfill these abstractions. So it doesn't matter if you store your data in a vector or an array or anything else.

A

As long as this data structure, fulfills the requirements or has the capabilities of the common abstraction, the iterator abstraction, you can always use this. Generic helper functions like summing filtering mapping and so on on your data structure. So what are the most basic abstractions in rust for iteration?

A

Now, the first one is. Let me show you that one here, let me go into my browser and fire up this link that I've prepared is this guy here, this guy, the one is the into iterator trait the into iterator trait is the most basic trait which allows us to ask an underlying type for an iterator.

A

It only has a single function, which is called intuitor and it will return the iterator. You see it here now. The iterator is an implementation.

A

This is an implementation here um and, as you can see here, self intuitive and the iterator again has to fulfill a second trait, which is the second basic abstraction for iteration in rust it is called iterator. So if I follow this link, this will bring us to the iterator trait here and this iterator iterates over elements of type item, as you can see here, and it has a lot of functions luckily, if we implement our own iterators all these functions.

A

Starting here are not interesting for us because, as you can see here with the these three dots, they have default implementation. The only functionality in this iterator, which is really important, is the next function. This is what we need to implement if we have a type which should be a we should, we should participate in this whole infrastructure of iteration.

A

We need to have an iterator which implements the next function, and this is what I meant with forward. Only iteration. There is only a next function. There is no previous function and all the rest like counting summing and many many other functionalities. We get them for free because they sit on the shoulder of this giant here. They are implemented based on the next method, so remember that one into iterator is the capability of a type to be asked for an iterator, and the iterator itself is a trait which offers a next function.

A

These are the two implementations. If we want to take a look at the source code, I have a deep link in my storybook, which I mentioned at the beginning of my talk. This is, let me zoom in a little bit. This is the into iterator trait in the rust uh github repository.

A

As you can see, this guy is pretty old. This abstraction has been there nearly forever, and here you see exactly what we saw in the documentation. We have the type of the items over which we enumerate. In our case it wasn't an array of numbers, integer 32. So this is the item type in our case, integers 32, but it can be any type you want, and here you see, the implementation of the iterator you see of the iterated trait.

A

If I scroll down a little bit, we see this into it function, which returns the implementation of the iterator. You see it here, it is the functionality which which, with which I can ask a type. Please give me an iterator for you for this specific type.

A

If you have a rust vector- and you want to iterate over the elements in the vector you ask the vector by calling this into it a function, and it will give you the implementation of the iterator for a vector that you can use to iterate over its value, and the same is true for every every collection type that exists in the rost universe that implements and wants to play with um with iterators.

A

So let me quickly go back to our code sample here and change this a little bit okay. So let me undo that one take the values here and get rid of that one. Let's put that one into a backdoor so that we don't no longer have an array here just to have something different and now we can say four number in numbers: you see it and then we can print line.

A

The sorry number here.

A

This is possible. This in numbers is possible because if we take a look, this is the type vec. So if I go to vac, then I can look up in the documentation that the structure vec implements the into iterated trait and with that, just because the vector implements into iterator the rust language. Compiler knows that in this for loop, it has to call the into it functionality to get the iterator and then uses the calls to the next method to iterate through the elements and what we get through.

A

What we get back are elements of type integer 32. This is essentially what's going on behind the scenes to get a closer understand, a deeper understanding of all this stuff, let's implement a custom iterator. To be more specific, I would like in my next demo, which will take a while I would like in the next demo. I would like to implement a generator.

A

A generator is a special form of an iterator where there is no real underlying data structure, but we are generating the values on the fly.

A

I've chosen this sample because I think it's a good way of looking behind the scenes of iterators and understanding.

A

What's really going on now for the next demo, I will copy in line by line some prepared lines and again I copy these lines from the github page that I showed you at the beginning of my talk, and I will also share this link in the discord server after my talk, so you can take a look at it.

A

Okay, let's get started first here we need a use statement, let's put that one, because we need a random number generator. Next, we need a structure and this structure will be called passwords. Why did I call it passwords? I called it passwords because I will implement a password generator. It will be a very naive password generator, so it will not generate strong passwords here, but hey bear with me. It is just for the sake of demoing iterators. It is not the world's best password generator.

A

Now this password generator has one parameter, which is the length the the the length that the user would like the password to generate its passwords to be now. Let's quickly implement this password generator structure, some basic functionality, we will implement a new method which will return by default, a password generator which passed, which generates passwords of length. 10, okay, and we also implement a helper function, which is called with length there. You can pass in the length and it will give you a password generator with the specified length.

A

You see that here, okay, so I think I can also get rid of exactly that. One. Okay, that looks uh more idiomatic. Let's put it that way. This still has not really some functionality in it. Of course, we could discuss whether it would be a great idea to implement the default trait here, but that's out of scope. For today we will jump right into the implementation of an iterator in order to make this password a generator, a password generator.

A

We need to implement the into iterator, trait that I introduced you to a few minutes ago, so, let's implement into iterator into iterator for our passwords struct, the type the item will be string because we generate passwords and those passwords are of type string.

A

Now the implementation of the iterator itself will be the passwords iterator which we'll we'll have to implement in a second for now, just except the curly. uh The the rats quickly is here. They are fine. We will need to implement this guy in a second, and now we can implement the core method that we mentioned before we can implement into iter like this. It gets a cell here and it will return the it to enter the iterator implementation.

A

Okay. Here we can return the password integrator iterator, which we will implement in a second and just for demo purposes. We will path pass on the length that the user has chosen for our password generator just like that. So this is the functionality with which the for loop of rust can ask our type passwords for an iterator. It will call into itter and we have to return an implementation of the iterator which we will have to write now. This was the first step implementing the into iterator trade.

A

Now, let's go on and implement the iterator itself, so let's create the passwords iterator this one as a structure- and here we again have our length.

A

Of course, we cannot, as you can see here, use passwords iterator already here, because it does not implement the iterator trait it's not yet an iterator. It doesn't have a next method. So let's write the next method and implement the iterator trait import iterator for passwords iterator.

A

The item we can copy it from here is still a string. Of course, our generated passwords will be strings, and that's fine like that. Now we have to implement the next method. Let's just implement the missing method here that looks pretty good. The only thing is this to do, which is not a good idea. We will have to write some code here now, let's step back for a second now, the rats quickly is here are gone because now our password iterator is really an iterator and offers a next method.

A

The next method will return. The next item. Optionally optionally, means that we can always return none if we have reached the end of the iteration now before the system calls the next method. The first time the iterator stands kind of in front of the first element inside of our iteration, and once we call next, we will either get the first element in the iteration in the underlying data structure or generator, or we will get none once we reached the end.

A

In our case, we want to have an iterator which can generate an endless number of passwords, so we will never return none. We will always generate a random new password. Okay, let's do that.

A

Let's first, let mute result create a string and we can already create it with capacity and say we can use the length because we already know how long the string will be, because we know the length of the generated password.

A

Now, let's write a for loop to generate the letters of the password for in zero self length. Something like this. This will give us a for loop, uh which it which gives us which iterates through this whole for loop self, don't length amount of times.

A

Now we can push into some new letters and here bear with me it will not be the best random password generator, as I mentioned before. So, let's quickly take um let's quickly take uh the byte value of the letter, a and add the random thread.

A

Random number generator generate range, a random digit here, and we do do that by saying. Give me zero to uh let's take this one, the c value minus the a value, so a a value between 0 and 25, including- and this will be this one as char- something like this. So, let's see what I did wrong here. um This is that one that looks good and we have this one something like this and we have the random number generator and at the end we can return some results. Something like this.

A

So let me quickly see what we have here and declared. I thought I used it up here. Oh of course, of course, I forgot something in the cargo tunnel. I need to reference the uh the rand crate, otherwise it will not work. So let me give it a second and cargo builds.

A

And which do I have, I think, ah let me quickly. I will not look for the typo that I did here, so I will just copy in I see what I did. I don't need to copy anything. I just forgot to be here and now it looks good now it looks good now the code compiles. Let me quickly show you again what I do.

A

I allocate a string with a given length and then I have a for loop, and here I just push in the letter a plus a random number between 0 and 25, uh get to 25 by subtracting c from a oh sorry, a from c and convert it to a char. That looks pretty awesome. So now we have the into iterator trait and we have the iterator trait.

A

So we now have an iterator in this case, a generator which can generate an endless amount of passwords, and now we can really benefit from the functionality that we get in rust, built on top of the iterator abstraction. Let me show you what I mean four pn passwords this one new into it, see that one and now I say- and this is the important one- take three- otherwise it would run forever, but by specifying take 3 like this one. Of course I need a dot here, not a not a colon here yeah.

A

By specifying the take 3, it is no longer a problem that we generate an endless amount of passwords. The iterator function take which is built on top of the next function will stop after calling my next function for three times, and here I can simply do a print line. uh The next password oops, sorry password, is that one.

A

Okay, let's run it cargo run, let's see what we get and get. Oh, my god, we got three different passwords three different passwords, which we can generate through the iterator abstraction by calling into our into eater scroll up into it will generate the passwords iterator. This is the guy here. This implements the iterator method. This implements the next method and the take 3 will make sure that the next method will only be called for three times.

A

This is what I wanted to show you. This is what I wanted to show you, so you get an idea of what the core abstractions for iterators in rust really are, and, of course, we can also write it in a different way. I will copy this one in because it is essentially the same just a little bit in a different notation.

A

You see again, I can use the width length functionality that we have implemented above and say I want to have five lecture passwords into it again, take three and then use the for each method to print the next password. So if I run this guy now you will see we have the ten digit passwords and the five digit passwords exactly what we wanted to have so the last 10 or 15 minutes.

A

Something like this, where primarily for rust, beginners or those who have just used functionalities like this or this or some or filter, and things like that. But I have never bothered to take a look into the details of how iterators really work now you've seen it and now we have implemented our own custom iterator.

A

So it's a good time to take a breath, because now we are starting in a new sample and we will dive into dive deeper into some more advanced functionalities of iterators and maybe some surprising things, especially for beginners in rust, surprising things with iterators and the first topic will be consuming iterators. Let me show you what I mean with that.

A

Let's get rid of this example. Don't need that one looks good and let's do the same thing that I did before. Let's create a vector off some numbers this time, let's take some fibonacci stuff, 3 5, 8 13. Something like this looks good.

A

Now, let's print the minimum number of this, this vector, okay print line, nothing special here: okay, the minimum is curly, braces and say numbers dot into eater, this one dot minimum, and we are sure that the element that there are elements in this array- so this should be fine check it out. Let's run this guy and okay, I don't need the use anymore.

A

Let's get rid of that one near it and run it again.

A

And we'll see the minimum is one. This is exactly what we expected.

A

What you might not expect, if you are a beginner in rust, is this problem, let's print the minimum two times, oh my god, so many reds quickly, just by displaying the minimum number two times well, the reason for that is the way that into it really works.

A

So if I go back here and if we take a look here at into itter, we will see that into it takes ownership of self. It consumes the underlying data structure. We cannot use numbers after line 3 anymore because it was moved. It was moved into the iterator and therefore we cannot iterate twice like this.

A

Of course, there is an easy way to get around that. If we say numbers is no longer the vector itself, but if we just borrow a reference to the vector, so numbers is now a reference to the vector everything goes away. We have no longer a problem, because now the type of into iter is no longer the vector, but a reference to the vector, and now I can run this guy and we will get the minimum successfully twice.

A

This is exactly what we need to do note that we did not have to change anything here, unlike c plus, there is no arrow operator like this. If we have a pointer, this is how the the dispatching of methods work in rust. It's really a core piece of infrastructure for making ownership work in practice that, even if we have a reference, we can just type in a dot and everything is fine, so again, cargo run and it simply works as an alternative.

A

We could also, let's put it that way, use two other functions. Well, I will demonstrate just one of them. I can use the inter function here, something like this and then everything is okay.

A

You might scratch your head and say what's utter we just had into it. Why now? Is it another trait? No, it's not the itter function. It's not coming from a from an uh from a trait it's by convention.

A

Returning a by reference iterator over the underlying data structure, so this one, this iter is automatically iterating by reference and therefore it doesn't take ownership of the underlying vector and you can call it two times, but the behavior is just defined by convention and not by definition of a trait.

A

There is also an ita underscore mute function which you can use if you want to have a mutable reference. Okay, this is maybe a little difficult, so I would like to walk you through some additional lines of code. Another example a more elaborate example where we take a close look at this into itter and references and iter and intermute, because I think at least it was for me when I started with all this. This thing this when to use it and when to use into it and what's exactly the difference.

A

This really gave me a lot of headache and this headache. I would like to spare you this headache: okay, by showing you um some some simple things: let's define a struct because always using just hints is not funny. So let's define a point. Struct and let's add some 64 floats here f64, it is just a very simple point here.

A

Good now, let's create a vector of points. Left points equals to vector, and here let's say we want to have a point. This one x should be 1.0 y should be 1.0, and then we add a second point this one this one- and this will be- I don't care it's, it's really not important too. So now we have a vector of points. The first one has one one and the second one has the value to two.

A

So what we can do if we want to have the first point, let um first point you can say points dot into inter this one drop next dot unwrap.

A

One and as you can see, the type which we will get back is the point, so in this case the rust, compiler iterates or makes and calls the into iter with the type of the underlying data structure, which is point as you can see it here. It is a vector of point. So this is what I get back.

A

I iterate over the points by value. This is what I do here by value iteration.

A

That is kind of obvious now, let's change that a little bit. Let's redefine the points here, I'm just shadowing here. So exactly the same points again, but this time I'm going to do it a little bit differently, let mute we want to have the iterator, and now I will do the following. I will say: take the address of points and call into iter, this one that will give me an iter which is the implementation of the iterator over point, but now check out what what's happening.

A

Let first point this one equals to it: dot. Next dot unwrap see. What's going on now, I iterate by reference, because here oops sorry here I have a reference to the the vector into itter immediately understands, although it's the same call as up here above, it immediately understands that it has to give me an iterator, an iterator which iterates by reference. So first point is now ampersand point the reference to point very important thing, and this is the magic of intuitor.

A

It understands whether you call it directly on the vector or on the reference to the vector and will give you either the value or the reference which is inside of the vector, and this is where the inter function comes in that I told you before so instead of writing this guy here, I can say points.eater see that is a shortcut for this rather complex syntax here in the background here in the back here of this line so ether by convention.

A

What did I tell you, ether by convention, gives you a by reference iterator over the underlying data structure. So if you definitely want to iterate by reference, call iter, and then everything is okay. You don't need to write something like this, which is smart but kind of different difficult to understand points that either is really simple, and now you know what it does. It gives you an iterator by reference iteration, something like this now we can also do that uh by oh. I don't need to that one. I don't need to do that.

A

One of course yeah. No, no mutable here um yeah. Of course they correct. I need to be mutable. Sorry, sorry, sorry, it's fine, a mutable thing that I want to demonstrate now. Sorry, I just got distracted um comes down in the third demo. Let's redefine the points again here this one, but this time let's make it mutable. Okay, now we have them mutable.

A

Now, let's define again the iterator, and this time I will not call points.eter, which we did before immediately here. But this time I will write it underscore mute and that will give me an iterator over mutable references of the underlying vector, so that would be equivalent to writing on the ampersand mute points dot into it. Okay, that is essentially the same just as here above we could write address of points dot into ether, and now we can say mutable reference of points into it. That would be exactly the same.

A

So now, let's get a, let first point equals to it: dot next dot unwrap something like this, and you see. I now get a mutable reference to the point: let's write it down: okay, just to make sure that we are absolutely sure what we are doing here and with that I can now say. First point: dot x, equal 3.0 and first point dot y equals 4.0 and I can print line.

A

I can print line.

A

um Something like this and say give me the first point and of course we need the derive debug up here. Otherwise it will not work. Let me give me let me give that a try and you see the point has changed so it worked. We could retrieve by mutable reference iteration. Here we got the mutable reference and then we can change the values in the underlying vector, and I can also maybe print the it's here. Maybe that one is also interesting and you see the values also changed inside the vector.

A

It was not just just changed here in first point, because obviously this is a mutable reference, pointing into the point of the vector. Okay, okay, um I have a question here. The question is: will next return second element or last next? Will, if you call it for the first time it will give you the first element. If there is one, if the collection is empty or the generator cannot generate any values, it will give you none. It is an option.

A

Okay, that is important once you are at the end, and you call next, you will definitely get none because then it's over the behavior by the way, if you call next again once you got, none is not strictly defined.

A

An iterator could decide to start from scratch and start from the beginning and return you. The first element in the uh in the iterator, for instance, or the iterator, could decide to give you non again that is defined by the iterator. There is no definite rule for that. I hope I could answer the question with that: okay, nice. So now I hope you got a better understanding what this into iter.

A

Is you understood that intuitor will give you a by value iteration or a by reference iteration or a by mutable reference iteration, depending on the context here we have the by value. Here we have the by reference here we have to the by uh mutable reference call the difference. There is no difference in calling into ether. Okay, there are multiple implementations of intuitor and the compiler will choose the correct one for you. If you want to make your code a little bit easier to read, then use the iter function.

A

If you want to iterate by reference or the intermute function, if you want to iterate by mutable reference, nice breathe in breathe out new topic.

A

Let me go into the rust playground for the next demo here.

A

And the next demo that I would like to show you here is the topic of serial cost. Abstractions many people ask themselves. Oh sorry, I have to copy it here. I can copy it now.

A

I copied it and I, of course I know I know I know I have to zoom in so this is a very, very simple, uh a very simple loop and many people ask themselves when working with iterators, okay, how much is the the runtime costs and if you take a look here, we calculate the sum of all values between 0 and 9. So I'm just adding to the sum and at the end I print it. Let's run it and without any surprise it returns. 45, that's okay, but that's not what I'm really interested in.

A

Let's do something else. Let's take a look at the assembler code here, okay, so if I scroll down and down and down and down and down and down and down at some point, we will reach the main method. Where is it? Where is my main method?

A

If I will not find it immediately, I will look it up and where is it? Where is it, I have to look it up ah here. It is uh without going into any details here. um Okay, here we call the something like this and if we take a pool look if we take a look somewhere ah is that that's typical? I can't find my here. We see it okay here we see it here. You see playground main. I finally found it. Sorry, it's not so easy for me to read uh assembler code while demonstrating.

A

So uh please don't be angry with me here. You see that we are doing a bunch of move statements and so on, but at the end of the day you see the call to intuitor. Everything is fine and at the end we will also see the call to the print functions. Nothing special. So we have some runtime costs. Well, not really.

A

If you take a look at what I did, you see I selected debug and if you select debug and you compile your ros code, we all know you get pretty slow code. Okay, you get pretty slow codes and it can be optimized a lot so bear in mind this one with debug calls the interneter and does essentially all the things that I demonstrated to you a few minutes ago. So bear in mind if we run this guy.

A

Come on run it the result will be 45.. So now, let's switch to release and take a look at the assembler code. Again, oh my god, I have to scroll through assembler code. Again, let's see whether I will find it yeah here is and see that one.

A

Isn't that awesome? There is no intuitor anymore. The compiler understood that it can calculate the result during compile time and give me the complete result directly as a constant in the code and immediately called display trait in order to be able to print it on the screen.

A

So that is kind of obvious. You see it really works, it still works and it is. It is. Oh sorry, I wanted to run it here. It still works, it works pretty well, and here you see 45, but now there is no intuit anymore. So this is a kind of very simplified example. I knew that okay granted it's very, very simplified, but it demonstrates the point. Zero cost abstractions in rust mean that when you use iterators, you should not let rus.

A

Let yourself be fooled by looking at assembler at generated assembler, ll db or llvm code in debug mode, because there you will see all the overhead of iterators, but once you compile to release, you will see a very different beast. Maybe you don't even see the iterators anymore? It is optimized to a very large extent- and I wanted to make it very clear to you by using this kind of naive example granted, but still it demonstrates. The point.

A

Don't think that iterators, when compiled to release mode will work, as as it is in your mental model that you might have in your head, the compiler will optimize it to a large extent, and the assembly code will look very different compared to what you might be, what you might have, as I said, as a mental model. In your mind: okay, okay, good, nice, good, good good.

A

This was zero cost abstractions. So with that breathe in breathe out new topic, let me quickly take a look at the questions. I think there is one um okay here we have a question. Let me quickly read it. Is there any convention how to write code, let mute variable into iter or let mute variable vector into a vector iter mute. I'm not 100 sure if there is a hard best practice here, but I would use the second version yeah, I would use the second version it's easier to read. It's completely clear.

A

What is done it has, for my personal uh experience, less magic going on behind the scenes, so I would definitely prefer the second version here: okay, good nice good, as I said, breathe in breathe out new topic.

A

The next topic- and the final topic is: why did it take so long for rust to enable this thing here by value iteration for arrays?

A

It is so obvious to write four number in array, one two, three, four: five, that it wasn't clear when I started dealing with rust, why this functionality isn't there. So two things were pretty difficult when building this implementation, and I would like to demonstrate both of them.

A

Okay, let's go back here and uh start another darker container run this one and run rust, uh 1.50.

A

Okay, let's run a darker container with rust, 1.50, let's create again a ros demo folder this one ros demo and let's go back to visual studio fire up my remote development tools and fire up a new visual studio connected to my container, which runs rust, 150. Okay, oh I'm in the wrong folder here. So let's go up oops up one here and go into ros demo and open that one here we are good.

A

uh Here we are, let's cargo new uh another demo here and go into the iterator, and let's take that one good for the next demo. I am going to write a few lines of code and also copy a few lines of code.

A

Imagine the following situation: we want to implement a trait that tells us whether an array is ordered. It's just for demo purposes. The code doesn't make very much sense, but it's just for demo purposes so bear with me. I want to demonstrate the problem and what has changed approximately in march this year? Okay, so let's write this straight: let's collect these orders.

A

And add a function is ordered, which returns just a boolean. Nothing special. As I said, the code doesn't make very much sense. There are other traits which which you can't use something like this, but we want to implement that ease word for whatever reason just to understand. What's going on behind the scenes now, let's try to um to apply this now. This is order trait to an array so down here we will say: let ordered equals two or left numbers.

A

This is what I wanted to say um this one one, two, three four, that's fine and then, let's print.

A

Weather sorry, print weather numbers is ordered. Okay, something like this. Obviously this doesn't work because the trait is ordered is not implemented on an array of four elements of type integer 32.. So what we have to do is we have to implement. We have to implement the is ordered trait for an array of t, so we need uh here type here, but the problem is that, as you might already know, that an array in rust has a fixed size, so we have to specify the size.

A

Let's do that, so we want to implement it for an array of size. Two we have to specify the size, it is necessary here and then in order to be able to implement it. Let's say this has to be ordered and copy. Okay, something like this. So the content has to implement the ord trait, which gives us a nice compare function and it has to implement the copy trait in order to be able to copy the elements from the array.

A

So, let's implement the is ordered functions. Something like this and let's quickly write a few lines of code to have this done. It's it's pretty simple.

A

First, let's say: if this one, if the self length this one is 0 or 1 by convention, we say then return true, then the collection is or that the array is ordered. Okay. Essentially, we don't really need that one, because we have the length here but bear with me. We will need this code in a second for now, just accept it it's okay, then we iterate over the elements in our array, not difficult.

A

Let mute preferences prep previous, is none and then say for item in self note that our item is a reference to t, because this is how iteration in rust prior to 143, really works.

A

If some item is lower than previous and we can say, return false, otherwise we just advance the previous one by saying some item and at the end, if we managed to go through all the elements, we know that there is no item which is smaller than the previous, so we know it's ordered ascending, so this might be a naive implementation of check whether our array is ordered, see it here.

A

Okay, good that looks uh pretty awesome, and with that we should be able. uh Oh sorry here, we should be able to call this orbit and yes, as it happens, it now works. It works perfectly fine and you can try it. Cargo run.

A

uh Sorry again: cargo run. Yes, it's ordered and if we mess up, let's see cargo run false, and that looks very nice.

A

But what happens if I add another digit here everything breaks, because here, as you can see, this doesn't work. This doesn't work because this has a constant size. So I would need to copy this guy here and implement it again for the size 5.. Of course, this doesn't make any sense, but now at least the code compiles it works for five.

A

What would you do if you have an old version of rust? You would probably write a macro. This has been done for you, so you don't need to write it yourself, but you would write a macro in order to generate implementations for this nice little trait for arrays of different sizes.

A

That is not a good idea, so what we essentially need is we need, let's get rid of this guy here, because it's so so so ugly. What we essentially need is we need here this one as a parameter, and this is what we call const generics, so what we essentially would need. We would need to be able to say this is when and when, as you can see, here is a constant number of type you size.

A

This is what we want to do, and this came in rust, 1.51 and was a very, very important precondition for implementing by value iterations over a race. So let me prove to you that this really works.

A

Let's take that one and let's forget about this old stuff in uh in in uh in rust, 150 or so, and let's replace this one, this guy here with a new version of rust- and you see the error is gone. It now works. This feature is called const generics and it came with rust, 151 in march 2021 2021.

A

So this was an important step towards this simple feature of being able to write four number in array. One two three four: five: we needed const generics and it came with rust 1.51.

A

So now you also learned what const generics are, and maybe you weren't you weren't. Looking at this rust feature before and now you got some additional knowledge of rust out of this demo, not just about iterators, okay, very good. um There I have a question.

A

The question is uh it's just back of t because the vect doesn't have a fixed amount of elements, exactly correct. That is exactly correct, and that is why why you do not need const generics for vectors, and that is why why iteration worked very early very much earlier for vectors than it did for a race, because arrays have a fixed size? Okay, good!

A

Oh thank you. uh Thank you, ono for pointing out that I missed uh us uh a bracket in the print line statement. I I found it myself luckily, but thank you for for pointing it out.

A

Okay, now, why didn't this by value iterator over constant arrays come with rust 151 because we got const generics. So why didn't we get it immediately? Why did we have to wait for another two versions until 153 to get this feature well, for that we have to understand that that that building implementing into it into iterator for const generic arrays would have broken a lot of code, and I would like to demonstrate to you so you understand what was the problem.

A

It is it's a kind of history lesson, because now it just works, but by learning this history lesson you hopefully get some deeper understanding in how method dispatching works in in rust and so on and yeah. Maybe it's just interesting, I hope so so, let's write and copy a little bit of code.

A

So let's get rid of that one, and I see I already spent an hour and I announced that this talk will be an hour, so I will copy a little bit more of the code and not write all this code so uh yeah. This is just a history lesson, but hopefully an interesting one. Let's define our own data structure, let's call it my array.

A

Obviously it's not an array. It's a is a useless structure because it contains only a single element of type phantom data. If you are not familiar with phantom data, phantom data is what it says it is. It is just a phantom data item. It has size zero. It goes away when you compile this code to release, it doesn't exist, it is just a placeholder.

A

Any underlying data structure will be here, and I call it my array because I implement the data structure. My array, please use your imagination. My array is some fancy own array and it has some fancy inside data structure. We really don't care. This is not the point here.

A

We also do not want to use the into iterator trait that rust has built in we implement our own my into iterator. This is exactly a copy of the inter iterator trait that I have demonstrated to you at the beginning of this talk so we'll not go into any further details, because this is our old friend in twitter, and you already know what this does same is true for the iterator trait that rust has built in. I want to have my own one.

A

I want to have my own iterator and I called it my iterator again, it's a copy of the iterator trait from rust, just to be able to play with it. Okay, so I'm fine with that one. Now, in the early days of rust, the iterator for a raise were implemented based on references. Let me show you that one.

A

This is an implementation of into iterator and then also the iterator itself by reference. You see that one. This is this exists for a pretty long time. It is there since 1.0 yeah, um and it was added with const generics in rust, 1.51.

A

So the into iterator trait for references to arrays worked for a pretty long time.

A

So we can add this implementation, this implementation of the into iterator trait for references to my array in our all in our own simple example. Here, so let's do that this is old code. This is just a simulation of what was there before we had rust 153.

A

So here you have the reference to the array and of course we also need the iterator. Let's do that iterator again, just the phantom data. We don't care about. The implementation behind the scenes, we just can't care about the data types, and this is the implementation of the iterator, in my case, my iterator, and in this case again it's about references to t. So this is a reference iteration.

A

Okay, are we good? So let's try this guy. Let's try this guy by creating this one here some instance of my array and then we call into itter. This is not a reference. My array is not a reference. Arr is not a reference, but still because there is only a my into iterator based on references.

A

Rust will automatically pick the by reference iterator and will give you an item of type reference to integer 32. So, although we don't have reference here, we get a reference iterator, because this is how rust works. This is how rust method dispatching works.

A

I could easily change that one sorry to a reference and everything will still be fine, nothing changes, but I can also leave that out and the system will automatically recognize. Okay. There is only an implementation for the reference here, so we are good remember. This implementation has been here for quite a while, so this code might exist.

A

People might have built code that relied on the fact that calling this into it gives me a reference and not a value.

A

So what happens? What happens if we implement a by value implementation for into iterator and iterator?

A

Let's do that? I've prepared this code copy. It.

A

Should sorry not here.

A

Should be here.

A

Let's copy the my intuitor again, this is by value and let's copy the iterator, you see. This is not a reference just like up here. This is a value and guess what just by adding the capability of doing by value iteration we broke existing code because now the into eater thinks it should take the by value iterator up here, because this year is not a reference, but it is a value.

A

So if I add the reference here, everything is fine again, because now the system knows I want to iterate a by a reference, but if it is like that, just by adding implementations here by adding implementations for traits, we break existing code, and that was the big problem.

A

The rust team, the rust compiler team would have broken a lot of existing code by just implementing a buy value iterator for a race, and that obviously was not acceptable.

A

So there had to be a different solution, and luckily there was there is now in rust a trick, and I want to show you this trick in the last demo that I will do today. Okay, let's get rid of this example. You can find this example in my demo repository and I would like to copy let's see here.

A

This little bit of code and walk you through it, it's nothing new, it's just things which we this one, it's just things that we have done before. Okay, and I want to use the std uh there, it was the std back into it now this is good.

A

This is nothing special, it's just what we did before, but I need to. I need to have this few lines of code to demonstrate the point you see the first one is a vector we have that before and we use integer to iterate by value you see value here. I have a reference to a vector, we call it into iter and we get a reference to the elements inside the vector. This is how vectors have been working since the dawn of time. Okay, now, let's try exactly the same with an array.

A

This one is, as you can see it here, no reference. We call intuitor just like here, no reference, we call intuiter and for vector we got an. I 32 ref a value, not a reference, but you see what's going on here in arrays, we still get a reference, and this is what rust, what a trick of rust is in rust. 153.

A

The compiler recognizes exactly that scenario where you're called into it and changes the behavior to be backward compatible.

A

So, in contrast to the vector we get a reference and not a value, but we also get a warning that this will break in the upcoming version of rust in the ros 2021.

A

This will break so it is okay to break this functionality in a major version of rust, but it was not okay to break it in a minor version in a minor release of rust. This was not okay, so therefore they did this trick. So if you do that one, the array will behave differently from the vector, of course, if you want to explicitly call into ether, you can do that by using this syntax.

A

Here it's essentially exactly the same as we had before, but only different syntax, and for this syntax the rust compiler will not do the trick and you will get a value and not a reference. So this already new behavior, no warning this trick to be backward compatible in the future. This will break and in the future this will give us an i32 and not a reference.

A

And because we finally got the implementation of into iterator and iterator by value, all the fuss that I did and talked about in the last hour was done in order to enable exactly that one see for I in array, and we do no longer for arrays get a reference.

A

We get a value, and just this simplex is already it stays there for backwards compatibility.

A

This is what I wanted to show you. I wanted to give you a closer understanding of what's going on behind the scenes, and I hope it was interesting for you. Let me quickly summarize we started with this one. I showed you what's new, then we dive deeper into what iterators really are. I showed you some interesting behaviors of iterators like consuming iterators, like mutable references by value iteration and reference iteration, and then we use this knowledge to understand why it took so long for us to learn this trick.

A

This trick that I showed you value iterations over race and with all this information. I hope you had an interesting hour.

A