Rust Programming Language Rust Cologne, 5 Sep 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Rust Meetup Cologne (2016-09-05): Arne Claus - Why I hate garbage collectors

Description

Rust Cologne User Group - Meetup September 2016

More on http://rust.cologne/2016/09/05/compile-to-js.html

Help us caption & translate this video!

http://amara.org/v/2Fi9/

A

Yeah so first talk I'm glad to be here so yeah, it's my first time here, I mentioned this and I'm very glad that I can help this talk. Yeah I have hope this talk already once in my life, but in a very short condensed form. So this is actually the first time I'm doing extended. Edition and yeah don't be scared about the title, because this is clickbait yeah, so I always attempted to add a little subtitle there so that you can see what it's actually about.

A

So it's not about not only about personal opinion, there's a lot of personal opinion in there, so I hope that would be discussed later on, but there's also some some code which I base my things on on my thinking self. So it's actually a thing for you to think about what you do daily life and what maybe can be improved.

A

Okay, so, as promised, I do a little introduction about myself, so yeah I'm, not coding for like like depends on when you start counting like 15 years or 10 years and actually started as a computer graphics programmer. So that's why they are things like HR in there and I did a lot of C++, but also Java Python, C sharp with all the stuff and I'm currently doing a lot of go.

A

You might never see that there is no rust on there just simply because I don't have a project yet so what I want to f1 and there's another thing you might notice there most of their language and actually garbage collected Oh what the hell yeah, but as you can also see the C process of the biggest ones. So that's basically, where base all my stuff, oh yeah, I'm in case what you want at your.

A

What do you want if you're wondering what I'm doing I'm, currently a site, reliability, engineer trivago, which means I, have to deal with all these except project break stuff? Okay, okay, enough of the start, have fun so okay, I want to play a little game with you and I know that you all have written C++.

A

So you can totally read this I guess everybody can read this I mean this is very, very simple, and so we have a little class here and we don't talk about sensors just a little example, and so we have a member and we create a new member of type event filter and we put it to some Singleton reduced it along with our this pointer, and we found out that this causes a leak. So what what do we do? We have a C++ programmer. So first thing we look at the first member.

A

We see yeah, it's a member, so I own this. So this is mine, so there's a part of my class, so I own. This and I created this. Even the constructor, so I take care of this. So this is mine. Normally it is so, but you see that this is passed actually to the register, function and you're just wondering yes, this may be cleaned up there, hopefully not because I own it. So you think about ok. Is it's really correct that that passes to that?

A

Maybe it does maybe not so I probably looked at to reduce the function and see. Ok, there is no delete in there now. So what is basically the question? Where is my delete now now there is no destructor which deletes my object, so I own it I need a destructor that deletes it. So that's rule, so that's not there, but this is actually not everything. There's a tiny little thing more and the question is: who is only me so what you start is you you think about rules? You think about a lot of little things.

A

Actually yeah. You mainly think about ownership at first, so you think who owns it? Who has to destroy it and so on? So you want to be clear about who owns what- and this is not only when you write code, but if you are also, if you write code for others, then you want to be expressive and say: ok, if I take this I, don't do anything with that and besides that, there is of course a few other rules like for every my look.

A

There must be free and we C++ there's also new and delete same rule, and you don't cross these stones. It's like Ghostbusters and I, don't do it, so it will break yeah and if you're a little longer the game, then you also know this little acronym ray so who knows ray Oh more than I expected, ok, so for all the others. I will explain it later, but you know the problem with C++ and all these rules that you might gather over the years of programming is they're, not really rules.

A

There are guidelines, so they're there to break so people don't follow the rules so yeah, it's basically a mess, so you try to make people follow rules as best as you can. So you come up with a concept that is normally called contracts and what is the contracts? Let simple thing: you have different kind of function, calls for example, so you decide based on your function, called type what is allowed or what you just want to display to your two year using programmer what is actually allowed, what you do with it?

A

The first thing is pretty clear: it's a copy, saying: okay, you keep your stuff, I keep mine, so I do whatever I want the second one is yeah saying base the optional, but this is like old style. This is a bit of a yellow, because basically I could do everything that I want, but I promise I won't do that. Fingers crossed the neck. Ball is actually cool. This is C++, so you say: okay, I borrow it and I can change it, but I won't delete it so you're the owner still but I.

A

Maybe you want to play around with it and change things. I can say: okay, I want to just read it. So you put constant front. Then you have some arcane stuff. This is the move operator. This is a little cane, so I don't explain it here, because it's a I had to read like three times through the standard two weeks to four understand what it actually does, but yeah the other one is a little bit more understandable basic says there can be only one.

A

So if I passed something I, take it away from you and then I put it into there and then he has it. So this is saying: okay I'll take away your ownership now, I have it. So that's how I express this and the last one is like hey. Let's share things. Yeah you can have one I can have fun the last one cleans up pretty on so pretty neat I mean everything is expressed. Yes, isn't it cool, yeah yeah?

A

But well it's not that easy, of course, because they said guidelines and the first thing is this really a deep copy, so the implant end of my class has to make sure that everything is copied. If you forget something you'll fight yeah, then you suddenly have ownership of something. You didn't know that what you have ownership on you know. Second thing is like this is the reference it's time to say? Okay, I won't delete your stuff.

A

Oh yes, of course, I can delete your stuff because I can get the address of your reference and then can delete it if I'm really the bad guy. So nobody prevents me from doing that. So mmm last thing is: it is Const, but in C++ you can take away the cons because you want it, you normally don't do, but you can- hmm and last but not least, of course, if it's a SharePoint, you can still access the data in this point and delete it if you want to.

A

But this is like hammering: okay yeah, okay, so basically you can define conflicts, but nobody checks this. You have to know what this means yeah and you have to learn, and normally you you'll learn it the hard way. So you fail a few times you're over and at some point after a few years.

A

You know: okay, I, do this, so people understand what I'm doing and all this stuff, but you get along with it and later you will stumble upon something that normally is passed on from Senior Programmer to Senior Program, and this is static code analysis. So when I started, nobody used this now everybody uses this, and this is good because static code analysis can help you now it can help you to define to see your errors, to follow the rules to and to enforce the rules.

A

Yeah so- and this is basically what you do nowadays- you have a set of rules. You'll learn them on the hard way and you use static analysis to fix this. This is C++ Estonia, so I'm going from that, so a mess yeah and some years ago. So in 1996 yeah, when I was still a yeah.

A

There was a new program that was created, and this was basically perceived as the dawn of garbage collection and they said, don't worry, we got you covered. Everything that is allocated will be de-allocated, so you won't have any memory leaks anymore. Everything is fine. Okay, good cool sounds great. Nice marketing won't have a look at that. So let's take Java 1996 first version, so same code as before looks almost the same. Yeah and yeah F knew we've got a garbage collector, so not instructor need it. Everything is covered, isn't it sent out?

A

No, there is a leak in here. Why is there a leak in here because we just transferred ownership? What this is me, so we have a crate object. That is hoped by someone, and then we pass it on to a global instance that now holds it to and we never remove it from the global instance. Now imagine you have a program with the treeview on the sides.

A

It's not that I worked on a company that had this and every time you open up a tree, things get registered in a global event registry and when you close it it doesn't get unregistered and now you're a year. Cto you're CTO comes to the point where it says: okay, we need a test with 50,000 objects and you open it. You close it your MIT and then your application stand still for two minutes. Why? Oh you have got a lot of things going on yeah and your memory is going like this.

A

It's saying: hey, didn't you promise me that I won't have memory leaks, but my size is growing and growing growing, so this is definitely a leak.

A

So what is happening- and this is something that I call a lingering objects so that the objects that are just laying around that are helped by some instance, but they are actually not in use anymore so because you forget to unregister them or remove them or whatever, and this is also a leak- and this surprise is not catched by your garbage collector, who just promised you to clean up after you. Oh that's not good!

A

So, and what actually happened so because the garbage collector told you you don't have to worry, you did not worry and you did not have a destructor, because the garbage collector just took it away from you, because you don't need it. You don't need a destructor, so you didn't care you just did it hmm bad thing.

A

So there is something that rarely happens to you when you C++, because you have to think about this all the time, because the consequences are so much worse and suddenly, all the mess that you have in C++ becomes somewhat a little bit better because you, as a pro I, think more more about ownership but resources abroad, resource and other stuff. So you keep your stuff clean in Java, you don't normally, because you don't care many many many times and even worse things that you used to clean up who are taking away from you.

A

Okay, so let's have an overview about what leaks we can actually have so because lingering is not the only one because lingering because there's another one which is resource leaks, so you open up a network connection to the database and you have to close it at some point same thing. So, ok, it scares of all these leaks. So this first one clear garbage collector. Can you cut a second one?

A

Well, something you have to do it yourself? Try find some way. You will do it. You can do it yeah and at that point that's not my personal opinion. That's where say the marketing lied, so you promise me to, but you didn't keep it up and you even waited voice for me, because I don't have tools for that. So but yeah, that's it! Let's, let's do any better than that! I mean it's old. So how does C++ work of that? So how does C++ fix all this thing? There's one answer: Ravi everything can be.

A

Everything here can become appareil, and why is this way? Because C++ understood one thing very, very well, and that is memory is just one resource database connections are a resource. If you have an event handler, this can be a resource to everything that has an open. Close semantics is a resource, and if you have- and if you see what results acquisition is initialization, which is the chronometry does, then it comes in clearer because it states just a few simple things: the constructor acquires open new.

A

Whatever started this doctor releases, this always the same yeah the constructor acquire destructor release and the destructor is guaranteed to be called at certain point in times, and this is the guarantee that is pretty pretty important because stack mechanics help you here. So have you put something on stack and your rubber very goes out of stack? The destructor is guaranteed to be called, and that is how C++ or modern C++ uses all this. This automatic cleanup mechanics.

A

So this is what actually I saw this first in 2002 or something like that and when C+ has 11 was read. This was a new thing, but okay people did it for quite a long time.

A

So what does it mean in detail? So this all relies on an automatic call at a very common function of a very common function like the destructor. It does define time like the end of scope, can be something else like end of function or something it by the define time and a function. That is well-known like the destructor yeah, and it's also necessary that this destructor call is guaranteed for chaining. So if you have an object, that has an acid object and that's the object, all these destructors will be called and chain. This is guaranteed.

A

You can rely on that at a specific point in time. You know this will happen now at this point, and this is pretty much cool. So let's have a look back at the code. So how do we fix our code? It looks a little messy, but it's C++ come on yeah, so, okay, so, first of all, we create something called a share pointer. So we share a resource. We don't use new anymore. This is C++ 11, so we say make shared, which internally calls new but come on.

A

Then we just said and we D registers in the in the destructor, which is basically the open, closed semantics yeah. So every time my full object is destroyed, I automatically unregister. So if I put it in a vector or an array, a sort of and put it and delete the vector it will go through the whole vector and remove your who call the destructor, remove it. You're done so clean up. Chaining works.

A

Yeah yeah, so we basically they only fix that we did it's. The unregister notice that we didn't call the elite here, because our SharePoint is taking care of that. That's us tech, mechanics.

B

A

Not because of the disrupter and the destructor is guaranteed to be caught.

B

A

Exactly yeah so to fix is actually okay and the cool thing about this is this is exception safe. So if I create something an object of foo and an exception is thrown and it's on the stack important, then it will be cleaned up because exceptions endoscope to ya, know new and delete is required. Well, it's hidden, but you don't need to use new and delete a modern C++.

A

You have no resource things anymore, and this is basically a resource leak yeah and are you over defined ownership like if you use this new point of stuff that I was taking over for boost? If you care about that, you have defined ownership and terms of saying okay. This is the shared thing and the last one cleans up. This is paid. We share pointer yeah, but you still have to stick to the rules.

A

So if you still have to call new, you have to make sure that the distractors call, if you don't do, then you still don't stick to the rules. So actually it's cool. It's not pretty, but it's cool yeah go on so yeah. You can do better, probably, but it's at least for many resource management. This is a very cool thing to do, and it's very consistent.

A

You keep thinking about the stuff and you have a consistent way of solving things, and this is what is pretty much important, I think so yeah now, if you now remember back, a called Ray is something that just requires some function at some point in time that is defined. So maybe we can use Ray with jobs, garbage collection- maybe it's still possible. So let's have a look at the leaks again and yeah.

A

We don't care about memory leak anymore, because this is a garbage collector done, but we care about those two, because these are things that we have to bear to have to manage manually in three C languages. So first thing is something called the finalizar, so it turns out there is a destructor.

A

We could use it, but you pretty much don't want this, because this thing is totally unreliable because the finalizer is called when the garbage stick that finally tries to to clean it up, and then it might call the finalizer which is done by some threat in some distant future. If you're a pretty bad luck, so you have no defined point in time, so the finalizar is a bad decision for that.

A

Like think of late now you have a database connection and you want to have to close it now, because you need to free up resources, because if a lot of things go into the database, then the garbage collector says well not now, maybe in five minutes not good, then here's something called defer, which is actually quite nice. We have to sing and go, and this originated in d- and this is basic, saying, defer this function.

A

Call here to the end of the function, so basically use something like file open and then defer file closed and they make sure at the end of the function. Close is called, and this is also exception, free and such so. This is okay. I can live with that I mean you have to know the defer, and you have still to think about that. You have to need to close things, but at least you know the close function already, which is a defined function, and you have a defined point of time. So it's okay again.

A

Sadly, you don't have this in c-sharp or Java butcher. Now the big players to be, you know to be fair and then c-sharp and Java. You have two things which is the disposable and the closeable interface, and these two languages are very close related, so they work in a very close fashion. So what do we have to do? Well, first of all, you have to implement the closeable thing yeah or disposable, and whatever have to do.

A

You have to create a destructor called close well same function again that you have just taken away because of garbage collecting you enter again by adding it interface, and then you have it here, nice, one so yeah, but that is sadly not all because who called clothes- and this is done normally by a try. Finally, Bach yeah, so you try something and if the exception of thrown, then you close it. So basically, you want to wrap away this exception safeness, so you make sure that close is always caught in here now.

A

This is a lot of code and you even have to defined who at the top, because otherwise you don't have it there on the top, because otherwise you don't have it at the bottom yeah and it's yeah pretty pretty unpretty in my opinion, but later on, like fifteen years later, they finally came up with a solution that is a little more elegant, and that looks like this, because new centers is always fun. Let's try at brackets and then everything is fine at the end.

A

This assumes that you know that your class, if it has closable implemented and closed, will be cost. If you use it like this, if and you know and yeah again wave routes yeah. So this is actually what you currently have so so what I want to tell you about is the problem. First off is, if you forget, to think about ownership, which definitely do all the time is he plus plus you make things problematic. You will forget things. You will forget to close things.

A

You will forget to unregister, because you're not used to especially junior programmers. We do that I mean we don't talk about what junior progress to to C++ code, but yeah. So this happens. They don't learn to think about that I recently, just two days ago, I had this problem because not as a normal program, another senior but normal program, I, don't know how to call this made an extension to a software road, and he made this mistake. He said: okay, we have a garbage, collector I, don't care about allocation, that's news!

A

This and put 80,000 messages per second through this and like 80,000 times per second, there were like I guess it was 50 allocations made or more. This is bad because he didn't think about that. He came from PHP, so you don't care about that. You don't you just do it in PHP, okay, so and the next thing- and this is the best thing about C++. If you haven't no enforced rules, you will miss things and all right.

A

Like a few days ago, I had a very nice tweet coming up and this pretty much sums it up. I like this very much it's like if you blame the garbage, collector you're, blaming the wrong guy, because basically the garbage like is steaming after you, and if you do bad things, then yeah garbage collector can't help. So it's not the garbage collector, that's bad! It's the thinking that is not working, but the garbage collector led to this thinking, in my opinion.

A

So what to do so static analysis, cool thing yeah, but some things can only be seen at runtime, so any checks and the language. So you need to put things like ownership and all this lifetime stuff into a language and whoops. What do we have rust and that's the point why I am interested in rust?

A

The cost because rust just thinks this totally to the end, and this is a very cool thing, and that is what is so fascinating about rust, because this is actually the first language in the years that doesn't say, garbage collector, solves all the things they say manually manual resource mentioned still has its place and we're gonna make this story great again yeah. So rust is actually interesting because of exactly that.

A

But there's one thing: I need to talk about that, because yeah I'm coming from a embedded systems, graphics part. So there is a point where I'm out of resources and I need memory. So what do I do and many remember memory management, I called delete, which basically says this is now free ..

A

So, basically, if I have like a 32-bit systems, there wants, like one gigabyte of free space, I delete one gigabyte in a habit and an instance I don't have to use, swap I, don't run out of LS press and what else for the garbage collection. However, if you say, okay I need something you tend to just now, things you say equals now, and then they say: okay, garbage, collector, GC, collect I, say I really need this now garbage collector. Please please, please do this and the garbage truck will say. Never not yet.

A

No, no, don't want this. No! No! No! No later, because GC select is more like a guideline, you know, I say if you've got time, do your garbage collector? Please please, please write someone. Please could you free up? I mean I, need it now, but maybe yeah, and if you got a real-time application that tries to render a few million points and 30 frames per seconds. This is really annoying okay.

A

So to sum it up, why I hate disease, oops, wonderful, okay, so I think developers get a false sense of safety yeah because they start to think GC was taught they'll help me all the time all the way and will save me of all kinds of leak there. It is not. It gets more complicated because all the leaks that are left are the complicated one that you can only see your soft in runtime and they took away our tools. So the problem is not the not solved, it is actually delayed and made worse.

A

So that's my opinion yeah and on the very very end- and this is what people might posit I talk about- that a lot is there's overhead. So you need to keep track of your references and all this stuff yeah. So you have like additional locations that have to be done besides your locations, that you can actually do the garbage collection and you have to stop the world stuff and all the son sums up to you don't want this in an embedded system yeah, but this is at the very end.

A

So why didn't I talk about that? Because you have the choice? If you do a non piu project, you can think about. Do it really care about memory management? Am I on on an embedded system, for example, or the graphics card or whatever? Then you choose something that does not have a garbage collector because you care and you want to care. The review writes some UI applications.

A

That is not that big, then yeah go for a garbage lecture and then please take care of your connections or your finals, but that's it so garbage collection is not bad at all. It helps you developing, but you have to decide and now hopefully I think that you have a good picture on when to aside what to take. Thank you very much.

A

So if there are any questions, I just want to say: I have a little daughter, that's 2 years old I'm used to very stupid questions. So there can be no stupid question.

B

My hf2 I think I'm not going to be like using the microphone and everybody else. You should too. If they have a question that would be nice for the recording. So first one you said: destructors are guaranteed to be run. Yeah and I know. One of the big bad epiphanies rust sorta had close to 1.0 was oh destructors on a parity to be run yeah because you can basically get the Reverend cycle.

B

If you do reference counting and then reference yourself, um it's destructors running actually any sort of guarantee didn't see, post passer, don't you have the same carbon with shet pointer yeah, so the.

A

Circular reference thing is actually one of the points where you have a problem when you start reference counting, because if you have only reference counting, then you have this kind of loop, so those reference each other and you cannot get rid of each other because you're referencing each other, so they have this little universe and they keep spinning and spinning. So this is not software this practice.

A

So this is actually a very bad thing to solve a very hard thing to solve, and if you look back in the Java version, history I think it was still Java 1.0. They had this problem with the garbage collector because they did resource yeah Remi reference counting. You can solve this, but it's not easy. So how?

A

How do garbage collector solve this? They only look at the top level instances, so those that don't have a parent and then there's throw those. So if the top sir and the top level, you don't have this kind of loops, so you can assure that all the objects below are actually cleaned up. This is a problem of reference counting service. Another problem is actually bound to destructive things, so this is just one hard problem, so how do you solve in C++ weak references yeah? So this is the way you do it I.

A

Don't like this, that much, but normally I'd take care of that. My reference point is: do not reference each other. So if you can do that, okay.

B

So that's that's half the answer cell right, because that means destructors are in fact not guaranteed to be run, but you have to be careful about. Our second question is I feel like and that's not my area of expertise either, but you seem to have a very like java understanding of GCS like from what I hear there is a lot of research into disease. There's a lot of research into real-time uses non stop the world. You cease all that kind of stuff.

B

Any comment on if that improves the situation at all.

A

One thing: why do I mention Java, because I started with Java and I began to hate it because at the time I used it? It was so so. So that's why I used to pick on Java all the time. So actually you saw that I use and go a lot. So go. Has one of the currently I think best implementation of garbage collection currently available for free- and this is totally concurrent- this is really running very, very nice, so some go is actually the first language way. I want to accept.

A

The garbage collectors is because I I don't have to write all this allocation stuff myself, so in C++ I have to write all these things myself, like I need to have to write a bucket, a locator or a body allocate or whatever, to keep teka for memory and all this stuff. I don't want to do this all the time, but I don't want to have the GC alright. Also so goes the nice thing in the middle so because of the concurrent garbage collection and in Java.

A

You actually have this too, but you have to pay money for that. As far as I know, so there is this, this concurrent garbage collector that you can buy from some company I guess this is like most of the the stock exchange. Things are using this because they need to be real-time yeah. So there are possibilities.

A

Yeah you still have an over it and you'd still feel it in certain situations, but it gets lower and lower lower. So that's also why I say the runtime part is not that important, because this gets better and can be changed that can you mitigate it and all this stuff?

A

It is it answer your question. Actually, okay,.

A

A

Okay, then, thank you again.