Rust Programming Language RustConf 2018, 6 Sep 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RustConf 2018 - No Spaghetti: Designing for Understanding by Brandon W. Maister

Description

RustConf 2018 - The Opposite of Spaghetti Code: Building for Understanding by Brandon W. Maister

Every moderately sized project is both a library and a consumer. Rust has an almost dizzying array of features that can be used to add structure and hide information.

In this talk we'll discuss using types, traits, modules and crates as the tools of encapsulation. To show how they work together to effectively superseed similar information-hiding tools in procedural and OO languages we'll grow a simple type into a collection of language objects. We'll discuss the thought processes that go into deciding to add a layer of abstraction, and how to use each layer to maximum effect.

A

Yeah, like Steve, said I'm Brandon I am going to talk about translating everything that the software industry has learned over the last 80 years into rust. So this is sort of design and architecture and rust, we're all very professional people. I assume. None of us liked writing spaghetti code. So so this is good, so yeah I wrote a demo and I called it. Lasagna lasagna is I, think a really good metaphor for good code. When you think about the kinds of things we try to avoid in spaghetti code, we don't like untraceable paths.

A

We don't like a lack of separation. We don't like, like the core architecture, getting just smushed up with everything. Contrariwise, lasagna is awesome. It has layers that are big and flat and easy to see. There are tools in place, ie, lasagna, pasta to keep the layer separate and the layers still end up interacting. Otherwise it would be just boring food. Interestingly, as I was writing this talk.

A

This slide, in particular, I discovered that lasagna code is a term of art which means code that is so poorly separated, and it's layers that you can't change any anything without changing everything. But by that point, I had integrated the metaphor into my talk so thoroughly that I couldn't change. It.

A

So what am I gonna talk about I'm gonna talk about leaning into strictness, how the fact that rust is difficult. Everybody knows but having to think about. Our api's actually gives us the ability to really clarify our thought.

A

I'm going to talk about using modules and privacy to keep our code simple and to keep our API simple, I'm, going to talk about using encapsulation ergonomics and that's separate to modules and privacy I'm going to talk about abstraction and dynamism, because abstraction is the last rung of the like totem pole, ik you're, very senior, when you're being very abstract, yeah. That's what it is to start I'm going to introduce you to my dominant metaphor. It is an onion. Just like this talk. We can't!

A

Actually you don't you don't know, what's in the onion, yet here's the code I'm, starting with a lot of code when you were when you Oh. Actually a very important part of this- is that my fool eye metaphor: in addition to being an onion, is a cache. I love caches. They are super simple. To start with. Every time you start implementing a cache. You realize that you have contrasting requirements. You have people who want to be fast.

A

You want to have people who want to be like correct for some reason, and so you always like there's a lot of things that need to happen, but it's a very simple metaphor to keep going through with so we're gonna implement a cache, and what's the first thing that you do when you implement a cache, you just stick stuff in a map and that's like in rust. This looks pretty similar to the way it does in every other language. You've got a bunch of code. You stick stuff in a map.

A

The downside to just doing this is that there is absolutely no encapsulation. If you want to change cache mechanics, then you are changing the code that is using the cache in this case, like we want to do something. If we want to change how efficiently we do something, we've got to modify our cache code. If we have a lot of different things that uses that operate on the same data, then we've gotta change the code everywhere, that's rough, so we are just like blending.

A

Our business policy throughout our code- if we do it this way, ideally we would avoid that. Even here, though, we have rust guarantees telling us something this, the ref mute here is saying: hey this function might be modifying our cache, it's not just accessing it, so even in the simplest possible API you should be thinking about. Is this telling me something about the way that I mean and maybe actually Russ is getting us a little bit more defined? Maybe it's a little bit more like this? It's not so blurry, it's a bit! Blurry! It's!

A

You know, you know our business policy, it has a clear boundary, but how do we take it to the next level? This is lasagna inside of a tray.

A

We use functions, right, separate, separate functions, and this code uses a few more features to sort of climb the pillars of abstraction. Everyone knows what a module is. Probably modules are rusts, smallest unit of privacy. They are the thing that you use to keep your code private. In that sense they say the same. They serve the same purpose as files and C classes in Java, capitalization and go and underscores in Python.

A

Module is also the smallest unit that can be broken out into their own file. You just extract the curly braces into their own file, name it calf, RS and but replace the whole block with a semicolon. That's awesome. I often realize that something that I'm doing should actually have a stronger privacy barrier. Just start writing module in my current file, and that makes it sort of trivial to copy paste code around and revert stuff without without it being super complicated.

A

Pub crate means that anything inside of the current crate can see this crate public item. This is a feature that's fairly unique to rust. There aren't very many other languages that provide a like compilation, unit privacy barrier.

A

So we've established a module. That's the bright orange part in the middle. It now has a boundary that is the privacy. It has something inside of it and something outside of it, and the pub crate is the tiny little hole inside of it. There are, though, problems with the implementation that we've got so far.

A

Ownership of the hash map is sort of spread throughout this code is circular it doesn't. The cache itself doesn't have full control over its own destiny. Specifically because do something knows about the implementation details. So what can we do or what's what's actually still good about this.

A

We do have a little abstraction boundary in the form of generics, so I'd know, 1977 is the earliest time I can find of like static polymorphism coming around in a de K and V here are just generic generics. They are similar to, but better than C++ templates or Java generics they're, not they're, not always better than C++ templates.

A

Depending on what you're trying to do. The where clause aka trace constraint is what distinguishes rust generics from C++ templates or Java. This particular use case here this, where K equals hash plus X plus clone, is a.

A

Demonstration of the composition over inheritance philosophy that runs through rust, instead of being required to create a cache into a hash map, can insert into hash map it or interface like it would in Java.

A

We just say that things that go into hashmaps require hash plus X, so you can sort of generate your own folks on ax me of small composable parts.

A

Additionally traits act as a type level privacy barrier, the same way that module privacy acts as a code level, privacy barrier types prevent you from knowing what's go traits prevent you from knowing the actual type, that's going into your code or being operated upon.

A

Modules give us the ability to hide items because they're not important to users. For example, I've got this pretty nifty syntax of just using dots instead of types. That's not legal I just didn't fit everything on this slide.

A

Sticking an item inside of a module I'm, not marking public means that we can depend on any invariance being guaranteed by a public API. That's just basic privacy. In this case, that means that get or insert know is in significantly more control over its destiny. That would be if ensure exists was public.

A

So why would we not just use pub for everything?

A

The only thing that is keeping you from making the mistake of interpreting of just providing the internal access to everything that you do is speed, bumps that you place for yourself. Discipline is hard and making a conscious decision to publicize your internals is very important.

A

The smaller the holes in your API, the easier it is to think through adding another one.

A

Additionally, modules are items too so a module, even though it is accessible, even if a private module is accessible from inside module and which it is defined, but items inside of it are not public. So here we have defined. This ensure exists as a public item, but because it is not Reax ported from its internal module, it's still private to that module.

A

So if we hadn't marked ensure exists here as public then get or insert would not be able to access to it, but because we've made mod internal private, we haven't marked it as public, it's not accessible to the outside world.

A

If we wanted to mark it accessible to the outside world, we would have to re export it some way. This is a super common pattern in rust. It is I believe called the facade pattern or just a sodding stuff. It makes it easy for you to organize your code as complicatedly, as you would like make yourself feel very, very, very mature and still expose a simple and easy to understand. Api.

A

So this is what we just wrote: it's a ton of syntax and important. The thing that I'm really trying to emphasize is that even when you're writing the same possible thing, Russ really wants you to be explicit about what it requires.

A

What's the next step, another layer of encapsulation in this case we are it's very similar code. There's a little bit new here, there's a struct. We now have an object.

A

What else is going on here? We are implementing stuff on this. Struct in poles are the way that you write. You access, method, call syntax. They are not fundamentally different, except methods aren't different from functions except that they're, more creative economic to use.

A

This is an actual design decision, though this the internal properties of the struct are public. That means that anyone inside of the inside of your crate has I can use this cash in any way that they would like up to and including initializing it separately or accessing its individual. Its internal items after it's been created. I would argue that this is a completely reasonable choice for an internal or otherwise unimportant object.

A

If you expose the public parts of your objects to things that you trust, that means that there's a whole lot of code that you don't need to write, and that means that you can have a more ergonomic API for your users if you can trust them any something else. That's new here is that we've moved the trait constraints onto the object itself. That allows you to move.

A

Even though it's possible for you to add trait constraints, onto functions inside of an poles, putting pushing the requirements up to the object itself makes it. It makes for better error messages, and it means that your objects are just sort of more straightforward to use. And you can describe what is actually important about the objects that you're describing defining a.

A

Subtle, but awesome part of rust, simple blocks is that it's actually possible to define require different constraints on in implementation than on the object itself.

A

In this case, the cash that is being implemented requires that the case be hash and ech, but the specific implementation of the cash is requires that the objects be cloned, so I'm just moving it there.

A

Let's talk about the error messages that I just described here, I'm defining a wrapper type that does everything necessary to be inserted into a hash map, but does not implement cache clone, which is required for the cache to work.

A

So, if I try to insert it, I get an error message step state trade center, clone clone is not implemented for wrapper. This is interesting because, like in most other languages, the error would actually come on a lot, the instantiation line, because you would be impossible to construct a cache of what you want here. The error is at actually trying to call a method.

A

There are lots of times where what you want to do, where you have either a more ergonomic or more efficient implementation that you can provide for a specific operation, but it only holds if some specific trait constraints apply. That's pretty much the only time you would want to do this every other time.

A

If you can move trait constraints up to the object, the you will end up with a sort of more pleasant API, because you don't want people to be using your object for months or years as long as rust has been around before they realize that they can't actually call the method that they expect it to be able to call only one more thing that I wanted to talk about at this level of abstraction.

A

Even here, the module is still the only unit of privacy. The getter insert is implemented on the cache object, but the it still can refer to other methods and internally. So this is a like it's a simple rule modules or board privacy as implementations or never don't affect privacy rules.

A

Another point, though, is that, even though the hashmap itself is now and working at floating around out in Maine we've sort of tied this down a little bit there, it's a little bit less of a loop structure, there's a little. The cash is in much more control over its own destiny and knows what's going on and we've improved things a bit, so everything so far has had the caveat of you have to trust your users, which is obviously crazy.

A

So, let's, let's add an invariant that we need to maintain in order for us to sort of demonstrate some some more features in this case it's a cash statistics. Obviously, in order for our cash to be accepted by our CEO, the hits and misses need to be correct. We can't trust other developers, so we mark you, don't mark them public.

A

That means that we know that we are the only people who can modify these attributes, but it also means that we need to implement a ton more functionality. An example here is that before we didn't need to implement a new method, but now now we do.

A

So since we've discovered that there was one method that we weren't implementing before to make sure that we are catching every possible method, dead-like would allow for our cash to be correctly usable or like pleasant to use. Let's say, let's clean up some of this API fundamentally I was talking about how cash is like we're tying business logic to itself hits and misses. Don't actually have anything to do with cashing we if metrics, are sort of an orthogonal concern.

A

We want to change if we want to change which metrics we capture or the reason that we capture them is going to be different to changing the cash implementation. So, let's move the hash map out here is a we've just separated we're using two separate modules next to each other. In this case,.

A

Okay, that gives us a fairly clean separation of concerns. Cash module is going to be changing because we're changing our cash implementation. When you're doing code reviews when you're handling stuff it it will you're much less likely to make a mistake if every object is responsible only for its own things.

A

The this also actually demonstrates fame, rusts famous commitment to like pay for what you need, and hopefully, zero cost abstractions in this case, because of the zero cost abstraction is this hashmap in this implementation is likely to be inlined and functions are going to be inlined and you're not actually going to have any kind of runtime cost for splitting things out this way, unlike in most other languages or yeah, it is also yeah.

A

So because we split the logic up into their own smaller units we were going to have. We have like this notion of a core caching policy section inside of which are we have our own privacy barriers. This is starting to look a little bit more like a shallot than an onion, but you do it. You can we're still controlling access. We still have these small holes in the public API, which means that we have a clear public API, no matter how complex the implementation gets.

A

Speaking of public API. So what is this answer thing that we've imported we inside of the cache module we've created me insert enum it. The name is a little bit. The fields are a little bit suggestive of use, where's it being used it's in insert if missing so okay, so that that's you doing it instead of missing has been around since basically the beginning, but all of a sudden, now it's public, and previously it returned nothing, and now it's returning and set insert, get or insert, though, hasn't changed at all.

A

Why did we change and sort of missing? It's a mystery. What happened? It's a bad mystery.

A

Cache metrics obviously needs to be able to determine whether or not the item it's interacting with has already contains the item or if it's a fresh insert so we're using the needs of consumers of the API to define what the API is. This is sort of in contradiction to what I said before, where I suggested that you really want separation, and you want items to not change unless their business policy changes, but this actually makes a lot of sense and is required.

A

The infrastructural Co, which actually is responsible for interacting with the real world is going to have needs and as long as it as long as the API pressures are always going from the like from the real world from like the things that change volatility in words to things that don't change that often so we're the implementation is going to change much faster than the definition then you're in a pretty good state. If we had if the cash was requiring changes on the cash metrics and the cash metrics were change.

A

We're client changes from the cash then you're in a situation where your code is in spaghetti and it's time to like evaluate whether or not you're you've defined, though your layers correctly once again coming back to this design, we just want the use arrows to only be pointing in one direction as soon as you've got a loop. You've got a problem and things get more complex.

A

Okay, so at this point, we've done enough work that we know the wrong ingredients that will allow us to create a nice cash.

A

At this point, and usually only when this motivation is when I start thinking about defining traits for my objects, one common motivation that would actually get me to define traits before I start thinking about reuse or common, like ecosystem concerns is testing traits are the best way to create mocks and rust and in general I, just try to stick to the philosophy of use traits as as early as necessary, but no earlier, unlike some ecosystem, where you really want to create an interface before you even have an idea of what the code is.

A

Gonna look like.

A

Coming to us from pretty much any language defining trace, looks a lot like defining abstract base classes. You've got a function and you've got other functions that can defend on depend upon those functions. That's pretty straightforward, but there are a couple more interesting properties and this woman there are no constraints placed upon care V at the top level, but we are saying requiring having placing trate trate constraints on a couple of methods in this particular case. There's absolutely no reason to do that.

A

It's a bad idea when I extracted this crate, this crate the cat, the specific implementation that I was working on required clones, so I initially expected the clone requirement as well. The good news is that in rust, removing a trait constraint is not a backward. It's a backwards, compatible change, so we can just trash them and call it a call. Today, there's only one more fairly obvious property visibility: modifiers go on trait definitions. Traits are interfaces.

A

There is no such thing as a private method inside of a tree, so you that's it also implementing a cache. Once you've got this, we have just moved the in Pollock from on from existing on the cache itself to an input of a trait for the cache, as promised, because get our insert was defined in terms of in sort of missing. It doesn't need to be defined.

A

One slightly interesting property here is that you're not allowed to while you're, while you are allowed to define new, where constraints on implementations you're not allowed to, unlike an input, lock raw input block or an inherent dimple you're, not allowed to add trait constraints on to methods themselves. The entire treatment implementation needs to be needs, the needs, the where clause or the trade constraint.

A

Now that we've got a treat consider now that we've got a treat, we can create a cache matrix type that can handle any type of cache. We've only got one so far, but that's okay, except that this does not work. Traits aren't types.

A

Even though here we are saying that the V is the compiler is telling us that the V is required, we're using the V, it shows up in there, but it is it's not it's not present.

A

The issue is that traits are not types rust only uses only considers traits used when they actually show up inside of the U of the implementation.

A

Luckily, the error message in this case tells us that gives us a pointer to what we need, so we can get around this with the phantom data. Cluj give the compiler a little bit of hints and work around it I'm going to ignore phantom data. This is just like a reminder that traits are type constraints. They are not types.

A

Implementing calf still looks very similar to the way we did before. Now. We've got access to the default. Getter method, get our insert method, so we're continuing to ignore it. One other sort of interesting property here is with the implementation delegation pattern that shows up and rust all the time our cache has a get method. So when we implement a trait for a cache for something that we're, the only thing we know about is that it's a cache.

A

We just call that, and here we are tying it all together with the core implementation, the inherent in pull on the cache metrics block the coolest thing here is we finally have some dependency injection. If you come from Java, this should make you feel a little more comfortable.

A

Yeah, something else is sort of interesting is that the ROS community has decided that functions like getters just get the same name as a field that they get, which I love lots of other languages support this, but it's not actually common in their communities and that's it. The overall. The implementation of cache metrics is straightforward. It just has a few design decisions that you are allowed to make here. We are tying everything together.

A

Is there anything interesting here we created one new line. That was six minutes of talking, and this line is more complex than the previous code. So what have we gained a single direction of implementation? The cache metrics knows about the cache shape, but it no longer knows about the specific implementation.

A

The black uses line is completely isolated from the implementation line, which gives you the, which is a demonstration of traits performing as a code type level privacy bear here now the actual work that we're doing doesn't know what it's interacting with any more than it needs to. You know, okay, now that we've got a trait. What can we do with it? She seems so happy a cache that falls back. This will really abstract our business policy. I, really like that. We've got the type type aliasing.

A

This pretty much just allows us to abstract is fairly complex thing into something, that's easier to say. Any cache is any cache focusing on the axle implementation. We've got a little bit of an odd shape to this struct. It's got an any cache in first position and a vac of any cache in rest position.

A

We, this is an example of using the type system to avoid having to write some tests. We want every cascade cache to have a at least one cache. If we were, if we had just a Veck of any cache, then we'd have to write a bunch of tests that verify that this is caching. This always has at least one cache here.

A

The type system will guarantee that, for us, the only other thing here is that we're using the Builder pattern every we just return itself, and that makes it like we can do with and and and it'll be nice.

A

This is a reasonable implementation, assuming a reasonable implementation of ket cash for any cash. We can now comfortably double our memory usage for absolutely no gain which I like okay. So let's talk about what it takes to build an ecosystem.

A

The most prop, the most important property in a large and complex ecosystem is the ability for users to walk to work together without ever having to speak to each other in any way.

A

The in this case, things that can contribute to that there's small, genuinely public items in general. Your code will just be much simpler to understand and use if it is simple making up items. Public means that it's possible for developers, so we use the work that you've done without communicating with you.

A

Building your own implementations in separate crates is a great way to guarantee that what you think is actually usable is actually usable means that other people can come in and build things for you and you won't have to worry about it, but you still have ideally a clear definition of what like what the core of your code is doing. Everything is going in one direction, and that is the direction that changes the least.

A

So what have I done? I talked about leaning in the strictness trying to encode business rules into the type and life time system. Keeping API small is really something that is hard to do in general and the best way to make sure that you succeed at that is to just make it as small as possible and give yourself for all those speed bumps. You can read the rest of this I'm out of time. That was my talk.

A