Rust Programming Language RustConf 2020, 21 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RustConf 2020 - Error handling Isn't All About Errors by Jane Lusby

Description

Error handling Isn't All About Errors by Jane Lusby

Error handling in rust is incredibly diverse and constantly evolving. As such it is often a source of confusion for new and experienced rustaceans alike. This talk aims to clarify what error handling in rust means, the patterns that exist, when to apply them, and what libraries currently exist to help with the task.

A

Hello and welcome to my talk, error handling, isn't all about errors. Let me start by introducing myself. My name is jane lesbie and my pronouns are she on the internet? I go by yah or yasi, and I've been writing rust for about two and a half years. Now, though, I was only recently hired to do so professionally by the zcash foundation, now quick, shameless plug. I also maintain awesome, rust, mentors.

A

Awesomers mentors is a list of projects and people who are willing to provide mentorship to anyone who asks, if you're interested in finding a mentor finding a project to get involved in being a mentor or getting people involved in your project. You should check it out also to be clear, they're more than just these two mentors. So please check out the website moving on, so why error handling I actually got into error handling on accident.

A

It started as a yak shape when I wanted to open source a library that I wrote for work, but I wasn't happy with the error handling, so I decided to fix it up first, that yak shave ended with me. Writing air air is a fork of anyhow with support for customized error reports via a global hook, similar to a panic hook. I also ended up writing color error. Color air is a library which provides a custom panic report hook and a custom error report hook for air.

A

With these libraries, I'm now able to construct error reports that look like this. Here we can see the basic usage example in it. We have an error section, followed by a span tray section and, if you're not familiar with tracing. This is an extremely cool back trace like type of tracing spans.

A

Then, after that we have a suggestion, section followed. Finally, by an emitted back trace section, I can also enable backtrace capture. So here we now have a back trace section and we can control the formatting of this factory section here, we've just pretty printed it in the style of color back trace, which hides the relevant frames. In this case, it's hidden five frames before read file and 10 frames after main. We can also take this further, though, by applying custom filters.

A

Here you can see, we've got 11 frames hidden after read, config instead of 10 frames, hidden after main, because I've configured it to hide the mainframe. We can apply this custom filtering consistently to all of our error ports. So here you can see we have the same report format for panics as we did for our errors and with our airport hook. We can also bundle arbitrary data with our error reports.

A

We can use this to implement things like custom sections in our error reports here you can see that, in addition to the error section, we also have a command section showing which command we tried to run and a stood error section, showing the output of the command when it failed. We'll dig into this example more later. So please look forward to that. Finally, though, you can also add errors as sections which we can use to aggregate multiple errors into a single report and format them all consistently.

A

Now, I'm not giving this talk to talk about error or color error. I'm giving this talk to share what I learned in that yak shave to fix the error handling in my library and how the process has changed, how I look at error, handling and error reporting. So what is error?

A

Handling ear handling is a lot of things when you zoom in up close error handling, is defining errors, it's propagating errors and gathering context and by context that means stuff like the path you tried to open or back trace, showing where your error came from, it's also reacting to specific errors and discarding errors doing so intentionally and visibly.

A

Finally, it's also reporting errors in their gathered context. Now this breakdown gets to the goal of my talk. I have a theory that error handling is made more confusing by people trying to simplify it because, among other things, error handling is kind of annoying. I worry. The fuzziness between the different responsibilities makes it hard for people to infer what tools they should be using when handling errors, and my hope is that, by breaking error handling into its component parts, we can make it easier to understand and explain.

A

So, let's start with the fundamentals, note that this first bit was taken almost word for word from the rust spokes chapter on error handling the roast's model flares distinguishes between two classes of errors, recoverable and non-recoverable errors. Recoverable errors are errors that can reasonably be reacted to or reported when encountered.

A

These are errors like file not found or connection closed. Non-Recoverable errors are errors that cannot reasonably be reacted to only report it before exiting the thread or program.

A

These errors are usually caused by bugs such as index out of bounds or integer overflow.

A

Now most languages don't distinguish between these kinds of errors, for example, c plus plus has historically used exceptions for both, but rust doesn't have exceptions instead, rust has panic for recoverable errors and result for panic for non-recoverable errors and results for recoverable errors. Sorry, so panic non-recoverable errors in rust are created via the panic macro. Here we can see an example of an index out of bounds error.

A

The only input for the panic macro is an error message and optionally. Some context to include in that error message, reporting and default context. Gathering is done by the panic hook and by default contours gathering, I mean capturing the collar location or capturing the back trace. If it's enabled, and once that panic hook is done, printing, the report, the panic handler takes over and cleans up either by unwinding the thread stack or boring the application altogether.

A

Next, we have result for recoverable errors. In rest, we modeled them with the enum result. T e the c num has two variants: okay, which contains the value of an operation when it completes successfully and error, which contains the error value of an operation when it could not be completed successfully.

A

We use results to combine two return types into one and assign meaning to each possibility.

A

In addition to this, rust has marked the result enum, as must use, which makes the compiler emit a warning whenever the result is discarded, implicitly prompting us to discard it explicitly or otherwise handle it. This helps us avoid ignoring errors accidentally and makes discarded errors visible to later readers.

A

The big advantage of using enums for recoverable errors is that we must react to all errors here. You can see we have to use match or anything equivalent to match to access the values inside of either variant with an enum. We can't access the inner values without first accounting for all the variants that enum could possibly be.

A

Next, we have the tritrate and the tri operator. The currently unstable tritrate is used to model fallible return. Types in rest indeed result is a type that implements the try trait, as does option, and some other combinations thereof with the tritrate rust is able to abstract the propagation of errors. With the try operator here, we can see two equivalent code snippets, the first manually propagates the error using match and return. The second does the same by simply using the try operator to propagate the error.

A

Finally, we have the error trait. The error trait fills three roles in rest. First, it lets us represent an open set of errors by converting any type that implements the error trait into an error, trait object.

A

This is important for composing errors, and it is what lets us expose our source errors via the error, trait, regardless of their actual type.

A

Second, the error trait lets us react to these specific errors by trying to downcast them back to their original type safely, rather than using match, as we would with enubs.

A

Finally, the error tray provides an interface for reporters now I'll get to this last bit, some more in a minute, but first, let's review so by now, we've covered fundamentals, and you know all the tools, the language and the standard library gives you to work with the different kinds of errors. So, let's see how these fit into our original breakdown of the parts of error handling, starting with recoverable errors, so we define recoverable errors with types and traits propagate them.

A

With the try operator for matching and reacting, we do that with match or downcast, depending on how it's represented, we discard them with drop or unwrap.

A

If we want to promote a recoverable error to an unrecoverable one and for reporting, we use the error trait for non-recoverable errors, we define them with panic macro, we propagate them. Well, we don't it's built into the language. Actually, so you don't have to worry about it for matching reacting, please don't! This is the whole point of this split between recoverable and non-recoverable errors. Like you, don't do this one thing for non-recoverable errors for discarding. You can use catch unwind, though I advise you to use caution before reaching for this.

A

Finally, for reporting, we use the panic hook. Okay, so now we know all the fundamental tools built into language and where they fit into error handling.

A

Next, I'd like to dig into the more complex ways you can compose these tools to write some fancy error reporting, but first, let's set straight some terminology in the context of error reporting, an error is a description of why an operation failed, whereas context is any information relevant to an error or an error report that is not itself an error and an airport is the printed representation of an error in all of its associated contexts.

A

Now the concept of error, ports and error reporters isn't a concept that is common in the rust ecosystem today or any language's error handling ecosystem. As far as I know, however, it is a vocabulary that I find particularly compelling in the context of rust error reporting, and this is largely because of how rust has defined its error.

A

Trait here's a simplified version, the error trade here you can see- we've got two super traits, debug and display which we must implement in the error trade, then below that we have two functions: source and backtrace, both with default implementations that we can override when needed. Next, let's look at a simple error.

A

Here: we've got a type with no members, we derive debug on it and then we implement display where we just write our error message and finally, we just implement the error trait now we don't have a source or a back trace, so we don't need to override any functions here, which is why the block on the trait implementation is empty.

A

If we did have a source, though we would need to override source function in order to return a reference to our source as an error trait object. Finally, let's take a look at a simple error. Reporter here, we've implemented our reporter as a short free function. This function takes an error and then prints that error and all of its sources followed by a back trace.

A

If our error captured one a more complex error, reporter might also try to check all errors for a back trace or if it were a type, and it was storing its own context. In addition to the error, it might print that as well now in other languages, there is no distinction between errors and reporters, and this is largely due to a lack of the equivalent of the error. Trait, the error, trait equivalent in other languages, is often quite simple. This is a single function to grab the error message.

A

These interfaces force you to either include your source error, your back trace and any other information you care about in your error message or to avoid using the provided interface altogether in rust. We don't have to combine our messages all into one. In fact, you're encouraged not to including a source's error message in your display implementation and returning it as your source via the error. Trait is essentially a bug, as it forces reporters to duplicate information when they print out the chain of error messages.

A

By separating the source and the error message, we move responsibility of formatting away from the errors themselves, making it possible to get fancy in rust. We can have the same error print to a log as one line, but to the terminal as many.

A

This wouldn't be possible if the error trait didn't separate error message from other contexts such as the source error. However, despite the fact that the error trait in rust is more flexible than most other languages, it is still restricted in some ways.

A

The error trade can only represent errors with a single source if you've ever written a parser. You might have run into this one where you have multiple syntax errors at once.

A

The error trait can only represent a chain of errors as a singly linked list, and so it doesn't work well for domains like parsers, where error causes end up looking more like a tree.

A

In addition, you can only access three forms of context via the error, trait the error message, the source and the back trace. We can't return types like span, trace or location without using hacks based on downcast to work around the error trade. This prevents us from having things like error return traces.

A

Now I dream of a rest where we can access things like the location. An error was constructed, http status, codes, custom report, sections and more. If we could access these generic forms of context in error, reporters we could implement error, turn traces, generically and much more now.

A

I do have plans for how we can fix these problems, though so in the future. This may no longer be an issue. Okay, so I think y'all have a vague idea of what an error reporter is now.

A

So, let's dig into an example on how to use one by recreating the custom section example from the beginning of the talk, so we're going to create a customized version of the command output function with a nicer to use interface instead of returning an output type with the stood error and stood out as effects of bytes and exit status. Let's just return a string for stood out if the command succeeds and return an error port if the command fails.

A

So we start by implementing a trait. We have to do this because we can't implement methods and foreign types. Then we're going to implement this trait for stud process, command and first thing. We're going to do is call the original output function. Then we're going to convert the standard output to a string and if the output was a success, we are going to return that string, otherwise we're going to return an error indicating what went wrong. So, let's run this and see what happens, error command, exited, successfully cool.

A

We got our error port, but also not very helpful. I didn't even show you the main function or tell you what command. I was running so, let's figure that out next, so first we're going to format the command as a string and then we're going to shove that into the error report as a section with an added header. Now these functions, section and header are provided by color error and they all just work with any types that implement display.

A

So, let's see how this changes, things- error, command, exited, unsuccessfully, command get cat okay. So we can see why the command failed hat, isn't a real git command. I wish it was, but it's not now. This error isn't very descriptive sure it describes what went wrong but is far too generic. So let's go ahead and find a new error message with a more descriptive or let's go ahead and define a new error with a more descriptive error message to wrap our source error.

A

So here's our main function, where we call color air install, which just sets up our panic and our error hooks and then we create a command to run git cat and use our output to function to run it to wrap the error. This returns we're going to call this rap error function, which is provided by air.

A

This function takes a result and an argument that implements the display trait and if the result is the error variant, it creates a new error using the arg as the error message and the old error as the source. It then returns this new error instead of the original one as an error report.

A

So let's go ahead and run this and we can see error. The cat could not be got because the command exited unsuccessfully and the command was get cat cool. This is pretty helpful. It's not quite there yet we're still missing the stood error output that the original example had. So let's go ahead and add that, finally, so we're going to convert, stood error to a string just like we did with stood out and here we're going to show both stood out and stood error as sections into the final error report.

A

Let's go ahead and see what we get error. Cat cannot got command exit, unsuccessfully commands get cat and cat isn't a git command. Here's the suggestions for what we could run instead, perfect. We finally have an error for including all the information we need with it. We can pinpoint what went wrong, why it went wrong and, as an added bonus, how we can fix it. So hopefully this example makes it a bit more clear how beneficial just a little context can be for air reports and help.

A

You understand why I think air reporting is such an important concept. This is just a small example of what error can do. This is just what I've configured the hooks for color error to support. You can do much more if you take the time to write your own hooks, though I don't have the time to go into details on how to customize air as part of this talk, so, if you're interested in learning how to customize air, please check out my blog okay.

A

So before we go on, I've got some tips related to air reporters that I think are useful to know. So, first of all, airport is almost always implement from e for all error types. Now. This is because air reporters exist to generically format, any error and they're built on the error trait. So it makes sense that we'd want an interface to create one from any error that implements the error trait. But as a consequence of this error, reporters cannot implement the error trait themselves.

A

This is true for all types that implement from e for all errors, including anyhow error. A report and box dine error. Implementing both of these traits ends up violating the overlap rule and that there would be two from implements to choose from when converting a reporter to itself.

A

So, thanks to the fact they can't implement the error trade. They also don't compose very well with other errors. This is just one of the many reasons reporters aren't great to use in library apis, because your users will have to jump through extra hoops to use them as sources for other errors, and they might not understand why.

A

So by now, you should know all the tools built into the language, how they fit into the various pieces of error handling and have an understanding of how they can be combined to write error reports. So, finally, I'd like to take a look at the ecosystem at large to see what open source libraries we can use to help us with our five parts of error handling.

A

So quick reminder: here's our breakdown. Now, I'm going to introduce these libraries by how they fit into the error handling breakdown, not every part, will have a library to help and some will be disproportionately represented. So I'd first like to start with this error for defining errors.

A

This error is an error derived macro and it exists to reduce boilerplate by implementing commonly used traits for you, such as error and display, and from to use it. We start by adding the macros identifier to our drive attribute.

A

Next, we can implement display with this error via error attributes, we can have a simple attribute, which contains only a string literal. We can also include variables in our error messages with this error's custom format, syntax for both unnamed and named members.

A

This error can also handle implementing our source and backtrace functions. For us, any member with the name source or back trace will automatically be returned from the appropriate function, or this can also be done explicitly using the source attribute.

A

This error can also implement from with from attribute and note that this also implies the source attribute. When used this, isn't all this error can do so, please check out the documentation for more details. Next, I'd like to introduce a display dock display dock is a fork of this error that provides only the display derived portion of this error, but uses dot comments instead of custom attributes to input the format strings.

A

I find this is great for simple error types and that often error messages make for great documentation.

A

Now, I'd like to move on to snafu snafu is another error, drive macro much like this error. We use it the same way. We add the macro to our drive attribute and we use some custom attributes to implement display.

A

The main difference is that snafu also has a special helper function, called context to further reduce boilerplate when capturing context for error messages. The context function takes a result and a context. Selector struct, which is auto-generated by the derived macro. You may notice that the source member is missing here. This is because the context function function implicitly passes along contexts like the source and the back trace making. So you only have to cut capture additional context that is unique to your error variant.

A

It then internally creates the correct wrapping error variant from your enum, and you can you can think of this as syntax sugar for map error anyhow and air also have helpers for defining new errors. However, these functions don't actually help you to find new error types. Instead, they help use their own private types to implement, to create the new errors and then immediately convert those errors to their error reporting types. This is useful when you want to create an error exclusively to report them common need when writing applications.

A

These crates do also provide some helpers for then later reacting to these ad-hoc error types. Now before we continue, there are some things I'd like to share that, I think are useful to know when defining errors. First, with api stability, there's an easy pitfall to run into. As a library author, if you just create a public enum for your error type, adding a new variant or even a new member to an existing variant is a breaking change and will make you implement your major version.

A

This can be avoided simply by adding non-exhaustive to error enums, where you think you may wish to add new variants in the future or to specific variants where you think you may wish to add new members in the future.

A

I think it's also important to consider stack size when handling errors. With result. The size of your return type is the larger of the two variants. If your error types are large all functions, returning, your errors will require more stack space, and this can negatively impact your performance of your program even when no errors are encountered.

A

The solution here is to allocate your errors on the heap when they're too large, either with a box or reporting type like air anyhow, which is designed to occupy as little stack space as possible. However, when boxing your errors, remember to use the original arrow type, not dine error, but the box still implements the error trait.

A

Okay, enough of the tips, let's move on to propagation and gathering context, starting with filler filler, is a library that adds support for throwing functions to rest through procedural macros functions. Annotated with rows return, a result where the ok type is the apparent return type of the function and the error type is the type listed in the throws attribute.

A

Feller then does implicit, ok, wrapping for return types and provides a throws macro for returning errors if you're missing throw syntax in rust. This is the library for you next for gathering context. I'd like to introduce tracinger tracinger is a tracing instrumentation library that exposes the span trace type. This library also has some cool helpers for wrapping errors with span traces and then later extracting them through dinar trade objects working around the restrictions of the error trade.

A

Here we use in-current span to wrap an error with our instrumented error type, then below that we convert that error immediately to a trade object and then extract the span trace from that error. Using some clever hacks.

A

Another useful library for gathering context is extract. Error extract air is an abstraction of the helpers written in tracing air force fan traces instead, extractor exposes a bundle type for bundling arbitrary types of errors, which can then be extracted later through die and error. Trade objects here is an example of bundled being used to bundle a back trace-rs trace with an error instead of a stood back trace, emulating the back trace function on the error trait in a way that works on stable today, next for matching reacting.

A

I mentioned this earlier, but anyhow and air also have some helpers for handling their own ad-hoc, created errors, errors created with the air macro or with wrap air meet downcast back to the original type used to create the error message here, you can see a type that just implements display, which we use to construct a wrapping error type. As the error message, then we can later downcast through the report and you specifically have to do it through the report.

A

You can't get a reference to the internal error, but we can downcast that report back to the message type that was used when creating that wrapping error now for discarding errors. I don't know of any open source libraries that help, but if you do know of any please let me know for printing, though you're already familiar with anyhow and air, given that they are the stars of this talk. These libraries expose air reporters that, as far as I can tell, are strictly superior versions of box diner for customizing.

A

These reports, though there are a couple of quick libraries. I've written color air you're, already familiar with stable error, provides identical features to the default error hook provided in air, but using backtrace rs back trace instead of the standard library's back trace.

A

So it works on stable, jane error is just a re-export of color air, but I think it's worth mentioning just for the pun, given that it's the reason I named my library, air and color anyhow isn't usable yet, but the pr to add the error, reporting hopes to anyhow has already been written and looks likely to merge as soon as I've gathered enough evidence to prove the design. So please look forward to using that in the future, if you're already using anyhow.

A

So that's it. That's everything I wanted to tell you, but before we go I'd like to try applying this breakdown of error, handling idea to a question, I see all the time in the rust community. What library do I use when writing errors?

A

The common answer to this question that you'll see time and again is used anyhow, if you're writing an application and use this error for libraries now. I agree that this is usually correct, but it's not universally true and we can come up with a better rule of thumb. If we ask ourselves what kinds of error handling do we need for libraries versus applications now for libraries, we don't generally know how our users will handle our errors. They could report react, wrap and propagate them or discard them. So we need error types that are maximally flexible.

A

This means we need to implement the error traits, so they can compose with other errors and be reported, and we want our errors to be an enum, so they can be reacted too easily. We also want our api to be stable, so we should use non-exhaustive, so we can add new variants in the future.

A

All of this pushes us to use error, defining libraries like derived macros or to just implement the associated traits by hand for applications. On the other hand, we know which errors we're going to handle versus the report. We usually handle errors close to where they return.

A

We will need to create new errors, but usually this is only for errors that we wish to report and when we need to be able to report arbitrary errors, because applications usually work with errors from many sources. All this pushes us to use error, reporting, libraries like air and anyhow, and that's it for real this time. That's the entire talk. Thank you, everyone for taking the time to listen, and I hope you find it helpful.