Rust Programming Language Oxidize Global 2020, 2 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RTIC: Real Time Interrupt driven Concurrency

Description

The use of embedded systems grows at an immense rate with 3 billion+ Cortex-M ARM chips shipped! However, someone has to write the programs that run on all these chips, where the trifecta of safety, performance and ergonomics comes as a wish, while in many applications an MCU hanging or crashing is simply not allowed. Currently the embedded Rust ecosystem provides the necessary foundational crates, but lacks the ergonomics and safe utilization of resources.

This talk aims to show that with Rust and the Real-Time Interrupt-driven Concurrency scheduler we get this trifecta, with how resources, memory safety, deadlock free and data-race free execution that can be guaranteed at compile time, no defining of thread stacks, and keeping the generated binary minimal without losing expressiveness! All built on a sound theoretical foundation of Stack Resource Policy.

ARM has shipped 100+ billion cores - let’s build the next 100+ billion on a strong foundation, ease of use and safe programs!

A

Hello back and we're going to continue with our next speaker, emil frisk and emma is going to talk about arctic. Real-Time, interrupt-driven, concurrency, previously known as rtfm, and I can personally share. I knew always that there was interest in the embedded, uh rust scene and there were always hobbies trying it, but when rtfm showed up, that was really the point where we're like. Well, this is getting serious and I'm happy that the project is still around and is in industrial use and emel was probably the best person to talk about that.

B

Yeah thanks, let's see, is uh sound working fine, just as a check perfect perfect. So I welcome you all to my presentation on arctic and demos around arctic, so my name is emma fresc.

B

I am one of the developers of arctic and I've been around with the since a little bit since the research days of this scheduler actually, and what we're going to see today is more or less. I want to give you all an overview of the arctic scheduler, and I really want you after this to even be able to use it yourselves.

B

So this is going to be a talk with very few slides, so let's hope everything works. Fine, so just to give you a small overview, what we're going to do here, so we will start with, let's say the info sheet of this scheduler.

B

I mean what does it support in the end, this is usually what once lecture scheduler on so we'll start there and then we're going to have a lot of demos, nine in total that I will do live and run here for you while uh online, and once we've done this you've got a feeling for how this looks in code. What you can expect a little bit, at least, let's see a little bit, how it works under the hood and finish with that.

B

So if you start with the let's call it the info sheet, what does this scalder support or what are its features?

B

The first one, which is probably the most important one that people get let's say not hanged up on, but it takes some you're getting used to this scheduler is based on tasks that run to completion or jobs. One might call it so it's not threads where you have infinite loops, but these tasks they're either event triggered, which means they're connected to hardware or they can be spawned on demand through software tasks.

B

It's very important to remember that this is run to completion. We do not have infinite loops in the tasks.

B

Arctic supports message passing. This is quite an important feature if you want to make any type of system, so between two tasks or multiple tasks, you can send messages freely.

B

And a scheduler is not really a scheduler unless you can schedule, so we have support for a timer queue, which means that you can schedule a task to be run sometime in the future. So you can say that task 1 should run in 200, milliseconds or similar.

B

And another big and very important part of arctic to understand is that this is a preemptive multitasking scheduler and not a cooperative multitasking scheduler. That means, if you have two tasks running at the same priority. The first task will first finish and then the second task will run and finish. They will not do like round robin scheduling or what you might be used to in some other schedulers, and when we come to some of the details in the end, we'll see why this is very important and why we do not have cooperative multitasking.

B

B

Something that is a very big feature is compared to thread-based. Schedulers is that we have data race-free memory sharing in this system. You could think of it, then well as mutexus, but the mutica mutaxes can never fail. They are guaranteed to always succeed.

B

So what we have technically are what are called priority-based critical sections, and this makes it extremely efficient in this torque cover. I will not show the machine code that is generated from this, but taking locks and releasing locks in arctic are a few instructions and awaiting nothing like that, and something that I'm very happy with this scheduler is that we have deadlock, free execution, and this is guaranteed at compile time, so you can never get, for example, two moot access or semaphores that get locked on each other or waiting on something forever.

B

This can never happen in this scheduler.

B

A

B

Is since this, how we're going to see in the end is that this? How this does the scheduling? We are have extremely low scheduling overhead, because we are letting the hardware do all the scheduling for us.

B

So in the end, there is no scheduler per se running we're just setting up the hardware to do the scheduling for us- and this is quite an interesting detail that turns out to make it extremely fast and the final one, which is nice, but um also to have let's say if you have threads, usually set up stacks for each thread in a norm or a which is common in schedulers.

B

However, since or how arctic is designed is that we are not setting up, call stacks or stack frames for each task. They just shared the same one since the hardware's doing the scheduling for us, so the hardware has control over the stack frames. So we never need to worry about stacks meshing between tasks stack overruns.

B

The only thing you can do is use so much stack in the system that you start going over the maximum ram or going into the data sections.

B

And this is what I would usually call the info sheet and if you have a look at the book, this is more or less the first page of the book, and I highly after this recommend actually to read the book. It's a it's a very good reference, but we're here for demos and with this I'm going to switch over actually and start running.

B

B

B

B

That's the second.

B

I don't know seems to.

B

Be okay now it should be working perfect. So, let's start with the basics of this scheduler, and that is what we call the app macro, the app macro which is included here- and this is usually the only include you need for arctic if you only want to do basic use of it.

B

What the app macro does, which is the outline you can see down here- is that this is where you write your application.

B

You might have questions like. Why is, does it say, const up and this and more or less? This is just to make proc macros in rust, happy in a newer version of rust that is coming. This is going to be switched out to a normal rust module. However, this is how we have to do it for now, and this is generally, if you're starting a new project, it's quite common to start here the important thing to know about the app macro.

B

What it requires is that you provide the device which this is running on, which means the peripheral access, crit and, in this case,.

B

There we're running on an stm32 and for those who want to look in the code later you're going to see we're running on an l4 board.

B

So with this, there's not much of a demo, but this is the basics. So if you ever see this app macro, then you know that you're an arctic land, but let's switch to actually something real where we actually run something.

B

So I'm going to show you the three main components of arctic, which is what we call the init function, the idle function and tasks.

B

And let's jump down to the code, so the app macro which defines the we have resources, but none for now and then we have actually in it in. It is one of the most important functions in arctic, because whenever arctic starts, this is the main of arctic.

B

So if you ever start any code, this is the code that will run first, so you put initialization code. You start your drivers, these type of things. For all of my examples, I'm only going to do logging, so you can see prints of the system and we can see a few leds blinking as well of course. So what I've put in here is only that I'm enabling a logging framework, so we can see what's happening now.

B

If we continue, we have the optional idle task and idle is that if no tasks are running idle runs, so this is quite common to put maybe background work logging, maybe writing to a logging chip. Maybe you have some background. Work needs to be on your system they're done here, and this is actually the only place in arctic where you will find an infinite loop.

B

One can see this as the infinite main loop in a way so just to get some feeling. However, it is perfectly fine to not have an idle task at all. If you don't have an idle task, then the what will happen is. It will only run a wait for interrupt instruction.

B

And what is also what's in in all of our functions here so you'll always see that all functions take a context and we'll see more, I said for the examples, progress how this context is used, for example, for accessing resources or other features.

B

And finally, let's have a look at the two task: syntaxes that exist.

B

If you have a software task, which means that this task is not linked to a hardware interrupt, then you only have the task attribute and also everything looks the same for everything in our pic. You have the context and a normal function which is has to attribute and we have what is called hardware bound tasks.

B

So, if you add this binds argument, then this task will be bound to an actual hardware interrupt. So in this case, if we were to get data or do anything with uart 1, this task would run and there's a lot of comments here for when you want to look at this after the talks. So you can see it go through and go through and take in all of this information and, finally, a detail which is necessary is that for software tasks to run they still need to run in and interrupt.

B

This is how arctic the core of arctic works. So what, whenever you use a software task, as we have defined up here, you must give to arctic what we call free interrupt vectors. This means interrupts that you are not using for specific hardware interrupts.

B

So in my case, these are put in an extern c block for now, but this will probably change to something more user friendly in the future.

B

So in this case, since I have one software interrupt, I have put in one free interrupt here. I only need, as many free interrupts as priority levels for software tasks and what this code in practice will do. If we run it is, we will get hello from init and a hello from idle.

B

Let's try and run this code.

B

And this is exactly what we get.

B

And if you run this or open this project yourself with vs code, all of these tasks for running all these examples are prepared there. So, as long as you have the same board, you can run this one straight off.

B

Well, let's go to a more interesting example. Maybe to see how we use tasks.

B

So in this case, we are going to have one task.

B

We are calling it our hello world task, and this is the task we're going to run and what we're going to do is we're going to run this one from main or the init function, so in init we're still doing a lot of initialization and here's the interesting thing in the init attribute.

B

We can add the argument to say that this task or function is allowed to spawn software tasks or spawn tasks overall, so we've said that init is now allowed to spawn the hello world task.

B

What this means is that we can access and spawn a task, which means that you are going to tell this task to run.

B

A

What is interesting here.

B

Is that we have hello from init at the end of init? The hello world task has hello world from task and idle has its hello from idle. But let's see how these actually run in which order.

B

So we get hello from init and then we get hello from task, even though we spawned the task before we sent the hello from init and finally idle. Let's have a quick look at why this is how it works. Is that no tasks can preempt init tasks. All tasks are disabled until init finishes. This is a very good detail to remember.

B

Let's continue to the next one.

B

And now we're going to have a little bit more interesting, we're going to use priorities else we're going to have more or less the same thing. So, let's start looking at our tasks, we have two tasks: a low priority task and a high priority task, which means simply that you add the priority attribute or priority argument.

B

And in spawn, we are first spawning the low priority task and then the high priority, but we're going to see that a high priority task. It will run first, even though low priority is spawned first, and this is because no tasks are allowed to run until end of init, which means that both will be started at the same time at the end of init, and then the high priority task will run first.

B

So we get our hello for a minute: the high priority task and low priority task and finally idle. After all, tasks have finished, and this.

A

B

An example I highly recommend you to play around with if you want to try it out to see how priorities work and how they interact.

B

Let's continue to the next example- and this is probably one of the more important parts here.

B

So we are going oops. Let's just do that. You didn't see that we're going to schedule. This means we're going to run something in the future. What arctic needs, if we're going to schedule tasks into the future, is that it needs to know what time base are we using it needs a timer practically and in arctic, we're providing the debug watchdog timer as one of these time references.

B

So what you do if you want to use the scheduling, is that you add this monotonic argument and this will enable scheduling.

B

So what we're simply going to do here and some details.

B

But what we're going to do is that we're going to schedule the hello world task and now I happen to know that this mcu runs at 4 megahertz at startup, so we're going to schedule it three seconds into the future and let's see that happening.

B

So idle, since the scheduled task had not happened, and here we got it in the future three seconds into the future, and this is the basis for scheduling.

B

What is the details, as a small detail? Everyone needs to know is that you need to start the timer that you're going to use, and this can be part of how it can be part of startup code. However, here I've written it out explicitly so for you who go through these examples, you'll see it's.

B

Directly, let's jump to actually using resources. This is something that is very important, so if we start we're going to define our first resource in this case, we name it value, and you just have defined this as a normal structure and it has a u32.

B

What we see here, then, is that init gets this late resources added as return. This means that after init, the return of init is will be the start value of the resource structure.

B

So we are enabling our scheduling and in the end here we are starting value with the value of 42. and more interesting. We have the two priority tasks: one high priority and one low priority. Let's have a quick look at the tasks, we're going to see that the high prototasks only increases the value and the low priority task. It does a lot of long computations, let's say so here we just have a poor man's delay, so this means hopefully that we will see a different value, as this will be preempted in the middle.

B

Let's run this.

B

Number six.

B

And here you see, the low peritosis runs first as after it was scheduled, the high prototasks preempts this one and updates this resource which has been accessed and then the low power to reuse it again, and the important thing to see here is that low priority task. It accesses resources through this context structure and it must lock it.

B

However, the high priority task: it does not need a lock, as this is guaranteed, to run at the highest priority that may take this resource, so it doesn't need a lock. This will allow change in the future due to we want the resource access to be symmetric so, but for now in version five. This is how it is.

B

And now, let's just do a quick example of message passing.

B

In this case, we're going to have two tasks: printer one and printer two, and this is an example. I recommend to sit a lot with to see how these work and let's have a quick look at printer one and two they have arguments here.

B

So if you look at their definitions, we see that they have now input values and they will simply print out these values as they come.

B

The important thing to notice is that this new capacity argument what this one does is that it allows printer 2 to queue up multiple messages to it, while printer 1 has does not have this attribute.

B

This means that the capacity of task 1 will be defaulted to one. So you can only put one at a time, and then you see up here that if we put a value into printer one, it will work if you put another one in the second will fail, because its q is full. Our printer two will be able to take all of these four values.

B

Let's run this and see it in action.

B

So we see first, the second spawn failed. So as we predicted, the queue was full printer. One prints, this value, 42 that I got and printer 2 prints is 4 values and now, let's do a blink key.

B

As and this one, it has a lot of code just in it to set up the pins, but what is interesting is as to the blinky task in this case and again we're using a resource which is the lead.

B

So in this example, we're demonstrating the save static modes, which is a transformation of rtfn, does for you. One could use a toggle led here, but I'm doing it manually just so, you can see it and.

B

We will simply run this.

B

And here we will see it going, but my hope is that this will show up well here.

B

And this is an example. I recommend you even more to play around with because what's funnier than blinking, let's say.

B

But we need one final thing and this is actually reacting to events.

B

So in the final launches button it has the same blink task and it has more hardware setup code and this will jump over for you to read later. This is a standard hall code for setting up, interrupts and pins.

B

So what we're going to do is we're going to a button event task which binds to an interrupt it will spawn blinky when we click the button.

B

So if we press the button after running this example, the old blinky code will run, but only after pressing the button. So let's do that.

B

So nothing is happening and I hope I can do this without and if I click the button link starts running.

B

And this is quite common if you have a hardware high priority task and you want to let's say, offload work to a lower task later, maybe get the radio message and then you just want to slowly process it. In the background- and this is all the demos I was planning to run- and let's now have a look at some of the guts of.

B

B

In arctic web preemptive scheduling this means, if we look at this picture, is that every line up here is a priority level, and jobs and tasks are interchangeable.

B

So if red dots are where we spawn tasks and straight lines or where we execute a task, we can see it as jobs of higher priority are spawned, they will run directly, and this is what means to preempt multitasking and, as the first one finishes, the lower and low priority will run again until eventually coming back to idle, and this is how all of the or the thought behind the preemptive multitasking.

B

Finally the same thing: if we have a low priority, the highest spread runs and the middle one is spawned, it will not run because a higher priority is running. This is also what one would expect from preemptive multitasking.

B

However, here comes the real, interesting part, which is usually the resource access problem and why it's so important with the locks, let's say: job1 and job2- can access the resource which is down here so task. 1 access resource does work on it. It gets preempted by task three. However, it's not doing anything with the resource. Everything is fine, however, task two does something to the resource.

B

So if we do not have locks here, we have now create corrupted the resource, because we have changed or interrupted the job one or task one in the middle of its execution, and this is not acceptable.

B

So, of course, if this happens, I mean we've broken everything. There are no guarantees left at all in this system. However, what one.

A

B

To happen with locks is, of course this when job one runs, job three still runs and two is pended for execution.

B

However, the job one is not finished with the resource, so only when it's finished with the resource will jump to be allowed to run, and this is how it works in arctic.

B

However, how can this happen, and how does this work? Everything in arctic is based on. What's called the stack resource policy, and I usually call it our savior, because this guy, you know he figured out a long time ago and now we're reaping huge benefits from it, and if anyone wants to read it, this is the paper actually where this was invented.

B

So what we're doing when we're looking a resource is simply this. We disable all tasks which are at equal or lower priority than the highest of all jobs. That takes this resource, and this one takes a bit to sync in, but we're going to see image what this actually means and we release a resource where we enable all of these disabled tasks, and how does this what's? This called? This is, what's called a priority ceiling, and this is something called a priority ceiling scheduler or it's also called a an.

A

B

B

So if we add the priority ceiling into this old graph, that we saw that when job run, one takes the resource, it raises the priority ceiling to this level. This means that job 2 is at this level, so it's not allowed to run. Jump. 3, however, is above this level, so it is allowed to run and then, when task 1 releases its resource, this even goes down to the lowest level and when job 2 runs we're back, and this is how arctic works internally with priority ceilings. However, how does this work.

B

So this is what how it works is. What has this feature in our microprocessor, specifically the cortex-m in the cortex-m. We have the nested vector, interrupt controller, which can do this type of.

B

Let's call it scheduling for us, because it is a priority based or the interest vectors, are priority based and will preempt each other. So the first feature we want is given for to us from the end the final one: what about the priority ceiling? What has this behavior in the cortex ends? Well, something that does. This is a feature of the nvik which is called base pre, and this is simply, if you set brace pre to a value.

B

Anything at this value or lower will not run from the nvik, and this is what we're using internally, so locks are only or in it is only setting up the nvik for us and locks are only accessing and changing base print. So in the end, you have only very few instructions, which is the entire scheduler, and this is the entire magic that is the core of arctic, so just to wrap up.

B

This means that all your tasks running interrupt vectors, which means that the hardware does all the scheduling for us. We don't lift a clock cycle to do scheduling only to set it up and we need to find these values of base pre, and this is why arctic is a huge prop macro.

B

The arctic, app proc macro, finds the values needed for basebree through code analysis, and this is why everything is.

A

B

B

And with that, I thank you for coming to this talk. I highly recommend you to check out the code. The documentation come join us in the chat and uh I hope we have good discussions afterwards on this, and we all thank the gods for technology and demos, working.

A

Thank you for this nice run through arctic and its ideas. um We actually have no time for questions, but there are questions in the chat, so I highly recommend you to to move over and start the discussion. It's quite active already or the matrix. It's in your it's in your matrix channel, exactly perfect.

B

I'll jump in there see.

A

B