Rust Programming Language RustConf 2020, 21 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RustConf 2020 - Controlling Telescope Hardware with Rust by Ashley Hauck

Description

Controlling Telescope Hardware with Rust by Ashley Hauck

Dive into hardware APIs as we explore how the hardware of an 8-inch aperture telescope can be controlled with Rust. In this talk, we'll explore using serial ports to drive complex motors and machinery, and using a native C API to download images from the camera and control the dozens of settings it has. As a bonus, learn some basics of astrophotography and see some pretty pictures I've taken with my telescope! All code is shared on github.

Code: https://github.com/khyperia/scopie
Ashley's twitter: https://twitter.com/khyperia
Pictures taken with her telescope: https://khyperia.com/space.html
All raw data that she has taken: https://khyperia.com/space/

A

Hello, everyone- this is the controlling telescope hardware with rust, talk, we're going to be talking about telescope hardware and controlling that hardware, with rust, from a desktop and interacting with our hardware, and how to do that.

A

So uh who am I I'm actually pronouncing sushi her uh all the code in this talk is going to be on github uh at the project name scopy. It's a project, name that I've been using for this code and on the right here is me in seattle, last winter, taking an image of the elephant's truck nebula and the result of that session was this image. So on the left, you have a very bright star that is blowing lots of solar radiation and whatnot at the gas cloud on the right.

A

More dense parts of that cloud um are more resistant to being pushed. So you get these globs of dust remaining behind and if you really squint, then those globs of dust sort of look like an elephant's trunk. Hence the name elephant's trunk, nebula cool.

A

So we're going to be talking a little bit about how astronomy and astrophotography works, in particular my setup. So we have some context of some technical words and an understanding of what's going on before we jump into the code. This is a newtonian telescope, which is what I have light comes in.

A

The left, bounces off a parabolic mirror on the right, then in the middle or on the left here is a secondary, mirror that redirects the light up top then normally, if you're looking through this telescope with your own eyes, you would have a lens here, but instead we have a image sensor that sits directly on the focal point and collects the data directly, and then we can pull the data off this camera.

A

What that actually looks like in real life is like this, so uh light goes in on the left. The mirror is on the right, then buried in the middle here is that second mirror, which redirects the light up top to that camera below. That is the mount system. So we have two motorized mounts the top. One is called the declination axis then below that is the right ascension axis, and these two motors combined. Let us look anywhere in the sky.

A

So what actually are right? Ascension declination, so the sky needs a coordinate system to be able to uniquely identify any point in the sky. We start off with using the earth's latitude and longitude as a base. Latitude is easy. We just say that anything over the north pole directly over the north pole is 90 degrees. North anything over the equator is zero degrees.

A

Anything over the south pole is negative, 90 degrees, but uh the equivalent of longitude is a little bit more complicated since the earth, obviously spins, and we can't have our coordinate system spinning throughout the sky. So instead the earth uses the zero degrees. Longitude is defined as being at greenwich in the uk.

A

Instead, we define zero degrees of right ascension, as it's called as being the direction of the sun at the spring equinox. It's kind of a weird definition, but that's what it is so cool. um Now we're able to uniquely identify any point in the sky and.

A

Have that coordinate system? Importantly right ascension declination are aligned with the earth's rotation axis, so our own motors need to be aligned with that earth's rotation axis, which means that we need to tilt our motors equivalent to the latitude that we're currently at so seattle is 47 degrees, north and so I've configured my telescope here to be used at seattle to tilt those axes to be 47 degrees, cool.

A

So now we're able to point our telescope anywhere in the sky and image whatever we want to now. Let's actually start getting into code, we're going to look into the motors first to see how we want to control those motors. So we do a google search figure out the hardware that we have and we dig up the specification manual that our hardware provider provided.

A

So it's a serial communication protocol. That means it's a tube that you send bites down and you get backs back. It's all serial and it's very, very simple, and it's pretty great. So uh in this document are things like 9 600 bits, a second, no parity, etc. It is very serial communication uh parameters if we scroll down in this document, we have various commands that we can send to the motors.

A

So goto is the term in astronomy to say, hey telescope, point to this point in the sky and the way we do this is to send down the serial pipe, the letter, r and then right, ascension encoded in hexadecimal, followed by a comma and then the declination encoded in hexadecimal.

A

Then in response the hand controller sends back a hash symbol to say: hey, everything's, okay, um things succeeded. If we scroll down some more uh retrieving. Data from the mount is a very similar system. uh We send the letter e and we get back a write, ascension declination in hexadecimal again that says where the telescope is currently pointing the sky. So that's really useful if things are drifting or whatever cool.

A

So we have a serial api and we want to write this in rust, and so we look online and we find the serial report crate, which is a fantastic crate. It's super fun to use. um Lets us use serial reports from rust.

A

So what that looks like is, if we want to open our connection to the mount, uh we give it a path, and we get back our serial report. Then we're allowed to do some configurations, for example, set that bit rate set the parity and here we're setting the timeout. So if the serial report doesn't respond for three seconds, then we kill the connection cool.

A

How do we actually send bytes down this serial port? It's pretty simple! um There's this write all method and we just dump the bytes in this buffer down. The pipe cool reading is a little more complicated.

A

So the trouble is that if we want to read something we can't just read up until we get a hash, because the data that we're getting might accidentally include a hash symbol, or rather the ascii value of a hash in the data that we're receiving um and that's no good, and so.

A

Instead, we need to get the length of the data that we expect to receive up front and the way we do that is that we have pass in a buffer, and then we use that buffer's length as the amount of data that we're expecting to read. So first, we read the buffer the here, so that reads: n number of bytes, then, after that we read the hash that is at the end of every single command.

A

The amount always responds with a hash, after doing whatever, to make sure that everything's, okay cool uh so connecting those all together. We go back to our pdf document and we go look up the go to command, so that is sending an r followed by the right, ascension declination in hexadecimal, and here we're doing that.

A

So we format a string with the letter r, followed by the writer extension with comma and declination. So then we write that send that to the mount, and then we read back that single hash. So remember a read method then, would read zero bytes and then read that hash for us, so cool uh getting data from the mount is a very similar system.

A

We send an e and then the buffer size that we expect is 17 bytes. So that's the eight bytes of the hexadecimal, followed by comma, followed by the eight bytes of the uh right, ascension declination cool. Then the rest of this method is just parsing to grab those two hexadecimal numbers out of that blob of bytes awesome.

A

So now we're able to point the telescope wherever we want. We want it to do. That's awesome, so the next step is to look at the camera. And so again we look up our cameras. Manufacturer look up some specification pdfs and we stumble upon this pdf and we start reading through it and turns out the way that we interact with this camera is a c api.

A

So the vendor of this camera provides this pdf as well as a c library that we link to with our rust program.

A

This function opens up the camera, so initializes the usb ports and everything, um and then it returns a handle and what that handle is. Is we then later give this handle back to the api and says?

A

Oh, you meant that camera that you previously opened up, so we're able to interact with a single camera based on this handle um then how we actually get data off of this camera is this function, uh so it gives us the width, height bit, depth, etc, and then the final parameter is the actual bytes data that we want to get, and so the c api fills in those bytes and then we have our image so fantastic.

A

Now we want to write this in rust.

A

um Thankfully, the uh vendors also provide a c header, and so what we could do is use something like bind trend to convert the c header into rust, um but instead I've opted to just hand write all these functions because there's only a dozen or so of them and it was easier to hand write these functions.

A

um So, for example, you can see that open function that we mentioned earlier at the top here we're linking the library that was provided to us and so that pulls in that code that was provided.

A

So how do we actually use this so c obviously expects null terminated strings, so we use the c string type to be able to.

A

Pass in the string to this function and then we get back our handle, and so then we do some error check of. If that handle is null um and then we have our camera open. So that's great!

A

So now we want to get some data off of that camera, and so we get the width, height, etc, and then we have buffer for that c api to be able to fill in that data, something that I found really useful for when working with c libraries is.

A

I have this check function because c libraries usually return integers as error codes, and so I have this check function that takes that integer compares it against error code, see if it's success or not and then returns a rust results, and that allows us to use that question mark operator and that's really really nice and ergonomic to be able to interact with these c libraries cool.

A

So now we're able to point the telescope wherever we want to we're able to get image data off of our camera, and that means we all have all that we need to create beautiful images. So this is an image that I took of the eagle nebula m16 at the center. You can see three little newts.

A

Those are the pillars of creation, the very famous hubble image, and I was really really excited to say that, like I actually captured this with my own amateur telescope- and I like saw a thing that hubble saw. It was like super exciting for me, so yay so cool. um The trouble is that just interacting with the hardware isn't enough. We also have to slap a ui on top of this, so this is a pretty garbage ui, you just slapped it together, whatever it has all the data that I need.

A

uh Unfortunately, this isn't actually live in data. uh This is just a simulation that I was running with my program and that's because at midnight uh stockholm looks like this because the sun is so like doesn't get below the horizon at all. So I can't actually do any astronomy right now to get any live data screenshots, but whatever.

A

So the fact that we're putting a ui on top of these hardware functions is kind of problematic because, for example, downloading the data off of the camera can take a couple seconds and we don't want to lock up our ui thread for a couple seconds, because that would be really bad at user experience. Even if it's just me using the program.

A

So we want to have a thread dedicated to doing the camera work or doing the mount interaction work. And then we have our ui thread, and so I want to share some experience that I had working with hardware and what patterns worked really well when working with, like blocking hardware and stuff like that.

A

um So this is a pattern that I really like. um So we have our dedicated mount thread here. That's just spinning in a loop and then we have a mpsc channel um that instead of sending data over, we send um lambdas or delegates, and then we call that delegate with our local mount, and so that means that this mount can only be accessed by this thread.

A

But the fact that we're sending over lambdas means that we can write really nice shim layers like this, um that if we want to slew the or go to the mount to wherever you want to, we just need to write their shim. Then the ui thread can call this method. uh It just sends this lambda over the npsc channel, which doesn't block at all and returns immediately, freeing up the ui thread to do whatever it wants.

A

Then, um when the mount thread can get around to it, then it actually does that slower operation cause this lambda with the mount. That is a thread local and rust, guarantees that this mount will never escape that thread. It's really really nice to be able to have those guarantees in this rust world.

A

Unfortunately, the camera is a little bit more complicated and I do have an enum instead of sending over uh lambdas.

A

This is because the camera is a little bit more complicated, so here we're setting a control and control in the sense means something like the camera's gain or the exposure or stuff like that, and it turns out that the camera hardware really doesn't like it.

A

When you set the gain in the middle of an exposure like things crash and stuff- and I just figured this out by using the camera being in the field- doing some testing- and I figured out that you want to cancel the camera before sending any control, so cancel the current exposure, um and so now we have this method. We have to make it a little more complicated, and so now we don't use a lambda anymore. We use an enum, so cool. So now we have all this flexibility to do whatever we want.

A

So um that was some threading talk, and now we wanted to talk about some of rus performance because it's really important. um So if you've seen this image, you can barely barely see the elephant's trunk nebula that I was imaging here. um It's a little faint, uh but hopefully you can see that, but this image isn't actually what's coming off the camera. What's coming off.

A

The camera looks like this: no idea, if you can actually see what's going on in this image, but it's basically a black screen with a few tiny, bright white dots and the reason for this is the camera.

A

Is has a 16 bit depth 12 bits actually, but it returns it in the data of 16 bits, and so that means it has a very high dynamic range and if we want to remap that to 8 bits of what our displays actually display, then we're only going to see the very very brightest things which in this case are tiny little dots of stars that are showing up and we can't see the nebula at all. So that means we have no idea if the nebula is in frame or not.

A

We have no idea uh how good of quality it is.

A

So what we want to do is when we're previewing this image in the app we save all the raw data itself, but when we're previewing, we want to remap this data to be able to see all the interesting parts of the image and not just have a black screen with a few white dots.

A

So we do that by computing. Some statistics about the image first, and so here we compute the mean and standard deviation of the image. These look like really really simple functions. I just loop through the data. Add them all up and then computing standard deviation is really really simple.

A

The trouble is that these images are not small, they're, 32, megabytes, big um and when I implement this in c sharp, uh it took like a couple seconds to turn through all this data and that's a problem when you're doing things like planetary astronomy, because in planetary um your frame rate is like 10 milliseconds exposure, and you want to collect as many many exposures as you can. So you want to have these previews come in immediately.

A

You want to process as quickly as you can um and so rust absolutely blazes through these functions, and it's really a joy to be able to have these previews to snap in as soon as they're available so cool.

A

uh So what that processing actually is doing is saying: okay, so we're assuming that the brightness distribution of our image is a gaussian which is it isn't, but whatever close enough, and so we want to say, like okay, let's make negative one sigma on this image to be black and positive one sigma to be white uh and clip everything outside of that and just see the interesting data.

A

That's in the middle of the spell curve, but we also want to be able to shift a little bit because maybe the interesting data is on like slightly high side of the bell curve or whatever.

A

So we have these two parameters of our sigma level of how big we want our gap to be, and then an offset of saying like hey, we want the mean to be twenty percent bright instead of fifty percent right or something like that. So we have this chunk of math uh because I wanna have this. uh These shifts to be a straight remath. That's a linear equation! That's just y equals mx plus b. That's really really fast to do so on the top.

A

We convert our um sigma and our mean offset to be a mx plus b equation. So we have a scale and an offset then at the bottom. We just do a multiply and then add, which is really really fast.

A

So that's really cool. So then, uh when we want to adjust our, for example, sigma or mean location parameters to adjust our view, we don't actually have to modify the image at all. It's still in gpu memory and we just adjust this uniform to be able to display it.

A

However, we like that means, we have real-time adjustments of saying like I want to see the bright parts of this image, or I want to see all the dark parts of this image and it's super snappy and great, um and the reason that we don't have to modify it is apparently opengl supports 16-bit, grayscale textures, which awesome, so we can just upload the raw data to the gpu and have that all work. So that's super fantastic.

A

So now we have a great application that can preview all of our astronomy images and save them all and image whatever we would like to in the sky.

A

Great, so yeah, that's my presentation. All the code that I talked about is all on scopy on github um there's, my twitter. My email is just wildcard.kapiri.com, whatever creative stuff you want to put in there and then, if you want to see more space picks that I've taken keperia.com has all of my stuff. Thank you very much.

A

um All these links will be in discord or youtube. Hopefully, wherever you're seeing this so yeah, thank you.