Rust Programming Language Integrating Rust, 9 Dec 2018

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: RustFest Rome 2018 - Yiming Jing : One Thousand Ways to Die in Rust FFI

Description

Efficient FFI bindings are one of the core features of Rust; however dereferencing raw pointers, conversion between Rust and C types, memory allocation, ..., all could possibly go wrong in FFI code and undermine the memory safety guarantee of Rust. The Rustonomicon and a few previous talks at RustFest discuss the topic of Rust FFI; but overlook the challenges in writing correct and safe FFI code. In this talk, we discuss a taxonomy of common pitfalls in Rust FFI with real-world examples. Moreover, we will review relevant principles and share our approach to effectively avoid the pitfalls.

https://rome.rustfest.eu/sessions/one-thousand-ways-to-die-in-rust-ffi
https://media.ccc.de/v/rustfest-rome-9-one-thousand-ways-to-die-in-rust-ffi

A

A

So our next speaker is Eamon Jean. He works for by do, and today he will tells us about the how to interface rust with C pitfalls problems, how to solve them, how to overcome them, because documentation is one thing, but experience is something else. So please welcome.

B

Basically, talk about the lessons, the experiences we gain when we try to rust a vehicle in our in our open source programs every in production. So my name is even change. Am security scientist engineer at by 2x lab so I'm? Also, the author of missile ink basically missile ink is a test library providing open, SSL, compatible CIPS for rusty, RS or Russells.

B

So with Missa link, we can bring rust into many places where open, SL and C used to dominate, of example, Android a lot of places and why do we have deployed miss a link in our applications running on Android powered, smart TVs? As of now from from my college, I heard that mr. Bing has accumulated about more than 4.4 million monthly active users and its rapid growing it's growing very fast, and in August we managed to to contribute code to the curve project.

B

So meta link is currently one of leap, curves 12, POS backends, and it is also one of the seven beckons that support geometry and it is the only one which is retaining rusty and speaking of our X lab. So we are a research lab based in California, United States and for bi. Do bi.

B

Do is the second largest search engine in the world and the biggest wine China entered app is led by our chief security, Oh links way and he's a big fan of memory safety languages like rust, and we have several open source projects which is a which are implemented by memories of languages, including rastko and our Python, and mr.

B

Ling is just one of them and if you were a trust first Paris earlier this year, you may have seen our work about bringing in hell as gx2 the rust ecosystem, so which is the rust SGX production here in the first row, oh and the second one from the left.

B

So in today's talk, I'm gonna I'm gonna share with you the experiences the pitfalls in the lessons we learned when we implement open cells, ap is in mr. link with 100% Russ code. So first we'll start from a simple hello world, F of example, and see how it ends in a disaster, mani box. So then I'm gonna cover Tim patterns and anti patterns in writing. Idiomatic fi code and finally, a quick recap and future work.

B

So through our code is a code examples in this talk. If you've seen a green checkmark, then it means this is a good pattern and we recommend using that and if you see a red cross mark, then it means it is an antibody and a pattern. It's not good news and one song means a pattern we suggest to use with culturing because it may involve some unsafe code and some assumptions you must be aware of if I use it so I start.

B

So this is a hard were example from the Russ programming language book of the textbook. So what it does is very simple, so it converts or I mean let me zoom in so in case. You can't see it so what it does. It is to convert a string literal which is her or us here to some errors to a reminder, so the string can can be used by some external C code. So the no.

A

Mango Domingo.

B

Attribute here turns off with Ross name mangling, so this function can be exported and it makes the linker happy so it works pretty well, but this function just says hello too only so here. So what do you want? A function to say hello to more languages? So we got this one.

B

This is, we just add another parameter called who enter our PDR, which, which is applying her to a string in C. You can put. Maybe Python, maybe go here to pass it to a tourist function, and this function body consists of only two essentially two lines of code.

B

If we put the breakfast on the same line, so the first line converts the pointer to cscr and we can were the cstr to a row, string Russ STR and we use the format macro to to concatenate hello with the with with the input and we return it as a pack. So this function has several box in this dead, simple to launch the code. So maybe, let's just spend a few second to see. If we can find out.

B

Okay, so um maybe you can guess: I'll put five Red Cross Monsieur, so at least five bucks. So first um you have notice the pointer. We have in a check if the partner is no partner, so that's the first one and the second one, the unwrap here may panic and, as you know, if you function panics, if your trigger is like unwinding and if the stack a once across an FFA boundary, then it is totally and even be able and the third one. The format mackerel here eric is a a string on the heap, but.

B

But string would be otoscope immunity after the function returns, so the return pointer is is actually a dangling pointer and as a lot 11.

B

Returns a pointer of every star, constant u8, but here's the tricky thing we want the function to return a star counts to see underscore char, which is a type defined in the leap secret. But this one CNS Kochhar could either be you 808, depending on the target platform, so on different platforms. This could possibly happen mismatch and, finally, because a c string is a is an array of ice which terminates with a knock f3 at the end and as you can tell from here, this string is now terminated with an open up header.

B

So obviously it doesn't return a valley, the C string, fine, so basically.

A

Five bucks here in this testing.

B

Go to Alaska, so you can see it's not that easy to write a correct, rastafari code. So next, in the remaining remainder of this talk, I'll be covering ten tips we gained when we write rosetta code. So the first one point pointers, anti checks, so on top of everything, always check for non-hunters, so it's not just for us is for every language like Lexi. Maybe Evie has a point so for this one I don't see more about checking, ok pointers. So when we implement a missile links, EAP is.

B

It is quite common that we generated an opaque pointer in Rus code and pass it to the corner in C, for example, the.

B

Umizoomi, so the SL city x1. This type, is an example of such pointers. Miss a link when the car in C uses this opaque pointer, for example, as in parameter of as a rate different here.

B

It is our responsibility to check if the pointer is indeed what the rascal degenerate for, because the C part of the C code may may just that may make just can't cut the pointer to some. Sometimes we don't know so here. The point is that we introduce magic, bytes photos on here. I use a you 32, as the Metro bytes, we use the Metro pass as type identifiers, so such metric bars are used for every W. Miss I think that we may generate a key point as well.

B

We also add a treat called opaque point guard and for each type that we generate. A big point is for we implement this treat for them and we also assign a different metric number, for example. Here the number can be anything it can be a USERRA. It can be an array of you a device. It can be anything you like and you can you can. You can assign it. Maybe it's dirty trust in the function, so as a as a constant, it miss a link, we do it or differently.

B

We initialize this, this this magic, bytes or with a random function. So when your libraries web start so basically every time you run your to our code, the measurement will be different, so increase it Ingrid some difficulty for some bad guys to reverse-engineer our code and to do something bad with the so or all the magic by scary type of information.

B

So we didn't lose it when we pass a row behind rusi in that fear, when, because of the the raw parent, doesn't because the sea can do anything to the fire and when the sea pointer is passed back to Rosco, we can check the metal parts to make sure that it is indeed what we generally form and for this one.

B

So tip tip is about type conversion. This is one of the most common mistakes. Rusted rings are very different from C strings, so C string are now terminated, erase advice while rostrums our unicode encoded bytes. So the good thing is that rust provides the cstr to handle conversion. So the bad thing is that from a nascar.

A

B

It only does that the row pointer has input, but if you recall in see, if example like strncpy, it also has a parameter of lens. This is because front underscore PTR internally calculates the length of the string using STR Leon. If you check the South's coddle, you see so in some cases, if the input stream the pointer here, if the input is not properly terminated at a right position, so you may you, may you may you, may you may incur some other burn rate so as I suggest here using the warning sign.

B

Additional checks may be needed to verify if the C students probably terminated our read price, so otherwise you will think you will get an Autobahn read and here's the another one I just mentioned before about the leap secret. It is either IAT go eight depending on the target. So in some platforms follow on x86 platform, a CI new supercar is i8, but on arm platforms it is u8.

B

So the solution is to implement to where runs of the function. So for you, 18, I, ate a copy. Another solution is to cast a slice with the help of raw pointers with the Panna. Here this pattern is recommended in the documents of STD kong-kong ma'am : contrast need without using transmitter. You can also do it as well.

B

Tip story is about getting see, pointers from rust rings and rust outlets, so we know that references carry lifetimes, so the bar checker can check the life times for us, but roll pointers are different, so we must make sure our arrival, always outlive, is pointers so for constant strings literals in maze a link of a lesson like this one. We represent the master. He you a slices and we manually add an altimeter at the end like this one.

B

So the so the arrival are always alpha: Dave, the pointer and four strings with a shorter lifetime, for example local variables. C string is a better way to do so in cruising now terminate a copy on the heap, and then we can lick a pointer with Inter all, but this, but in the this approach, may leak memory. So if the leak string is not free later, we will know later in. In our talk, in forgetting C pointers for us objects, the similar rule applies, a variable must outlive aspires, otherwise you get a dangling pointer. So.

A

B

Here we are casting a reference as a row pointer and use the pointer as a return value. So ban is bad so because the food does not out little pony or you got a full underscore. Pdr you've got a tangle planet here. So what we do in measuring is we frequently use box into wrong to link a pointer, transferred.

A

Ownership from.

B

From Russ to see who is into wrong so then we so in C code we pass the pointer back to Russ and we reverse the role pointer and to.

C

Our self again.

B

Looking, for is a very interesting case, the calm is not visible here. Actually is it's the part of active and active slab you've ever heard of it issue 289. So some someone complains that a lot of bad patterns were use in a in active app focus.

B

Focus on me like this, so we declare foo, which is, can hit a string and we define immutable. I mean horrible here phille here and we use this kind of trick unsafe.

B

You, you cast the you cast a reference as a circles pointer and Costigan's as a star mute, pointer and dereference it. You get immutable reference and you can just.

A

B

A

B

Cell and rev sale to help you to use this one.

B

Memory allocation and de-allocation for tips 5, so that read if they, if you handle them improperly in unsafe arras, so they may cause a lot.

A

B

Memory leaks and dangling pointers, so in say, frost code. The boat record usually identifies and prevents more than 99% of memory box. But if unsafe ross code and row pointer is kicking, so things start to get very complicated and a special culture, so here was show an exhaustive list of things that needs your attention. For example, the for many micro.

B

Rc and a RC are also on the peep and otoscope marbles are also implicitly allocated. So the last one into raw needs some debate, our discussion, so we know that the box into raw can be used to leak a pointer. So the question is: can we call into raw in ross and free in NC? So if Ross and C use different memory, alligators, like a few months ago before global a dog is stabilized, so the answer is definitely no because your your systemic trader and Ross the tree melaka main they don't share information in general.

B

We also don't recommend doing so because we think the best way is to advocate in Ross and B. I can rust and arrogancy and the Arcadian C. So rough and C are two separate world. Only they communicate with the FFI interface and tip six is about panics in Fri code basically was the bad thing is that a panics may trigger take on winding and unwinding past the boundaries and different behavior.

B

So the suggestion in the documents is to use catch, a wine to catch the steak and wine ahead of honorees enemies are linked. We also used rustic air handling, a pattern along with missing links, internal air q, to handle this kind of panics.

B

So, despite that, we can catch and stop second winding using catch unwind. It doesn't mean that we can just let our rascal panic. Panics are kind of good for in terms of preventing undefined behaviors, but paintings also have a negative impact on user exchanges.

B

So in missile inka we always prefer to write some robust panic free dress code. So these are the scenes that may panic at runtime, for example, ref cell panics. If the dynamic power checking fails panics if the sizes of the source and destination slices don't match, the unreachable, Metro also panics is enriched and then dynamic overflow checks also manning.

B

If it's turned on by default, it is off its or tries to turn on or turn it on or off, or not so also to vote for its dependencies and callback functions because you don't know if they are panic, free or not so really be careful tip. 7 is about converting fail descriptors, so we are talking about a CNS current and the leap C Konkan file.

B

Absolutely in rust, you can use the from raw RFDS or a toggle here or from raw idly, at least for eunuchs, to convert a file descriptor into rust objects like file or TCP stream, but promise that this function usually doesn't validate the file descriptors. So we don't know if they'll file well, 30 superior is good until a later operation return. Some errors- maybe maybe sake, cannot be connected. So what we suggest here is we've annotated in the affair descriptor with sleep C's, F cntl. So we can know if the Faris Twitter really exists.

B

It is varied or not and for the leap, cease-fire, pointers because rust in rus STD do not have such times, so we still have to use the unsafely PC using FD, open and phone number. We can convert a file pointer to integer, for this hunter and tip 8 is about copying data from C into rust. So basically we have three approves or to do so first it can. We can use the lips as we did in C. We use mem copy and move it works. Andrew cross is not the rustic way to do it.

B

The second approach we use the PTR can come copy, narrow lapping. It also works. It is certainly semantically equivalent to a mem copy in C, but personally, as you prefer, the third one using a combination of PTR can come from Roper's plus slice kanakam copy from size. The reason is so using from reports. You can get a slice and you got a chance to validate the buffer before you do some further actions on it.

B

For example, if you are referencing a pointer which is a C string, so by from using from reports, you got you a slice you can check if the the buffer is terminated with the null character at the end and using copy from slice, you get a free, you get a free boundary checking because copy forms, the lives of you check the lens of the source and destination slices. If they match your didn't match either. Panic so are all using the third approach. We can mock intro and also free secrete checks.

B

This is the one is the most interesting one about converting see in arms to Russ, announced, see and Russ see seen. Arms are nothing like rose in arms. This fight, Russ approach, this Tao in amps, so Russ in arms are essentially a range of integers where you can now define duplicate values, but in C because they are just simply integers. You can do so very easily. For example, you can see I have a lab or default equal to two.

B

Also level high is 2, but in Russ you just cannot do it because, because in Russ, Russ in amps are a range of numbers, noting values around. So the workaround to this problem is to use difference in header files, this header file, so we use the preprocessor Marco to define level default to live a high. So we can kind of bypass this problem a missile Inc this one because, as I said see in arms, are just integers.

B

So if you are trying to convert a signal through a rusty knob, don't use to Neos transmitted, because if you don't, if you transmit a integer to a rusty now you may encounter some kind of undefined behavior. So if the value is not in the range of the racino, so what we do in missile Inc is we use the from in to treat to check if the value is in the range? So it is not.

B

Oh, it is level and defined here so always validate a see now before you convert it into a Rossy now, and this one is pretty interesting. I will share with you so mr. Lincoln mr. linghu. We sometimes need to starter pointers for some for some names for some named names on of some in a morons in Rus. We can do it easily with the gravity back and we can use something like this format.

B

A column question mark to get a name of the variant, for example, congressional Davos, like level high and I below and never level before, something like that, but in but not to leak. A pointer of it is very hard to do so. So we actually struggled quite a lot. We have several attempts so the first time that we just used the formatter, it Erick's the string on heap and we turn pointer and we cast it as a row.

B

Pointer but as I have said in tip two and five is not right, because the the archive string does not outlive the pointer, so basically Katanga pointer gear. So in our second attempt we learn something from our mistakes, so we start with you see string, so the string is probably terminated with a null character or at the end, and we enter app and we intervallic a pointer, big burner. So now the pointer has a longer lifetime, but I would say it works, but it Eric's memory on the heap.

B

So it may lead to some memory memory leaks. So please see our tip 5 & 6, for we find it's very hard to do so and finally, we came up with our own crate, with procedural macros and custom derived so find our create on Chris Nile, and it's pretty easy, and here are some external tools. We find pretty useful when we write a truss I've our rows, FFI code. Please give it a try and see if they fit our needs.

B

So, for example, I always use whatwind to find memory leaks caused by improper use of box into raw and cargo clip is lin's often give me surprises about writing idiomatic russ code, and also it helps me avoid some unnecessary unsafe code and some acknowledgement on the materials I found pretty helpful. So the Rustom on our roster nomicon is definitely must read, and micro bronze, alpha trole fi book press me a useful insights into error, handling in Ross and Z, and also some tips about integrating cargo into the sea maker.

B

Build system, and my things also go to the previous talk on R as fast and Russ conf. So many acts and talks on this topic. So a quick recap so tip points about checking on pointers. We use some magic buys as toilet and fire to do so.

B

Tip 2 is about converting converting between commercial between C and Ross types. So we mentioned about strings and tip stories about cutting a see. Pointers fall from Ross objects like strings or some anything like anything, and he for portable pointers, not for bypassing the protractor I showed you a counter example. So do not do that and he first bought memory allocation and de-allocation. Something bad can happen if you are not aware of it and also fi code, virtual panics, because unwinding past the boundary is the undefined behavior and tip 7 is about converting file.

B

Descriptors, like integers or lipsi con-com firefighters, tip 8 is about copying data from C into.

A

B

I showed you three master to do the two so and a person I prefer to serve on and now is about converting C nouns into rust and file wines by external tools.

B

This is it young, since well, since all coming.

C

B

D

B

D

D

Okay! Okay! Sorry, for that, do we have any questions right.

D

C

We'll have any tips for handling see libraries that internal use, static, mutable, state.

B

C

Do have any ideas about how to create FSI bindings to libraries that internally have static mutable state, because I actually have such use case that I use library that has static mutable state and know that it's tough to do in rust. So whether you have any advice on this.

B

I, don't know if such crews exist. I know some good ones like is the FFI and and also like C pungent price, a good example of automatic doing so, I don't yet, but I we are still in on instruction. Thank.

C

D

A

Questions there were there's a hand over there.

D

Thank you for the talk, I'm interested to know your ideas on how you handle errors across the FFI, for example, when you dealing with the seco that is using like status code and how you convert about to like proper rust errors and also, if you're doing the other way around. If you have something that's calling rust via FFI, how would you handle our.

B

Ross, the side we have investing, we have several macros to help. You turn rust result to some like sierra code, because Owens's they all use error codes, visitor integers, but arras has is results and also some errors and failures. This kind of error handling creates so basically at a boundary. We convert them between from rust to see as I said, I have a matter of doing so and from Caesar rust we define a few from and in two traits implementations to convert the sea-air numbers to some rust errors so been used to thank.

D

D

Other question.

A

Okay: okay, okay, we're done yeah! Thank you. Thank you. Thank.

D