Rust Programming Language EuroRust 2022, 4 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Rust and C++ - Tobias Hunger - EuroRust 2022

Description

Tobias Hunger
Software Engineer at SixtyFPS GmbH

Rust was designed to interoperate with other languages, but this interoperability is primarily based on C data structures and functions. This is limiting for C++ applications, because instances of C++ classes do not have a standardized representation in C. How can we expose Rust interfaces to C++ and the other way around?

At Slint we develop a UI toolkit in and for Rust. We also provide quality C++ bindings and integrate with existing C++ libraries. We do have to deal with interoperability issues on a regular basis. In this presentation we are going to review the crates we use to achieve interoperability, as well as the tricks we learned using C++ libraries and how we expose a modern C++ API to our Rust code, including a first-class CMake integration.

A

Hello, thank you. Welcome to my presentation. Let's get started um when I handed in this. When I saw this call for paper, um there was a lot of interest in the C plus community, at least about um how to write code in other languages on C plus plus, which is still fully compatible with existing C plus code basis.

A

So, for example, uh Chandler carus introduced a language called carbon, which is apparently developed at Google, which will be fully compatible with C, plus plus, but better in so many ways which are not yet defined foreign language specification, which is not that full feature yet but they're working on it. So I'm really interested in that language.

A

A bit later, herb Sutter came forward with the basically a way back to the history into the history of C, plus, plus uh C, plus plus started SC front or front end that converted the new language into C code and I hope Sutter had a CPP front which converts a new language into a c plus code uh again having a nicer language and still being fully compatible with C plus.

A

So we on the other side. Here we do have a nice language at least I think so um so I was interested. uh How can we actually interact with the bus? Go Echo code bases that exists out there, so rust was originally developed at Mozilla or Mozilla is a company with a huge C plus plus code base, so they actually developed it with intention to write code for Mozilla in a safer way.

A

So there was some idea about integration of C, plus, plus and rust from the very beginning, so I want to introduce some ways that can be done and let me get the let me get started with a few words about me.

A

I'm, a long time, C plus plus developer I spent, like 20 years of my life, working with C, plus plus or working developing UI code, working with UI code or building tooling for languages and I, was uh oops wrong direction.

A

I'm now a city arrest, developer um and I work on a project. I find really interesting, which I'm going to tell you about now and that's a toolkit written and implemented in Rust. So we are scalable. It runs from microcontroller like this here or a little picture on the board up to Graphics accelerated workstations and it's 100 rust.

A

So why are we interested in C plus plus? You asked so for two reasons. First off we also want to provide an interface for C plus Developers.

A

We provide a toolkit, that's just it's basically language neutral and it would be really cool if different languages could use it and one of the languages we want to support the C plus plus. So we have the challenge of integrating our existing rust code and providing a nice API for C plus plus developers to work with it.

A

On the other hand, are we optionally use existing C plus plus code, there's really good uh UI development toolkits for C plus out there, and we want to reuse some of the codes they have for our stuff to basically into platform integration into accessibility, Frameworks so get the styling right and all that stuff? It's optionally. We don't it's not the final solution. Of course we need something, that's fully implemented in Rust, but it's at the moment. It's a good solution to get give us a lot of Head Start.

A

So that's what we're currently using so we're I'm interested in both sided directions, basically using C plus plus from rust, and also exposing rust code and providing an IC plus interface for it.

A

I have planned to talk about two things are basically code level integration, I have some C plus plus code I have some Rust code. How can I make those two talk to each other and I also have some awesome prepared some slides about the build system integration because once you have code in Rust, energy plus plus, you need to build it into some combined product. How can you do that? The interest in that topic or should I just skip it for the okay? Well, I'll, try to leave room for it.

A

Thank you, yeah I do now my window closed. Okay. So, let's get started with uh the code level. Well, we have two languages. Why is it challenging to integrate some of them with one another? So first off rust has no defined API. Api is basically when you program code, you define functions, you define data types and the compiler Works processes that and produces a binary out of that, and this binary has also some kind of interface to assess these functions and methods and data types and the rest.

A

Compiler, basically does not define how that looks like that makes it interesting to have one binary talk to another binary, uh but on the other side, C plus doesn't either so they do have a compiler. It's basically compiler defined there as well. They, of course, have lots more compilers than we do so. uh In my opinion, that does not really improve the situation.

A

Then we have two languages with very different concepts, so we have uh inheritance and multiple inherits in C plus plus, which is something we don't really do. As on the west side, we have trades, they don't have that we have live times, they don't have that, but they have templates, which can also be quite challenging.

A

So it's very different concepts and it's hard to map those on top of each other, and we have another problem. So we have data types. So, even if we have the same data types, let's say we have some Vector, which is basically in all language. It's just some memory, a capacity and a length, or you can still have problems, communicating this data structure across language binaries, because our it's there did as they might Define the sequence in which those uh this data is in the defining the structure differently.

A

So rust side expects to reach length, so C, plus plus side puts a capacity there. The whole thing blows up or the data types might mismatch. So there's a lot of details here and even if the data types would match exactly, you can still have problems, think about strings or in Rust we have a string which is utf-8 sequence of characters.

A

B plus plus has a string which is a sequence of bytes which may or may not be in utf-8, and so programmer knows what he's doing so. He will know which encoding. That is so it's really challenging to mix these two languages.

A

um We do have one basically, the least common denominator and that's basically the C language interface.

A

That's not really that well defined either or but or it's the basis of a lot of definitions for binary formats are then use like Elf or ee on Windows, though, that it can be used as some um common basis that both sides can agree on, and indeed there is a c uh foreign function. Interface as it's called is the backbone of all the different approaches I'm going to introduce today.

A

So how can we generate binding? So why do we need bindings? Let's start with that, so we have some function in the C plus method. We want to call it from the rust side, so we need a bit of code to tell the rust compiler that this function exists, and this is a binding.

A

So we need a way to generate those. The best approach for all the people involved is, of course, having that done automatically, though um we have several tools for this. uh There is spine gen, which takes C and simple C, plus plus code, and produces rust files. Out of that, that can then call these functions.

A

um Binding is a wonderful tool or it works really well, except so it does. There's some construct that it doesn't support. So if there's some multiple inheritance, it can be tricky, um and so the tool is basically designed to skip over anything. It doesn't know and just ignore that part. So when you use it in practice, you will need to click it a bit and Define the configuration.

A

So it's not fully automatic, but it does get you a long way already or the other direction can be covered by C by engine which takes rust and produces C and C, plus plus headers. This works actually reliably well and that's because it only looks at functions and data structures. You have marked up as rest programmer with wrapper C. So basically you made sure the thing can can be expressed in the C function for a function interface. So it's really easy to extract that and produce basically C code. All of that.

A

Well, that works really well. So the next approach I want to introduce uh is basically semi-automatic bindings.

A

So the first approach we had the full automatic one is basically ideal when you have header files and just needs a quick way to get started with uh resource in Rust.

A

This approach is more for people, designing a new library, for example, or is a basically um the idea here is to Define or the data types and the functions that you can be called from: one side on the other side and generate bindings for both sides and basically automatically generates ffi interface in between, in a way that you don't really need to worry about it.

A

So here's a prominent example is a cxx grade, uh but there are, of course, other crates that build on top of cxx there's rates that handle special C, plus plus dialects or whatever there's extensions to cxxo. But this is just a basis here. How does this work? So it's takes some Rust code here um and it you need to introduce a new module in this case it's called ffi and you mark it up with a dxx bridge macro.

A

This basically tells the XX gray to take over and interpret have all the other codes that follows here and then you can go ahead and describe our common data structures. So in this case the data structure metadata will be available from both sides.

A

um Both rust and C, plus plus, will know about it. They can both interact with it. You then, can also basically Define what you expose from the rust Side by marking it extern rust.

A

These are, in this case, I expect or exported data type is a multi-buff, or in this case it's also just an opaque type, so you can pass it through C plus plus you can pass a pointer to it or whatever, and you can pass it back into rust and the rust site can then assess it again, but you cannot do anything on the C plus plus side with that.

A

Oh, the cxx grade is actually so intelligent that it recognizes that next chunk takes the multi-buff as a first argument and actually turns that into a member function. So it's really neat there.

A

um Then you can go ahead and Define the C plus plus side. So these are this is this: is our functionality that's exposed from the e plus plus side to the rust side?

A

Here we have a client again, it's an opaque type which cannot be assessed from the rust side, and the interesting thing here is that you're encouraged to mark up the things that are relevant to the rust programmer or the rust compiler like.

A

Is this function or a safe or unsafe to call, or is there a Lifetime on the different reference pass and all this uh which is completely ignored by the C plus side, because these Concepts just don't exist there, but they make it easier for the on the right to work with the data on the rust side.

A

So um so this is, you can generate findings for both languages and start to use them, and the cxx great here tries to be safe and fast. Safe in this context, is a typical, rust, safe or the rust compiler has checked out the code and is sure that all the invariants it expects from it are held.

A

um Of course, that cannot be said about any C plus plus code, simply because rust compiler can't understand it, so it cannot uphel any invariance there. So the safe in the cxx grade only applies to the binaries. It generates themselves so that will not introduce any additional unsafety. It's there's no um there's nothing said about the C plus code itself, so you still have to review that as any other C plus plus code in, and it's still have to treat it as unsafe, interesting.

A

um The first thing basic uh means that uh the the XX crate will not do any conversions, so it will, it will do zero copy between the C plus plus side and the rust side.

A

This also means that the data types from one side site will shine through on the other side, for example, if you take a C plus plus string and pass it to rust on the rust side, you will have a cxx string because a string in Rust or needs to be utf-8 encoded and some guarantees are need to be upheld, which are not guaranteed on the C plus plus side, and if you don't want to transcode and copy things you have to do it. That way.

A

So the next uh approach I want to introduce is basically no binding and I'm cheating here a bit. The idea is, uh can you actually write rust code and include C plus plus code? In there? The fun thing is uh yeah. We have a CPP crate. You can, and this totally blew my mind when I saw that the first time so you have a rust function, taking a self object, a reference itself generate a rust variable object, which is take something out of that self and then oops.

A

What did I do here and then you do this.

A

What this code does um is it internally? It adds a function, call um which generates a binding to C, plus it generates a C plus plus function which takes object as a q object point an object pointer and it generates the code. It compiles that and then it adds the code. You write there into that function and it compiles.

A

The C plus plus code, generates the bindings for the rust code and puts it in place of the CPP micro, when I first saw that this little might exploded so and you can go, go one step further. Of course,.

A

You can have a rust macro in there, a pseudo macro which allows you to switch back into the rust world and again also you have the C plus plus data type, which is void pointer on the right side and the rust a data type on the left side that will be overmapped.

A

uh This crate is not fully saved, so if these types are not aligned properly, it can crash or what can be done to improve that, but at the right time it's not as safe as a cxx grade, but when I first saw a code like this I was totally Blown Away. The rust macro system is so great.

A

So uh let me or summarize a approaches I wanted to introduce to you, so we have several options, so you can use code, generators or you can use macros or if you want to you work with the library you can or a C, plus plus library and call code code in there. You can also use it, go with the c plus a CPP crate and basically avoid the bindings.

A

Of course, you also need to Define data types on everything to make it if you want to use them from both sides- and there are more macros in the CPP crate to help with that. But if you want to use Library, you can use a CPP crate. If you want to write codes that integrates both sides, you are have the cxx grade as a great option and you can always fall back to code generators like the bindgen or C by engine and others to generate more code.

A

None of the options we have are fully automatic, so you will need to do some work, especially if you want to have really greater native interfaces. So in slint we use find gen. We use the CPP crate and we also have headers C plus code on top of the generated bindings, to make that really nice to interact with.

A

But the cool thing about this. uh Nothing in the rust language is in is concerned with C plus C plus plus compatibility, but we have a language which we can develop completely independent of any types or influence from the C plus language, and we can basically go a completely direct way. A independent way or all we need is external code generators like winegen or C by engine and macros, of course.

A

So that's what I wanted to say about uh there's a Code site or any questions so far, uh I don't see any so we stick with the user format and do it at the end. So let's look at build system integration, uh that's actually a challenging topic.

A

uh Usually you will end up with one build system that you have to integrate in another build system. That's for two reasons: any real life project and any non-trivial project will have a build system. That's just too much work to we Implement in a different way, just to build it in a weird setup with some other language, so you will need to embed one build system in another for the rust code.

A

That's especially true, or mostly because the cargo is the interface to build rust code, but users I expect to use, or so that's makes it a bit challenging where, with C plus plus build systems which kind of have this approach of running compilers directly, which is something we basically discourage people to do in the rust world.

A

So in most cases you will end up with one build system integrated in another one. So let's get started oops. That was the wrong direction again.

A

Am I, let's get started with cargo. This is a good approach. If you have a little C, plus plus code that you want to integrate in a cargo project.

A

If you go for that approach, first thing: I recommend to do go to creates.io check whether somebody else did the work for you. That's the easiest way to get bindings possible, just use whatever somebody else has done before and cooperate and help some fix bugs. If you encounter any, brings the entire ecosystem forward and it saves you a lot of work.

A

If you still need to do it, uh we have the approach to do it in cargo is build RS, which is a rust of a piece of rust code that gets compiled at compile time. When cargo sees a build, RS file, it compiles it, it executes it, and this this is this code can interact with whatever can do basically do everything you can express in Rust code, run that and then return some information back to Cargo about when to run the the build address again and similar things.

A

So it's rust code. So, of course, we have grades for that. Well, the first one is the CC grade, the wonderful grade, for whenever you have just have a couple of files you need to compile. It basically allows you to drive or start run a C and C plus plus compiler directly, but basically includes the CPP crate as a built as a developed depth dependency. So it will be available for the builders script or tell it here, find a compiler build these one. Two USB files for me generate a static Library.

A

It will generate a static Library out of that and that can be included in other parts of your project.

A

If it's more complex, you want a more specialized trade. For example, there is a cmake crate which basically drives a normal cmake or configure build install process and then makes the build results available to your or to the cargo project to include in other places,.

A

If you need to go the other way and have a little rust code or like let's say one library or something and want to integrate it in an existing or cmake project, you will need to integrate in the cmake build system or in the e plus plus build system I just assume for now it's a cmake, of course, other options or or cmake. There is actually a really good solution.

A

We found in slint uh that's the corrosion project, which is basically e, plus C, make code to interact with cargo and drive cargo for you, a minimal project looked like this, so the first two lines are the boilerplate. You have basically in every cmake project, just telling it the minimum CMAC version and the what kind of project you use or you tell it or I have a project it uses a C plus plus a C plus plus language with language is cxx.

A

You don't need to mention rust at all, no need, um then you do find package corrosion, which will go and pick up the corrosion you have to have installed separately. Are there other ways to actually make cmake and download and install it for you during at build time? If you need that, but uh this this approach here assumes it's installed on your system somewhere in a place that cmake can pick it up and then all you need to do is basically point it to a car.

A

Automal file and corrosion will do the rest for you, so you can, of course, also set features and all the other things that you need to do orbit afterwards.

A

You can or there's a build results of the cargo or build project will be available as normal CMAC targets, and you can just link to them as with any other CMAC Target, so that works really well. We use that a lot basically to enable C, plus plus developers, to just interact with them with our code without needing to know that it's invest, so they can ignore the fact the way if they want to yeah- and that's uh all I wanted to tell you about.

A

Thank you for your attention on this QR code should take you to slides or copy of the slides with speaker nodes and a lot more text, because my slides are a bit sparse.

A

um Please feel free to reach out to me Twitter whatever, and so these are links you will find as a presentation. Thank you for your attention.