Cloud Native Computing Foundation Cloud Native Wasm Day North America 2021, 30 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: Wasm for Heterogeneous IoT Devices - Dan Mihai Dumitriu, Midokura (Sony Group)

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Lightning Talk: Wasm for Heterogeneous IoT Devices - Dan Mihai Dumitriu, Midokura (Sony Group)

Software for embedded systems has been developed in a monolithic way. Firmware is written by a single vendor, and not updated frequently. Modern IoT devices are constantly connected to the cloud, and OTA updates are possible anytime. Modern IoT devices may be customized, by 3rd parties, becoming true platforms. However, the software development paradigm has not kept pace with this trend. Typical IoT devices do not have full Linux capabilities, no MMU, and thus no HW based isolation. Wasm enables the creation of truly customizable devices. On the server/cloud side, the platforms are relatively uniform - x86 ISA and Linux OS - and the OS can be the abstraction layer. In the IoT domain, that is not the case. IoT device platforms are very heterogeneous. On the ISA side there is at least ARM (32 and 64 bit), RISC-V, Xtensa. On the OS side, there are a myriad of RTOSs. The platform abstraction must be at a higher layer. Wasm can be the abstraction layer for applications on such platforms. We have created a framework and SDK for AI App development. We have benchmarked the performance on ESP32 and on ARM A7. There is a performance penalty and memory overhead vs native, but it is very much acceptable - within a factor of 2. We believe that Wasm will play a key role in IoT development.

A

Hi, my name is dan, dimitri and I'm the cto of midokura, a sony group company. That's focused on iot uh platforms and visual sensing, and this is some joint work done with my colleagues, dongxiang yan and pojejang from sony semiconductors solutions, china, development center.

A

So some of the pain points of iot manufacturers are that it is difficult or impossible to change functionality after deployment and there's no standard component model. So it's pretty costly to customize devices from a base platform, and cyber security, as we know, is a big problem, especially for for iot. There have been many many instances of compromised, iot devices and vulnerabilities really can all be found before shipping. That is just not going to happen.

A

So, in fact, the industry needs to move to a secops model where, as vulnerabilities are found, they're quickly, fixed typically the companies in iot. They don't have these competencies, so one of our goals is to build customizable iot devices.

A

That means that we need to change functionality without doing full ota over the air update, which involves replacing the entire firmware.

A

It means that we need dynamic loading and linking of application code which again traditionally in embedded systems, it hasn't been available and we need memory, safety and control flow safety on iot hardware that doesn't have mmus, which are basically most 32-bit mcus.

A

We also would like to enable application portability across hardware platforms and across operating systems, uh which you know in the iot domain. The real-time os ecosystem is quite fragmented. It's not like in the linux world, where all the cloud world, where everybody's running linux or windows, there are many of them and finally we'd like to reduce the development effort and support high-level languages.

A

So on that on the topic of the platform heterogeneity again in the cloud world, it's pretty much all x86 and linux in the iot world, there's a very large diversity of architectures. There is a lot of arm but arm v7 rmv8. Now these are different. There's a some x86, potentially there's extensa, which is a cadence isa. That's used, for example, in the esp32, and then there are a variety of operating systems, so linux might be there, but also there are systems like nut x, free, rtos thread x.

A

We don't have a stable application platform.

A

um We don't have mmus on most mcus, as mentioned. That means no virtual memory, no process isolation, and it is known that the number one source of security vulnerabilities is actually memory bugs. So if this is, this is true in the server side as well. Certainly it's probably true in the in the iot domain too.

A

So what we propose is to make an edge stack that separates the application layer pretty cleanly from the from the os and creates a sandbox and a set of services for these edge applications that are portable across multiple platforms.

A

So we looked at several uh options for safe language, runtimes or, let's say safe, run time execution runtimes. We looked at microaj, which is a commercial option based on java. It's a 32-bit jvm for mcu devices. We looked at micropython, we looked at webassembly jerryscript, which is basically like, javascript and and also lua.

A

So my colleague hendrik sundstrom in the sony research center in lund, sweden. They did some evaluation based on a bunch of different criteria like power and performance general system requirements, uh the runtime model, the ease of integration, the security, the software update model and also the hid support, which is not so much a concern for us in in the embedded iot world.

A

But it is for some types of applications, so long story short, uh we decided to go with webassembly and we decided to go with webassembly because it seems to be the most future proof and it allows us to support uh multiple languages with a with a single runtime system.

A

It's also because it's compiled uh to native and it's it's high performance as opposed to, for example, uh the micro python which is interpreted so here are at least three of the uh web assembly runtimes that are available on mcu class devices that we consider. So we looked at wasm time, webassembly, micro, runtime and wesm3.

A

And finally, we decided to go with the webassembly micro runtime, because, first of all, it supports the all the instruction set architectures that we we care about um extensa arm in the future risk five. It has an interpreted mode, a jit mode and an aot mode ahead of time compiled and typically, we are going to use the aot mode in for our iot devices and it runs on multiple os's, including linux and netex.

A

It also supports load time dynamic, linking of modules, multiple modules, which is interesting for our application platform.

A

So some of the pros and cons that we that we looked at the pros it's easy to embed with with the c api, so using c code, it can be embedded in in another application. You know like, for example, the whatever the bootstrap application agent, you know on our iot device.

A

Has a relatively small footprint supports various languages and it has a built-in libc support and it definitely ensures the memory, safety and control flow integrity that we that we care about cons that we discovered is that there is some overhead for the memory safety, because the bounce checks are implemented. In software it has no built-in gc that hasn't been standardized yet in webassembly, so we can't directly support languages like python yet, but that will come in the future.

A

So just some interesting numbers you know to look at so we benchmarked uh the uh we benchmarked this. You know using the core mark benchmark on uh the esp32, which is a dual core extensor uh running at 240 megahertz, and also on an all winner v3s, which is a single core 32-bit arm v7a running 800 megahertz and the numbers are pretty interesting. So, first of all, we can see that the aot mode is far better than the interpreter, that's expected, and also on the esp32. We see the aot mode is basically one-third.

A

The speed of the native the native code on the all winner, it's about half of the speed of the native code. So all together, this is pretty good. You know it's a it's an acceptable level of overhead for our application and we haven't really done much to try to optimize it so far.

A

One of the other issues, though, is the program size, especially when using c plus plus the the program starts to get quite big the code size. So especially the in fact the aot compiled code is pretty big.

A

The esp32 actually has a pretty difficult memory architecture because it's used it's a hardware architecture, meaning that it has separate instruction and data buses, and so not all main memory is executable and the instruction memory is relatively small. You know it's less than 500 kilobytes and in fact we we just ran out of space, so our solution is to execute from flash memory.

A

So, just to just look at this, uh you can see that the aot code size is actually even bigger than the interpreted. You know size of the of the webassembly by code.

A

So our solution for the execution for from flash memory which we implemented in the webassembly micro runtime, is first when at load time we will relocate. You know the um relocate the uh the tech segment and then we will write it into the flash memory and then we will use the esp32 mmu to map the flash memory into an executable region of the memory, and this seems to work and the overhead is actually pretty reasonable.

A

So what we plan to do with this is to build a full, automated cloud to edge pipeline where the developer writes some code, compiles it into webassembly, pushes that webassembly compiled module up to our cloud platform and then the cloud platform based on the target type of the device will take care to compile it to the final target could be the extensor on esp32, it could be 32-bit arm, it could be x86, etc and then finally deploy the the device. So the developer is really decoupled from the the target.

A

So in conclusion it works. uh The performance is approximately 50 of native code, uh which we hope will get better. uh The memory is bigger than we would like, especially for c plus plus, which we have a lot of, and at some point we would benefit from having hardware support for bounce checking, but that, if we don't it's still, okay for now and the automation of the end to end is very important.

A

So I hope you enjoyed my talk and thank you very much.