Fluence Labs Wasm in Web3, 14 Oct 2021

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Mike Voronov – Wasm at Fluence

Description

Speaker: Mike Voronov, system program developer at Fluence Labs
Topic: Wasm at Fluence

Wasm in Web3 Workshop https://avive.github.io/wasm_on_the_blockchain_2021/#/

A

Hi guys, thank you, annam, I'm going to be here for a second time, and uh today I'm going to tell you about how we are using webassembly at fluence in particular. The agenda of my talk is as follows. So, first of all, I will tell you a bit about fluency what we are doing and how and then we are talk a lot about what was marine runtime, how its looks internally and explore some tools around marine marine is our general purpose.

A

Sometimes that is based on interface types and the model linking and also it expose, shortness and linking scheme a possibility to link modules and, as the last section will cover a bit uh second, the last topic that is about our intermediate representation code here and about aqua vm, our virtual machine that interprets this air and that's located inside marine on in our network and, let's start with fluence this fluency review and fluence is a so-called open application platform uh in stern, open application platform is a thing that consists of three main that consists three main other things.

A

First, one is usual web 3 project. It is p2p, it's completely decentralized, and this p2p nature allows fluence to compose different apis different services, data processes inside network and so on.

A

It could be said that say that, from the developer point of view, fluence is about composing. It's about composing everything we could compose modules in the servers and the services with help of our new language into some, so called super service, uh and second, two things here is about uh is about marketplace is about monetization.

A

uh We believe that the future fluence network will allow developers to be paid automatically by the fact that their code are used by any one influence network, and the second important things here is aqua is aqua is our programming coin, which is that uh inspired by picaicos, and especially by protest calculus, and this language allows to program network requests across our network.

A

It's designed to coordinate and orchestrate a network calls without any central gateway. So all peers inside our network are peers. There are no any leaders, they play the same role they could host, um they could host services, they could be relay and so on. So they completely decentralized and aqua is interprets and is uh japanese by every service at once.

A

uh The third important thing here is services. Services could be considered as usual as general microservices. They first of all were assembly-based, and that's why this is unboxed by design. I will show you later on uh dedicated slide, that a developer could set settings for each module inside services.

A

It could have different constraints, different abilities and they could play a different role inside service.

A

On the slide, you could see a model of fluency network that consists of three peers and two clients, clients, usually usually browsers uh and the aqua. Could an aqua allows a user to compose these services located inside peers and compose their uh through webassembly interface types.

A

Each service could be either stateful or stateless. They could access hard drive through wazi. They could also call some external services uh through a spatial gateway functions. People also cover their later, and here you can see on the right side that uh one service could interact with three other different services uh that are in orange boxes. Here you can see ipfs some unknown micro servers, for example, databases like mariadb, mongodb, oracle and so on, uh and this uh like microservice written on webassembly, allow you to interact with some.

A

uh I don't know, well-known services that should that can't be, for example, compound web assembly uh and it's the power of fluence and power of composing and let's dig into morning runtime in details. uh Actually, you could see my previous talks and very marine was called fc. We have changed name about several months ago.

A

So please, uh when you see maureen, when you see fce in our code base or in the slides, uh please uh like notice that uh us and the things about it is about marine uh and from the server surrendered view flames could be considered as a in the following scheme. Here you can see that we have several uh several modules form a service here.

A

Modules is the usual webassembly modules that compound with our sdk and they link together and form one service and in certain, several services could form so-called super servers or back end, uh and it's important that these services could be located somewhere in network. It doesn't matter where they could be on one pier. They could be on different peers, it doesn't matter with help of aqua. There could be composed to some super service and for users.

A

This super servers could expose functions and they are like they're transparent for users and also it could be seen as a functional service.

A

But it's a bit more, of course, than just the usual function of the service um and, as I said, uh fluent service is just a group of modules that link together with help of sharpness and linking scheme and, as you know, short nothing. So actually there are two types of linking the first one is shared. Nothing. Second, one is short terrorism, and the difference here is what module exposed to external world, for example, to host runtime or to other modules in sharing nothing linking scheme.

A

There are only experts and imports that are visible for different modules for other modules, except this one and uncharted everything uh mojo could expose everything- yes, for example, constants, globals tables memory and so on.

A

Marine currently support only shortness in linking skin and the scheme actually could be implemented on the scope of interface types. So under the hood we are using web assembly.

A

Oh sorry, we are using fastener as a base runtime and the vasner craze that was highly modified by us called interface types, and with help of that, it's possible to make such a scheme where, for example, you have a service that consists of three modules, uh one like uh facades, it's one that just exposed api of all services and the second one, for example, could be some database like white.

A

We have a sqlite that ported to possibly also like in such a way that is compatible with other modules and these interface types wrapping in our marine and the third one, for example, could be some authentication module. Yes, it doesn't matter why.

A

An important uh thing here is that uh the linked by a linking corresponding imports with corresponding exports is shown on the scheme so like module, each module encapsulates all state inside. So no other modules could access to some memory, data and so on, um and we have three types of modules. The first one that already mentioned is facade.

A

This facet module is a exposed cpi of all modules in service, and only uh there could be only one facility modules service uh in config, their last one, and only uh these modules could be called from aquascripts. No other modules are, like all other modules are unaccessible for aquascripts.

A

Second, one is pure modules: uh they can't access file systems, they can't call external binaries. They just should contain pure logic, and the last one is a factor effector survivor. Could access file system could call external binaries and some other stuff? So the difference here is that pure modules could be safely loaded into each service into each runtime that located somewhere on the user machine, and they are completely safe and effective modules.

A

They could access file system, but they could access only uh these constraints with some settings and let's show how it's uh how it's look like here. You can see service configuration. Each service has a configuration file.

A

uh It's based on a thermal uh format config, and here you can see that uh this example of so-called euro download service. It's the simplest one. It consists of three modules. The last one is already set is first 8.

A

It should be set as the last one and can pick, and here you can see that there are no other special settings. Second, one is a cruel adapter.

A

uh This one is different from facade because it has uh mounted binary sections and the sections allows uh this adapter to call external binary through coi interface for standard operation system, civil interface, for example. This module could call coral, that's located with like through this path uh on operation system. The last one is local storage.

A

Remember that this is a simple example. Just like it's intended to show species of linking there are no any. I know any uh logic to split this into three modules there. We could see on example, that they're rather simple there are contains, I know about 20 lines of code, but and that just example- and here you can see that there are a bunch of additional settings, for example, for this module for local storage. We could enable special logger, I'm not.

A

I will not cover this worker, because it's not so interesting and you will find in well described in documentation.

A

uh Next setting is, I think, about uh count of memory pages that ex that can be used by this service.

A

uh These pages are not assembled pages of 64 kilobytes and we could set this uh setting for each module, so we could like set maximum amount of memory, that module could use inside the marine and a site, operation system, and also there was this thing here is wisely visibility we could set, which files and which directories are accessible, could be accessible from this module. For example, here there is this directory site that located somewhere near the binary uh of module, and also we could map it to like.

A

We could make aliases for this directory to access our analysis, but not through a whole name. um Okay and from general perspective marine could be presented like in this slide. Here you can see that in marine we could load several services at once and they could consist of separated modules. There are not. There are not any interdependence between them, they're, just each module uh which model like uh belong to only one service, and here you can see also that survey, effector modules, called access to operation system called external binaries and so on.

A

Marine has so-called a layered architecture. We have three layers to feed different purposes uh on the third way, areas on that layer that contains app service and a command vm.

A

This were intended to run services, let's aim of app service and to run aquavir to machine. That's aim of marine vm, but actually marine. uh The second and the first layer could be used in another different scenarios.

A

I think that's probably most white's most important one and most uh no useful here- is a so-called entity company system pattern that widely used in games and in scenarios when you need some plugin system inside your other system. So I believe that marine could be easily integrated to other projects and used as a used to allow ac pattern and on the um like, uh first level, level of engine. uh We have like a lot of apis here. You can see only important ones.

A

For example, you could load module uh these some mojo can pick um in mojo config. There are some settings that we covered on previous slides and one that dedicated to cost closures. I will cover it later. Also. You, of course, could unload module by name and while module loaded, you could call it code by name called by function, name and with arguments here. Please note that there is a value. There are no in arguments. There are no any row types. I value easy num.

A

I will show you later the what fuse that consists of, and also you could expose state of voice, state of module and module interface. Here by interface, I mean interface of modules. Exports functions that module expose that could be called from a module uh and second layer that also could be used not only by us they intended to like provide several variants of constructors.

A

They also provide several functions to work with complex and it has a function that mimics for call but receives arguments not in this form, but in form of json. So you could just uh code with json arguments and get also json arguments and also it defines several host imports. That's special for for network um anyway that I value that I mentioned previously looks like this here. You can see that I value consists of uh general rust types.

A

Yes, it's here, you can see like some primitive types, strings, vector of bytes, vector of different types and records uh and host closure. That I also mentioned looks like this, so here this one is intended to like to to code a closure that could be called from a module, as it could be called from mojo imports in main projects that they have seen that based on different runtimes vlasmer, wasn't time and so on. um Those closures were based on uh row types, so you should manage uh your own by your hands.

A

Yes, how arguments will be passed from blossom to host and so and in a different direction? Here you can see that this mechanism provides you a way to abstract over it and work on these eye value. So there are no any role types. Of course you could access to memory. With this context, it's was american text, uh but in ministry, what almost all station it's isn't needed here uh and on the slide, you can see uh how import for mounted binary that, like option on binaries that each module uh could be extended uh works.

A

So here are functions that return this host import, descriptor and inside it there is a. There is a like closure body. This cause your body, receives arguments and receives contact that uh isn't used inside this closure. Actually, and here there is a special function that implements uh choline mountain binary. It's doesn't important, just usual rust function, and uh here you can see that we that it so the results here.

A

Actually it's just a rust structure and uh like we have special function code to interface, value that overlaps this result into eye value and, like this return result and the whole function return this host input, descriptor. That consists of some additional fields uh like these hosts.

A

This course host closures is what is that I covered previously uh argument. Type should also be specified. Yes, because it's dynamic, linking that's supported by wasmere, but uh like to link it properly. We need to know uh types that closure should that caller should receive.

B

A

Allow, for example, type checking and also the same for output type, and if there is some problem, for example, in warring or lifting types between row web assembly times at and some uh non-simple rust types like string, vectors and so on, there is special error handler that looks like this. It's also closure that allow you to do some. I don't know some recovery after year or print some error message or something like that uh and the whole uh like whole import looks like this here.

A

Please also notice that there are no any, I know any any things related to raw arguments. We work with I values and we like. We receive five values and we return my values. There are no any issues with reading memories, writing memories and so on.

A

uh Okay, let's cover the next topic that, uh like dedicated to marine sdk and those around uh hello world for marine runtime, looks like this. uh It's similar to what's mentioned and to other different projects. I hear please note that there are no any traits that you should implemented. There is only uh that differs from like a vanilla, rust function. Here is only this procedure macro, so this uh micro means that recording in progress uh that this function greeting will be exposed and view like.

A

It became expert from this module and here just a whole world.

A

Yes, it's I know six lines of codes and it allows you to code on when you arrest, with some limitations, of course, because marine macro, for example, doesn't support, traits generics and like stuff like uh boxers and so on, but this will be added in future um and, like previously, I view um previously, I covered world download the service and showed a config for it, and here, let's see how it looks like from a code perspective, because here remember that it consists of three services: they're, just really simple ones and contains uh 20 lines of code and how it looks is from a code perspective.

A

It's this function is function from facade module. This function uh allow you to download some. uh I know some bytes from url with cool and then save within the file and this to function. The node and file put there from different modules and to like allow this module like to use it. Yes, there is like a known, um rust, ffi mechanism, that hero is is also used, and the only thing that you need to add is also marine macro here in procedure macro and also specify from which module uh this function should be.

A

It should be called, and here also link name- is also works. For example, this uh functions that in this module code, file put actually it imports with an input from module, called local storage and, like uh coral and local storage modules, looks like this like in real uh in real uh example. It's uh a bit longer. You can find uh it's this by this link, but uh here is the main logic, and here you can see that this download function actually we'll call it this this function without any additional uh additional work.

A

They it's it just work with marine. uh Also a more complex example, of course supported. For example. Here you can see that records could be used and the records could could be also contains fields that in its store and other records, and also you support, passing by reference in imports and exports.

A

uh But please note that it's since it's a sharedness and linking scheme uh memory is coping and the passing like in experts by reference is, is like a syntactic sugar. It's it doesn't allow you to prevent copying additional copying.

A

um Okay, and uh here I want to cover something special. uh Actually, it's special, because I haven't seen uh such possibilities in other projects. um This is how we approaching testing so to like to allow users to test their code um in the same way as it would be. It could be world in uh with developed marine.

A

uh We introduced a new macro called marine test, and here you can see how it works uh here, also our whole world that contains only one function greeting and imagine that we want to test it and to test it. We write tests as usual. We wrap it with qfg test and so on, and here, except just test that we like should write and when you were asked here, we need to write marine test and in attributes we should specify paths to config file and the pass to modules.

A

Yes, what folders that contains all modules of the service- and uh we also should specify an arguments like this one- this is all codes should be generated by this macro and inside it we could operate with this with export functions from this module.

A

It's different this uh this code, like won't, be compiled to web assembly and code. uh It's works a bit different um uh here. New marine instance will be uh created. This marine instance will be reward all modules from this deer and according to this config and all modules from this one uh will be supported from wasm. So here we test not a native target. It is the wasm. So by calling this greeting, actually we call marine and in marine we call functions that come out of assembly.

A

This allows users to make comprehensive tests for service that contains several modules, because you could access different modules. Modules should be specified in the arguments of function, and only this module could be access could be accessible inside this function itself, and this approach is very, very flexible.

A

Also, we are working now and this new version will be ready in one two weeks. uh This version of marine tests allow you to test several services at once. We could specify services as a previous one, but we specified their names like here and also we could interact with services and each module that that each service contains.

A

uh Here. You can see example for uh where service one um is called, especially it's. This first eight modules uh provides function called process. Data is called from service. One and the result of this code is then passed to process data of service. Two, and uh with this result, we could do something. For example, compare it with some. I know expected value, or something like that.

A

So hero scheme is uh different from other projects, because here, except this again, this function uh like isn't compiled assembly and then run it run. This function is run as native, but all calls of the function they actually called web assembly about the web assembly and this tests allow you to test services as more close to how it will look like inside real runtime as possible, and, let's cover the like last topic of this section, how the module multimodule codes work internally.

A

Also remember our example: these three modules uh of rail downloader service and actually for each uh function that imports or are either expert. There is a uh dedicated interpreter these interpreters is it operate, interface, types instructions, for example. Here we have seen three modules and for each module there will be a different web. Wasper instance.

B

A

For facade, coral and local storage and for each exporting for each import in this instance, there will be corresponding adapter, import and adapter expert, and imagine that we could call. We want to call function kuro from a module cool and to call this function.

A

There is a bunch of code of interface. Type code should be executed, so at first there will be executed uh these four instructions. They are intended to lift a string. ah Remember that kuro has a curl receive string, can produce string, it's one argument and one output value, and this code is indented to lift string from memory of guru, module and then call cool adapter this one and next step there will be uh like these instructions.

A

uh This instructions are intended to lower uh string into a memory of guru module. Yes, here you can see, for example, functions at over memory. Actually, this instructions do uh like this. First, they allocate memory uh of string size that passed from this uh adapter from the left side.

A

Then it called over memory that copies string from a host memory to a curveball memory. This module memory. Finally uh like a wheel function, function from instance, will be called.

A

uh It's like, after wolverine, yes, we could um use row types, and these are all types we could call row wasn't function and uh the last step here is to pass result string back to the first 8 module here. We like in this in expert adapters. Only thing that we need to do is just to lift memory back, leave string and then we should lower it back, should lower it to a facade memory.

A

So that's how interface types works.

B

A

Should be not not here that we reuse, we are using now the first version so called first version of interface types, because the interface types proposal was completely changed about a year ago and now, uh um like we are waiting when it's became stable uh to use the second uh version of interface types, but currently the first version is work. We added several instructions. For example, you can see here dab and swap tool there there are. There were no such instruction in the specification, but it's impossible to implement such a scheme without them.

A

um Also, we did some work to optimize the scheme uh to like uh uh specialize some. I don't know some instruction to work to allow them work better. uh Then, then it was initially uh written and so on. So it's just work actually.

A

So, regarding those like, we have two tools. The first one is called marine and it's in like uh aims to compile modules. It works as a wasm pack, or you know, or other tools, that, for example, as the wasabi engine is, it uses two types, uh two stages compilation scheme uh marine is also exposed such a scheme, but I think it's not so interesting.

A

More interesting here is ripple. uh This two is intended to test the services by hand. Yes, here you can see that services code ap first loaded inside rebel, and here it contains two functions. If a sphere- oh sorry, two modules epiphyse pure and previous effector- and we call the first one with uh json uh like uh serialized arguments. That is actually an array with three numbers and the uh this is put this code internally through cy interface.

A

uh It's called ip4s uh put, so it's like store inside apis network, this piece of data and by using cache that provided by this function. Here we could get back our three numbers, yes and could see that their number is the same um okay. How fast? Is it uh like these among visas? Quite we all supported radius registers are also accessible in our network.

A

uh You could use it as a like a building block of services, and here you can see some numbers. For example, radius, instantiations instantiation takes about uh half a second and by instantiation hero means, first of all, compiling from webassembly to native and like some work to instantiate import experts, uh interpreters for experts and so on.

A

So function call usual function code. Here you could see that takes 67 uh it's micro seconds. uh The first call takes a bit more about three milliseconds, uh because we have some lazy linking inside and like all next calls will be faster.

A

Okay, let's cover our last topic that dedicated to air and alco in our acquired machine.

A

As I said, we introduced a new language called aqua, and here you can see a simple example uh like a simple sir or simple quote, written on account uh like the full code could be right or could be found by this link at the bottom of the slide, uh and here you can see like an application called fluentpad.

A

That's like cooperative, cooperative editing and this function handles joining user joining. Here you can see that this function receives one argument: user and the returns join. Results that have this structure and the this function is the same. It goes to pure. The time is fine. By up user release, p ready and on the sphere, it calls service, called user released that identified by this service id this one and on this service function join this code, this argument, user and the result of this coin is returned from the whole function.

A

So that's how you could transmit through uh aqua and the visa operator on you could transmit execution flow to appear that identified by this paradigm. It's pretty easy. Of course. I won't cover the whole topic about aqua.

A

You could find like a lot of like documentation and a lot of videos on our channel, um but like aqua, uh why? I mentioned it because aqua actually compiled to our intermediate representation called accommodation, a commercial intermediate representation here as a compiler here, written on scala and adjust compiles, and this year then interprets by our interpreter this interpreter written in rust, and it's then compiled web assembly, and then it could be loaded inside mourinho time.

A

So here we can see like our complex scheme, where code written on narco compiled to air and then it could be like launch it on our interpreter. That's in its turn load into marine, but it's for user. There are a bunch of tools that a lot of sims optimizes to automate all this, and also these.

A

Again, like here here, this interpreter is loaded into marine and marine runtime are allowed only to launch into nodes, not in the browser, but also this interpreter exactly the same works in the browser. This browser run time and also it could be loaded into node.js exactly in the same ring, and this interpreter like works in such a way that it interprets about 10 instructions as they're like inspired by process calculus and picalco's.

A

They just allow you to program how network requests will be uh you'll, be moved across your network uh and from I know, from a formal point of view, this a converting machine is a pure function that takes two data previous one and the so-called current data.

A

This current data is a data that comes from a network and brief data should be stored in the node that manage such interpreter, and the result of this is the new data and a list of peers, whereas this new data should be uh should be centered, and this this function actually is like state transition function. It's just some form of like finnish state machine and it's uh it's a pure function. There are no any imports here and that's why it's really easy to run it on a browser or not js, and some something like this.

A

uh It allows also host host like uh note or a browser works in a synchronous way uh because it there are no any working calls here. It just interprets instructions and provides a uh like some form of comments that know that the cost should be should follow.

A

Okay, here, you can see another picture, uh it's how, like our network could be represented.

A

uh There are two layers: first, where is the execution layer that uh is handled by marine and into like, and we have peers that could be, could uh loaded several services and all service services? uh Marine, uh sorry, aqua virtual machine is also ordered and in stern the spears could be composed by aqua.

A

uh That's also another way how to like present present yourself fluent network. It consists of several notes again piers and we have sever dedicated peers code clients that usually like browsers.

A

uh Each like peer here runs uh a converting machine uh on on note on the flick on node set node browser there uh run it through marine and on the browser they run through. uh Browser runtimes like uh a v1, v8 or spiderman.

A

And from a marine, a point of view interpreter is just a module just sorry, just a service that consists of one module but with special host closure. This hosts closure is something special because it allows interpreter to call different services, because services itself can't uh can call other services they could call. uh They could like communicate only uh through modules that form the service.

A

um Okay, and today, uh like I presented you several things that we uh that, for that we use webassembly influence. uh I covered marine virtual machine. You could find it by this link, always the key and those you could find this by, like rust, sdk link here aqua and, if you're interested in our project and if you want to follow, updates, please join to our twitter subscribe to our twitter uh and if you have for any questions uh like in your, if you, for example, watching this in the record, you have any questions.

A

So please ask me, by this email or by this telegram, nic or through issues or somewhere, so we're very open for any questions for any uh proposals. So please please uh and now find us and ask them. Thank you very much. I would be glad to hear there so hope you enjoyed.