Fluence Labs FOSDEM, 6 Feb 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: How to make a Service and distribute it with the Fluence network

Description

This section aims to show how a service can be created from scratch and then deployed to the Fluence network. We will start with a discussion of FCE – special runtimes that designed to run multi-module Wasm applications with help of interface-types. Then we will create several simple services and discuss how to compile, run locally, and debug them with our tooling. Finally, we will deploy these services to Fluence nodes to use them next by the front-end application.

A

Hi guys, my name is mike, and this section will discuss fluent services and fluent compute engine in particular. The agenda of this section is as follows. First of all, we will start with a little introduction to our assembly just couple slides.

A

Then we will walk through some fluent services, internals and fc runtime and finally, I will show you how to develop and debug a simple whole world service for the fluence network from scratch.

A

Let's start with the answering to the simple question: what is web assembly? Actually? I have taken this marvelous slide from the brandsmith talk on speedy clone 2019. You can find the link at the bottom of the slide, and here we can see the directed graph and moving from left to right. We can construct our own definition. What is web assembly?

A

I ended with this one. So web assembly is the open standard of stack based virtual machine that has different parts such as stack tables, global variables or new memories, and so on, and there were some specifications introduced concept of modules with imports and experts.

A

Webassembly modules could be considered as a black boxes, whose experts contains entry points that define how exactly it could be code from some external world, for example, from a browser or operation system and imports uh define which functions from external world. A module could call and also important that web assembly specification defines only primitive types.

A

By primitive, I mean that there are only integers and forwarding point numbers. For example, there are no any arrays strings records, and so on just these types and of course this make it difficult to make a handy api and api of modules, but uh web assembly provide us a way to abstract our this constraints. With help of interface types on the side, you could see a complex scheme where, on the middle of this blue box, represent a row of assembly, module with row imports and exports so by raw import and export.

A

I mean that this function could operate with only primitive types. It's only floating point in integers, and this adapted module could abrupt this raw module and provide some adapted import and adapted export that could awarding and lifting times between complex and road types.

A

So half of work by uh type conversion could be done by module itself and the another gem of web assembly. World is vising white stands for web assembly system interface, so the standard of posix likes imports that the module could go from runtime and on the slide you can see the compilation of two whole world program written on rust and sipos pass, and uh actually this gives um webassembly binary with the same imports thanks to yj.

A

So here we can see five uh imports, yep and webassembly provides us a way with help of yz and help of model uh module series, imports and experts to make heterogeneous one time that could run modules written on different languages.

A

So, but actually at the moment we provide sdk only for us that so rust is our primary language for backhand service developing, and uh I think that it's right type time to dig into the details of this sdk and the service internals.

A

So fluent service is a group of modules linked together by sharing nothing linking scheme.

A

So this linking scheme is based on linking only imports and exports of modules.

A

This means that all modules state keeping private, for example, web assembly module, doesn't expose memory, tables, globals and so on and all all of this is possible because we use interface types and each module contains adapter functions, that code lowering and lifting types and also with help of interface types. Our module keeps states and private and linked together only by imports and experts.

A

And actually, we have three types of modules: the first one called facade, so uh this module uh actually exposed the api of entire service so and there in this each service there could be only one such module.

A

uh Next one is pure modules. Pure modules could contain all business logic and they can't access file system or call external binaries and last one called effectors, and the purpose is to do some uh stuff related to file system operation, other operation system, services or cooling, external binaries and also each service should have a configuration file on the slide. You can see the example of configuration file for url downloader service.

A

You can find also the full code as a service by the link in the bottom. This service consists of three modules: the first one called local storage.

A

As you can see, it is a factor because it requires access to file systems revising.

A

So in particular, it requires access to directory code sites.

A

Here, pre-open files is a list of directors of files that a module could access during into its work and mapped gears here could be used for setting aliases for each pre-open files. The next one is auth effector, because it requires access to mounted binary code curl.

A

So here this mounted binary option say that a module could call a binary located uh by this pass. Here is a user bin curl on file system and the last one is always a facet model that exposed api of an entire service.

A

And on the side, you can see principle scheme of fluence compute engine, so fuse can compute engine allows to load several services on the same moment and each service could could consist of several modules as we discussed so the module.

A

This modules, linked together by important experts and effectors modules of each service, could access some operation system services with help of vision and uh also, um as you heard on previous slides uh acumarine, is our language based on picalco's for complex multi multi-agent scenarios across the fluency network.

A

But what is the connection between fc and aquamarine, and so actually, a co-marine interpreter is compiled to webassembly and could be loaded to fc runtime as a module.

A

But this module have a special property that allows the interpreter to call other services that are loaded to fc at the moment by special instruction called call. So you can find more information about equipment next on next sections and also in our dock and finally, let's consider how services for fluence could be built and debugged.

A

So all complexity of generation, web assembly, binary and generation and compiling interface types is encapsulated in our sdk and building tools. On the slide, you can see how how word for fluence could look like. So this example contains two functions main and greeting and the first one is needed for uh utilization of voicing.

A

So it's called only once on the module startup. The second one greeting is wrapped by our special procedure. Macro fc, and this makes this function exported from a module and allows other modules or are common scripts code.

A

And so, as you see, you can find the complete example of the service by the link on the bottom of the slide.

A

So this fc macro uh could be applied to function, extern book and structures, so um applying functions is, as you saw in the previous slide, make this function. Expert from module applying to extern block tells our business tools that a module requires imports from such module. For example, in this case it's curve adapter and the last one. The last one option provides a way to use structure types in signature of import, imported and exported function, so in first and second variants.

A

So and finally, uh in this section we discussed uh fc and our sdk a little bit and you can find a repository for them by this links and also we have for documentation about module developing that could be found by this thing and so on the next we will take some hand on sections.

A

Developing is simple for world fluent service: let's see how simple whole world module could be built for fluence and then could be turned to a service, but first of all we need to install rust and also implementary tool and then install our build 2 and 2 for dividing.

A

So actually you can find all commands on the on our docks. The link was on the previous on the last slide. So actually also. I have all these two installed on my stub on your. It could take more time, of course, and also please note that it's important to install nike rust, because our repo requires only only nightly two chain of rust.

A

So then, let's install wasp32yz target, it could be done by this command. So for me it's already installed, then we need to install our building tool. It's called.

A

Fcli, so this command adds a binary code fc to your system, and then we also need to install repo that could be used for debugging, and it is so we installed rust, uh wasn't ready to buy the target for us and our building can divide those, and actually here we uh we could create a new product. So if the rust could be done by a cargo new, let's call it greeting and make it binary so hero binary means that uh final uh webassembly binary. You contain some uh intuition code for writing.

A

So it's important to make it binary so, okay and then let's um edit, a little bit cargo tumble to make it uh compile to webassembly so and to specify our our fluency decay. So actually, uh let's what change it in some, for example, something like this one and also, let's add our sdks dependency.

A

Actually, it's current version is uh 2.17, but uh it doesn't matter for our example. We could use uh the last one. So, uh let's see if we can exit and then let's open our greeting research c and main the main project main insert zip file, and so here we need to include our macro.

A

So it could be done by this and then we need to uh leave main empty so uh and then we could run our main function that wrapped with fc. So let's follow the example that was on slides, so it causes function like greeting that takes a string and return a new string and internally. It could, for example, a plant to string something like this yep.

A

So and it is- and actually it is all code for simple whole world- there are only two functions, main frontalizing, quasi and greeting as a main export functions. So it is.

A

And then we need to compile this uh compiling is done by lfc, so we could compile it by fc build it. Just drop your own cargo that could embed interface types so after compiling it takes some time different on a cache of compiled cargo artifacts so uh also fc. Allow you to to, for example, um find out what interface type is generated.

A

For example, let's see uh for our case because we haven't specified release, so it's should contain it's located in debug folder. So and here we can see generated interface types for our simple binary.

A

So then, let's convert it to a service to convert to this binary, this services module to a service. We need to do only one thing.

A

So actually we need to write a config yep, let's create a new config tumble and so just write something something like this. So here we specify that.

A

Binaries for this service is located on this folder. It's called artifact and the service is consistently on one module called greeting and it is so that's all. Let's say with so. We successfully created a new service greeting and then let's see how it could be called. uh Actually there are several ways to code. One way is to uphold the node and then call it with aquamarine or something like that.

A

um But for debugging purposes we have a special tool called ripple, uh but first of all we need to create, activate uh and then copy our greeting binaries to it. So something like that, then we uh could run fc repo and specify config our configural to it.

A

Actually, repo supports several commands for loading and loading modules, calling obtaining their interfaces, environment, variables, file system, state and so on, but today for us important to call module and see see the result of clank, so calling module should be done with specifying module name in our cases. Greeting export function name in our case, it's also called greeting and arguments and arguments should be encoded in json.

A

So it could look like this.

A

And, let's see something like john and here we see that the result is a high john yep. It's the same string that we expected. So thank you very much uh on this on this. My section is done on next on the next section you see how more complex services could be built, then how they could be deployed to fluency network and called by aquamarine.

A

B

Hello, my name is dmitry. I'm a software developer influence. Let's continue our journey to the fluent system. I hope that you already saw previous videos of our speech series today. I'll show you some tips and tricks on how to build services in the fluence network. I'll, take services that we used in fluid spot project, for example, to start the creation of our service. We simply initialize our last project and influence library to dependencies. As you see on the screen. First of all, we want to initialize our fresh services in service lifecycle.

B

It could be logger initialization, as you see, on the screen or creating a database or initialization some preloaded data, and we use the main method for this, so how to build functions that we could call from airscript. As michael already said, we need to mark our functions with fce tag. Fc environment doesn't support, result types as a result, so we need to wrap possible errors in structure.

B

For this, we create custom structures, as you see on the screen, with the return code to understand if it is an error or successful, result for every function and convert results to these structures for the convenience of application development, and this is called for conversion.

B

So, in our methods we just use intel and convert our simple functions. That return result to our structures and to use the structures from airscript. We mark them with the fce tag as well. This project is quite simple. We just add some data to our storage and return it if needed. For this, we have a join function, to add user to database, to some sort of storage, leaf user to delete user from our storage and some helper functions that we later discuss.

B

In this example, we use in-memory storage that will be cleaned after service will restart, not restart or some other reason. As you see, it's only a hash map to store our data, but if needed, we can connect and use database modules. We have scolite and registers was modulus for now, but it is possible to create approx models for every database and other stuff that can use a file system and store data and long-term storage to connect other modules. We need to describe a p of a model to connect it to your code.

B

For this we need to write this code. We need to tag our code with fse tag, write our model name, as you see on the screen and add functions from this model. If you make a mistake in your code, you'll see it when you compile your was modulus in a frapple, as michael said before, or after deploying a node will return an error that it cannot create such link of models. After this, you could simply use these functions in your code. As you see on the screen.

B

To use cli, or maybe some operating system commands you need to import host functions like this.

B

So main difference is that name of model is host and here is a cli tool name and the comments are passed as string. If you want to use file system, you can use it as you would normally do it in rust. But all this like file system or other modules or cost functions should be specified in the config.

B

That michael mentioned before another important thing that we need to build our services properly is authorization and authentication first of all, to check if a caller is an owner of a service or it exists in some sort of list. We need to get the caller's id this and other useful info could be found in call parameters, call parameters we could get like this and what what is called parameters, it's all necessary information about caller and about arguments that we pass through our function.

B

It is like initial id service id service creator, peer id host id particle id and tetraplets, so here in user list, we only need an init peer id to check, but what, if we need to verify into that, the user is authorized in other services.

B

So, let's have a look in a history service that store our messages, and these messages could add only that users that is authorized in user list service in user list. We have a function that called is authenticated and it return result of identification.

B

So if a user is in list- or he is an owner of this service- we return true. Otherwise we return false. So how can we check it in history service? This is our function to add the messages and we have argument called alf. That is a boolean. How can we check from this argument that it is a valid info from user service? We check it in is authenticated function, so let's have a look.

B

First of all, what we need is uh get our call parameters, as we seen before, and then we need to check our tetra plates tether plate is a security data plate that store necessary data to have a proof of calls. So here we have pure public k, service, id function, name and json path. How can we use it so tetraplex, a vector of vector first vector, is as our list of arguments in function and second vector is for the case. If our argument is a vector and we need to check each element of this vector.

B

So simply we have an index of our argument and we get tetraplate for our argument and all that we need that the triplet for our argument matched the specified parameters. As you see, we need to check all fields of our tetraplate. We specify specified parameter and that auth is true. After that, we can be sure that the call is authorized after we write our code. We can build our services with f c e and upload them via js javascript sdk, to simplify this process and use simple commas from the terminal.

B

We wrote proto-distributor that use jssdk and could upload modules create blueprints and services. Let's have a look on this thing. This command line tool could be found in our github repository I'll. Show you how to do this in a few steps.

B

First of all, we can look at the help and use commands to upload our module with this command. We could upload, for example, user list bus file that we will name as user list.

B

So model uploaded successfully and after this we need to create blueprint in which we list all our modules that we want to use in our service.

B

So it looks like this command called add blueprint here we need to list our dependencies here. We have only one dependency, but if we want to use maybe sqlite, we need to add this before model that used this model. After that, we need to specify an id of our blueprint. It could be some custom id, maybe uuid or something like that, and the name of blueplanet so blueprint usually started successfully and after that we need to create service. So here we use blueprint id to do a calls to a service as an owner.

B

So let's just create our service, and here is our service id to use in our air scripts or in jssdk. That's all with these services. We could build big network, big applications that will use services from other nodes services from other providers that can use data storage, local storage file system, operating system comments and other stuff. Thank you for watching, and let's move on to the next video.