Fluence Labs Start with Fluence. Tutorials, 10 Sep 2021

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Adding A Compute Layer To IPFS With Aqua

Description

A workshop on how to add a compute layer to the storage for your application using composing language Aqua developed by Fluence.

00:00 Intro
00:58 Fluence network overview
03:25 Particle, the foundation of Fluence
05:09 Aqua in a nutshell
07:49 Fluence and IPFS
09:07 Storing and deploying Fluence services with Aqua and IPFS
09:55 Demo

Useful resources:

Open bounties https://gitcoin.co/hackathon/gr11?org=fluencelabs
Fluence documentation https://doc.fluence.dev/docs/
Aqua book https://doc.fluence.dev/aqua-book/
Fluence Devcontainer https://github.com/fluencelabs/devcontainer
Fluence IPFS Adapter https://github.com/fluencelabs/aqua-ipfs
Discord Support https://fluence.chat/
Book A Meeting With The Team https://calendly.com/fluencehack/

Speaker: Bernhard Borges, Fluence Labs
Original video: https://youtu.be/8MSioER3dUA

A

Okay, sweet uh looks like you've got a critical mass, so I'm just gonna kick this off. um So yeah really excited uh that everybody's here. For this workshop, um we're going to be going over fluence and the new aqua and basically adding a compute later to ipfs, but I'll. Let bernhard do all the talking on that, but yeah. Otherwise, of course, same rules apply as usual. So if you have any questions or any thoughts as we're going through the workshop, please drop a note at the end.

A

Do some time for q a and then of course, once this is over, if you ever need to grab anyone on the fluent side, there's a channel in the hackfest community. uh Of course, this is part of the hackfest event that we're running starting kicking off officially tomorrow, um so yeah excited to learn about aqua I'll turn it over to you, mr burnhardt.

B

Hey thanks everybody thanks for having me thanks for being here, and the workshop is really to show you how to combine compute with storage and compute being fluent and storage being ipfs and let's get right into it.

B

So fluence fluence provides an open, permissionless, secure protocol and the associated network and open source tooling, and the protocol is actually geared towards and focuses on, compute leaving the developer, take care of of store in a peer-to-peer environment, so obviously it makes sense to start looking uh to match your store with the compute peer-to-peer and ipfs is obviously a clear candidate for doing so before we dive into that. I want to just give you a few things on how things actually work on the compute side.

B

Basically, compute is done by services. Services are deployed on peers and services are comprised of wasm interface type modules that are executed on a general runtime. We call marine, which has been developed by fluence, is also open source and services, even though the web assembly. So you know there are no sockets in sandbox. They can actually do execute external effects like downloads, uploads, modify, replicate and that that allows you to fetch, urls and, of course, work with ips ipfs files through cids.

B

So on the right, I got a little uh peer-to-peer network where the greenish things are, the clients, uh thin clients, and then we have the service, of course, they're all nodes, one is a relay and on the right, you get the service that it's basically an adapter that can call out to various apis and, of course, ipfs, and this is what we're going to do now. The question is: how do you program that flow? So we have the services we want to interact.

B

We want to compute on a file, for example, we want to interact with ips. How do you do it, and this is where aqua comes in aqua- is a purpose-built programming language environment that basically allows you to program, network routing and which functions or services, to call in what sequence on which peers and it makes it extremely easy and it's open source, and we believe that over time it will truly commoditize peer-to-peer programming, because uh it's extremely ergonomic, especially if you've ever done periodically.

B

You will see this in a minute now, one more foundation piece influence the compute side.

B

We have the notion of a particle and the particle is a conflict-free replication data structure that combines data, the compiled aquascript and some metadata, and what that allows you to do is basically you're looking at a push processing model instead of a pull processing model.

B

So if you look at a client- and you really want to go by analogy- you basically literally can think that a client starts composing that first particle and then literally flings it on to the network which usually is through relay and then on. This relay the aqua virtual machine starts looking at the attached, it's not by code.

B

We call it air aqua, intermediate representation goes through the compiled aquascript at that stage, looks if the service is there on that node, if not it determines which node it needs to go to, and some of this dynamic resolution and service discovery, and so what you end up. Having is that particle moves from node to node, to source, to servers and executes and updates data along the way? It's eventual consistency.

B

So eventually, what comes back to the client is the final result of your computation.

B

That, of course, is in in direct contrast to your average uh client server implementation, where you have request response going back to the client request response going back to the client, so this is a very, very interesting and very very effective peer-to-peer model which allows for very thin clients, browsers iot devices, what very thin low bandwidth requirements and low compute requirements and aqua is at the core of all that so in a nutshell, to actually start looking at aqua.

B

If you look at the little diagrams, we got a client, we got a relay node and we got network. Of course the relay is stylized. The relay is part of the network, but basically what happens is we have a service? That's called a greeting service and the client here flings that particle that represents that greeting service workflow onto the relay node, the relay node then checks whether or not the service is on that particular node. It's not we're going to know a die whatever it is or in the network.

B

We call the service, we execute now the the web assembly on marine and then we return the results back by relay to uh the client peer at the bottom. We get the greeting service. So it's super easy. This is aqua now and basically, what we need to declare is.

B

We need to declare our our interface with the service keyword and we got our namespace greetings service, which eventually will tie a bind to a service id which is unique for every service applied on any one node, and then we have a function which takes two arguments: a string and a boolean and returns a string. So, basically, what the greeting service, which we don't have that's out there on the network that has been deployed, does it basically has a a boolean grid, that's true or false, and nameless string.

B

So if the name is uh is hack, fs and the the grid is true, then the response would be high hack if s and if the greed is false, then the response would be by hack of s. So it's a very, very advanced hello world example.

B

If you look more in the function signature you have the node reference, the peer id and the actual service id, and then that function, which constitutes our our workflow for our application, and if you implement it the right way, it's a decentralized application that you now go on node. So you could be on the relay. You could be right here, but we want to execute on the node specified here on that pier because that's where we know that the service is which is defined by the service id and on that node.

B

We now establish that binding to that service id. We call the namespace with that function. We get the result back on that node and then we return the result back to the relay and eventually to the client. This is what was uh sorry.

B

This is what we uh indicated in the diagram, ah quite a natural, if you, if you've ever done any peer-to-peer programming and you look at the diagram and you look at aqua- you will be more than uh pleasantly surprised of how easy it is to program networks and compose services into applications in the peer imperial environment with aqua.

B

Of course, this was the compute. So how about the store?

B

The way we deployed it is we deployed a fluence node with an ipfs node as a sidecar, and the reason for that is is that we have a bunch of web assembly services just like the greeting service. You just seen that provide subsets of the bindings to ipfscli, among other things, allows you to add and get from ipfs.

B

This is called the fluence ipfs adapter and it involves the fluence node file system and what that allows you to do is it allows you to use aqua to compose ipfs, related services and influence compute services in one seamless interface, one seamless, programming environment, one seamless set one seamless workflow, which is, uh if you ever try it, which I hope you will for the next few weeks.

B

It's super super exciting and the way it actually works is, if you have a file in ipfs, it needs to go from ipfs to the fluence node, where the service is hosted. That makes the call so it needs to go from ipfs to the fluence node file system and then it's available for the webassembly code to compute on it, and this is what aqua allows you to do now from the academic to the pragmatic. We, uh basically, the compute services, as we said, are web assembly modules.

B

They need to be stored somewhere and, of course, you deploy, you write them, you compile them, you have them on your local machine. That's not really a great way of doing it. A much better way of doing this would actually to store your web assembly modules on ipfs.

B

Of course, once you store them ipfs, you also want to deploy them to whatever node you want to go, and you want to use equity to do that and luckily for the rest of the presentation, there is a demo for that. So let me switch to uh the terminal real quick.

B

So all right, so basically, what we're doing through the next to the demo is we have a service service is written in rust and it compiles to uh wasn't 32w and the marine macro takes care of that that that that enforces our web assembly. I t requirements and uh gives compile direction. Directives into the wasm 32 aussie target, and all we want to do is basically with this service. We want to take a file and we want to compute its size. That's it not a big deal.

B

So what we do, then, is we uh all right? Let me see where I'm at.

B

B

And we have a pre-built script to compile it. It should be super fast because I've already done that and it is- and this is what the script looked like. So we take this ros code, we compile it into webassembly. We do this with marine, which is part of the fluent stack, and then we copy the file into the artifacts directory, which is right here, and then we have one more step, and basically this is super nice.

B

What marine allows you to do it allows you to export and export the marine structures the interfaces into aqua ready code. So this is what we're basically doing here. We are exporting those interfaces, those type interfaces and we write them to a process file aqua, which is.

B

Right here here, it is sorry one up here, so you don't have to do a lot of cut and paste and you can reuse everything you've done in uh in your web assembly component. Okay, so now we're all dressed up and nowhere to go right.

B

We have this service and it's sitting on this module, which we want to be a service, and it's sitting on my laptop and what we really want to do is we want to get this onto the fluence network, so we actually can compute something file size that is and for that we want to get it onto uh onto onto ipfs.

B

Now before we do that, we have a web app for that and we're then going to use the ipfs desktop to kick off the storage of the module okay. So we have a web app, it's react. It's coming! It's coming! It's coming all right! Here we go okay. So, in order, as I mentioned before, uh ipfs is high card to a fluence node and in order to get going, we need to get going with a relay and we've we've just listed three relays here and actually let me show you something.

B

We have a network, a test network called stage, and that only has that has six instances of fluenced fluence nodes with ipfs sidecars and the first three are: are the ones we picked for for picking a relay to do the demo. So we connect now this this client now is this. This browser is now your client peer into the peer-to-peer network with fluence okay, so this client is now interacting with the fluence node and, among other things, what we get back from this interaction is we get back?

B

We got back the come on the rpc address of the ipfs sidecar, and what we can do now is. We can go and hook up our desktop.

B

To that sidecar ipfs, node, okay and we discovered five piers, which makes sense right we're on the relay it's associated with the uh uh that node and uh we had a network of six and our peers are five.

B

So now what we can do is we can put a file down there and uh the file we wanna put down. There is obviously our webassembly file, because that's the one we eventually want to upload come on, drag and drop.

B

All right it's here and it should be processed any minute all right here we go so now we have our our webassembly module on ipfs and using the desktop applying for this part, and we got the cid.

B

So what we can do now is we deploy the service. We deploy this service from ipfs onto the fluence network, on that particular node, and all we need to give it is the cid and we have the ipfs rpc address and okay. So now we have this service deployed, and this is the unique service id and now we have a service that calculates file sizes. So, let's just we could use the file we already have, but uh I'll drag and drop something else in there.

B

And okay: that's uh 428 bytes! We add the cid in here this so now that the service we deployed from ipfs is now ready to go and we can calculate the file size and it is 428 yay.

B

Sometimes things just work out all right. So let's have a look at behind scenes, because uh the whole point of the exercise wasn't just that uh this works, but that this works with aqua, that it gives you a unique and uh uh highly ergonomic interface to come to manage not only your compute workflow, but also your ips workflow, and uh let's have a look at uh our process files.

B

So this is all it took, there's a little bit more to it, but this is fundamentally all it took here to uh to deploy the service so just like, before in the greeting service, we have our relay on id. We use the cid, we get the ipfs address, and then we have what we call an error, which is an arrow function or actually a callback.

B

We can call it callback and then we return optionally, the service id, which is what we wanted, and so what we do is we specify our service ids return value as a stream and service id literally is just an alias for a string and then on this relay. We want to get the results for the ipf's get, so this is from the previously mentioned, ipfs adapter code that comes built in essentially ready for use, and it gets us the result.

B

Basically, and the result is, can we download the wasm file from the ipfs to the node and if that's successful, we calculate a few things, including uh wasn't file hashes, and we create a blueprint because services influence our logical constructs depending on one or more modules, and one more modules may be part of one or more services. So you basically build a json config file simply said it's called a blueprint and on that we can create a service which then returns the service id and that's it.

B

This is all this is all there is to it to deploy a service from ipfs onto fluence compute peer-to-peer network, which is uh I mean this is like what 10 lines is awesome again. Obviously I'm excited, but it's really really cool. And now, if you want to calculate the file size, it's uh it looks longer, but the signature is broken up on multiple lines. Basically, the same thing, however, we have two uh error functions or callbacks. One is the log size because the application we just saw was in uh in react.

B

So we can basically use a co routine to push this down here. The co co routine to uh initiate the callback while the rest of the aqua program uh runs, and we.

A

B

One uh callback for our error and again all we need to do is: uh can we get the file if we can get the file?

B

We now call that service we deployed and uh basically using the path onto the fluence node file system and we calculate the size and if this calculation's successful, we log it back straight to the uh uh react app in this case, and uh we write our uh our our file size also somewhere else, and we can put it back on to ipfs if we want to and then there is a remove service uh capability, which I probably should do because, okay, because I got to clean up after yourself right so are there any questions on on this end from anybody?

B

I don't see the chat. I don't.

B

Okay, well, if you have any questions, uh please now or or if it occurs to you later, uh please don't hesitate to uh bring them forward, so that was that was in a nutshell. So, just just going back. If you coming from the fluent side, you can now deploy your modules from ipfs to any node.

B

You want it's a it's a real improvement on the workflow and if you're coming from the ipfs side, you have a really easy way and to to build the applications that use ipfs to store and uh bring fluence compute to the table and uh your your gateway drug is aqua, which is amazingly awesome in order to do that.

B

So for the hackathon we got three prizes uh each two thousand dollars, and the first prize is for the best use of aqua to use fluent compute for ipld.

B

So anything you want to do in terms of indexing and searching of uh of hosted blogs websites, uh blockchain ipld data, which would be really really close to my heart, not that anybody cares, but I think it's a super, exciting area and, of course, uh content, addressable archives, readers and writers can also be should also be able to uh be built on uh fluence compute.

B

The next two thousand dollars are available for uh the best functional solution solution, providing a performant ipns solution with aqua, so ipns isn't the most uh stellar performant performer, and we feel that the alternatives right now are somewhat limited. I haven't seen personally, I haven't seen anything on the blockchain.

B

That's performant and general dns is, in my opinion, a rather centralized solution which really gets in the way of your decentralized web3 application space, and I believe we could use aqua dht as a name registry or a name registry cache to potentially speed up ipns, and this is basically a price for somebody trying to do that now. Aqua dht is a distributed. Hash table that can also is fully programmable by with with aqua. So your workflow stays exactly the same.

B

In terms of tooling and even interaction of the compute, the ipfs and then the dht, so this should be this should be doable. And uh if anybody uh wants to tackle this I'd be super excited and the last uh price not last, but the third price two thousand dollars is best integration, affluence with ceramic or textile data layers or identities. Obviously, both projects are also building.

B

Ipvs are super exciting and provide the missing store and then some element to the fluence protocols compute and uh any work in there would be super super uh exciting for all of the both fluents and that's a miracle textile, obviously and uh there's a repo, and uh it has pretty much all that written up in there and uh if you, if you need to follow up on that- and we have a bunch of resources available to you to hack through the uh next few weeks, uh documentation, uh discord, pulled out discord and, of course, the hack fs fluence discord channel, and there are some youtube uh videos.

B

If you want to learn more and if you get stuck, please feel free to set a appointment with uh the core team and we try to help you out. Obviously, we're hiring who isn't but we are, and I think you should fear in rust or compiler work. uh You should definitely look us up and I use aqua to basically say thank you and good luck. Now I see the chat and the question. Is this like a remote procedure? Call uh well, is it like one? Well, it feature functionality.

B

Yes, you make a call, but uh it's it's not it's really not. The difference is the particle, and does that answer the question or do you want me to get into it.

B

Go into it, okay, rabbit hole, all right, yes, rabbit hole, so um basic rpc is based on uh on client server and client. Server is a request response model which, which is a pull model which we don't use. So in that case. In that sense, it's not like rpc. However, of course you do make remote calls. However, the remote calls do not go between the client peer and the node. It goes from. It operates as a workflow along the way, so.

B

The p, the the particle that travels from node to node is your medium, that uh that takes the uh immutability or immutability data usage and uh and then updates the data in that spot. So it basically travels where in rpc you have this strict uh client to server back forth back for it back forth, and in that sense it's not the same at all. Actually, however, the outcome is somewhat similar.

B

Now we can use rpc calls from compute services to make calls to to apis and solutions that are not part of that are not natively deployed on as compute services on the peer-to-peer network. So in that sense you can use them, but is what fluence is doing like a remote procedure? Call no! It's not.

B

Did that answer the question cool all right, any other questions.

B

All right, I I have a surprise, I hope, that's okay, I think we we didn't, spend all our allocation. So there's this uh say thank you and good luck service here, right which I use. So thank you for having me thank you for being here and good luck and uh we'll pay 250 to the first person who provides the peer id and output of the say. Thank you and good luck service in our haptic fs, sponsor fluence labs channels before tomorrow, 2400 edt so midnight, and basically, what that requires you to do.

B

Is you have to read it up a little bit and figure out how service deployment works and uh how to uh how to find them, but the first person who posts in the channel the peer id and service id and the output of that good luck. Service. I mentioned before 250 bucks before midnight tomorrow,.

B

B

Okay, well, I could have gone a little slower.

A

I was almost perfect timing so, and I mean that's a that's a hell of a good surprise at the last minute, so I heard it here. First folks, um everybody else is gonna, be catching up on youtube. Afterwards uh is not at the same advantage of you. So it's a good opportunity to look through everything and win 250 bucks um awesome! Well, uh big! Thank you for the workshop big.

A

Thank you, of course, to fluence for being one of hackfest's sponsors for this event really excited to see what people build um using compute later on top of ipfs, um and of course, uh if you joined late or are watching this or want to watch this later, it's going to be up on our youtube channel right away, so east global on youtube and you'll be able to catch it. There awesome thanks again and hope everybody has a lovely rest of their day.