Filecoin Retrieval Markets Workshop: Intro to Filecoin Saturn, 3 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Saturn L1 Node Architecture Deep Dive - Diego Rodriguez Baquero

Description

No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).

A

Oh no, there we go so level, one notes. What are level one notes: uh they are the outermost notes in saturn uh they have a parisian hops. uh Think of north america, south america, europe, uh their access, viago dns, as asgard mentioned, they serve car files via http 2, with tls using nginx.

A

Why change what's already working right and they fetch car files from level two or greater notes uh on cash? Miss in case they don't have it like readily available and in exchange for those retrievals, you earn file coin right, amazing. So, in this diagram we see the retrieval clients fetching through http car files from the level one nodes which, in the v0 implementation, if they don't have it, they just go to the ipfs gateway.

A

How does the sign up happens? We'll go through that on the registration with the orchestrator which uses route 53 for the geodns setup? It also contacts a lock service um to send how many retrievals have that have there been made for which cids and from which clients and there's also the last part, which is the fraud detections. We don't want to pay you if you're faking, those retrievals.

A

So what goes inside a level one note: we have a big docker image that runs nginx and a node.js process. This node es process uh pretty much handles the cache miss which contacts level twos. It has the login gesture which consumes logs from nginx in a specific format, and it just throws them to the log service.

A

We also have the registration part, which pretty much pretty much allows to contact the orchestrator. Hey, I want to be part of the network set me up, give me a tls certificate and join my ip to the dns records in my region.

A

There's a registration check, it's just a tiny endpoint, so that whenever there's a sign up, we want to make sure that you do own, that gateway that level onenote and that it matches and there's a health check every minute. If you, for whatever reason, go offline, will remove you from the network and hopefully not affect any users, and you there's reputation on that, the greater app time you have, the more traffic you'll get and outside that docker image.

A

We have the auto data, which will keep your docker image up to id up to update and we'll check the registry for newer versions in an interval.

A

So, what's the setup like we have hardware requirements for the level one nodes, uh you need a linux server with a public ip. Ideally, you have 100 gigabits per second, but the minimum is going to be 100 megabytes.

A

You have 2 gigabytes of ram, that's that covers most of desktop servers, bps and about 8 gigabytes or more. I recommend it the minimum. This is going to be 128 gigabytes of ssd, but ideally you have a great setup of raid zero nbme ssds that are uh two terabytes or more in the footnotes. You can see that the more you can surf so the greater upload speed you you have the more falcon earnings you get and the bigger disk it means bigger cache. That also means greater falcon earnings.

A

Some general requirements before you can even set up a node. You have to have a falcon wallet, address and 10 minutes, and hopefully that will be like one minute in the future.

A

So how does the registration happen when you set up this docker image? You bring it to your bps, your dedicated server, your raspberry pi at home, and you say you wanna become part of the network, so the initial steps, it's it pings, the orchestrator.

A

Who will go back to you in the public ip that you're coming from and uh verify that you have you you own that node id and will give you uh a token. You have to persist the tls certificate that the orchestrator gives you on demand.

A

um You will join the geolocated dns, so if you're coming from florida, you are going to be serving requests in florida. uh If you're. If there's no notes in georgia, for example, you are also going to be a point of presence for that traffic.

A

Lastly, you will receive a token, so you can send the locks back to the log service and you register every five minutes. They have that token refreshed.

A

In order to have a smooth transitions or blips in the network, you should be doing a graceful registration as to not kill active requests. So there is a graceful registration process which you let the orchestrator know that you are going offline for maintenance or just updating the docker image itself. You leave the dns for that region.

A

You gracefully finish the active retrievals and then you shut down. If you don't, then you have a hit in your reputation.

A

So what are their responsibilities with the network of the level one nodes? The first is the registration where you get the tls certificate and the dns setup, so you can start getting requests.

A

There's a continued reregistration, plus the health checks to verify that you're still in line you're still serving content and answering those retrievals which, as answer mentioned, they are through cids. You want to maximize cash content. That means the greater falcon earnings you have to fetch cads in case you don't have the content readily available from the level two or greater notes, and you have to continuously verify all the blocks that are coming from those providers, because also you don't trust them.

A

You have to report the retrievals to the log service.

A

That's done on an interval of five minutes, and you also report the load and the bandwidth that you have so you can either receive more requests or we send you less requests.

A

uh Updating uh the docker image should be kept update if it goes out of date for more than 24 hours. You get removed from the network and the tls certificate is renewed every 85 days, just under the threshold of expiry.

A

And earnings, which is the most uh interest thing uh with the lock ingestion uh the engine logs, are parsed aggregated in batches and sent to the log service, using the token that the orchestrator has provided and filecoin will be delivered to your wallet address in a daily.

A

Basis, uh we hope to be launching v0 soon in the test network that we have uh on testing, not production, testing but uh and mainnet, uh hopefully coming in the few months. Thank you.

A

B

Questions nice thanks, diego um yeah. I've got one to start off with you mentioned that the orchestrator is giving a tls certificate to each of the l1 nodes. That's correct! um Have you looked at any ideas about how the certificate management could be more decentralized.

A

uh We haven't yet uh right now, as everyone is joining pretty much the same, um the same uh dns record, which is uh cdn.saturn.network, um everyone has to have a verified, tls certificate and sadly, that's not decentralized. Everyone could ping, for example, let's encrypt or such options, um but then it would be much more slower and the dns verification of that part uh would not be able to be completely decentralized. Yet we just haven't researched on that.

B

Nice thanks yeah any questions.

C

Yeah on one of the first slides, where you have, I think it's the first one yeah you're talking about car files there, that the car file um gets requested um from the client right and what happens with these car files. Once they're like um sent back.

A

In the retrieval clients, yeah yeah, so retrieval clients start consuming, divides and continuously verify all the blocks, which is one of the advantages of cids and that's done on the service worker.

C

Okay, but normally um the file that the client would request would be like a jpeg or whatever or video, which would be something completely else.

A

Yeah, so the service worker grabs that car file and pretty much unpacks it uh while continuously verifying the blocks. It will then render the jpeg or png or the video that you have.

B

All right another question: at the back.

D

Yeah thanks talk is really useful. um One question uh I think the the goal of using nginx there as a cache layer is a really good idea. um I was wondering if you could expand more as like the trade-offs there and so on. I think it's a very good design decision um and I think a lot of people should hear about it.

A

um Decisions that were made right.

D

Yeah, like just talk about like lion gen x, um that's a really good idea, but just kind of like help unpack for for other folks watching.

A

Of course, so nginx was chosen, uh especially because it's a tested solution, that's open source uh and I've worked with it a lot uh through the years uh that was my go-to. I guess we could go with varnish or other solutions, but uh engine x is uh proven and tested um battle tested across a lot of internet services. um Anything else you would like to for me to dive yeah.

A

B

Any other questions.

A

Well feel free to reach out to me on any social network or diego at protocol.ai, and I will answer any questions that you might have.