IPFS Practical IPFS Guides, 23 Sep 2022

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Deploying a Kubo IPFS node and gateway to Fly.io

Description

In this video, you will learn how to deploy a Kubo IPFS node (which doubles as a gateway) to Fly.io and background about the utility of running a node.

Source code repo: https://github.com/2color/ipfs-deploy-flyio
Kubo repo: https://github.com/ipfs/kubo/
Fly.io platform: https://fly.io/

00:00 Opening
00:05 Preface: Why would you deploy a Kubo IPFS node
02:26: Intro: What this video covers
02:48: Prerequisites
03:16: Creating and configuring the Fly app
04:40: Creating a volume for Kubo
09:06 Testing the deployed gateway node
09:50 Using the Kubo RPC API
11:24 Adding a file from your local machine with the Kubo RPC API
11:54 Summary

To learn more about how you can use the Kubo RPC API programmatically, check out the following video: https://www.youtube.com/watch?v=Ga9gfoZSm0Y

A

A

So why would you want to deploy a Kubo ipfs node to the cloud? Well, the ipfs network is a large Network that is open and is an opt-in Network that anyone can participate in. However, there's a common misconception that, if you add something to ipfs it'll, just be magically hosted for you, this is not actually the case. While there are many pinning services that will give you some free hosting for your files, they actually have to run infrastructure like ipfs nodes, so that they're made available.

A

Now, of course, you can run an ipfs node locally on your machine. However, your machine is likely not to be consist continuously online, making those files and blocks and SIDS available continuously. So by running a Kubo node that is continuously running in the cloud you can actually ensure the reliability and the availability of your SIDS.

A

There are two more reasons why you might want to deploy an ipfs Kubo node to fly.io. The first is that you get an ipfs Gateway, and that means that you can fetch all of the SIDS that are pinned to that node immediately via HTTP and any arbitrary set. That is available through the network via that Gateway. Finally, the last reason is that when you deploy to fly.io, you automatically get a TLS certificate and one of the benefits of having a TLS certificate is that you can connect to it from the browser using websockets these days.

A

Most apps and web apps are running on https, which means that, due to the secure context of the browser, you can only establish websocket connections to servers that have a TLS certificate. Now most of the Kubo nodes or the ibfs nodes in the network. Don't actually have a TLS certificate. So by having a node that has a TLS certificate, you can actually connect connect to it using Js ipfs.

A

One final note to make here is that, if you're, using a pinning service like web3 storage, NFD storage, file base or pinata you're, essentially delegating the responsibility of running and operating an ipfs node to a service provider.

A

So you have that choice to make you can run a node or you can use a pinning service and essentially they're running the ipfs node for you in this video I'm, going to show you how to deploy a Kubo node to fly.io fly.io is a platform that makes it really easy to deploy containers to the cloud, and so for this purpose, we're going to use the official Kubo Docker image, we're going to customize it a little bit in order to be able to apply custom configuration.

A

There are three things you need in order to do this, you need to have the GitHub repository cloned. You can find the repository in the video description. You need to have a fly account and you need to be also logged in with the fly CTL. That is the CLI command for fly for interacting with the fly platform, and you can find the instructions for that in the fly.io documentation. So, let's get right in the First Command we're going to run. Is this fly launch with the copyconfig copy?

A

Config is actually going to look at this fly.tomo. That is already in the repository and we'll use it in order to configure the application from a high level. Most of this configuration is about setting a couple of environment variables and also opening up a bunch of ports. uh We need the 4001 both on TCP and UDP.

A

um These are the ports that are going to be used by Kubo for peer-to-peer communication and there's also port 8080, which is the Gateway Port um Port 5001 is actually not open, and the reason for that is that we don't want to expose the Kubo RPC API publicly. We only want that to be available in a protected Manner and we're going to I'm going to show you how to do that using proxying whenever you want to interact with the API, so I'm going to go ahead and write this launch this command.

A

It's asking me for uh an app name, I'm, just going to call this my ipfs node, and it's asking me for the region I'm going to deploy that in Frankfurt, which is close to me. We do not want to send to set up a postgres database and we also don't want to deploy it just yet. The reason is that we need to create a volume- and you might see this here so inside the mount um in the fly Tomo. We have this ipfs data.

A

This is a volume that we need to create and the command for this is also inside the readme and so I'm going to create this and, as you can see, ipfs underscore data is the same name that I'm using inside the mount section of the fly tunnel so I'll create that, and that obviously, should be in the same region as the app which is Frankfurt.

A

In my case, and now that we've created uh the volume we can actually go ahead and deploy, but before that there's another thing that we need to do, and that is, we need to actually update the uh IP version. 4 that is uh announced by the node and uh the reason that we need to do this sort of manually so to say, is because the container itself Kubo isn't actually doesn't have inside the container an interface with that IP.

A

It's actually sort of happening through a load balancer, and so this needs to be configured manually again. The instructions are inside the readme, but basically we need to look at the IPS and list them we're going to see two okay, oh so because the app hasn't been deployed. Yet we don't have the IPS, so indeed we do need to deploy. The app one thing to note here is that Kubo cannot be scaled horizontally in the sense that you can't just launch more instances of it.

A

So just be aware of that, it's not like traditional sort of stateless applications that you can just sort of scale them by launching more instances of them. This is going to do two things. This is going to actually build the docker image and we have the docker file inside the Repository.

A

So, as you can see, this is just building on top of the official Docker image, and it's just adding this ipfs config shell ipfs config shell. This is the extra sort of customization layer where you can add any necessary configuration and, as I mentioned this ipv4 that we need to add into the configuration. This is going to be added in this last line here into the addresses, dot, append, announce and this will actually publish when sort of the Kubo node publishes its IPS to the DHT.

A

It will also uh publish this ipv4 that we're going to add so now that uh our uh container has been deployed. I can open up another terminal and I'm going to go into the same folder. It was the ipfs deploy fly.

A

I o and I'm going to try to run, fly IPS list and we already have it so here we have the ipv4 and, as you can see, it was created 27 seconds ago, so I'm going to go ahead and add that here uncomment this line and because this ipfs config file is actually added into the docker image, we're going to need to deploy it again. When we do a deploy, it will rebuild the image. So we can run go back to the original Tab and Run Fly, deploy.

A

One thing that's also worth doing is probably scaling the container.

A

Scaling the memory I think by default, it's uh getting 256 megabytes um and so I'm going to scale that to 512 fly apps list. By the way we can see here that we have this ipfest. If we go just to flight logs, we can actually see the logs running so there we have it.

A

um So next we want to know what's the actual host name for this and by default flywheel, actually uh link a hostname that it has with a certificate. So let's take a look at that we can use the dashboard and, if I open up the dashboard, then we can see here. Indeed, we have the two IPS one is an IPv6 and an ipv4, and we also have the hostname.

A

Now, if I go to this hostname I can see that it's not found, but if I go slash, ipfs and I enter a uh Sid I will be able to fetch it so I'm just going to find a sid that I always have pinned, and now what I'm trying to do is I'm trying to use the Gateway that we just deployed as part of Kubo and I was able to fetch this image. So there we have it, um that's the Gateway working now.

A

Of course, you can assign additional host names if you want, but you're going to have to sort of mess there with the certificates and so on, um and the next thing that we want to look at is how to use the API. So we have this Kubo RPC API, and this is usually the kind of thing that is protected, that you don't expose to the public and so there's sort of two ways to to get into it.

A

The one is to just use the SSH console and that's essentially sshing into the container, and then I can run ipfs ID and, as you can see, I have you know the the node ID, and so, if I go into swarm, peers, I can see all the different pairs and we can see that it has 378 peers that are connected to the second approach to interacting with the Kubo RPC API is using a CLI that is running locally on your machine and then just pointing it to the RPC API, and for that we're going to use the fly proxy command so flight proxy command.

A

Basically allows you to map a local port to a remote port and once that's been established, then I can use, for example, the ipfs ID command and using the dash dash API I'm, going to pass a multi-address and that will essentially connect to the remote host and we can see that because it's running in Docker and I don't have Docker running locally and also the node ID seems to be different from my local ones.

A

You can also add files using the proxy so, for example, here I'm going to add a file and it's uploading it over http and uh there we have it. We've got the seed returned. One final thing that you can do with the proxy is: you can also open up the web UI for ipfs. So here you can see the status of the node. You can also look at the number of pairs and you can also add files directly using MFS. This is mutable file system, so you can add that using the UI.

A

To summarize what we just did we cloned the repository? We then created a fly app using the configuration inside the repository inside the fly dot Tomo, we created a volume and then we deploy the app and then once we deployed Kubo. We also went ahead and updated inside the ipfs-config shell script. We added some custom configuration now at this point. It's worth noting that you can add any sort of custom configuration. For example, I have hole punching disabled, however I could enable it and then just run, fly deploy again and that will restart the container.

A

We will rebuild the image with the new net script, which will reconfigure um ipfs, and so you have the full flexibility of configuring it as you need, if you have any questions, feel free to drop them in the comments. um So good luck and enjoy deploying.