IPFS IPFS Implementations - May 2022, 23 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Mixing Peer to Peer Protocols on the Web - Mauve // IPFS Implementations Workshop

Description

In this May 2022 IPFS implementations workshop, we heard a number of first-hand accounts from builders & thinkers working on and with IPFS, taking stock of the current implementation story, and imagining what’s possible through shared effort on a protocol with such broad applicability.

A

Thanks for everyone for coming out- or I guess watching this recording after the fact, so I'm going to be talking about mixing different peer-to-peer protocols with the web with agricore, I move my pronouns. Are they them everywhere on the internet? I'm at ranger move and I do consulting for decentralized software with mo software inc, which is pretty much just me um and I'm kind of obsessed with peer-to-peer protocols.

A

I really like them, because they enable me to stop hoarding user data as a developer. So my applications can like stop worrying about managing back-end infrastructure and privacy and all of that- and instead all interactions between users are done directly rather than being mediated through a third party. One of the cool parts of this is there is no longer a central point of failure to go offline either, and so, if you're within network range of somebody, you can still use all your peer-to-peer apps without a hitch.

A

So since this is an ipfs meetup, we're mostly going to be focusing on ipfs in agricore, but it supports a whole bunch of different protocols which I'll kind of like slightly touch on later. Ipfs is great because it's an example of a really versatile protocol which can store a bunch of different data types, and this is thanks to having a layered model with content addressing and mutability.

A

So one of the cool things is since content is addressed by individual chunks and you discover content individually. That means that you can maximally reuse data across large data sets so say you have, you know reusable libraries or reusable images. Your clients can automatically re-share that data and avoid re-downloading it when you don't need to.

A

um This, of course, does take a little bit of a performance hit in that your ipfs node now needs to do a lot more traffic as well. Another thing ipfs lets us do is in order just outside of loading data arbitrarily that's uh content, addressable.

A

We can also load mutable data through ipns, which is also super useful because oftentimes if people are working with um data, they want to update it, and so you can actually see this protocol being used in the wild right now in places where they do stuff like web archiving with web recorder or a lot of web three slash. Blockchain things use it as a reliable storage mechanism- that's off chain on its own, it's pretty useful, but what would it look like to integrate ipfs deeply with the web?

A

So the first part that you kind of see when you interact with some data is urls and most protocols in agricore and really out.

A

There have some sort of url scheme for identifying data and urls consist of the scheme, parts which identifies what kind of data is usually an origin of sorts, which kind of represents a data or can be used by web browsers to section off storage for different domains, a path which might give you more specific data and the search params, which can give you extra kind of like querying and metadata about the information you want.

A

So you can see some examples of peer-to-peer urls that follow this scheme. So with ipfs we have cidv1, hashes or identifiers for contents. Bittorrent has info hashes, which are pretty similar. Hypercore protocol has public keys and with ipns you can use good old dns if you want to set that up. These urls are used all over the place on the web for loading web pages with html or embedding them with iframes for loading different images for audio video styling web pages and also scripting and making things interactive.

A

So if I'm loading any sort of code on the web, I'm probably talking to a url- and you can talk to these urls from code using the browser's fetch api, which lets you do. Http requests normally or through other apis like event source which, let you listen in on incoming uh data from a url one step. Further past urls is http or restful interfaces, which is kind of the thing. That's been um a little bit more new in uh the ipfs face that I've been exploring with agricore right now.

A

Most of the gateways that people work with are really used to get data out, but http actually has a bunch of extra methods that we can use so with http. You can use different methods like get which is default to download data post, which is to upload new data put, which is pretty much the same thing delete and other such verbs.

A

You operate on urls and you tell usually a web server or in this case your user agent, uh what you want to do with a url- and you might add some extra metadata for headers and optionally. Also, if you're uploading content, you might give it a request body the server or in this case the user agent or protocol, will then tell you a status of whether it's succeeded or flopped, maybe some other headers for extra data. And if you're downloading data or downloading some sort of information, it will give you the body so most commonly.

A

We see this when we want to get some data say from http, but the same thing could be said about using get for ipns, where we might want to download data from ipns as a protocol, um and similarly, we might think about uploading data to a peer-to-peer protocol like hypercore, where we'll post hello worlds to a text file and then expect others to be able to then get that back out of the peer-to-peer protocol.

A

So this is kind of how web development already works, and this is just one more step that I've taken to kind of bring web development to the peer-to-peer space and kind of mix. Those two together, there's been a few web browsers that have kind of tried doing peer-to-peer.

A

But the latest effort, I think- and the effort I'm focused on- is the agricore browser, which is a really minimal web browser which didn't want to think like any like super fancy uh functionality other than just experimenting with mixing different peer-to-peer protocols with http like interfaces and instead of making assumptions and adding features. It uses web extensions which also have access to these peer-to-peer protocols to customize the browser so say.

A

If you want to have bookmarks, you can just have a bookmarks extension that can publish to ipfs or bittorrent or whatever else we actually support a whole bunch of protocols. Every time I do these talks, we get more and more protocols, which is very exciting, so we have ipfs and ipns and more secretly there is ipld and pub sub, which blog posts coming um hypercore protocol, which some folks may have heard of which is you know, another peer-to-peer protocol, bittorrent, which honestly, I think is super underrated.

A

um You know bittorrent ran so ipfs could walk or the other way around and remember, but um there's a lot of stuff in there gun, which is uh kind of like a graphics, peer-to-peer thing gemini, which isn't even peer-to-peer but is kind of hip, and you know we love that and secure scuttlebutt, which is also an oldie.

A

So all of these protocols have their own trade-offs and their own things that they're good at or not as good, at and what's cool is that having a similar interface that uses urls and http, like methods means that developers can more easily approach using them, rather than having to figure out how to install every single ecosystem and the different apis and approaches to working with the data?

A

Now you can think of it in the same way as you would with any web server or with any web content. So that said, what does this actually look like? um I'm gonna do a live demo, because you know gotta attempt the demo gods, I'm actually gonna cheat and actually already have some code that I pasted earlier, but let's go over and see what it does we're going to be using the yeah.

A

Let me actually increase that we're going to be using the browser's fetch interface in order to uh upload some data to a peer-to-peer protocol handler, so we're telling the browser we want to talk to ipfs and we're going to put some data onto ipfs.

A

So there is an existing um cid here which might look kind of funky, uh which is like an identity cid. But basically this represents a block of uh unix, fs, v1 or v2 or whatever of uh an empty directory, and so rather than having a cid that points to a block which you download that contains the directory. This just does it entirely inline, so we don't even need to touch the network. So it's just like a nice improvement.

A

But what we're saying is take this empty directory and I want to add a file to it called index.md.

A

So uh spoiler warning uh agricore supports markdown natively for rendering, because you know html css, that's nice, but sometimes I just want to do stuff. So here in my body I just place whatever mark down. It can be like hi there, ips folks, what's cooking, I can put in a uh key tag because it's still technically.

A

Html related, I submit. I wait a second and bam so what this did was uh it took that body and it fed it to the local ipfs daemon that agrigore comes built with, and it's gave that content. The damon gave back the new cid for the directory that contains the markdown file and the file path, and then the browser included it in the http response and gave it to me here so now, if I copy this link and enter it in my browser bam here we go. I just live coded a website.

A

I think I messed up my marquee tag, though oh nope there, it is so I'm going to attempt fate and post the url in the zoom chat for anyone that wants to try that out. um It might not work and that's because uh this is all super fresh.

A

So there we go, it works, but there's still work to be done so as part of the work that I've been doing um with different peer-to-peer protocols mentioned before how they have different trade-offs and ipfs. Although I absolutely love it and it does so much, it does have some things that um are kind of like screaming to be improved upon, so one of the main things is mutability in ipns.

A

So if I want to publish to ipns at the moment, it's actually super slow. um When we were testing it, we were getting like a minute and 30 seconds just to upload a cid, and we were working with some folks to speed that up and even then right now, after a bunch of improvements, we got it down to like just under 60 seconds, but there's stuff that could be improved there.

A

Similarly, with mutable data, um there is a lot to be desired, with actually pinning it, because right now, if you're updating a data sets you either need to use ipns with a public key or dns, and when you use the public key, there's a limitation where you can't just tell a service like hey, keep this online for me forever without either giving them your private key for that ipns record or periodically publishing yourself with a node that has a private key.

A

So there's also some active conversation with this, and I think we were talking about this in the ipfs discord recently or matrix channel. um So there's a paint point, but it can be addressable and the last thing is hole punching where uh actually connecting between computers, even though it is fairly good.

A

uh It's only been recently that we got this new hole punching functionality that makes it more reliable to connect between two home networks, and so once this is enabled by default and stable connectivity is going to be a lot better because at the end of the day, from working with users, the important thing is: can we upload data? Can our friends just download the link without any third things, and can we keep stuff online and mutable, so we're getting really close and it's very exciting?

A

Why also does any of this stuff matter like? Why do we actually care about addressing these things? So to me, the peer-to-peer web is something that is very useful for this concept called local first software, and so that's software that works.

A

That's user owned where, rather than a third party, owning all of a user's data, a user controls it to start it's software that can work offline with that user's data, so that, if I want to update you know some markdown on my blog, I don't need to be online and connected to a service all the time. Similarly, it's the software that can work on a local area network, so say we don't have access to the internet, but we're on the same wi-fi access point. We should be able to still exchange data because we're right there.

A

We should use our networks and, lastly, um it's local first, but it's not local only in that it can work over the internet. If you have that network connection, this really fits well with uh peer-to-peer and the web. In that we can have these peer-to-peer protocols for storing and exchanging data. We can exchange entire websites using the same protocols and everything works like offline and local network and internet network uh by default.

A

So it makes it easier to write local for software rather than in the cloud use case where you have to like hack around a whole bunch to get that same experience.

A

Similarly, one of the things that I'm really excited about is how this will interplay with community mesh networks, where these are situations where they're often um prone to either brown outs or internet out bridges or blackouts, where you want software, that's reliable and that works locally.

A

Also, you know it's places where internet might be expensive, still or slow where you want to download stuff. But then you want to reuse your local network as much as possible and similarly building apps is a huge pain in the ass, especially if you want to go cross-platform so with aggregor, we support androids and linux, and mac and windows, and hopefully more things eventually so that now, as a developer, you don't have to care about building an app for every platform.

A

You can just again focus on a web, app use that peer-to-peer and you just get all of those niceties for free, that's kind of agriculture. That's the idea of all these urls and the protocols. That's what I've been working on, um I'm very excited to be working with ipfs folks to push this forward and it's overall a really exciting match yeah. So thanks.