IPFS 0.5 Meetup, 22 May 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Upgrade Your Developer Tooling with Textile - Carson Farmer

Description

Providing a first-in-class set of developer tooling that invites devs to join in on building decentralized software with IPFS has been one of Textile’s central missions. Carson Farmer talks more about how Textile has been anticipating the 0.5 release, and what they have next in-store.

Textile is a set of open-source tools that provide a decentralized database, IPFS-based storage, content hosting, and more over libp2p, IPFS, and Filecoin. https://textile.io/

For more information
+ read the go-ipfs 0.5 release announcement: https://blog.ipfs.io/2020-04-28-go-ipfs-0-5-0/
+ visit the project website: https://ipfs.io
+ or follow IPFS on Twitter: https://twitter.com/IPFS

Sign up to get IPFS news, including releases, ecosystem updates, and community announcements in your inbox, each Tuesday: https://ipfs.us4.list-manage.com/subscribe?u=25473244c7d18b897f5a1ff6b&id=cad54b2230

A

Okay, thanks very much everybody for having me so first of all, I just wanted to say huge, big congrats to the whole ipfs protocol, labs team for this pretty significant milestone and thanks a lot for bringing us along for the ride so yeah, just a big congrats to everybody and then I'm gonna talk a little bit about what textile has been doing in anticipation of and on top of and with ipfs and ipfs tooling, and that sort of thing so I can't see any messages or anything.

A

So if and if you can't hear me or something just yell at me, instead so hi I'm, Carson farmer from textile, we as a team have been working with and on ipfs for about about just over three years now. I think so. That's reasonably long, I think in the scheme of things community wise and we've seen a lot of changes over that time, documentation to billing all that stuff.

A

Big improvements and we've also done a lot of experimentation ourselves in terms of figuring out in what environments ipfs behaves nicely in what environments that doesn't behave very nicely and we've thought a lot about the developer experience our own experience and the experience of other developers around this.

A

So that's kind of what I'm going to talk about today is a little bit of you know how textile ended on some of the products and tools that we've built on ipfs and then go into some details about what they do and how they work and why we built them specifically, so you may or may not be familiar with some of our early experiments that we did on top of IP FS and the sort of D web space in particular.

A

Textile photos is something that some people might be familiar with, and textile photos was essentially a fantastic experiment into just what you can what you can actually start building on top of you know web to are sorry web 3 or peer-to-peer technology, and we learned a ton. You know throughout that experimentation we learned what does work. We learned what doesn't work. We learned and developed a bunch of pretty strong opinions about you know: mobile environments, web environments, desktop environments and what makes sense from from actually building products.

A

And so today, I'm gonna talk to you about, essentially the outcome or the learned tooling, that we developed out of that initial experiment. So I'm going to focus on essentially three legs of a four leg chair that textile is currently building and so we're going to focus on essentially solving problems for real developers on top of IP FS, so I'm gonna start with threads DB, which is something that our team has been working on for a while, and it's we've written a white paper about this.

A

So you know it's it's good science and there's a bunch of different implementations and I'll talk about a little bit about those and how you can get involved and that sort of thing. But the the take-home point, which is here you've got a problem. You want to build an app, and you probably you know you want some sort of dynamic data so that you know someone's uploading photos or someone's chatting with friends or someone's having some engaging experience with other people or with your product or with your service. There's some dynamic data involved.

A

So you want that. You probably want some level of security and assurance that the data is actually getting where it's supposed to be going, that your users can maintain control of their own identity in data and interactions.

A

You probably wanted to be reasonably scalable, there's nothing worse than a sort of experience that falls apart as soon as you start to add new users to a chat thread or something like that and for the crowd. Here. You probably want all this to work on top of IP FS.

A

You want it to work in a peer-to-peer way. You want essentially a very solid web to experience, but you want all of the ideals and benefits of web 3, and so we spend a lot of time figuring out exactly how that might work and what that looks like in a lot of our early development and when we sort of went back to the drawing board and really started to figure out what exactly this needs to look like. We wrote this white paper and we came up with something we call threads DB.

A

So threads DB is a here's. The tag line here, a secure, decentralized peer-to-peer database and it's built on ipfs, Lib, PDP and IP LD and the idea is here, is basically if I'm a developer or if I'm, a development, team and I want to build an engaging experience for my users. I, don't want my users or my developers to have to learn a whole brand new language in a whole way of thinking to get on board and get started.

A

So we've developed a essentially a MongoDB like interface, where it feels like a local database but operates like a dist, decentralized or distributed database. You've got you know: very modular access control, you've got implementation, agnostic, identity, conflict resolution, CR, DTS, all the things that you need in a decentralized database, but over top of an API that feels, like you, know, you're, just adding a document or modifying a document and then for our team.

A

A really important thing is ensuring that users have control of their own data and that's actually a huge part of what brought us into the ipfs community in the first place. Is this idea of content addressing providing a way for people to really control where and how they address and access their data? And in order to do that in a nice safe way, we've built encryption into the actual protocol level, layer of threads DB that provides ways to manage access control, but also to then fill silicate services.

A

On top of that, so you can have trustless third party pinning service or relays or archiving so that your users know that their data is gonna stick around, but they don't need to worry that someone else is snooping on what they're creating so here's an example of actual code that you would write using threads DB and you know initialize a new database. You specify a schema using well-known jason schema.org for your database. You create collections, you insert values or astronauts.

A

In this case you can query it using MongoDB, query, syntax and you get all the nice sort of modern web style interactions.

A

So you get an async iterable that you can go through and and all these nice things and then every time you add something to debate database it in the background syncs across all of its different peers, using a bunch of the really cool technology that we've already heard about on the call here and there's a bunch of really nice things that happen in the background here in particular, it's designed to be offline first, so essentially, you've got a database that not only feels local but actually operates local.

A

When it doesn't have a network connection so that as soon as you come back online, you can start to tell everybody: hey I've got some data. You might want to pull it. So I'm not going to dwell on this too much, because there's some really exciting new things that I want to talk about in the next couple slides but check out the white paper check out our go implementation check out our JavaScript implementation and come build some stuff on top of threads DB. But what about?

A

If you have a bunch of like really large files or directories that you want to manage our IP FS, so IP FS is already really quite good at this. You know, ipfs ad, you get some data, it's chunks it up into a beautiful merkel tag for you. You can share that. You can pass around just the top-level hash, that's fantastic!

A

But again, if you want to work with dynamic data or you're, collaborating on a single folder in a Dropbox like way or you've got a team that were you're trying to manage team assets. You want something: that's sort of persistent, is dynamic and feels again and I you're working on a local folder, and so that's something that we, a product that we've developed, called buckets and buckets, is just a really nice slick wrapper on top of the ipfs mutable file system.

A

Well, what it's designed for is essentially representing dynamic folders on top of threads and on top of IP FS. It's got a API, that's not unlike say, like an s3 bucket or something like that. It's designed for collaborative multi-user folders that you can persist on remote, IP FS peers, so it works really nicely with pinning services. In fact, the api's are designed to be adopted fairly easily by pinning services or our own pinning service, but the really really cool thing about buckets and and will probably don't have time to do a demo today.

A

But this is kind of this has blown my mind. So here we go. Hopefully you guys think it's pretty cool, so a buckets are designed to be highly interoperable. Every bucket is essentially a folder, ok and when you add data to that folder, it's actually an instance. Inside of a thread DB running locally, so it's essentially like a document in a database and you can query it and you can treat it effectively like a document.

A

Ok, except that this is like binary file based data or a whole folder of binary file based data, it's also its own HTTP domain or gateway, and it actually takes. We take a pretty cool advantage of some of the new sub domain properties of ipfs 0.5 to do some of this stuff. So, essentially you know your folder. Is your? You know static website? Well, actually you can just address that as if it's a website online it'll be rendered like a website and yeah.

A

It's gonna get all the benefits of that, except that it's a dynamic folder as well. It there's a phone in the background, Paula jeez, so you essentially can deploy your blog as a static, static site that gets dynamic updates. But here's a really cool thing: every bucket is also itself an IP and s address, so the unique bucket ID is also an IP and s ID.

A

So, every time you update your bucket, you also in the background, update an IP and s address and therefore you can do all sorts of like DNS link type linking back to your bucket, so that every time you update your bucket, you automatically get. You know your URL. Whatever will will automatically point to the latest: IP NS, hash or you're. Sorry, you're, persistent, IP, NS hash, so you have literally a dynamic folder in ipfs with all the benefits of ipfs and that gets updated automatically in the background.

A

But the big piece there is like, with with peer-to-peer systems, there's lots of different ways that you can address something, and our big philosophy on our team is that you need to have different ways of addressing the same thing so so that if one of those things doesn't work, you've always got a backup.

A

So with buckets you have IP NS, you have actually all of the updates are also published over pub/sub, you have direct gateway access, you'll still have the IP FS protocol because of course, if it's an IP NS address, it's also an IP FS address. So you've got all these different ways of accessing content. So, no matter what, if you're a developer and your users want to be able to access, you know their data or your deployed service you they always can, depending on what sort of connection they're able to manage.

A

This means that you can deploy a lot lighter weight apps, because you don't need a full-blown ipfs peer necessarily to access your website or your data. So we're actually I, guess kind of launching this softly right now, as I'm speaking out loud, so definitely check it out. We just updated a bunch of Doc's for this. We've got some blog posts that are becoming out so check that stuff out and then I can also talk about something very exciting. Do I still have a little bit of time, Molly just say yes or no awesome, okay, cool!

A

So the other really exciting thing that we're also launching as I'm speaking, is something that our team is calling the hub textile hub. Basically, this is a infrastructure and tooling and persistence layer for a lot of the threads DB and buckets tooling. That textile is deploying as a cloud service, so developers can access all this really cool stuff and right now you can access this really cool stuff for free, and you can do things like persist.

A

Your buckets and you can persist, thread data and the really really cool thing and the piece that all of this design is built around is the idea that you could you as a developer, can persist and store data on behalf of the user, but so that that user has full control of all of that data and information, and in fact you can design it such that the user holds all of all of the keys to access the raw data and you the developer, don't necessarily need to, or you can, depending on what that, how that relationship needs to be set up.

A

So essentially you have a remote IP, FS and threads DB peers that can change the way you build your apps. They make it so that you don't need to worry about spinning up your own infrastructure and they set it up so that it's very easy to pin and ensure that data and content for your users sticks around it's. You know we basically designed this whole thing with the developer, experience kind of as the first thing and then okay, how do we we actually make that happen in the background?

A

So there's a lot of experimentation, a lot of testing out with partners, including working with folks like fleek and, of course, the team. Any type team has been building stuff with threads and now the new threads DB for a while now. So a lot of what we do is just facilitating developer experience and that kind of boils down to textiles kind of opinion about how all this stuff needs to work.

A

If we want a mass adoption of of ipfs and of web 3 technologies, we need really great experiences to be for users to come to and until we get a lot of developers building those experiences. We're not going to see that mass adoption so we're tool builders, and we hope that you know application developers build on top of our tools and then people build on top of that and everybody wins.

A

So do ask me about the hub stuff and what we can do there. The last little piece I want to say about that, and this is something that we're really excited about as well. Is we're talking about user control of data but potentially paid for with file coin, so the hub is designed to work with the product.

A

We've been working on called power gate which basically lets you backup and store buckets and threads on file coin, and that will be available essentially at the launch of maintenance and even before, on test net, so that users can, our developers can essentially develop applications that work now on ipfs that will pretty much seamlessly and with a fairly low barrier to entry, allow them to start playing around with file coin.

A

So, if you're at all curious about how might I be able to take advantage of file coin in my application, how can I make sure that my users data is persisted? How can I allocate storage space on behalf of my users as I test this out and I? Don't want to charge them all that sort of stuff?

A

You should be able to experiment with really easily so that we get more developers adopting some of these new ideas, and so that's the whole idea of the hub and we'll be adding these features for integrating the file coin right away as soon as that stuff is available, so definitely come check out our github and you can see the the progress that's happening there and I can, if I have time, just a little demo of how buckets work so that you can get an idea of what it's the sort of user or developer experience looks like there.

A

So this would be like a developer, creating a new bucket for a particular demo, and you know maybe they've got a whole static site that they want to deploy or even just a little index to tell you to know that they want to pop up- and this is just you know like you- would expect on ipfs, except every time you update this, it's going to push and then automatically update all the underlying IPN and pub/sub and all that good stuff in the background, so that users can access their webpage and then, if you use our gateways, we do a bunch of magic rendering, so that websites look really nice and automatically do local or sorry relative links and all that stuff very easily.

A

But of course everything is also ipfs, so you've get all the benefits of the whole ipfs infrastructure as well. So I think that's pretty close to my time. Thanks a lot for listening and yeah have a good rest of the day. This is all very exciting again. Thanks a lot to the ipfs team for bringing us along and huge congrats to the release.