IPFS Weekly Call, 8 Jul 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IPFS Weekly Call 2019-07-08

Description

IPFS Newsletter: https://tinyletter.com/ipfsnewsletter

A

I may or may not be about to be live streaming on YouTube, we'll see if this works. It was a different flow than I've ever done before.

A

B

I just got a notice from YouTube saying that it is live streaming. Huzzah.

A

Then it works hi world awesome.

C

So I guess I'm the host for July I wasn't the host last week because I was on a plane, I'm coming back from Barcelona. So because I didn't really line up a speaker. I'm just going to give a represent the presentation I gave at the ipfs Meetup that was in Barcelona a couple days before ipfs camp and I released a new version to Pierre pad. So that's what I'm gonna show go to my sides.

C

Okay, and can everybody see this oh I have to share the screen.

C

C

C

Don't know how to get rid of this thing here: okay, okay, this again!

C

Okay, can everybody see that.

C

Yes, yeah I'm, going to assume everybody can: okay, so Pierre pad and pure base building collaborative apps using IP, FS, +, CR DTS, so pew pad net. It's a project, that's been around at protocol labs for about a year and a half. It was originally started by Pedro Teixeira and it's a collaborative markdown editor. If you've ever used a system such as Google, Docs or etherpad, it's very fairly similar to them.

C

Bunch of people can log in at once in edit something so here's a screenshot collaborative real-time markdown encrypted so I think I'm gonna just give a quick demo of this.

C

So I have it loaded here and if you look, if you type in peer pad net, oops or gateways having in this series now.

A

If you remove the s and HTTP it'll work, just fine well.

C

C

Or not, okay, I haven't loaded in a different web browser. Okay, let's go here and basically you can click on this little symbol here. Alright, we just really really stood quickly and used to have a landing page and the landing page, but the landing page will come back soon with some more brandy and it's just markdown editor. But in your title.

A

C

There's a no document, title loaded- and you know it's you know- is markdown.

C

C

Other things you can do markdown so and if you want to work on a document with someone else, you just copy copy. The URL and paste give it to somebody say over chat.

C

And then you can see.

C

Take the so you can see whatever you type, the other person will see and that's Rick. So that's, basically what it is. I'll go back to my slides.

C

And Pierre, the collaborative editor was basically built as a demonstration for a programming technique which uses a data structure called CR DT, which stands for conflict, free pre, replicated data types and Pedro built a library called peer base which allows you to combine these data structures which may make it really easy to build the sort of real time collaborative applications and run them in your web browser, but using the IP FS primitives for all your network communication.

C

So this library is called peer base and also to Oh sugar here, okay, the here's the webpage for it it has written in JavaScript, runs a new browser. Also runs it no GS. So you can use this in server-side thoughts and Pinterest and things you might want to write that would run in a server that would also collaborate in work work together in a collaboration, the parts of it doesn't use all of ifs. It uses primarily limb p2p for all the network, communication and WebSockets, and it also uses IP FS pub/sub for gossip.

C

So when you start document other people want to are on there a different web browser they want to collaborate on the same document. They have to sort of rendezvous and find somehow and it uses IP FS pub sub channels. So you can run ipfs in your web browser when pure bass starts up. It like pure pure pad, has a pure pad pub sub channel and anyway, that starts a pure pad up.

C

He goes into this pub sub channel and it's like hey I, want to edit this document, and this somebody might be in there already say: oh what we're already editing that document and then they basically join up. It's a bit like a chat channel and then once they do that they're a bunch of data structures that they used to maintain connections with each other right now it uses WebSockets and a library called WebSockets tar to in order to get through firewalls, because web browsers are unable to run TCP sockets, which is what ipfs uses.

C

They can't listen on a connection, so they have to talk to a central point. There's also some support and Lib p2p for using WebRTC. So they would still need to be a centralized signaling server, but then they could talk directly peer-to-peer, but we haven't set that went up in pearpad yeah, so pure bass has several other the example apps that people have built with it.

C

So Moxie studio built something called discussed by, which is a browser extension and it uses pure base to allow people to go to any web page, and you can log in- and you can comment on what's on the web page and have a little discussion thread and that's available in I, think Chrome and Firefox app store, so just a true discuss. If I we built actually one Bennett, he the CEO of clinical labs, he built a chess game that uses pure base.

C

So you can it's more for more than just collaborative editors and or file coin I'm, making an experimental text client that uses pure base that allows you to negotiate deals with other file, claim miners and store your files, so CRT, T's, C or D. T stands for conflict fleet, pre, replicated data types and they were introduced in this paper from 2011, and they it's the structure for apps that sink that it was really great. You can have multiple writers, there's no leader and if, once everybody syncs together, everybody will see the same document.

C

So there's a visualizer here that I'm going to show you which I can sort of show how they work.

C

Playa haters, so this is a from an CDT visualizer, the Lorenzen web browser from the antidote DB project, and this one. They have several on here, I'm just going to show the simplest one, which is a CR DP counter, and you can see how three different replicas or you can think of them as different clients. You know who is in the web browser would be three different people in three different web browsers and it implements a counter. So if I click here, you can see incremented and the state here is one.

C

So if I add another one, stateless now to see oh yeah.

C

Okay, States, not three and then the community. This replica knows this. There's the counter is now three but the other replicas it hasn't talked to the other ones. Yet so you can simulate message passing here, so you can draw in there and you can see the state is now three and then, if this one is to increment the counter, the state is now four and this one knows nothing. So you could do this one. Let's get it and uh wait doesn't okay, that sort of thing they. Then you know if everything secret, synchronize us together.

C

Everything's gonna have the same state. That's basically what's their duties, do they it's really great for offline, multiple device, type of applications to be typical? What you'd find in modern mobile apps today that need to synchronize with web pages and whatnot so.

C

So there are several different types of CID teas. The the original paper in 2011 introduced, two different styles of CRD teas, one was called state-based, CRD teas and the other one. That's called Operation VCR Haiti's state-based work by copying the entire state from one replica to another, and but that works good.

C

If you stay this small, but if your state is really large, that can be a large amount of data to move operation base, CR DT is worked by recording each and every little action that you do and then just synchronizing the individual what they call operations over, which is really efficient for things like collaborative text. Editors, where you making lots lots of little changes and synchronizes lots of little ones, but they they have the drawback in that when you just first synchronize it you got to play back.

C

Every operation could beginning a time and there could be like tens of thousands, even like a million operations, and that could take take a really long time. So they both have pros and cons. Delta CR DTS was a follow on paper, which was an attempt to find sort of a middle ground where he has sort of good properties of both models.

C

So Pedro implement the paper and implemented a JavaScript library which implements these, and you can find this at this github page and there's many when you use a CR DT, it's a little bit different programming model for programmers. You got you have to pick your type ahead of time and by picking your type, your also picking like how it resolves conflicts and things- and this library has many different types- sets counters registers, arrays and maps.

C

So when you build a pure base application that uses Delta, C or D, tease you you model, you have to model, you have your data structure first, and you have to pick your data you. What sort of types you're going to use so cm time check here? Ya, got 20 minutes, so I have a home automation demo, which I'm not going to go. Show it because it's just running at a time, but it's at light starch and picked calm and if I connect to it with multiple web browsers.

C

I can toggle lights on and off and there's some source code here, which is I'm, probably going to turn this into the hello world app for for a pure base, because it's much much simpler than what's currently listed thing now.

C

So, in addition to pure base, which you see pure pad, this is just a static, react es app and it's just you generate the static file and you upload it to ipfs and Peapod net is just a hosted on the public, IP FS gateway and we've just mapped the domain name on to that. So it's completely static. There's no server side, apart from the WebSocket server that we talked about previously, but we did, it do have another service. That's hooked. Up to this is called a pin er. So when I'm typing things in.

C

You can see here, it says, will save soon and if it's working it says saved. So if I flip over I can there's no user interface on the printer, but you can see I.

C

This Pinner is running on Heroku and it's joined there, every single LP pad collaboration. So if you click down here, one of these will be the Pinner and as I type.

C

There's that there's about a 20 second delay, it'll it'll, try to save it and now I can shut this browser down shared the link with somebody. Somebody else somewhere in different time zone can get that link and open it up, and even though my browser isn't open, you can see that the data here was pinned and it's pinning all the data into ipfs cluster and we're running ipfs cluster, that which is collecting all these.

C

All every document is encrypted, so the the data that's on the cluster, the we can't read it, but the people collaborating on the apps can can so so.

C

That's what the paneer does.

C

And we're doing peer bases in the only CR DT project happening at protocol labs. Dr. San Juan from the ipfs cluster team has built a system called Merkel, C or D T's, which stores very similar data structures, but it's storing them in IPL D, which has potential to support very, very large, potentially very, very large data structures, which is very, very interesting and I. Encourage you to go read the paper.

C

There's we're also. Protocol labs is also providing a research grant to Viktor Khristenko, who is at ipfs camp I got hanging out with him a bit and he's working on something called replicated object, notation, which is another CR DT he's actually been doing. Crd T's, almost as long as C or D T's have been around in other over on the dot side of things. I previously worked on a project called a hyper merge which takes the auto, merge, CR DT from Martin clapman and runs it on that.

C

That, theoretically, could run on ipfs as well, but there's a lot of people doing some interesting things with that library and there's a lot of other prior art with CR DTS people built like a lot of this work came out of work that was done to put these their duties in the react database and there's a follow-on project called antidote DB, which has some uses some of the same data CR DT data structures and akka distributed data actor system for from Scala. This also has some of these in it.

C

So that's it so I like to anybody has any questions. I can can do that now.

D

E

D

Question I quit to question, but you already had one so my question is: are the operations right now described on equality.

C

The Pinner is storing data using IP LD, but the in the web browser there. It doesn't use IP LD. So that's.

D

So interesting because I'm not really understanding what that is possible like so, if the peanut doesn't know the key to decipher and if the operations are sent not as I feel the objects, then how can the peener without knowing which operations are there converging to like, create the states that then get spins as I killed me so.

C

The pillar, the pillar is limited in that it's only dealing with encrypted blobs. So it's not. It doesn't get a lot of information. It just basically gets the name at the document and then he gets these sort of encrypted blobs.

D

C

It's got, it's got to store those, so it's my initial attempt with the Pinner was I really wanted to like sort of try to store sort of like Merkel, C or D T's and store all the information. But you can't because it's all encrypted so it's it, doesn't really have any information. So so it's basically just storing so with Delta, C or D T's. It can work with, like the full state of the document, which is basically everything. That's in the document or it can work with the little deltas.

C

We've decided to only deal with full state. So every that's why we don't pin continuously only pin every like minute or so, and we just put the whole state into the into ipfs cluster thing.

D

That's intriguing me: they don't understand so like if you'd, storing all the operations, all the deltas, then then I could touch to the user to the new browser to kind of like just receive all of those operations and and resolve the like converge, the states locally but but like if the peener is getting. If the pinning is getting effectively snapshots of the state, then what is telling the Pinner? What is the weighted snapshot like? Is there a converse in between the browser saying, like oh I, just converts the state I converge, yeah.

C

So if you have like two people, editing document at the same time, they both have the private keys, so they're, sending the little individual deltas back and forth to each other, but the Pinner is just taking a snapshot every once in a while. It's just basically so it's the backup it's like. If those two people go offline and then they send the link to a third person. Third person comes online. They have to get the statement.

C

Someone and the regional two people aren't online anymore, so the pinners role is to grab the state which would have been saved into ipfs cluster, so to rehydrate that and send this encrypted blob to this, the third person, and then they can go edit. It make their changes, their changes can go back to the Pinner and then, when the other, two people come back online. They can get there. So so.

D

This is kind of like a growing ball or like encrypted cipher text right like over time. What the Pinner is doing is like, whatever is happening on on this document. It just keeps like smashing more and then it's up to the new clients, to kind of like just spread this ball of things and just power that and then I just figure out how to pay. So it's all these pieces together, yeah yeah.

C

I think for it works well for pure pad documents cuz, they tend to be. You know, small documents of the text, but you know if you're doing like there's a much more elaborate data structure that might be like potentially much much much larger.

C

You might want to change the pinned around so that it could save all the encrypted deltas as well, but it doesn't really have a lot of ability like you can't merge the encrypted state with encrypted Delta, so we just have to keep all these like little Delta's around and then, but it could potentially do that. It I think it was a simplifying assumption. Just to like not worry about that particular use case. Is it's not something that's needed right now.

D

I didn't go anything.

E

Yeah I don't know if you've got time for this really but I'm, not sure. If I fully understand it does. The Pinner has to then join every document to be able to read to rehydrate.

C

Yes, so it's basically the the gossip channel, the pub/sub channel and every single, so the Pinner is listening to every single peer pad document ever but it like it only listens to it for like a minute or two. Then, if there's no activity, it drops off so like it's. It's really only has to listen to the live set of documents. So but like everybody, that's using the public peer pad, we'll all like, say, hey here's.

C

My document in this public channel, so there's a little bit a little bit of a privacy thing there like the thing is. The names of the documents are just like a random thing and the contents of the document is random. So it's there's not there's not not a really big privacy problem there, but yeah, but.

D

C

They are sort of globally broadcast hey. This document is live right now, yeah.

E

Okay, so it's the pinna went offline, you and everybody else went offline. Also, then, would the document be lost forever.

C

Potentially, but what the the document is also saved in your local browser indexeddb. So even if the pin is sort of flaky as long as you're online and the.

D

Person's on, at the same time, you.

C

Don't really need the pin early. You would still like the one critical part that must stay on line is the the WebSocket server. So potentially we can use WebRTC, but we'd still need a signaling server, which would be like a smaller thing, so WebSocket server everything all the traffic gets sent to the central hub and then goes out so with the WebRTC. Potentially, the packets could go directly from web browser to web browser, but you'd still have to have fall backs that were centralized. It's.

C

It adds more complexity. Eventually. The hope is that this there'd be a rendezvous protocol implemented in ipfs in the p2p, which could take away some of the some centralization. But then we could maybe use circuit relay some like your people had IPF a secret circuit really turned on and it could avoid the whole WebSocket business we'd still be WebSockets, but you know more. There just be like a whole bunch of servers out there that you could connect to, and they would be like when specialized, when hardwired into your applications.

C

You and to be dusting WebRTC signaling over Bluetooth, I, yeah I think like it'd, be really great if this could work offline so that work hasn't been done yet so it yeah so the times running a very.

D

Fun unhappier yeah.

C

The problem we're working in a web browser environment and web browsers are limited, lydell's been doing awesome work with ipfs, companion and brave, which sorta can open up some of these possibilities to actually work offline, I.

C

Guess we're over time. So let's call that and see you next week.

D

Thank you, Jim.

E