IPFS Weekly Call, 3 Dec 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IPFS ALL HANDS 🙌🏽📞December 3, 2018

Description

newsletter: https://tinyletter.com/ipfsnewsletter

A

A

A

Excellent quick question: does anybody know whether we fixed sprint bot and that is making the issues for these calls or not? Because the links all the links this week go wrong? So we can.

B

A

B

Is on me, I haven't gotten around to fixing it, but at the very least we're doing it manually, we can have the right links. So thank you for bringing that to our attention. Yay.

B

Hello: okay, let's begin hello there, my name is Portia and welcome to the ipfs weekly call. This is a call where we find out all the interesting things that are happening in the ipfs community. How we usually do these calls is that we start with announcements. Then we get into the main presentation and we finish up with Q & A, so we're looking for an agenda. Let's see you.

B

So Argenta is usually here.

B

ah Thank you. Thank you rob and before we begin I'd also like to thank Alli for taking notes for today. So thank you all excellent. um Do we have any announcements that we like to start off with.

B

I have an announcement make sure my screen, you you you.

B

So tomorrow, I am holding a contribute to open source ipfs day, it's going to be from 6:30 to 9 and it's a great chance for people to dive in into the ipfs codebase and contribute. So this is the actual event and if you are maintainer and if you're interested in what we'll do during the event, I have a brief outline of the three different tracks to contribute.

B

I've gone to contributing events before and in many cases, I was not able to actually contribute or I was at not that level. So, in order to not have that frustration for the people who are arriving um I broke this event up into three events: I mean three tracks, so I will give a presentation of what is ipfs and what we've been doing so far, and then I will break it up into three different tracks, which is what is get for the beginner. So this is B I'm trying to make this as beginner-friendly as possible.

B

The second track for people who aren't familiar with our project, it's ipfs. How can I use it so even for the folks who are not able to contribute they'll be able to understand what is ipfs and look at some example. Applications that use ipfs and for everyone else, I'm going to add some more links, but I have contributing to ipfs first issues, which include contribution guidelines, and it also includes issues that are labeled as easy.

B

So, if you have I mean if you're a maintainer of ipfs project- and you would like some help- you can put it file an issue here under under contribute to open source project ipfs and we'll try to get some people tomorrow to look at the project and hopefully contribute and hopefully land in a commit.

B

So that um that is me, does anyone else? Have yes, it's from 6:30 to 9 CET. Does anyone else have any announcements before we begin? If you do you'll raise your hand, you, let's.

C

See I don't see any raised hands Portia my militia Nana taught was, if there's any way for people who are not physically present in Berlin to take advantage of all the work you're putting together to make this event happen.

C

B

So this is a first event and we're not going to livestream. That being said, we are going to post the presentation of what does I, professed and such and for people who are working on the je s ipfs or working on go ipfs. If you can just like look at some of the pull requests and just look it over and see if you can actually commit it, that will be great for our next event, we're trying to figure out if it'd be feasible for us to actually stream the event live.

B

B

Okay, I see that this issue's, not public so on I, will find a place to make this issue public so that everyone can actually look at it and actually put in comments and such so. Thank you Matt any other comments or announcements which is in gallery view. Weekends know already thank you and we are going to get into our main presentation from Jake and he's going to show us some of the awesome work he's been doing with the live, p2p JavaScript API, so Jake. Please take it away.

D

Share the screen.

D

Yeah so about two weeks ago we released Jaya slip, ptb Oh, 24, and so we're just going to kind of go over. What are the major things in that release?

D

There are a lot of bug, fixes and stability improvements in there as well, but we're gonna go over the high level stuff and see if anybody has questions so the big things that went out with that were finally delegated peer and content, routing I previously demoed that in ipfs All Hands, so we won't be demoing it today, but I've got links in the presentation which will share out that you can go, and there is a working example in Jay s, Lib PTP, so you can look at that and pull it down and play with it.

D

We also have circuit relay, has been enabled by default, lit p2p and Lapidus, which are now state machines and then CAD DHT now has Interop with go for ipfs. So yeh. Let's talk about those yeah so delegated routing like what are we trying to accomplish with that? Because a lot of times we have the, we can use the DHT and we get a lot of this out of the box with the DHT. But there are some instances where the DHT doesn't get us everything.

D

So a couple scenarios for this are low resource nodes nodes that are in maybe IOT devices or restrict browser space that really don't have the resources to handle maintaining a DHT. This supports that or low availability nodes. This is going to be probably a very common thing with browsers where they're you know you turn on. You create some content, whether it's in pure pad or something like that, you create a new pad, and then you close your browser down. You shut your laptop down. Now nobody is able to get that content anymore.

D

Pure pad is working on like pinning services to solve that, but this also provide its ways for us to have another node provide our content. So when we register as a provider, we actually have another node, come fetch our content and then serve it themselves.

D

So this is a great thing for, like the Gateway nodes who everybody a lot of people connect to that we can have leverage of them as potential providers have them pull our content and then, if we're, creating content, compare pad or something that we know that as it's served well elsewhere, when we shut our laptops down and then custom writing needs, there's very likely going to be the more people start using IP FS and let p2p they may have different needs for providing content and for finding peers.

D

They may have maybe specific business needs whatever that is with Lib p2p. The delegate, routers are all part of the custom peer and content routing, so people will be able to create their own for that and then supply those and we'll run through a number of them to serve up that content. So.

D

There is an example on github I'll, just pop this open real quick. So you can see it is in the J, asleep p2p repo. It has all the code there. You can run through it simple working one thing to note right now. This is dependent on a update to go ipfs which is pending for it to work against a running gateway node, but it will work out of the box as long as you point it to an IP FS nodes that supports the endpoints.

D

Currently Jas, ipfs API doesn't support that because we don't have refs, built-in and refs are what's needed to actually register another person to provide your content. So if you go and say, hey provide this content for me, it has no way of automatically like going and fetching and pulling your content so there we would need to change how that works or introduce refs to JSI PFS, the delegate modules for peer and content, routing they're already on NPM. You can go use them, pull them down.

D

Yeah circuit relay, so one of the things we noticed is we in both go and jaea slippy to be we weren't. Turning on circuit relay dialing by default, so if you didn't speak the same over the same transport as another node, you wouldn't be able, nobody would be able to dial to you, even if you were connected to a relay that could translate that communication.

D

For you, nobody was trading at home default, so we went ahead and flipped that, on one thing we didn't do is turn on hop because that registers you as a relay, so we didn't turn that on because that can provide a lot of second cause, a lot of resource resource constraints, because you're relaying a bunch of communications that typically we tried to do that reserve that for knows who can handle that kind of throughput.

D

Well, click, maybe.

D

E

D

We go all right state machine updates, so Jess live p2p and Lapidus, which are all state machines now and the reason we did that is so that we could get better flow control both at the start stop layer as well as like the connection layer. So in with this update, we expose a new dial FSM method in Jace lupita P, in what that does is, rather than getting a like normal interface connection, you actually get a state machine connection which enables you to hook into a bunch of events, they're linked here, the variety of events.

D

It's like warning events, so we got a lot of requests for hey I.

D

This thing says that circuit failed and nobody was able to connect to it so and we didn't get enough errors from all of the different transport failures that could have happened so now it will emit additional warning errors so that you can listen to those and then better, diagnose, what's going on and make better decisions about that with that, we also introduced on the state machine connection.

D

We introduced a closed method because previously we didn't have a closed method on the normal connection, and so it was kind of a annoying for users to be able to close the connection. This makes that a lot easier. So as soon as you get that state machine, you're done with the connection, you just call closed and it will shut it down and the other side will hang up and it's a lot cleaner. We also improved debug logging for connection, because this was this was a big pain point for a lot of developers.

D

So there's now a lippy to be con, setting that you can set and that will actually log out much better information and I will just pop over and try change my screen real quick. So we can see that in action.

D

So this is just the the p2p chat example. So I'm just going to go ahead. Can everybody see that.

D

Way, alright, so with that, we can go ahead and just run debug we're listening for a wild card on the Lib PD PD connections for the source listener that will boot up in our other window. We will do that for the dialer and you'll see these messages. Spamming out, I'll go ahead and make this one window bigger. Oh, no, that's not very okay, so we can see that we have messages here. Tells you which node is. This node is dialing out. It's dialing to this other node.

D

It's going over the TCP support when it's successfully dialed. If it successfully privatized the connection.

D

If it's finished crypto, which crypto it's selecting when crypto is done and then just every step of that process is a state machine. Any time it's state is changing or is in process. We log that out. So you can get a lot more verbose, debug logging to figure out. What's going on with the connections.

D

And I will switch back.

D

So with that, we also with the state machines for Lib p2p start and stop states are now idempotent. We had issues with users where they were trying to. They wanted to know. If the lid p2p node was started, so they would have listened or they were doing. Some like Genki calls, because the callback stuff, which will be illuminating, hopefully in the not too distant future, but this solves that by, if you want to know if live p2p is started, just call start wait for callback and it will done it be done. It's already started.

D

It will handle that, for you and just say it's already done so that should make that a lot easier to deal with.

D

And then the last thing CAD DHT interrupt so Bosco has done a ton of work working on getting the bug, fixes and API improvements for CAD DHT and Lapita P.

D

This has removed a lot of blockers also for the implementation for IP NS, which is getting closer and closer so working in JSA PFS, so awesome work bashful on that and then our next step is going to be turning on the DHT by default in JSI PFS, because right now, it's experimental and hopefully very soon, we'll see that is no longer experimental and just running all the time. So you can get peers and content yay.

D

Does anybody have questions about anything? Thank.

B

B

Okay, if you have questions, if you could put it in a chat, that'd be great, we'll start with dubby.

F

Hey awesome work super exciting update. Thank you so much for doing. Oh, like on your tests, always a net reversal. No problem.

F

Do you get lots of success dialing to other nodes that are behind a photo today or.

D

Yeah, so, ultimately, when you're doing delegate routing the first thing you do: is your bootstrapping to bootstrap to single notes. Typically, this is a gateway node. So this is something that is, it's still a problem with NAT reversal. But this is it's a lot more specific so that, if you're an environment where you can control your port mappings, you can delegate hey I'm, going to be dialing out to this Gateway node and I'm going to be doing an HTTP API request to that node.

D

That's the advantage! There is as long as you're able to talk to that one node. You don't have to worry about talking to the rest of the network because you're registering it as your provider I'm in fetching content from it question.

F

Not necessarily my question was specifically after you do the the content routing right like you, get a lease of peers and those peers, probably our other people's desktops. So now your desktop has to die out and that's like where the net reversal problem really- and my question is: is there any test to see like the level of connectivity from a node on gpfs to the rest of the network? Busy use IPC be a whole bunch. A does it defaults to secretary like.

D

So we don't have tcp hole punching into ATP, yet that's one of the things that we want to look at getting added in the near-future circuit relay was a big thing that we wanted to make sure was on by default, to help with a lot of that, because that is a huge problem of being able to do with that. We also have the nat manager work, which we're going to try to be working on for early next year to help get around hope with some of that tcp hole punching.

F

D

Great opportunity.

F

There for like avoiding second relay, is by just checking in on UTP like it seems like you, do a lot of updates recently and then, probably as the bug fix that we needed, and I qtp already as hole punching by design and so HTTP. So it makes things simpler, something worth checking out. Okay, I will not steal more time. There's like three thousand questions.

B

E

Two questions: one is just a request for clarification. When you say Libby 2p is a state machine to define what you mean by that. But then my question is: is around delegated routing you the way you described it? It sounded like the node that you're delegating to is actually holding on to that content for some amount of time. Is that true? Or is it just repeating your provide statements and providing routing to you as a new yeah.

D

So I'll answer in reverse order, so for the delegate routing when you provide content and you make that call that HTTP call out to your Delia node, say: hey I want to provide this content. That node will then come back and it fetches the content from you and becomes a provider, and then it registers as a provider for that content. So once that transaction has happened, you could shut down and never turn your note on again, but you that content will exist on the network. Well,.

E

So the main computer is that then it's basically providing a mic pinning service. So then is there. Is that I guess my confusion is? Is it's almost even though it's a lip feeder P feature? It's almost more like an IP FS, pinning service feature like because you're providing this yeah.

D

So this isn't this.

E

G

D

On ipfs, so it's it's like a routing plugin for ipfs that plugs into the p2p so, and this is why we introduced like the custom routing, because there are probably specific, lit p2p use cases for people that aren't using IP FS, where they might want to build their own solution around that, and then they would be able to go and create their own content, pure pre routers. For that, ok.

E

C

E

People who don't know what a state machine is like simple definition of what do you mean by that yeah.

D

So one of the things that we run into with a slim p2p for a while and even some stuff with JSF EFS, is like transition of states and controlling what what state connections were in when they were dialing, and so what state machine allows us to do is it allows us to initiate transitions, and so we can go and we say: hey, let p2p node I want you, you start and then, depending on what state that note is currently in will determine the action of it.

D

So if it stopped it will move into a starting it'll transition into starting and then it will do a bunch of accidents and once that's complete it transitions to started, and so, if you call start on it again- and this is the item potent thing of, if you continue to call the BTB start, if it's already started, it says, yep I'm started I'm done, you can go on with it and the same thing for stopping and what this allowed us to do.

D

Implementing it in the connection is we actually had a fun issue and a slip p2p where the we didn't enforce encryption on incoming connections. So when it came in, if the node didn't negotiate and say, okay I want sekai. Oh, we wouldn't enforce that it's so now it can actually go on and say no I'm going to forcefully go through that state. Like I've dialed, we've successfully dialed now I'm moving into the crypto phase, and if you try to use a different protocol, that's not in the crypto protocols. It will fill the connection. Oh.

B

Okay Ally next question: oh yeah, on.

H

Relay and hots if I wanted to run a PT node as a relay can I configure, which nodes that I'm gonna relay for? Is there a way to do that not.

D

Currently, currently, that's just a global that you're just registering as a global relay relay.

H

For everyone, yeah yeah, okay, really.

D

For the greater good yeah.

B

Jonny crunch next question.

G

Yeah actually had three questions. One is for the states. Are you using IPL D, no.

D

So what we're doing is we're just using a library called FSM events, and it just helps manage our state transition. Internally, that's Jason, yeah.

G

I think divvied my mission day you to be this support.

D

We don't have UDP support yet I think there is a transport somewhere, that's been kind of dead for a while, but I think that's one of our upcoming priorities. I think that might be in our roadmap discussion for next year is getting that UDP support and, lastly, which crypto algorithms do you support. So currently, are you talking about four keys or.

G

Just imagine definitely sorry field. What about Edie? Two five, five, one: nine or a sec p25 6k 1! So.

D

That is coming. We have I've got a pure a pull request open for the e25, everyone, nine keys, the only issue there is lip PE, the keychain, which we use mostly in ksi PFS that currently X needs to export the keys to save them into the keychain, and the only key that has export support right now is RSA. So I just need to go fix that, but my goal is to have support for that released by the end of the year. Cool.

G

B

Allen and last question goes to dahveed, so Allen, do you have a sorry.

H

I'm terrible at let me button what does is: is J asleep p2p using dial FSM by default. Now.

D

So, yes, it anytime, you call we actually like melded a lot of the logic. So if you call dial dial will actually in turn call dial protocol which in turn calls separate dial FSM, it might not yet actually might be lying I. Don't need to look at that, but the goal is to turn that all in and white blue and move to the a sink, interrater work which hopefully will have some open proposals for that soon. That will just move all of that logic into that to just simplify that API.

B

And last question: Bobby.

F

So it's not necessarily a question it just like two quick points for clarification by the way open Bazaar is happy to hear that about the new keys added support. That's for that I know that also excited. So in terms like clarification, so two things we will not directly support UDP as a transport for a peer-to-peer, and that's because, like we boutta peer design, assumes a lot of like stream based protocols, and so you have to have some kind of notion of the second circuit in a stream.

F

So so they're like all the clips that we don't all the providing etcetera words. We do have a side quests of like enabling packet switching on on only peer-to-peer and there's a lot of work, expecially I've done by wires and others, and when we get there, then we will have a UDP.

F

What we can have today is UTP and I know like these three weather elements, with very pretty much like the same letters just confuse everyone, but like UTP, ease that transport developed by the beam current group for BitTorrent, that is on top of UDP, but adds the the stream flavor that I think likely to the pyramids. So you can get the benefits of UDP like it's going to be on a UDP port. You can get the hole, punching, etc, but I can get as the properties that you need to actually use.

F

It hope it was useful information. The other clarification is people were asking about the difference between content, routing and pre loading notes, just like a very quick distinction. So in a leap you to peer feel today we are this concept of like records, and you know my name record for a penis can be a provider's record for the DHT. For like saying about you store some file and those things like they don't know the structure there is the plan to move them into I feel the align but I a lot.

F

Yes, and and that's the thing that gets used by content so like when I'm asking other nodes to provide a file or when I'm, when I'm, adding a file to a professor I'm going to do a provide query and so I'm going to other nodes in the network that I have this file and then is about puddings provided records out there. Then that's what goes into this. Like delegated routing scheme, it doesn't move the file to the other.

F

Like you still all the file, then there is another concept on ipfs lens, which is free, loving and pre-loading is when I am adding or touching fouls from I can use some preload notes, which I like nodes running on beefy machines, with a lot of storage and bandwidth and with more CPU, RAM, etc.

F

To fetch these file for me, so that, like when I'm fetching it from a low power device or a constrained runtime like a browser, you can have nodes effective responsibilities, I'm just like assembling all the pieces so that they can ban or extreme the file faster to you. It's not necessary for the full operation, but I can something that just makes the experience a lot like the proceed. Speed increases dramatically but like this constants are different like when you do a provide in whipped with EPO, and you are not setting the power anywhere.

F

Just I just want to make sure like we we are like here for everyone and I hope it was helpful. Any questions, maybe Jacob Akira and the extra points that go for it. You people that ask a question on I achieve now make more sense.

B

I'm going to put a video up later and if anyone has any questions after the presentation, they can put the questions below the video we are going to conclude the ipfs weekly call. Thank you very much Baek. Thank you very much for your presentation.

B

And I hope to see everyone next week have a good week. Bye, bye, alright y'all.

I

Thank you for the awesome work.