IPFS Berlin Developers Meetings - July 2018, 9 Jul 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Demo - libp2p Interfaces Part 2 - David Dias

Description

Originally recorded during the Berlin Developers Meetings from July 9-13, 2018.

A

Alright, ok, I'm going to do two demos.

A

One got me inspired to do an extra one Q and the first one is going to be a continuation on this interface discussion and I'm, basically going just like to show things are done, but from the Jeff or at least how we have prepared the things to be done by multiple languages, but they're not mostly used by the GSI and so yeah. We prepared there's one of things. We just came from a talk on interfaces on peer-to-peer interfaces.

A

We discuss our our reputed fear is like this set of modules that work very nicely with each other. If that each module has a very specific defined interface so that we can just like swap them and put new pieces upgrading. Our networking stack I should show some images I like what the city faces look like, but just to give an idea like for each of these pieces, like for each other, like this building blocks that you are used to see an Olympian appear.

A

You get like one of these repos and they always start with a two face and then the pipe and then on that repo. You get like modules that people make the interface like. If you ever went your own you're welcome to submit up or requester that to the list, you guys really not a nice match, so that people recognize on your report. I back, you implement interface, it's like a reputed for your compatible module.

A

You can like some code to just set up test, and this is like those kind of things that when you do JavaScript you have to deal with, because we don't have nice interfaces and so the way that we overcome nice way to describe interfaces for programs implement and so a way to overcome that just like create a set of tests that people can import and use, and one of them very good examples here is the stream boxer. So the stream max here was actually the original interface repo for a leap utopia.

A

There are currently one two three four or five six modules between girl and, yes, it will mend this interface. They also get a nice badge. You got like some code to set up this test on on your own test.

A

Runner go as a specific way to set up tests and you can I find it out to do it on there girl, repo, it's all described here, but when you scroll down kind of like at this description or like, we don't want to be a super explicit and like sometimes these interface changes and I know we have to update them like what we want to communicate it like what should be the things available to the user like attach much circle, connection and ingest.

A

We we say: okay, it's matter, you see the connection and you tell if it's a listener or a receiver, you know could be a completely different name could be a completely different way to attach a mixer to a connection that doesn't matter, but at least you expose something, and then all of the pieces that implement that interface in a language should exactly expose the same function, same thing for dialing and listening, and and so we with this interface and these test suite that's here.

A

We not only sorry like I, wanted to check the source folder, because this is the module that is a test suite for the code of the module. Is the test not the test of the module? That is, it.

A

A bunch of tests tests like basics, open stream from the dialer up in the stream from the way snare. You know it's like a connection like attaches from both hands open. The stream from both sides make sure they're like we can open stream from both sides.

A

There is some other things like closing like if we closing the stream like do we close the mixer, because some boxers, when you close the the streams that were open like the master, auto destroys and we wanted for only Peter Piot, like you, can add the mixer open, close all the muck streams and keep the boxer, because you might still want to forfeiture things you just have don't have like any use for it right now, then, there's like stress test and this we call the mega stress test because, like he takes an hour to run, it's like how we detect my problems and so on, and so with this battery of tests, we implemented two three multiplexers for Jess, one based on speedy one, based on a thing that we created in-house complex and we can just like run the tests on these modules and see that they are compliant to that interface right.

A

This is how we, with we test for that interface in Jess. We see passes here if I go to my ex.

A

To do the same thing like both of them are passing the same interface. So no I know.

A

It's important so so now I know I can go into my lip. You do peer bundle and just like replace one three months by the other and both will work on my stack so yeah. That's that's how we we get to play with multiple building blocks on peer-to-peer and cool. This was one you can find all the other interfaces for peer. All things for content routing for experience, call ray. You always get a nice badge and yeah, that's it okay. This was the first part of the demo or the first demo.

A

The second one is really all to plug these things together and I'm going pretty quickly. You will find all the code that will run on that gently peer-to-peer repo on the examples folder you can find a bunch of examples like I'll transport protocol and stream. Multiplexing works good. You have a voice here, so if we go to that kills, I was already there.

A

If we go to death killers and let's try with the transports first.

A

Second, so most of these examples always like a create notes function, and this is just like reading a peer info and adding a multi other to it. There is a and there's always a part where, like the action happens- and here we are just like starting nodes and like creating a connection between both, but but what I really want you to pay attention? Is this bundle right like this constructor for the way peer-to-peer node? So here we can see that it's pretty simple, like we actually I think this is a laser pointer.

A

All right, I got your tip, and so you have the bundle here. It accepts the way peer to peer class, get a constructor any of the defaults, so I as a user of a peer to peer exceeded my own Whipple P&L. That extends that way peer to peer skeleton and I, say I want to use transport TCP. So this is pretty much like a weight will appear stack that does the protocol multiplexing, but doesn't do any. Second, you know doesn't do any peer discovery, just like TCP. So if I run this code.

A

It basically will do that create turnout style from one to each other.

A

Okay, so no one is listening. No two is listening, one dials to the other, one on the Nile protocol thing and right, ll peer-to-peer world and the message passes so very simple got pro comedy passing over TCP. This is not very exciting, like anyone could just like require that module enter the fingering. So let's go to the stream max our example protocol and mixing.

A

So here we have again, we have again like my default. The the rest of the code is pretty much the same. It's tying on protocols and I, think I know now I'm, but now I'm using TCP and I'm, saying that I have a stream at sir, am I using speedy for this one, and so, when I do multiple dial protocols to from node 1 to node 2, instead of creating multiple raw sockets I'm, actually creating much streams of over that stream axis and reusing the connection and for the user. This is pretty much for yeah.

A

The user is pretty much transparent right because you can't, you feel the same thing and we pure fear is smart enough to understand. Oh I have a stream, except to this pier I'm, going to use it. If not I try to open a new connection, and we are all happy so if you run that now.

A

Put OB another corner new per copy, so, by the way, just in terms of API, you have tile protocol, you basically specify the pier and if you want to dial the protocol in which you want to talk with appear here, I say like I want to talk on protocol a it could be any kind of string here. Protocol B and here is like so when they'll die on student note, you can open even multiple connections to the same protocol, so multiple wasters for the same protocol.

A

The other example is here for stream. Boxer's is.

A

Okay, this one is pretty much the same, just like we, since the addresses I'm going to run this one. As you can see, it works.

A

So it did the thing and in either of these examples, one thing I can do is instead of accusing speeding, I can do here and change to multiplex and just make it obvious. I also rename these variables.

A

Now I'm going to run.

A

And they did the same thing like again like super swappable, like the module implements the same interface, it works.

A

Gets even more conflicts, so there is many other things discovery so like detecting, using something like multicast dns to find peers on the local area network. Same thing like peers will find each other just for a second time. Players won't fight each other to disappear discovery and once they discover the peer, they just say: okay, I found the peer on this local area network, but also you can do things like pyridine.

A

In periodic, so we this example is more advanced one. We are already saying: I want to use TCP, and this could be multiple transports again. I'm saying I want to use em plex I'm saying buy one buy encryption of a my connection. Con encryption to be secure and I want to use this DHT module, which gives the peer routing and content routing superpowers to my ability, a bundle then I also have my config values, and this is like the standard config like the giant objects.

A

That's like has some config options and I do the same thing if I create nodes, I use these properties to connect them. So here, like knelled wand, I also know to know mm o tree and then the old one is going to use content routing to provide and then like the content. Low tree will find the providers of that no other one getting into the folder pirata your content, driving well nevermind mix. It shows that.

A

Yeah example: one spear well things about to is content routing, so just make sure I didn't lose you, because no one is going to say hey like if you're looking for something come, find me and I know tree is going to find the nodes that have that are providing that see ad. Does it make sense to everyone? No cool, so again, creature nodes does the thing and eventually we'll bring the results.

A

This is the proof either from.

A

This we found this this morning, there was like a module.

A

Yeah, okay: this is something that was broken this morning. I'm sorry. This is a boards yeah, okay, which it is evening you'll be able to find this example working and so yeah, like you get all of these properties. As you saw it's very easy to like configure, we could appear in the remaini. You get like an example of like all the things you can do: yeah yeah. If the transports you can pass the class types or even a nuisance.

A

So if you have a transport with a little bit of setup, you can pass the instance of the transport you can even I use multiple stream multiplexers. At the same time, why not? You might want to talk with different parts of the network that have to support boats? You can also that's why this is always an array like you can support multiple encryption encryption channels can support multiple peer discovery mechanisms.

A

You can have multiple periodic things you have like all these configuration values, some experimental flags, yes with beautify, also exposes pub/sub and then, if you want to learn, identify some, we peer-to-peer we're not a bunch of things here from my starting emails and something in there. That's raw like this is a another thing in the 2000 table, so we have a distinct p2p nodes start and then we could do like a tile and Daniel and I opera call.

A

The distinction is really like this warms up the connection so I can down to someone and like do the second you and shake this remark, sir. All those things but I'm not going to create a MOOC stream to that nodes. Yet I was just going to warm it up and then, if I want to dial on a specific protocol me as a user of a peer-to-peer like bits up, for example, I, say I want to dial bits up to this pier, so it does the final protocol multiplexing to appear.

A

We have a the the close connection. We tell it hang up, because if we, if I hang up, some people have different opinions about the name of this. They don't fear routing with frontier content. Routing, like the two things I was showing you you can handle multiple protocols. This is how you do it, like you say the string for your protocol, the and the word fund and the matching funk and the match. Funk is.

A

If you want to do something exotic, more exotic, like semper, for example, for your protocol, you can use a matching function to do the semper so that you can have four calls if endures, the support, multiple versions, same thing for I know annoying. They like we have these events. Always there.

A

We have it a meteor.

A

Is that right, when you discover it here when you connect to appear and when you disconnect or appears, if you want just like for statistic, reasons, statistics, that'sthat's reasons which is like listen to this or if you want to do something more proactive when you access cover a peer connect to it, you can do it by by default just happy fest, for example, as soon as it discovery just connect to it, because why not you can do the pings, you have access to the pure books, like all the periods you know like the P reinforce like information about yourself like they asked other is.

A

Is that where you are listening, the adverse is that other peers see you from the outside world and you also get the Maps API, which is the same API that I just like. If s exposes so, if are familiar with that one, you got this one as well. There's other things here, there's also a stats API, so you get kind of like data from oh. You are interacting in the notes that are on your swarm and in many other things, I think this is it showing I'll to configure only peer-to-peer nuts any questions.

A

So if you want to discover what the here protocol.

A

Here did warming up the connection based like warm earth connection, say what is it? What protocols is it so.

A

Available so when you dial to the pier so I hear the only thing this pier could be appearing foe, which has the protons at the pier supports, or that we know that the peer supports or its PID or it's not the other, once we dial to them and we do the identify and check its where the other peer tells the protocols that they support. So we can just go to this object, which is in our pearbook kind of like the yellow pages of would appear and you can see which protocol to the other peer supports.

A

But then again it's identify and identify only happens once when we first connect to each other. So imagine your application that can like end all protocols and I know multiple times. It doesn't mean the fact that is up-to-date, so the best thing you have to do is really well. You can use that as a like a and every six to know if you should dial, but you can always try to dial on a protocol and the other thing I can tell you that it is or it isn't supported, did I.

A

Yeah yeah, so we type EFS, LAN or even the internals of leap appear. We do a lot of like flash, the name of the protocol and an app version so that we can upgrade in the future.

A

Any more questions.

A

All right, thank you so much. Oh yeah, microphone, please, sir. Here.

A

Do you think we should make version of that yeah? Absolutely you mean.

A

Yeah, yeah and I believe you open an issue just for this.

A

Yeah I love that because then it's obvious for every user, like Auto, configure every single.

A

You know independent of the language we've seen yesterday, how it is how it is easy to add a transport to go nowadays, that's awesome, but it would even be better if it's just like a description, the recipe, and then we tell it to we, give it to the language in the language give is, if that gives us an instance, or we could appear as all things we need at some point, we even had the idea I'd like to have a website that works like an ice-cream machine, and thank you can I say: oh I want some more TCP and some were three mature and some were second, oh I, like it just mix it up and I keep sort of like putting like a sundae out.

A

It was like a JavaScript bundle. All the configuration of we did appear. You want it and just like a funny idea.

A

Okay, all right, yeah, I guess it would be pretty useful for tests. So you could very easily use these configurations and then run them across the same tests. Yes, yeah and so I guess. The question is like how complex does that language? You need to be like if we have identifiers for each of the protocol implementations if we are just like a a moody cause I'd like for the booty pop rerouting the multiple transfers, etc?

A

Then we just need to have this, but with those IDs, so that both hear what she has rust and all the others can just see those IDs compared to the table, we're at the model, module from the shelf and mixed ingredients, and so yeah like I, think we can very quickly start with that and I quit the configuration to be something external, like the configuration of each module to be something external.

A

It becomes more complex if you want to say in that language or I want to use PD, but in a special way right. That's where it gets a little bit more complex to do that. Multiple language, alright, like I'm, trying to avoid like having a huge by Chen, I, didn't know we can do something very quickly that serves a lot of purposes.

A

We can optimize.

A

Yeah and so I don't know just curiosity, for example, we have this periodic and continuity, um and we like to get these. We get it to the VHT and if you go to the VHD.

A

You'll see like all these badges because it really does all these things. At the same time right it has periodic in front of the peers. It is cut and routing because, like we get to know where the content is and as well as peer discovery, because when you run a more waka DHT, you are discovering random peers and, like refreshing, your routing tables, which is super useful. But ideally this is not bad but like.

A

Ideally, all these pieces that are here inside this spawn package should be also separated, so that when you are loading it into a pure peer, you can say: I want to just load these periodic module and it's to other content routing right now. We do that by enable disabling parts of it like we, you get the whole package and then you can disable a little bit of it. If you want to other, probably most of people that will use the digital to use the whole thing anyway, 10 little cool, any more questions.

A