libp2p Team Meetings, 1 Apr 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2019-04-01 libp2p Sync Meeting

Description

Weekly meeting. Notes at

A

Be on your best behavior, alright, alright, hello! This is the April 1 / April tulip p2p meeting and we are ready to start discussing the grip pad which I guess both you are in all right. So should we go an order, um let's skip Martin because he might show up Yusef, I, guess, theoretically, he could show up so anyway. Well, let's let's skip them uh and then, if they, if they don't come, people can just read their update so and your next enter Cox. Do you want to go first, yeah.

B

Ok, so just go through those things in order, yeah.

A

Basically, and then, if something stands out is something that we want to talk about it weekend: okay,.

B

Yes, so I did a few miscellaneous bug fixes on things that I found, while exploring metrics I've been exploring metrics with Prometheus and open census, and then also making use of them and having a look at what kind of behaviour we're getting.

B

One of the things I've been using them for is to test alternatives to the way we're doing stream management before that pipelining stuff and some other alternatives and the results are interesting, but not what I expected some messages are very slow.

B

Some messages have strange delays. I'm, not sure why an interesting thing, I found was request, response message. Types though, like get providers and fine mode, are actually faster than one-way messages like ad providers, so I'm still exploring that I. Don't know why that's the case, it seems very strange and I guess just exploring the stuff in general I'm, adding metrics here and there that make a lot of sense. So eventually those things will get merged into the DHT I'm, not sure.

B

If tracing is gonna be helpful in this regard, maybe it will be, but now that we've chosen to go with open census, maybe I should start looking into using tracing to help explore the stuff that could be a bit of a time sink, but maybe it's necessary I have to do some more research, there's a few repos that probably want metrics exposed as well, but I'm gonna, wait until the open census thing is settled before adding metrics to those those armed repos in the short term, I'm hoping to work on the I think about they all know the two issues that I have there I think their routing table related just general failures in bad circumstances.

B

That I think fairly easy to fix. If they're done right, so they are having fallback nodes in case the routing table gets completely empty. You have a user, that's not doing that manually, that's sort of an unfortunate side effect and the other one is moving towards the behavior that routing table entries shouldn't always need to be connected to kind of avoid. Weird network layouts that occur, if you don't have that that property so yep, that's all.

A

Okay, I Johnny, you are go sure.

C

So last Friday I met with one, and he took a look at my elimin airy, DHT redesign proposal and he actually seemed very positive about it and he seems to agree that things are in pretty bad shape and we should just I should just try doing something rather different, moving things around modularizing things, so I think I have kind of a list of things. You know next steps here, but the first thing I want to do is look at some papers that one specifically recommended there's one called vuvuzela, that is a privacy oriented messaging system.

C

I, don't think that's going to be relevant immediately for the DHT, but just as she isn't be interesting to understand it and rehash, which is paper I, didn't it's. A DHT paper that I didn't run into the wine was pointing out is actually similar to some of my ideas about making the the metrics that we use to decide who the query next much more flexible on also recommended.

C

So sorry, yes, the next thing we need to do is, from my perspective, is to figure out how to do messages efficiently.

C

So I want to talk to Lars who's working on multi gram, just like multi stream for datagrams I talked to one about some of the technical difficulties there in terms of sessions and losing state and getting confused when you potentially reconnect after reboot- and it seems like at least while and I are very much on the same page about what the potential problems are.

C

So that's one angle, that's and then, and then the actual DHT parts of it I need to look at the way advertising protocols ought to work because, right now, it's completely implicit in our DHT, just as it was in the original cadonia, where anyone issues the query is willing to be to rock data I, don't think! That's what we want and I don't think that even belongs in the DHT, so figuring out. What that looks like thinking about resource management interfaces.

C

One seemed to like my preliminary suggestion of essentially you have each end user facing part or the application or service provided by a node would have an account that you can charge Reese or I'm, not sure I like that, but I think it's at least an interesting idea. So I want to think about that. Some more and I need to figure out what my actual goals are going to be. This quarter.

A

Okay, that sounds good. I took like one note if you, but if you wanna, if you want to amend it John, you can hear the official note-taker, okay, so alright yeah, so basically um here's what I've been what I'm up to so two weeks ago, um roll and I went to our romesco and I went to Ireland and met with the design firm. That's building this lippy 2p visualizer, um it's just at the like visual design stage right now.

A

It's not there's no code is gonna, be written in this in this phase, but basically I think it will based on the directions, they're going I think it will you'll be able to attach. To a note, it's like a debugger. You attach to some instrument, lib p2p note.

A

It sucks out all the metrics, whether it's open stances or Prometheus or whatever, and then it like makes this information sort of more like visible, but in a way that newcomers might find easier to understand than you know, like text trace, execution, trace or something so I'm really hopeful about this project. I think it'll help us I, think it'll help make little p2p bugs easier to find and also make it easier for users to understand like what is the expected behavior.

A

It has modules for like the DHT, for the routing table for showing all the connections you have open and what streams are on those connections and how much data they're pushing back and forth so I think. Hopefully, this thing will make the whole system more easier to understand. That said, this is like I mean the design phase will ends at the end of this week. But the actual implementation of this is a massive web, app.

A

So probably six months or something so you have to figure out how we're gonna do that, but and then security audit is going to start on April 15, which I think it's like two weeks from now and there's the security audit of lip pp will happen in two phases: phase one is they're going to find as many bugs as they can and then report the security bugs as they can report them to us, we'll log them in a repo, probably a private repo.

A

Just so that people don't until we have a fix, it's I, don't know that we want to be advertising if serious security issues and then Phase two, which will happen later this year, probably in q3, they'll, come back re review that stuff and hopefully also review TLS 1.3, which we're hoping will be merged and usable by then, and then it at that point. They'll issue a report like a standard security auditor report of any new problems.

A

They found how we what we did to mitigate the old problems and that's pretty much the process so yeah well I'm not blocked on anything this week, I want to finalize okay ours with the help of you guys and the rest of the team.

A

Those are due by the I think by Friday and then yeah. This is internal to 2pl, but for people who are directly employed by ppl, we work on the performance review stuff. Just trying to you know give good people like helpful, helpful feedback and and I just noted that yeah this week, I'm in Seoul, with parody so and my times like a little limited. Although it's great for timezone alignment with with Australia and wherever okay yeah. That's all I had.

A

I wanted to ask one question about something: you were saying: antic relics other than that I. Don't have any agenda items so we could. We can call it I'm, not some questions. um So is the I, so I I keep it like. There seems to be conflicting things about open census, Prometheus I know it's like a sensitive issue for, and some people feel very strongly on one side or the other, but have we'd like actually made a decision to go with consensus.

A

Is that was that the point we've gotten to or is it still kind of undecided, I think.

B

We've decided to go with it and do one of those we'll go with it for now and then review it. You know I think a month, so I'm guessing the end of April and decide if we're happy with the results. I think it's one of those collisions so I, don't.

A

B

We're going with no consensus for now: okay,.

A

Cool cool, yeah, okay, that sounds good I, don't have an opinion. I just just wanted to know what the status is and.

B

I think all the features that we want are prison. The only thing we're lacking is a bit of documentation and some stability around some of the features, but I believe that everything we want is there and ready to go into sue yep.

A

Okay, cool cool um yeah I have one question.

C

Yes, for an acrylic and mic I guess it definitely I definitely get the feeling that one would prefer I work on the go version of the DHT, but I was just I would personally prefer to work on the JavaScript version. I just wanted to get your guys thoughts, I.

A

Have a thought well here: I'll go and then I'm leaning on the go direction. At this point.

C

A

I think the go direction is better for the project. The reason is that, as you know, right now we have specs that are in a state.

A

That is not it's it's coming along, but it's not it's not really something you could implement just from reading the specs and for most new language and planners, which they've proliferated massively like this two plus plus there's at least two different JVM implementations, Python, obviously and I feel like I'm, forgetting like two more languages, but if we point them to the go, implementation basically say like this is the reference implementation, so I think for you for your work. It would be more helpful if we could get it into the go implementation more quickly.

A

Okay, just my personal opinion, though enough.

C

But that's perfectly I think the only other thing that I'm a little more familiar with JavaScript well significantly more, but the other. The only other argument I have for it is there's a lot more stuff. That's only implemented in go, that's sort of not core and just having something simpler, I think would be kind of nice, but I think I'll go with go.

A

Yeah I do see what you're saying I mean: I, don't know, I, guess you have to think about your own, like if you, if you are so much more productive in in JavaScript, that go working and go, is gonna like me. Take three X's long or something then just find out worth it, but no, no I, don't think it's gonna be.

C

That bad plus I'm sure I'll get faster. It go quickly.

B

um So one observation I had you're talking about resource management. I'm just running Netflix on an IP FS instance discovered that it seems to be quite clever at managing file. Descriptor resources- I, don't know if there's something already there I haven't had to poke around. There might be something something in there that does that, but there.

C

Is actually there is something somewhere that that tries to optimize for file descriptors.

B

Yeah, whether or not it's optimal I'm, not sure, but it does seem to to do very well or at least maximize its use to the available resources. It's quite impressive.

C

Okay, I'll take a look.

A

Okay, if people don't have anything else, then I think we should call the meeting all I'll post the creep had to get home. Oh I'll post this video on YouTube and then we're gonna. Try this new thing where we try to get everyone on the team to approve the the github pull request of these notes, just signaling they were at it, I mean they can comment if they want to, but just we don't.

A

The worry is that these two different calls will cause the team to like bifurcate and so we're trying to keep people like equally informed. So so so you should approve this. You don't have to reread your notes since you attended and if you didn't attend and but you're watching this, then you can hit approve by this point in the video.

A

All right sounds.

C

Good there guys.

B