IPFS JS Core Dev Team Weekly Sync, 24 Jun 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: ⚡️ⒿⓈ Core Dev Team Weekly Sync 🙌🏽 June 24, 2019

Description

https://github.com/ipfs/team-mgmt/issues/650#issuecomment-502741584

A

All right, we're recording hello, everybody and welcome to the James core developers. Team weekly sync is June: 24th 2019, it's almost like a fest camp. So there are people frantically preparing their stuff and not here because they're in Barcelona already and doing god knows what but we're gonna catch up anyway, and what we normally do. Is we look at the prepared?

A

Don't me I've been here for a while look at the crib pad, if you, if you're here but you're, naming attendees list, I, think you've already done that that's good and put your weekly update down.

A

If you haven't already- and we will start going through everybody who's here and we'll go through what you've done, what you're blocked on and what's up next, just as we can sync up cool, so first up vasco has put in an update down, but he is not here, he's been doing, he's been doing, presentations and things new PTP and I confess, presentations and Viera University, that's cool and yeah. He did a presentation, a Tokyo Jonas. We provide a work. Ok, so he's working on DHT re, providing which is awesome.

A

It means that once we've provided something to do, Sheehan told it that it's there. We will eventually tell the DHT again that it's that it's still there, because it will eventually just drop off. So it's good to have it. We renounced cool, so I guess this was from previous week's final presentation and APO get the HTV provider. P I was ready for a review, I, think I've seen them they're, open, busted and draft.

A

So hopefully that will be ready soon and he's been doing for intro intro test for hybrid pub/sub network, that's cool whew, any questions for master. Then please get in contact with him and either via IRC or how else are we open? Initially, we open a corpus on the team management records, though this and you can probably comment on that and ping him. If you have any questions next up is folk BMX. What have you been up to.

B

So my update is also a two weeks update because we didn't meet last week. I haven't done that much in the ipfs part of things, but I've seen that who was working on part of the bench anchored to a assembly, script and I just wondered how fast the C version would be if it's directly compiled and I would read a blog post just two days before I started about how to compile C, to represent it without em scripting. So I was excited to just try it out.

B

It took longer than expected, because it's not as easy as it seems, but the outcome is that there's no difference which is good and kind of expected, but still what's the fun exercise and I learned a lot of things about pep assembly and then I'm totally blocked on making the trace, IPL D, stuff promise free and only a sink await. It's a pull request on Ellen's iPod Ian's memory thing yeah to make it as sink await as finally, all the others dependencies are also sink await.

B

Once this is merged, I can upgrade everything and then yeah. It will be nice and clean, and next is yeah doing this, and also potentially removing name links from deck PB.

B

What it means is currently there you can access the so we've been ever use, MIP, FS and UX some file. It's not how it's actually stored naturally, and the question is whether we should keep support for this in IPL, D or not, because union service is actually not using it and it's a performance bottleneck but I'm checking with the go em IPL did him to see if they think it's a good idea to remove it.

B

A

Cool, thank you. I just figured out what module it is you can have of mine in in exchange for bits, law.

B

I'm happy to take it.

B

Alright good, as almost equals size.

C

There's a little difference there, alright.

A

Thank you. Thank you. Boca I have no questions about that. I will I, guess. I will transfer that module into IP Odie and then I'm happy to review as well the portal quest when I get a sec.

A

What's next Doc you're up next, would you like to give us your update, yeah.

D

So last week, I was at the MS teamwork, and so most this is from the previous week and so I fixed a couple of pinning bugs in Ikea. This so I've been looking through the code quite a bit too to work on the jeaious pinning stuff and create an issue about blocks. So so we have this issue where we're sharing on bit swap some stuff. That should really just be internal.

D

The painting stuff- and this has a pretty severe performance degradation effects when we're when we're doing anything really bit, but particularly on some DHT tests, so um so yeah. We should be able to improve the performance of basically everything. If we can, we can figure out how to do that. So I've written up an issue about it, I'd like to get some people's feedback so then yeah so now, I'm working on improving painting performance, it's pretty complicated, there's a lot of things to think about because it affects nodejs.

D

The browser, you know, works differently and you know there's some trade-offs there between whether we're gonna cash it up in memory, whether we cashed up one disk, so I think I'm going to write up an issue about the various trade-offs and ask people for feedback before continuing with that and you're. The last thing is I, as you guys know now, I'm taking over as lead maintainer further bit swap implementation and I know that in go lands, they've been adding some things to bit swap leg sessions.

D

So I don't know like how that all like how this fits in with kind of JavaScript priorities, but I just wanted to sort of check in on there. There's a road map.

A

Yeah I think there was a section of team week that you missed out on.

A

That was a whole bunch of things to do with, like all restructuring and reprioritizing, and that will have to bring you up to speed on and it will eventually make it out will be communicated out to the public pretty soon I think camp will take precedence an out but and yeah in terms of your priorities and things I just yeah just be aware that that's that's kind of happening and everything's kind of up in the air in the moment: okay, yeah thanks.

A

Otherwise the issue we're talking about about this weapon that being provided to DHT. That sounds like.

A

Disgust for your professionalism, yeah.

D

You know there's an issue there, so the magico a.m. yeah a couple of comments and then I just wanted to you know run about you guys and see if.

A

That's if, if we have time because it's.

C

A

Well, let's let Jacob go first and I can quickly share an update, and now we can review back to it. If that's.

B

A

If is there any other questions to dirt before we move on? Okay, Jacob is up. What's going on yeah.

E

This will be quick, so I was out most of last week, working on ipfs camp stuff. Do more of that this week and good ipfs camp and then I created a PR for the delegate.

E

Routing support in JSF EFS adds in the configuration for that got some combats back from Dirk, so we'll probably move the addresses for the delegate nodes into the config so that we can update those and make that configurable for people and I'm going to Michael sent over the instructions for getting through being able to run their own service and go so I'm going to test those against local install at some point this week and then get those updated in the modules so that we can make sure that people are able to run their own services and not everybody's depending on the preload nodes.

E

So we don't take them down. That is it for me. Any questions.

A

I guess I have a question about the implementation and.

A

Thanks for sending a PR I'm just wondering if, because they are currently shared on the preload nodes and not split out, interrupts their own as their own service. We've got two basically two different services running on the same machines and if one should get really busy, then it takes down both of them and that's really bad for like if everyone starts using the delegate routing and that causes them to fall over there's no backup which is the like thing. You know the pre-loading stuff. That's this rape.

D

A

Already there and I'm just wondering if we don't I haven't, had a chance to look at the PRS. To be honest, but- and so is it? Is it basically configured by default to use um content rating or yeah.

E

So if you it'll enable content and pure routing by default, it'll turn those on so.

D

If you do any find peers or anything like.

E

That, if you have a DHT, enabled it will attempt to run through your DHT first. If it fails to find what it's looking for, then it will reach out to the delegated routing.

A

E

But because we don't have the DHT enabled by default, that's by default as people eat, it's gonna cause problems, yeah I, think having our own service having individual services running would be helpful. I think also encouraging people to run their own for the infrastructure, but obviously that is gonna be dependent for not everybody's gonna. Be able to do that. Yeah.

A

Okay, so what so, I'm thinking this would go out before before camp I? Just don't want, like everyone in the world to start using that and for the experience at Camp not to be really good, but what we could do instead is to just inform people how to set their art and have it you know, ready to go and then, if you want to use it in their workshops that can then they can configure it yeah I'm Glen.

A

We only then we potentially only we have a like far fewer another people using that, and so it might be less likely to fall over. Do.

E

We want to throw it behind like an experimental Agri, something for right now or just make it. Configurable is.

A

It well yeah, that's I, guess that's my question: is it? Can it be configurable yeah.

E

So we could actually just have it go through and if you have, we could have it so that if you actually have what we're talking about doing is adding delegated. So in your config you'd have addresses delegates, and if you have any in there, then we can just have it run it. If you don't put any in there, then we won't have it run, so people will just have to go in and pop the preload multi adders in there yeah.

A

E

Could do that I think.

A

Well, I, don't know I'm interested in your opinion, but that sounds like a good idea for a camp and until they get separated out into two like the prelate nodes and the delegated reading notes. Yeah.

E

I think really yeah not having those turned on by default. Not having everything turn on but fall is good and then having. Then it was just configurable like here or a list of resources. We recommend you spin up your own, but if you absolutely must here are some that we have yeah just note that support is going to be degrading if the whole world is trying to use them. Yeah.

A

Exactly and I just I just want to make people are using them in their workshops, and that's that's rad, but there's you know it's much restricted numbers of people using it. Yeah.

E

And won't the advantage to if I can get the just some clear instructions on getting the go one. Then, if people need to run their own delegated routing notes for the workshop, then they'll be able to do that with go yeah.

A

Yeah exactly yes, that would be rad. Okay,.

D

It seems like a lot of the time we we specify posts directly, like you know, most one host to host three I. Don't I, don't know if I know enough about load balancing, but it's impossible to just give a DNS address and it'll like the DNS will like load balance for us and it'll choose like you know, round-robin or whatever, so that we don't have to specify the exact number of posts, because that way we can kind of scale it up and down the.

E

Biggest thing would be dealing with knowing who you're contacting and routing that, because a lot of times you'd have to have if the biggest challenge I think that that is having continuity in the peer that you're dialing. So if you hit a load balancer either, that is a sharded peer which I don't think we have support for right now or it's somehow able to determine like okay, you coming here.

E

This is a specific period Eiling, because when we dial out, we expect that we're dialing a specific peer and if we hit the load balancer and get a different peer, we're gonna, shut down right.

D

I said you're saying, because it has a specific ID right: okay got it make sense.

A

E

Yeah, ideally being able to sharp ears behind load balancers would be super nice.

A

Yeah I feel like the prehnite stuff it when you add or catch something from my PFS. It will send like an API request to it, sends it to the first one at the moment, but there's a peer open to just round-robin them, but it's just a HTTP API request to to those preload nodes and that that in itself could be like it could be DNS or or load balance to any number of peers.

A

But we would still have to specify every peer in the list of peer in list of in the bootstrap nodes as well to connecting because they we actually have to be connected to each one of those peers to start receiving their content over the bits. Well, you.

A

A

That was Jacob any other questions for Jacob all right, sweet answer: okay, I'm next one week, hopefully pretty quick as well last week- was teen week, which was amazing so much so much content. So many things learn and, like I said some reprioritizing restructuring and things and towards the end of the week, which will which will all kind of come to the forefront. I think after the camp is out the way. But now it is, it is like finishing our final prep for the camp.

A

There's still a whole bunch of things to do and like we want to get a chance of the best release out the door, because there are camp courses that depend on the new release. Add some refresher, so I needed to get that done. The big things there were there's my pianist so named resolve with DNS DNS, hoping those things and, and then there was the delegated routing, which is now kind of also in the program in the pipe should be hopefully really really soon, which we've already talked about.

A

And then the third thing was the kind of DHT and whether or not that should go in and I was slightly considering this morning. Whether we should just add it in and set it to the EDS to client mode by default, but not not full mode. But yeah I, slightly held back on the whole.

A

Getting the DHT really considering the issues and problems that we've been having with the gateway and the degraded performance, and it doesn't.

E

A

Like the right time to be introducing a new DHT implementation, even if I was a hundred percent confident it is completely right so I'm thinking- let's not, let's not do it for camp we'll have delegated routing, which will be hopefully almost as good and let's get it in the next release instead concern because there are fires that are still still there and I wouldn't want to aggravate them in any way so yeah. That's my current thinking, I have to finish off my core course.

A

A it's nearly done, slides a good and well they're, taking a long time because I'm handcrafting them, but then they've done and then the I'm, the DRI for the deep dive. So they are. They are also almost all ready to go on track relatively cool and I. Think any questions.

A

All right cool there is no cross team updates. There are some people away. I said Wesker is still away next, two weeks. Okay, you look back again. Sorry welcome back again and Turk is away. Okay, that's! Okay! Let's get some 80 all right um cool should we've got you've, probably got five. Just ten minutes left. Would you let's talk about the bits, work, yeah.

D

Let's do it, so maybe everyone can take two minutes just to read the issue that I've linked to from my notes. Yeah.

A

Yeah we had the same problem when, when pre-loading was first well, I had the same problem include any was first, first added and then like I realized that there was a whole load of things that we didn't actually want to be on the Gateway that sort of preload notes that are internal to your notes. So it just meant that that option got propagated in quite a few places.

D

Yeah, so um so that's one where we go, we could also do the same thing. So basically, the issue is that we don't want to provide blocks that are used internally. We don't want to provide them to bits one, and so one way would be to kind of pass an option wherever we're we're doing that. Another way would be to create like a special instance of like LD and especially a flying version of the tag component and and then in the peano.

D

We just used that offline day component so that it knows that it doesn't send you any of that stuff out to the network. So that's kind of a question: I guess is whether whether that solution makes sense. You guys you.

A

That sounds pretty sensible to me. Yeah.

D

Back I just had some concerns about, like it just felt a little weird to him to be like creating two versions of stuff one item on online: it's how they do it in the girl code, but yeah. If you guys, are okay with that I think I'll just go ahead and make a PR and.

B

I think it also sounds good in the perspective of the future of IP LD, so just em, so the plan is that I know if it's the next version or the version after even is that really IPL d is not really responsible for how things are stored. So it's rather, you pass in some foot function and some CAD function, and then it would totally make sense to have basically two instances because the the april IPL d won't too much io its. Would.

B

Rather you pass in the functions you want to use and then it totally makes sense to have one using bits: four and one using just local things or yeah. So therefore, it for the vehicles, yeah.

D

That's what I was thinking that it like IPL D, doesn't need to know about the internals right. So probably we should just keep it in an external place. What.

A

Kind of overhead would you get with having an and another instance of IP okay. What is what does it mean like? Is it just a bit more memory, because there's another instance so is there some like? Is it.

D

Yeah I mean it'd, be pretty negligible, I think.

B

A

B

Wouldn't expect much much really expected to be a resource, intensive I. Think it's I, guess we'll see: okay, I guess we'll see, but I don't expect any any major problems. So.

A

One thing that does come to mind is IP OD formats and lazy loading them. If we create another instance, is it going to have to lazy load.

A

Again, probably right.

B

Yeah, probably M, but it's for the for the internal stuff. We don't really it's support. So so what does internal mean? So that's internal me interest for like M keeping trade of things, because then it probably will be seaboard anyway. Alright defo.

A

No no types anyway right, yeah,.

B

They were probably just to create this instance with just aesthetically or like statically loaded seaboard, and you will be good yeah.

A

D

I'll make sure when I do the PR I'll check, you know check what happens internally, make sure it's not not necessarily loading stuff, I.

A

Think the IP ID, like with the volcker you're saying about it, just taking a get in it, getting a set and I think like the block service, you give it. It only uses getting set already, but I'm pretty sure I've passed like a mock block service to it that just just has I get an asset, but I doesn't that's not a big change to what's already happening?

A

B

Correct yet, but but but there's still, this rear, like this, this dependency between egg block service bit swab and IPS repost, so I kind of like yeah historically grown, end I, think super ugly, but as home yeah. But this will hopefully go away in the future.

D

Cool okay, I think that's I guess mean enough to go on. I'll, make a PR and I'll I'll. Send you guys this reviews yeah.

A

Does it have any repercussions for like locking and and stuff if we're using a separate well.

D

Question I'll I'll make sure to take that into account, but I think the pinning step end zone locking anyway, so it it should be. Okay, I mean.

A

So like on the face of it, it sounds. It sounds reasonable.

A

It's just the I, I guess I just want to know what the where the potential problems could be like what you know is it gonna use up 10 times memory? Is it going to cause problems like we just discussed, yeah, yeah I? Guess you get in there and get your hands dirty and see how and see how it goes. That's.

D

A

A

Sweet is there anything else, you don't want to say I'll talk about my shall we shall we call it a day.

A

Next time I see Jacob, it will be the opiate. That's yes, eventually, all of the viewers.

A

Alright, well have fun all all of you people and until next week, have a nice have a nice time and happy. I confessing.