IPFS JS Core Dev Team Weekly Sync, 19 Nov 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: ⚡️ⒿⓈ Core Dev Team Weekly Sync 🙌🏽 November 19, 2018

Description

https://github.com/ipfs/team-mgmt/issues/650

A

Cool okay, hello, everybody and welcome to the Jay s core dev weekly sink, it's November, the 19th 2018, it's nice to see your faces. How are you're doing yes, rad good cook, please put your name on to the list of attendees.

A

Jacob has volunteered to be note-taker. Thank you already.

A

What should we do? Let's do what we always do and do a round of updates of what you've done, what you've done this week and what you're blocked on and what you're gonna do next week, let's start with Jacob, would you like to give us your update, yeah.

B

So last week on Friday, we released the PTO 24 that includes our custom content, pure writing support, including delegated routing, with PDP switchover as a state machine. That also exposes a new dial up the send method. So you can get a connection state machines for better control of individual connections.

B

We also have stability and error handling improvements for that there was an issue with circuit relay where circuit relay wasn't listening last so cods to addresses to be wrong, particularly if you were using a TCP, wildcard IP address it. Wouldn't the period for your circuit wouldn't bind to your your auto addresses, so that's fixed now, so we should see better improvements once everything's upgraded to that also submitted PR to GSI PFS.

B

So we can get that into the next release, also fix some issues with switch, which is in the Lib p2p release, and then some multiplexer support fixes as well. I also started the J asleep p2p daemon and we'll continue that this week and I am out Thursday and Friday. Cindy buddy have questions Oh.

A

Thank You Jacob: let's move on to Volker.

C

So I found a bug about slow transfer, speed that I've already discovered like half a year ago, but now actually spend some time on performance, work, I thought I felt bad about it and and then yeah finally, and the IPL D team has a beauty reading and management repository. So you can check it out. It contains all the the in from the LG arms, like it's a famous for IKEA Vespa v Billy, so we finally basically spawn out being our own or own organization. So that's pretty cool and yeah I looked into the IPL dapi.

C

It's because we want to do change the API a bit in this corner, or at least a view of it, and especially in the context of graph sync because I like to find out like if we would have graphs already one with the IPAs API speed like, and then we can just yeah um update everything in sync, basically to make everything great and then I also found out doing the Beauty meeting that on zoom the gallery view doesn't work in the live stream. If you are on.

C

Linux this was foreign one. Our chat with the zoom support, true that they can confirm. There's the back so yeah we'll see how this works out and I'm not done anything and I will do further work on the iPod, the API improv, some stuff yeah. That's it.

A

Any questions for VOCA.

D

Just like week, one so now with the new IPL desync, does it mean if I build, the team will focus on that sync up like leave this one, or will you continue to be part of this one for the Giants parts.

C

And so I like as so I so ice, but that I and this will be part of both because yeah I need to sync with other people and so I don't see anything changing so yeah.

D

Got it so I'll guess we were just like see how it goes and see which duplicate information. There is probably not that much.

D

One thing that you mentioned was you found a bug that were sitting performance if I heard correctly, one thing that I saw on the dynamic data and queries working group update was that they found that the way that we smash index TV it's kind of like problematic for performance reasons, just like drags down any kind of like operation or like when you are like doing a CREP thing, because, as so many rights that they kind of like vogue sound into our application, and so one thing that I don't know if these like falls into responsibility or I, feel be people JSI back because people or both I think it's both one thing to look at is how interface artists are works in what things we can think there to make things faster like carrying occasionally or whatever something good to look at, but but I just wanted to think it's to this group, give it a few mention performance and I feel these storing objects.

D

Its other question.

A

Okay, thank you. Let show you move on to vodka.

E

You know so last week, I fixed review of balance regarding the rotten or factor for I pianist and Jessica Fest, then Steven made to fix for the interrupt problem in pub. So I did the same for the Jeff site, but I need, as you need to test you for using both fixes. It solves our problem or not. Then I was also implementing the openness over DHT implementation. It's almost ready for great a whip PR, because people from dynamic data went to start using it, so I'll create a PR.

E

Hopefully, today, Jan in the h3 stance, tests and test bed for the HD I had a talk with John ESA I asked some feedback about what should I read to understand what should be good test for the this DHD stress testing test, but recommended me some papers, I already read them and I also had a sinkhole with Jacob and Cole about the test that unfortunately, I didn't have time to create tests. But I will look at it this week. Generally.

E

I also did lots of reviews for the lipitor pre-release and the fix today back in the NCI for Jessup FS. So for this week, I want to fix some issues that David reported in the serviceworker gateway and then continue. The I penis over the HTTP are says. The Ibanez over cops have interrupt with fixes that we made and continue also the working the stress test for the nature. Any questions for me.

A

Okay, if there's no calf questions, let's move on to. Let's move on to me: hi.

A

Cool, so files API is being reflected a little bit to move the to move. The well basically move the files API to the root of the new space. You just do ipfs dot, add don't get ifs, not cat, rather than ipfs dot files don't add, seeing as it is ipfs with files and stuff. We privilege the file we're going to be privileged in the file methods, so there's there's a refactor for this and we're moving the regular files stuff to the root first and that we eventually followed by the MFF stuff.

A

So there was a PR for that in interface, I've got a score which I looked at changed, fixed up and merged and I did the same thing for J, so BFS, API, I, also reviewed, emerged, Alex's changes for IPO, d-doug, PB, so Dec nodes and pro buff tag. Nodes no longer have a multi hash property or a CID property, so that was a PR interface, has called merged and and released, and the same for J, so FS API.

A

The next step, for that is the the kind of fallout from that is that the object API is a whole bunch, less useful because it essentially just returns. You back the data you gave it in because nothing has a CID anymore. So what the next step of that is is to make a breaking change to the object API to return C IDs instead of instead of dag nodes, and then you can get your node.

A

If you want to get your data back again, but it's more useful to know the CID and then well, you should really be using the dag API anyway, but this is an interim thing and it's um the the idea with this changes that it will make reads of of these particular types of nodes, a whole lot faster, because we don't have to actually do the hashing of the of the data when we read it out of the datastore again.

A

So that's all it so any weight that is reviewed emerged then, okay, so these two things are kind of fighting each other a little bit. So the files like you're a fighter and the those egg PB changes. I had to request pull requests open, but they are sort of affecting they are because of dependencies.

A

They I've kind of merged them into one mega, P pull request and we can and that's currently running through CI right now, but I think we can get that mad most pretty soon for JSI, PFF, say okay, so then what else was I doing so? I was working on the CI, a CID base, PR still I've added all of the HTTP API tests.

A

That I wanted to add I just need to add core core interface tests, which there are fewer of to do, but that shouldn't take long now, I reviewed vasco's, a pianist reading, logic refactor. So that's pulling out the routing logic to from from the IPA and s module so that you can use different methods of doing IP, NS resolves so either by pub/sub or by DHT and.

A

Re-Enabled right so last week, I told you about the lazy load, the IPO D formats that the now you're now able to lazy load them, and this was just finishing off that work by re-enabling it in Jes, ipfs I've create the issue for the 0:34 release, because there's now a lot of lot of things that are breaking they're, going to go in it and plant and I also spent an hour in nearly two hours with before, share doing some pair programming, and it was, it was rad fun and you should all do it.

A

I recommend it cool, so I'm kind of I'm kind of blocked on the the objects API breaking change. That I mentioned returning CID, it's like at the moment. We've got a version like we've got Jes IVFs API, which will return a dag node without a CID and I draw not do a breaking change release with that in it and then do another one immediately after which changes the CI. It changes it to return, see IDs, so I, I'm, sort of soft blocked on on being able to released.

A

Yes, I've give sa p I with without basic I, can't I, don't wouldn't don't want to do to breaking releases to change the same API so I'd rather not do that next week. I need to add, in the add from URL, so in with with the files IP IRA factor that I mentioned.

A

There are a few API methods from that ia used to be in the utils object of Jess ipfs API that have been moved up to the root namespace and they aren't add from a stream, add from FS file system and add from a URL, and they have become part of the interface IP fest core. So they need to be implemented in Jess ipfs as well. So I'm, probably going to take much of the code, that's already in JSI PFS API and put it over and then finish off.

A

My CID base option PR by adding the core interface tests for that and if I have time, I would like to write some interrupt tests to continue that work and allow us to put with a particular CID version and get it back from our repo using a different version of the CID. So that's me any questions.

D

Probably are better off like checking just a beef SEO I killed for like ad for my vests, then the one named Jeff s API to the coin Flannery, because the client library is kind of like just forming all the files into a multi-part stream, which is not what you want. What you want is just like, grab all the files and create the stream that just a diva score expects, but that's a little uptight.

D

We don't one of the exciting things seems that, if things align, we might see another really spike in of the months early December. Something like that with all these features. Yeah.

A

D

All the things faster, yes,.

D

A

You have questions.

A

All right, I see blank faces is a good moving on Hugo.

F

Hi guys so last week, I was able to finish the work on on the IBF 50 CTL.

F

Basically, now all the three implementations are normalized behave same to me was much easier to work with. Also it's more about two dependencies on a face was we had some problems about having the WebRTC stuff in and out and being pure dependencies or optional dependencies and drinking is failing to be old stuff with O's, but hopefully that will we figured out fixed a problem with the if it's not working on on the latest version, and that basically was about locking the repo the public wes is already on.

F

My PFS repo should be ready to merge, as it's already was reviewed by jake replaced. So I just need to move that to my PFS and it should start working again. There was a most recent version also ever work in progress.

F

Both requests about reducing the fantasize on CTL still, neither the other one, the first one, the 308 to be merged. To finish this one.

F

Also, the public we're studying the renovate I, don't know anymore. Everyone knows about it. I already talked see her about it. It's like going keeper but with filters I request only with like one pattern, so we can start small and test if it works like we expected twork. What do I need to someone with permissions to activate the renovate service on EFS ripple, so we can test it.

F

I also updated the github CI, a config and added some docs according to the feedback Allen gave about it, oh yeah, and also started working on he stinky traitors and the damp like slave, just like at the beginning of its doing some research making sure I know what I'm doing I'm locked on yeah on the unit config on CTL.

F

This is basically the GS part is done, but we need to like go to the go team like implement the same argument on their CLI, or we probably just can decide to implement it only on our way and ever issue open. So we can follow it up with the go team. I, don't know if that's an option, but it might take a while to get the go team to spend some time remaining this on their side.

F

Also, the circle c---eight stuff is kind of I, don't know pause because the four teams not giving any feedback on that stuff, not in their own PR, not to me, because I am waiting like three weeks for the machines to finish the developer type. So we can like two versions, so cozy I, envy, bye, bye, neighbor one can just give feedback and choose with which one is best. So I left a comment in both public words, so making me a better world to anything they didn't.

F

They need to make this happen, because I think we all agree that we need to move away from Jenkins either to a circle, CIO it alive, say or whatever just something that works and it's improved our flow.

F

So that's about it and this week, I want you to fall upon still pending requests, continue the asynchronous work and the bundle, sighs work and also scouring the mid terminal cos any questions. I.

D

Just have a technical point because, like this is designed to be, is super quick, fast update, call, thank you so much for everyone, and that's like shared update so far like it's very useful, to have like comprehensive update, but I get some time like the team, it's kind of growing, and so we have to go to everyone's updates. One one thing I want to give you a confidence about.

D

It's like rest assured like if you put something on the blog, please we follow up like if you put something like if it's like getting Cir, give me something to get back to you like whatever it is like seriously like going through the notes like it's like me and others. You know point by point and make sure that you get the answers you need so you're.

D

Just talking to my lights, let's say like I blog, because three weeks are fast and I haven't received a response from this enough and so I try to like for the people that like will share their plate after I. Try to compact as much as possible share. What you have done. She'll want is in progress. Share what your blocked and share what you're putting it, and then we can like dig into the issues that you linked to like to earn more sounds good cool.

D

All right shall we continue.

A

Sorry I'm guilty of that as well. I, just can't stop talking anyone. Anyone more questions for Hugo quickly,.

A

Okay, red: let's move on to Alex hello.

G

So lots of stuff I'll try and not talk about all of it, so I split the UNIX FS engine into imports and exports of modules, which is really cool because now the smaller more focused I went to town on the exporter, because it's the easy one so he's an in-memory I, peeled the annotation OCP first course and dependencies.

G

You know added extra tests for shardene which weren't there crazy anyway, so there I just went through and tied it all up to the same thing for a bunch of old MF s, issues yeah, so changed I, peeled D to return us the idea. When you use IP, you don't get and there's a path which is cool, yeah I've changed exporter to stop talking out. Looking the interesting thing is directory. Sharding, so is I've deployed to change the ipad2 remotely be to remove the hashing stuff to NP on my PFS and is great.

G

It got really fast, not fast. They got faster, we're still slow, alright, so thinking, why is it so slow and then looking at the new patterns and nothing we're instantiating the loads and loads of diagnose them as a? Why we Dana- and it's probably because of directory sharding, and it's not in MF s now, David said a while ago, make sure you enabled are actually sharding. So I was like cool.

G

We had a problem and passed the options into the importer, but Emma Fest has a whole bunch of like custom tag, manipulations and Nana was sharding away, so I'm, making that sharding aware as quickly as I. Possibly can so that's going to be me for the next couple of days and it's kind of like retrospectively shard directories. It encounters they're getting big so over time. These things should get faster as well. That's gonna be me for the next couple of days and hopefully then we'll be able to use an idea first for everything, any questions.

G

D

Great Abed I'll just say that while you are edit, if you can like factor out the age, the model module outside, you will make Michael very happy, because then you can access that module and if you can even make it agnostic to use that PP or like Lexi bar like you will make that appeal. He can even more happy. So just let in mind.

G

So from the home but the deal and.

A

You have a quick question for Alex.

A

Okay, what are we go, so this is over Alex the your name space has been overloaded. Thank you for sharing your your update here. Would you like to talk this Rick Lee.

H

Broke up when the name out there was it me: it's your name: yes, Lee, okay, cool, because there's Alex went anyway. Yeah I well, first got another project. A little bit went through all the material I made a I started the document kind of describing what it is that we're trying to do. I shared it and I'm, not sure, if that's the right way to share documents in ipfs.

H

Just please tell me if there's a better other way, it's a Google Doc now so, basically working away with, together with Ron, on getting benchmarks to run locally in a kind of a developer type setup, and also on a remote host to eliminate pollution from other processes and right now, I'm integrating some of the tests that Ron made kind of aligning the things that we did and moving on from there. Any questions.

H

A

No, no questions, it would be cool to catch up. I guess at some point, maybe this week and we can yeah.

I

That was my kind of my list of things to schedule so yeah yeah that.

A

Would be awesome and then maybe you can take us free what what what you done and now that we're at yeah cool yeah I could so then, hang on so next matera. Would you like today, Oh more.

I

Or less like to work together with both Alex and Ron, on reviewing code and discussing that piece of architecture that Alex has done so mostly code reviews and reviewing things not much, not much activity really I plan to do more. This week, though,.

A

Okay, cool and then finally, finally, not finally penultimately Ron,.

J

So we started working on the tests right new tests for for the adding a file using UNIX FS engine, so I'm, taking kind of like the performance that was currently out there. You guys there looks like a small file and large brow, so I'm using the same files so doing them in an empty repo to start with, and then also populate and reco.

J

In doing this thing, this thing ads against the small large file then results for that also added the get so reading from the local repo same files and just putting the test on that nothing, blocky and just working with Alex on just that. You know the output stream, I kind of kind of have an initial version now for gonna probably have to change it. Look at this week, so I'm working on hardening Matt and then also, let's go see, sounds let's see so. Ok, we're writing.

J

Yeah writing results to the output directory and also a testing in a write test against multiple peers this week and also review the browser test. Yeah, that's about it.

A

Sounds really exciting. Any real, quick questions cuz we over time already I just.

J

Have one quick question on he was yeah first I was talking about the changes they to importer and exporter on the UNIX. Fs is when you guys talk about. Is that like how far away is it like? What brain is it going to master when it's done or is it then I guess when do I see those changes when I'm, you know if I'm gonna pull something.

G

Those changes being done well that there are changes in most right now.

G

A

There'll be a new release.

G

Of a huge toll to.

G

A

Might be me, okay, so yeah, there's gonna be a never released fairly soonish, but I would I'd expect it to be at the end of the month, maybe a little bit into December and that's that will probably have most of the stuff that Alex is talking about now.

A

All right cool- we are over time, so we could an update from Travis, but I don't see him yeah, I, guess, okay, cool! So read that in your own time one final thing to say is: if you haven't already done, your ok are scoring further mid-quarter. Then please do that. Thank you all for joining us and it's been really nice to have to have the new phone guys also joining but they're up days here as well. That was very nice surprise. So thank you and let's we'll offline scheduler cool cool. Thank you very much.

A

Everybody and we'll see you next week. Goodbye.