IPFS Weekly Call, 1 Aug 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IPFS- go-ipfs Roadmap Aug 1 2016

Description

See https://github.com/ipfs/pm/issues/130

A

Hello, everyone welcome. This is go I PFS! You were here for the last one awesome. Basically what we're doing today, instead of having the normal hangouts or generally don't know what he's doing or David or whatever we're going to be going through specific roadmaps we developed over the past week or so at the workshop and elsewhere. This will not be seen on get up on the PM Rafael in the fs organization.

A

um If you look at the sprint issue, / today, issue number 130 you'll see a link to it as well as link to computer pad and basically, what we're going to be doing is going through this roadmap. We have the next couple months or so, and look at like whether or not it matches up with our design for the future and what we want to do about it. So I'm gonna, let Emily, and why are you sleeping for most of this?

A

If you have any question, feel free to throw them on IRC and I will look at them Henderson and otherwise. Thank you very much for watching and uh Emily I think you want to take away how it's going to go well.

B

Jeremy, if you want to review the roadmap, that we have very cool- and you can also talk about the outstanding excuse- me- you two and what all needs to be added to the roadmap. Alright,.

C

Cool, so on the issue we have same for end of July pre q3. We wanted to ship both the fso 43 wanted to have sharding for sure, and then there's some other things on there, which I don't know why they're put on pre, q3 I. Think that was a mistake. I, don't think that was written up, but as far as shipping 0 for 3, we have some UX issues that we are waiting on. We got the release candidate out, which is good, and we've identified and fixed a few bugs all ready for that.

C

The second release candidate should be out probably an ad in the day or two um down to chardon. um We well I've got some good working on that.

C

Given I have enough time this week, I will be able to have that done and a pull request out for that this week, that's going to be post for three just so we can not push a not quite tested feature out into really so quickly past that we don't have anything that we scheduled through the first week of August, which is great because all the stuff romantic July is going to be first week of August second week of August, we want to ship interbay shins, JSI PFS, but I'm not sure well.

A

It is, there were some scrabble arabs and taking it down from the wall, um issues that we had and onto this thing so.

B

Much you not have the clearest handwriting might.

A

Be a good idea to actually take this issue and put it in the etherpad and edit it directly. Should we do that? That's.

C

Probably a good.

A

B

A

C

Do that give me one second, okay, um so one of those checkboxes does not get the public key from sekai. Oh and the second is web sockets, um not getting the public keep in sec. I, oh I think is not something that we're going to worry about, because we want um yeah kuba, that's so we want to be able to implement Sakai. Oh Friedel was working on that. So we don't have to worry about that, because we don't have any interoperability between the two without sec, I Oh in the first place.

C

So it's kind of no point there. The second bit is web. Sockets and I. Have this working just need to get it merged and tested? As with everything past that multiple protocol version handling one to bead and I made some good progress on how exactly we're going to do this um the other night? And so we have that figured out.

C

It should be pretty easy to implement and I think we'll probably be able to make at least that part second week of August, and then we have a data store too many file, descriptor fixes which actually get done so I'm going to check those boxes right now week 3, we want to have multi codecs implemented. um This is.

A

Milwaukee, the pad, instead of on the on github, did you post a pen, yep cool.

C

A

And in the side chat bar here and on the expenditure I'm.

C

Looking for it on that,.

A

You do it. Okay, there we go.

D

A

D

Can actually hit its ok.

C

Cool thanks hmm all right, so yeah multi codecs implemented. This is um going to change to to be not just multi codecs implemented, but we also need to implement well first, we need to decide on our content, ids spec. As far as the changes for IPL deep, adding that we discussed- and I think one just posted an issue to discuss this- see if I can find that real, quick and post that here, because it's relevant.

C

Cool yeah, so that's this.

C

1-2 issue comment going there, so we need to figure out and implement the multi codecs and the cid be one as long as calling it. That's gonna probably take a bit more time. I, don't know how like the overall timeline and that's gonna work, because.

A

Many ship intubation was actually shipping to grace into tears at the fs.

C

C

C

Alright, so yeah implementing kids see IDs be one is going to be good. Then we have benchmarking setup, so automated sweet on Team city, as well as benchmarks across the entire API space. So this is going to be.

C

This is going to be dependent on kuba for the most part, I'm gonna be able to help out a bit, but I will be busy so kuba. How do you? How likely do you think in this will be? Should we move try to move this back, or is you also were going on.

E

Like I'm work, no symbol, t ash right, yeah I have first like I like. I have the first mmm Pete like it's ready. I have got a few tests and it's ready for a review by people. I have the basic implementation done absolutely ash, so I can start working the benchmarking platform and yeah okay.

C

Keep in mind that there are other goals that were working here.

A

Now I know: okay.

C

A

C

Find you can buy you selling looking.

A

Benchmarking is not a priority; um it is quite a priority. They're usually decided to search by the team yeah. It's it's fairly perfect.

E

We have like it's blogger for many PRS, so it now like at least three PR super block currently, because we don't know if they are beneficial or not yeah. So.

C

Week, 3 or week 4, we.

A

Also have it on September week, one by accident as a duplication, so.

C

Interesting, let's just put move it to week, four and duplicate that okay, as well, have better metrics as part of week. Four, but.

E

Metrics are not connected fetch barking, any in any way that resists next think I blew. Some tile also will be working on.

C

Such me eyes open. Oh.

C

So um that's that we have extracting gateway to its own tool using kar di. It was.

E

C

This job- this is a large thing when.

E

Lars complex yeah.

C

Does anybody know when Lars is getting back.

B

Howley's two weeks, thank you kidding.

C

A

C

I'm not 22 hold.

A

On cover, he took three weeks, so the 23rd is when you're. Finally getting back so we'll have the last week of August, the drive finishes: okay, I'm looking to do it. A hundred cool yeah.

C

Let's leave it there for now. We can always move it back later when he gets back Nicole, better metrics. um We should one of the to do. Some here is.

C

But if I better means.

C

So, let's figure out what we want to do as far as improving our metrics, whether that's more integrated, I, PFS things or get a response. Timing and such stuff cool, sep tember week, 1, I've.

B

Killed glory Jeremy, since we have August. Can we touch base about the outstanding items to see if there's anything else that needs to be added for August.

C

Meaning meaning.

B

We're all out of the gate pad at the top. We had a list of things from the 2006 cute redock that we reviewed um things that were not completed.

B

Which it looks like somebody already went in and add some check marks it's at the bottom of the eighth pad, so it has a list of go I can't fess bug fixes reduce idle bandwidth, usage net travelers whole six memory leaks, improve testing for 0.40.

B

C

Yeah um so I mean testing, that's so the third. um The bug fixes are all complete enough to be no longer prioritized all right. um We none of those got prioritized during the meeting I'm, not super concerned with them other than reduced idle bandwidth usage, which we was brought up and we decided that I pld was more important than that.

B

We're completely profit.

C

B

During it well.

C

It's it's going to be on the long list, but it's not to be on the short list, which is what we're going over right now.

B

Okay, um so what about under the improvement yeah.

C

Under improvements up testing, yeah I mean we always need to get better at testing, that's kind of need. We should probably quantify what is meant there.

C

You fit expo up our PT on this. This actually means is intended swap has on.

D

C

We talked about this, the.

C

F

C

We came up with something. Basically, there are some bugs in bit, swap that mean fix alongside this and o speakin o much here's one. So we need to basically have a request ID for each list, entry and that needs to be sent back and forth and then use when we're sending lots of people to make sure that we keep track of them and can hash things properly. So thats kind of alongside the bits, false paths changes because we're gonna have to change that all.

C

At the same time, when we change the protocol, that is that's priority for out I pld, because we're going to need these changes for the a pld integration.

C

C

Past that improve this performance, this is blocked on metric or the benchmarks. So much parking needs to happen. Then we can do that eating too. Maybe I potranco progress and stats.

C

Yeah we need to, let's see you mate, outline, use cases and expected.

B

So when should that be slated for about August thing or QP, roughly.

C

Q3, that's something: we've want.

A

C

Could be october note and use.

C

This is nice for myself and others.

C

Well, that's the snowed versus app no problem. This is another one in the lungs where we need to define these cases and like what.

C

C

It's best sharding. um This is already slated drop. That I mean resource limits, um yeah resource limits.

C

That's a thing need to work on, but is not been slated for q3.

C

So storage limits shouldn't take too long. We can probably throw that one in September somewhere memory limits is really hard. Bandwidth limits have hard performance issues or if we start the meeting, then Witten we have to worry about limiting different protocols versus other protocols, and you know one protocol choking the others out blah blah blah which gets difficult to do hmm so. That's those that review thing alright,.

B

So none of those affect the timelines for august's no.

C

Other than what I just removed of the interspace sharding, what is already in the August yeah cool now so I pld blocks for integration. um This is going to require the content, ID stuff to be finalized and implanted leave.

C

With Michelle, since with listing multi codex, wait-listed multi codex, twice in August week, three and in September with one.

A

Which we did so quick one.

C

That depends one. How long do you think the content ID bug shedding is going to take the.

F

Question and I just shipped us back, that's what I was wearing this. What I just wish! That's uh we'll see, I think. If the initial review comes back positive, then we can move pretty quickly and being a short news like, oh, my god. This is insane, then it might take longer yeah.

C

Ok, let's often, let's realistically leave it in September week one so, let's remove let's yeah remove it from August week three, let's leave obviously 3d more blank.

C

A

Have had it in August week three on the board for those in the lower section we did Frank say that again you're rather than august week, three on.

C

The board last week, ok, cool yeah, that's weak on a September, ok sep tember week. Two should grab us 0 for 4. That is a dork.

E

Should it be ship 0 for 4 or shape RC, one yeah? Let's.

C

Let's say the RC there and then because basically the whole reason for this one is to have a fifth unix of s charting. That's our core. Like thing we want in that release and it shouldn't be a huge release. We shouldn't have too much stuff in it, so it shouldn't be too hard to get through and I think that's realistic, honestly, um cool now, September week. Three here comes the fun stuff. We want to buy September week, 3 be done with our code coverage. Push this means across all of our goat repos.

C

We have over 80% code coverage on all packages. I have in my goals thing that I'm formulating and going to post as a Captain's Log update a very large list of every single package that that implies so I'm going to put that and then we're going to go through and it'd be nice. If we could figure out how to automate this, and so I think part of the part of this goal is figure out how to automate code coverage across all of our repos and maybe even have a dashboard forward on a project.

C

Repost pager, something who knows but.

E

It would be nice to clift simply click in other languages, who would have just badge yeah see.

C

I so coveralls does it's a badge so for going today, yeah for go so we need to figure out cover all stuff. A couple of our repos have cover I'll set up, but I don't know if it works so we'll look into that and that'd be really cool to have like coveralls for all of our egos.

C

What this means, though, is that we want it done on September week 3, which means that it has to be in progress through the last six weeks before their that's going to be roughly three or not. It's gonna be like five repos um completed a week which estimating is going to be 20 hours worth of work per week, which is a lot, um it's probably probably less than 20, but that's just kind of a little closer.

A

Than other people.

C

A

With that, yes.

C

And this is something that can be easily shelled out to any contributor. So I'm, not super worried yeah, you can do Richard, you could go test, so I'm not super worried about making it a deadline because yeah it's something really easy to ship and push to other people.

C

Ship complete, go I, PFS API.

C

We need to find so we can either take this over more ourselves, which is going to be a fair time investment or we need to source the community to find somebody who's willing to take a more ownership on this either way we want to have this complete documented nicely: have all the API calls implemented and actually have tests. We don't want that for this right now,.

C

Next up a go: /, /, typing I, think this is yours: I finis yeah, um so we want better performance. Yes right now, the performance I think people complaining about performance on publishes and people saying.

E

C

10 to 30 seconds on publish the results times have been pretty fast talking to people who use it. It.

E

Isn't like event in this resolve time with a local cache. The investigation is around every time a few tens of milliseconds. So that's that is selfish. You need to work on because it is for every request. A few tens of milliseconds, yes resulting name. That's better.

C

E

Oh, no, that's that's! Bns DNA! Only DNS based I penis, not I pianet, not the HD base of my penis yeah I'm, talking about the HD basis in HD and it's I was getting tired. So it's couple of seconds yes, yeah.

C

So that's not the huge bottleneck. It actually gets really fast when you're fully connected and have made queries previously. So I, it's not priority to fix this all time, but it should be priority to improve published I'm. So look into that. The other thing I want to start doing is I want to use I pns for one of our websites.

C

We should start thinking about a candidate, for this should be a website that is used with some frequency, but not horribly bad. If something goes wrong for a few minutes, obviously we don't want anything failed, but we're testing out a feature that we haven't put a lot of pressure, and you know time on so maybe with Richard.

F

You know soju doesn't make a make an alias to dis da visit I- oh that's like this too, or something and then use that for builds and stuff and so download stuff from there. Yes, a mirror of the regular dist, but it uses it go throughout the MS. Well, then, you can see whenever it fails, which is the NCI yeah.

C

E

Works then we random failures in CI, which is something we really don't want, which will get idea aspect really fast. Yep yeah.

C

Cool so yeah. Let's that's a good idea. Let's do that. We also want to have integration testing which putting using a pianist for our discs page for her see I, would that's a level of integration, testing, I, think so that would be. You know two birds with one stone their reliability. Let's try that front of file descriptors. This is a hard problem, because there's many layers with Norma to file file descriptors at each of them handles the issue differently, the block store or flat.

C

If s right now handles it very well, it has kind of an exponential back-off to retry putting block. So the worst thing that happens when you run out of file descriptors with flat FS is it will just get slow until the JavaScript or pressure is alleviated.

C

Leveldb will fail to put which kind of sucks, because it does throw that to the upper layers and causes whatever operations we were doing there to fail. We might want to try handling that better like catch that an error from of the be in try or something, then the other one is in lib p2p. When we're dialing out that's hard, because there's two different ways is: it can fail when we're dining out which we can handle just by trying to dial again handling their butt on the listening side.

C

If somebody tries to connect to us and ratified the scriptures, then that just sales- and we don't really get a clear idea of if it like when that happens, pretty sure that listen will not return an error. If somebody tries to dial us and we can't accept due to file descriptors cool IPL d-block store integration. This was somewhere else already thought yeah, that's in September week one. So it looks.

C

Well um right sharding data sheets, but um does.

A

Anybody know what this one refers to: I'm looking, so just move on to different one: okay,.

C

The September week for shift go at the fso 44 yeah that sounds right, um Wow and then to their next week. We're going to ship the 50, rc1 cool um I. Don't think that's super realistic! We can try. um It really depends on how all of this goes. But there's no there's no green for slack here. So.

B

You have your shipping go I, PFS, 05 and week 4. So when would be realistic to do the 05 RCL 10, if you're gonna meet the ship date of week for 405.

C

Realistically, we should.

C

It's really going to pend how all of the rest of sep tember goes and if we're able to meet the IPL d-block store changes and implement, go I pld, which isn't actually on this list. As far as I can see, and it's kind of a hard requirement for implementing I, pld and shipping, the RC 1. So we need to have like.

D

Back here, I can't find the card off with the poor. So little bit about go IPL, be.

C

That needs to happen, but that also depends shipping go. I, pld I believe depends on the content, IDs stuff, so realest means this has to go after multi codex get implemented. So.

B

Richard one of the parts it would be at the bottom of the board that can sweet hardcore, yeah.

A

I couldn't see it there, I look.

B

I Pia I thief as I pld right starting data sheet.

B

Helps give my friend.

B

A

Yeah I see it now. I still know that means I'm. Sorry I talk, it is filling her regarding.

F

The sheets I think it's probably writing up how we choose to shard things into a data structure.

F

Okay, maybe like basically describing how I have to do the charting so that other implementations can can do the same thing. Okay, oh I'm,.

B

F

Sure that this is what comes to mind.

A

Do you now almost ten minutes over yeah.

C

C

So! um Well, you guys were talking about that. I moved I, pld blocks or integration back two weeks to sep tember week. Three because it depends on go I pld in go. I pld depends on having the multi codex Content ID stuff finished, so I move those to give us a week of time for each which is generous or not that's kind of like Titan, but we'll.

A

See I'm sure to mention this earlier: the bowls denoted here and mark down by two asterisks, it's on our main priorities for those sessions, so well everything else to be on there, like the bowls. What we already decided like this needs to happen.

A

You want to keep running through this, or is this enough to go from for now we can revisit in a couple weeks, yeah.

C

I guess we should probably revisit this in a couple of weeks because we'll have a better idea of timelines.

A

F

We're hunting one thing worth mentioning is that if something runs late to building some buffering into the schedule, so that, if something goes late in one week, it can be picked up in the next one without removing anything critical for that week. Otherwise, the whole thing's gonna start sliding, and that's not very great.

C

That's what I'm trying to push things back for it.

A

Also be good to remember at the beginning of the pre q3 section that I could be moved to August week, one if we haven't finished any of those hit, doesn't look what we have.

B

Like you said that none of it needs to be carried over okay.

C

Not not not the q2 stuff, but the stuff we had slated for Pete Creek, you three yeah.

A

C

Probably not this week, one yeah brah see this stuff was not part of that, so this is homeless, stuff I just.

D

Put that back there, because it's almost we put a bottom to a girl to do.

B

Yeah looks like ok, so actually missing sense, so those were three orphan cards. They were just at the bottom of the board. They weren't slated for any schedule, so we can figure out where those belong now that because or where.

D

Do you think that they.

B

Would fit in this schedule so there were cards that were written but we're an assigned a timeline.

E

C

Hit anything more.

E

C

um Ok, then, we have October for getting all of IPL d done and chipped.

A

Which one point out that the notes for the performance task was, the performance will be performing Lee performance and I. Don't go close by.

C

A

Is there anything else going to cover in this call not.

C

Necessarily know I already way over time, so we should probably it.

A

C

It's just I filled in accident.

A

E

We want to note that we want to release our see every week through October, because I think we will need for our seas will probably end up with for our seas, and we should just try, keep trying to writing it. Make sure we release our see every week that.

A

Sounds good to me as long as it's not like, like it's I, think it's better than more RCS and more testing before it actually something it's the best release of having been awesome. um So yeah I think you should note that.

C

C

Well, that's that thing that's good enough! For now, we should probably get to talking about a guildie.

A

For us all right, thank you, everyone, sorry. This was kind of a technical talk about schedule. What was all useful, um Emmaline or yeah Emily? Do you want to take these notes and put them on the issue um and split them for I can do that now by the way I feel D you got it want me to do if.

B

You could do it very great hold.

A

It okay, thank you and you have any questions, feel free to ping them at me, and I will give a canister as possible, otherwise have a good day and we'll see you all next week, unless you're going I pld see ya.