IPFS Weekly Call, 10 Jul 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IPFS All Hands - 10-07-2017

Description

https://github.com/ipfs/pm/issues/470

A

All right five pass: let's get started, welcome everyone to this week's All Hands.

A

Let's take a look at the agenda. All right, we are recording I, believe this recording excellent. So if anybody has agenda items, please put them in now at the end, if possible, so that we can cover them. Otherwise, maybe that a lots of agenda items for us today, so that's good! So asking can you talk about?

A

Well, let's get started with the first one read talking about. You could rely specification, I, don't yeah yeah. We had five minutes. I have.

B

I, what I mean it um I had Inc five minutes is enough. You can set me up to five minutes so hi everyone I, have published a new version of the secret relay specification, basically tune iteration over the old one. That call SS all the comments that were made across multiple PRS and multiple issues and like it was getting very hard to really get a good idea of what the thing had to be and like doing it. So synchronously was not working because there will be multiple people talking over the spec in multiple channels.

B

So what I did, as we agree? The last week was: okay. I was just like take the responsibility and call s all of them myself and create a new PR that is clean and I closed. All the other issues and I point all of those issues to this PR, and then everyone do a review on this PR I think we are pretty close, like we probably are like 95% about getting the real expect finished. At least I don't find any major issues and I currently other people that have reviewed didn't, find it as well.

B

Nevertheless, I welcome everyone to like really good. Do a fine-grained review make sure we are not losing any edge case. Make sure that we are not missing. It was already thought and that we can incorporate it all in the spec because, like once, the spec is merged, then like people should be able to implement it. From that specification.

B

Hope this was clear. Handing up everyone. Let me know if you have questions. Thank.

A

You yeah yeah. It's really good to coalesce those into one place, because I know that I got that five minutes on five Reapers for relation to discussions. I could work.

B

So I was staying out of really spec discussions. I think I might try giving like giving it 12 wait and try to like see what's unclear for me, and this way this way like as people Wars or something they start to miss the to the details right, mm-hm yeah, exactly like we might be already so used to talk about certain things about relay that we need to explain some of the bit The Bacchae necessary for the people to understand.

B

Also it is a spec and we currently have several thousand specs that are scattered over the IP perspex repo and no equilibrio specs. You can see this pack as kind of like a defeat style from like iterations from previous spec satire roads. If there is something that you don't think it's clear from the structure of how the spec is laid out, if there is something that we think we did best, it's a motor spec which point it to be because we can always like restructure and I, get some points. There's enough to be now.

B

But at some point in time we should find this structure for writing these technical specs for instrumentations. That suits multiple needs and not just like f1 spec template per spec, so kuba I know that you have written the Private Network spec. Compare those two I see what you could take to the Private Network specs and what you can take from the permanent effects to this real, a spec that will be AK series all right, 1r.

A

Than that it might be very useful to start collecting facts. We like especially also specs, that we read read that are not written by us so that we have a place to search from, because there are quite a lot of people on the internet, writing specs, and so it might be useful to search on those to converge onto a general spec style every we want to go for in the near future.

B

Yeah, there's definitely a lot to learn from out there and there's a lot of like formal templates like RFC, so I would say like now. We have the opportunity to actually like rethink that a little bit until like there will be parts of the spec special.

B

We have the pro multi-format plans that will be published under some like standard bodies, and we will want to follow their format but like if we have this hybrid between, like formal specification and specification for implementers, we can experiment a little bit and I try what C works best, especially with the whole community. The whole ecosystem in get up, but but yeah I agree that we should like make the best we can from others cool. Thank.

A

You any other comments sauce all right.

A

Next, one I guess the p2p dates read here. You have lots of new things for us.

B

Oh yeah, so, by the way someone like has items to the agenda, please add them between mine to the day: I, don't give a goo goodness, okay, so I've been updating like GFC could appear at before. We cousin I've had a model where we had like this base class, where you could extend the base class and implement your own p2p bundle, but that was not clear to everyone and it required a lot of maintenance, because, like every time, you updated something on the WIPP peer-to-peer tree.

B

Now you have to plate all the bundles, the people polish has modules and what we realized was well. It is probably better to just like make one single entry point where people can create examples, they can read the whole documentation. They can read how to X in the boys class and they can see the tests like the tests instead of, like the rest being on each of the bundles, like the desks, are all on this base class and I.

B

Basically, there's a test implement their own bundles and like testings individually and together, I PCPs, with remixers I transfer system markers with the channels and so on. So now, if you go to, let me hear you scream: will you my screen? So if you go to the Jess we preview period problem, can you see my screen fine here like this right here.

B

Like as a fresh look like, you can see the new logo, which I really like it has like a background like explains like while EPP exists in the first place, it has pointers like two full bundles, like the ones that I pianist uses still in explains how to create a base. I can also extend the base class I was just speaking about and, like you can physically bring your own modules here. We are bringing TCP and WebSockets like we are adding them and like we are using this PD stream.

B

Luxor and SEC is the decrypted Channel and then, like you, have all of the API call. The commutation like after you extend the voice class. These are the primitives you get with, which peer-to-peer start and stop dial and gap 186. It's a cycle here, peer routing content, routing protocol, marketing and so on. If these are like all new terms to you, one of the things I work a lot, especially during this weekend, was getting really nice examples.

B

So if you go now to weep YouTube here, examples and tutorials there's a lot here and you're like this kind of like walkthrough. So what is it transport, and why do we need to support multiple transports and I'll? Do you create a bundle with more than one transport and, like all of these tutorials, always my steps and explain what you have to do?

B

There's a lot of information here to digest and actually I always have the final version of that example, so that you can like run it and learn from the point of example yourself. So there is like transport, there is protocol and stream axing.

B

There is increased communications, so like oh, and why do we have SE cayó there we speared Cauvery somewhere, okay, there, you go your discovery, so this explains how to use the bootstrap, how to use multicast dns and also gives you a hint that you cannot find discord, mechanisms like the WebRTC transport and also the DHT module. There is two more examples: I'm working on for publish/subscribe and for the peer content rodding.

B

You should see these ones being merged soon like this week, and the other thing I also did was create, like a general update, the things that we call typically the catalogs, so we peer-to-peer hasn't seen a catalog in a while, but now we are like a log like from yesterday and basically gives you I can run through all the things I just explained here, there's also a reddit account. The preacher contacts been like for a while, but now it's like more actives, the single entry point, the tutorials.

B

What we are working on and yeah this is this, is it I think I would wave nighter? It is like maintenance. It produced Mendez along because now, like I, just have to write test once here and I need to just update things. I'm, definitely, definitely for it. It's flatter and yeah.

B

We love to hear everyone's thoughts like please check it out and try the examples and yeah like I, think it's like a very useful tool to actually teach networks to other people like if they are not familiar with like this comes on like peer discovery and peer routing and transports, and so on and I think the examples make a good job of like explaining why they are necessary, why they exist in the differences and so on.

C

Map, so you know of anyone outside of this, like with PDP team, who has done their own limitations. Based on that, the previous version we had to implement your own stack, the all Matt.

B

You are cutting a little bit, but if I well, no, you are totally off Matt I.

A

Believe we lost not.

B

Well, I think I managed to understand what was the question. So if I understood correctly, Matt was asking if there was other communities other groups, other projects that created their own, with peer-to-peer bundle to use in their own application. Oh man, your back is your connection better. Now.

C

I, don't know it's supposed to be 200 megabits, I, don't know. What's going on, probably.

B

The Wi-Fi usually I.

C

Can I guess.

A

Are from another yeah.

C

Yeah I'm back now sorry I was.

B

Just like repeating your question, if I understood correctly, you are asking if there were other projects, other communities that built their only peer to peer bundle without these pieces right, yeah.

C

And are you aware of anyone outside of the Liberty division using this stuff yeah.

B

So, for example, the Korean team has been playing with the p2p for a while. There is computer dial that uses ipfs lashley peer-to-peer pub/sub to do I, you T computing strategies and what I've been seeing is. Yes, people are using the peer-to-peer, but they are not necessarily building their own bundles. It's more like will just like use ideas, because it's the one that runs in OGS and the one that runs in a browser like the one that has all the features that I need.

B

So how'd you like to use that one and not worry about building my own and yeah that works well. Why not maite all.

C

Right so the switch doesn't actually disrupt anyone who is relying on the p2p. No I could maintain yeah.

B

Yeah and Ike we we have checked the dependence to NPM so NPM. You can see we depending on your module if you have published something to do outside world and like so far like the the bundle the previous one octave works like we have issues a notification and back on the Remy now so like. If people were using that in a private way, they will see that now they have to to migrate over and a migration is just like doing a bunch of endgame installs and copying one file like it is of this rapid.

B

Actually, we didn't change, we didn't do any breaking changes in API sense, cool, yeah,.

A

A

All right, Levi was the PGP updates, so figuring basic figure into nice shape.

A

Next up to be, we are first from you in a long time, so you're talking about JSF, GFS reason, 24, that's 25, I believe alright.

B

Yeah, can you go next I see that you edit a lot at items. If you want to go next, please because I have another one opportunity fun. um Let's.

A

Do this one and you get no gold, Ashura okay, see.

B

Okay, so I announce today the like, as you might be familiar by now, when we release just a PFS, we kind of like always do an issue like describing what is going to be released like a pre announcement, so that people can I can time to test it out. Tell us if they found on the bugs and and just like, get the community ready for for then for the next iteration. The next version is 0:25 and it's more like a maintenance release than actually like a breaking change in terms of API or just like.

B

That adds a lot of features there. There is one significant change, which is if you are using WebRTC in node.js. So why are kissing? The browser will continue working as it was working previously, but if you were using it in node.js, you now have to like plug in your own WebRTC team, and we like to really explain, tops to that and the reason why we have to do.

B

That is because, wherever you see in node.js, when means that you have to install a native dependency- and that makes it really hard for some like runtimes to like catch a thief- is running, for example, Linux or Windows.

B

We have been also like trying to get the experience of the electron users using GPS on the main process. So not the browser process again, like the browser process, is always fine. This is just like the note GS part, which requires this dependency, the selective dependency, which is very alpha. It is a woman that more tricky and that you have to go to more steps, but again it's under AMI, and you can always open any ship channel running promise. There is also an issue with all of this. Let me share my screen again.

B

There's also an issue like losing all this applet there you go yeah so sundar. Really this year, I lied. What's going on the web RTC situation, I was just talking about what we are going to do in the future or like the soon feature. The rule is checklist and yeah.

B

If you have an app using GPS, it would be great if you could like release it here, because it's the app is open source and it SS I'll be more than happy to when releasing the ipfs to go and try to run your tests or even just like open an issue on your project to say: hey, we are going to create a new version of just a bit as soon you might want to try it before so that you know what you need to migrate and yeah.

B

It's more of a maintenance release like if it includes a lot of bug fixes and it like, brings the newly peep here and yes, actually, that's it.

A

Great. Thank you any comments, questions.

A

All right go check it out coming soon to NPM.

A

B

A

Sorry I had some items myself, so the first one is essentially call for help when we ported the biggest story from go to JavaScript, we essentially reimplemented the same interfaces for query that go implement. Now. Header has pointed out that those interfaces are making because they're, basically most part functions.

A

Callback like functions that you pass in, make it very hard to actually optimize heavily if you would implement data stores, for something like this equals store is for a sequel database.

A

So the question Q becomes how to move forward with this and how we can ensure that still keeping all necessary functionality that we need and might be still, maybe you be able to change the interfaces in a way that it is easier for implementers to make these performant when computing data source. So there's a bunch of comments on that issue that I linked there please feel free to go there.

A

I will also especially like to hear from the NGO side of things, because I believe it makes the most sense if both languages decide on how to move forward. This so any background, any additional information that we can get there would be very appreciate.

A

Thank you all right. Any questions. Three thank.

B

You awesome, I think yeah. This is awesome and I quit at no issues. I created all this API. It has kind of been like it's like I just thought. It was something be fine in 2011 so and it was designed initially for Titan, like from Python to go and from go to Jess like something just just need to be real, like redesign to that to the language.

B

The thing like thanks to Jess and like it's highly dynamic one time, the second, we can actually inject like different repo instances on the fly or on ipfs nodes creation. One of the things I talked with Pedro was the third option of eg. We have like our default data source for the IPS repo and they offer the primitives that, like we need like that, I define on the interface, but then for applications that wants to operate over the same data, but one, for example, to create a next to get my queries faster.

B

They can inject their own data store. That also exposes those features for that app to consume, but for ipfs IP, better than even camera right, because I as long as I'd be best as those primitives step back you looking for defined by the spec. It's fine. However, the key thing here is like the spec doesn't say these calls and not anymore calls like the sex. Is you just do I have these primitives and if it has, is primitive? It's fine.

B

If you add more, it doesn't fine, because it's JavaScript and like you can add whatever you want and, and that enables cases like Club sub room and like the live DP. That literally, is building to add their indexes happening underneath and like the very fast queries for like making that alive to be more interesting and still make it work with the rest of the hypothetical system. We are bringing anything.

B

So it is like I think, to explore and I think that, like we need to do comments better and I, just like show by example, because it is strictly like it's not it's not intuitive, that oh I can inject my own version of the storage back-end. That exposes the interpreters I want and implements the query engine in my own way and like still offer the same primitives that the rest of the ipfs, what like in secedes it started to deliver all we need to do recommend this. Everyone there's.

A

Also something that I did not pour it when my idle, your initial port, because it's not fully completed yet, as far as you know, go ipfs is working on implementing configurable data stores so that your IP FS config will contain a field which is data storage, and so this is something we also probably want to eventually pull in and implement all right, cool yeah.

B

It's getting quite quite close to them, so though it will have just a data store stack in ipfs config and if there is no existing data, storing in your, we are initializing to your packages, node to just create those data starts from space, and otherwise there will be migration tool that will be able to migrate between two different data store, specs transparently so like. Currently, we have some speck. It is just implicit and we'll make it explicit and then allow tool to migrate between them.

B

We are like- and this will allow like two users to tune data stores for performance or like so, for example, we have very good data store upcoming, but it probably won't work with like congress's terabytes of data, but will work liquid, like hundreds of gigabytes so like we can make it default, but from the other side like it would be very nice default, because it's very fast.

A

Cool, thank you we understood need is something we should also keep track on. Near-Sacrifice said absolutely.

B

I like in the end, we just need to explain that there are multiple options and that we have been working all them and then, like shell, what are the metrics that we are like trying to optimize for an Indian like it's a waste of like names of the data storage? Videos we like read like capacity to, like you know, maturity, maturity of the accommodation itself, performance for a larger set or small other sets, etc, and then developers can pick whatever you want. The most yeah, that's rakul cool I, just record no very silly. Oh.

A

Cool alright yeah so go check out these issues about datastore. It's pretty cool so and help us make them better. Also, please, post any use cases you would have for data storage, maybe in that issue, because that also will help they find api's in terms of what we need going forward.

A

Alright, so, but one more issue for you: AGA the build tool that we using in the jeaious ecosystem, to build by PFS and the p2p things is getting a new boom. It's coming soon. Well as soon as all the bugs are fixed and the features implement, it is going to remove gulp, which is going to make a lot of people happy I believe, and it's going to be mostly a drop-in replacement.

A

There will be a little migrations here and there, if you rely on sure things will hopefully just make it a video if you're maintaining a module yeah. Also the with what is already done is the Linton configuration is now shared, this link, config, which is published champion. So if you want to reuse that you can just install it there, a click send it and ipfs /. Yes, then config each year, I believe.

A

Yes, thank you.

B

That's awesome: I cannot wait.

B

A

Way, I will also help of things like help like help swag and will actually tell you what's happening. So it's going to be nice yeah, absolutely.

B

I guess, like Age year, has been super super useful throughout the entire projects, like fraud, all the JavaScript repos. It is yeah like he just needed this crap like this polish okay for a while I know it is finally getting it. So super excited.

A

Yeah, alright, the read believe you have another item for us: OPSEC, okay,.

B

Yeah so plus item on the agenda today, by the way, I think, like some people that demos for today I don't see the method on the agenda, maybe I'm wrong, just checking. If you have.

D

B

Now so OPSEC operational security we have been chatting over like we had a conversation going on for quite some time. We had the opportunity to discuss it in some issues. Sometimes we also at that discussion here at the LMS, and essentially it's like making sure that we don't open the room to do any mistakes with regards or like shipping codes into production after our defense gets more and more used by. There is more people like putting it in production.

B

People like experimenting with it, installing it in their own machine so that we start like we need to be extra careful with what gets merged into master, what gets release and so on, and so we have been like trying to figure out what are the best strategies to make sure that we enable everyone in the community to contribute to these projects, while not getting into a state where, like everyone, has admin permission to every single ripple and can do like every single thing that they want right.

B

Yet, unfortunately, let us roll back a lot of these things. That's all it's not a person control system, but but by there is some admin tasks that cannot be rolled back and there is a lot of distress that can be caused by pushing the wrong code at the wrong time.

B

And so we started to see alternatives to make sure that this works well, and one of them is that now we dub, for example, enables as organization owners to set default option that says everyone in this organization that has write access to a ripple should have two-factor authentication at all. It is a simple step. It is a very powerful security step like it essentially ensures that, like your account, cannot like be compromised by brute force attack for a it's very easy to set up.

B

You just like annoying because like setting that up without selling the entire community might mean that, like some people receive some weird emails from Goodell saying now, you don't have permission to write this repo anymore, because that option was set up. So we need to make sure everyone understands that we are thinking about it and that the everyone else ends when we turn that option on I. Think, like the best is just like, as usual, to open an issue and I raise this.

B

This concern and this option and make sure that, like everyone, that has an opinion can voice it before him. Otherwise it might be just better to enable the other thing and I know ours is working on this is we have like teams for organization like the JavaScript team? They go team. Some people like work across multiple teams and sometimes like saying language team, is not the best way to organize people, but so far it has been the easiest way to get a clear perspective or like who is working on.

B

What is ok, this person is all know, JavaScript team, so they have access to the JavaScript peoples.

B

We have been discussing the idea, like once a person is not active, contributing to code like daily for three to six months, to consider like reducing the permissions of that person to something like read access, instead of like having the right access to many other ripples, just as a security measure, something that can be again bumped up as soon as the person wants to go through it again, but just to make sure again that no one gets account colonized in the future, we can switch a person from a Java Script team to a contributor scheme, which means you you still.

B

You are still part of York. You still keep receiving the same notifications. You can still subscribe to other repos in the same way that you are subscribing. You just like, lose the the right permission to some of those repos for the time while you're not going to being. It seems like a sane approach and like a clear one.

B

Sometimes it's disinformation, if it's not well communicated, can likely to some very sensitive conversations. So we are just now preparing everything to do those roll those up it kuba yep. Some thoughts on this yes convinced device, but the default one, you type it again.

B

I'll write access to deploy, you're stretching to watch the repo, and when you, those the right access fears, are stopping to watch the repo which will be quite annoying for some people, especially if they are like in my PJs, subscribe to all the ripples because and they gain the right access and when the right access is revoked, they won't be watching those repos anymore, but it can be solved with two links so like LR on those things, the two link is the most important thing.

B

Yeah, absolutely and probably like I would guess that, like we are not only open source project that that's these kind of questions, probably there is already like other services that extend github to have this kind of user management models right now we are just in this situation where, like sometimes the way to add someone to have push a push.

B

Access to repo was adding them to an admin repo and in minting, which would give them admin permissions to every single people, and there is not like a like a sane, secure, safe way to actually distribute permissions. So what we might do like in the following weeks is reduced.

B

People like change people to like take people out of the admin team and I put them on like contributor teams, like JavaScript team, go team and so on, and then once we find the tooling or once we develop the tooling, then I get a more clean process of understanding. Without coding. For six months, sending an email say: hey I was going like notifying. We are going to like remove the right permissions and then do that cool effect.

B

Nothing important to do before reducing someone's admin permissions is to make sure that we added like in process of removing keys and permissions that are adding his him to like the language specific team. He doesn't lose access to multiple vehicles that are currently not in the team, which is like the state of of multiple vehicles right now, yeah, that's the sofrito just like answering working together. That's like some of the things I had done this weekend was making sure that, like every go or Jess, they want the JavaScript or the go team. So that's like.

B

If we do that, we know that those contributors don't close access. Pre, oh yeah, one.

A

Thing I want to now. The first thing is I believe it would be good. We we have a list of who is right, access losing the app team, etc. We go to open an issue for permission, changes and tag all the users that will be affected by the change. I, don't know a couple of days. You like send out a message like look. This is going to be the change don't be alarmed.

A

This is going to help me if please comment here, if there's some problem with that, the other thing is in addition to the admin, Commission change, probably good. We should also probably safer to enabling protected branches by default for master such that push to Masters and the allowed for admins once and only once. Ci is passing. There's some very nice checks. You can. You can enable which even force on admins to adhere to them so that, for example, if you're in admin, because you have to manage it, you still can't merge unless C is passing.

A

Unless you go to the settings, remove the protection which Sam and you always can be right, but so that you don't accidentally image stuff that that shouldn't have image and you can also enforce at least one- could review acceptance, for example. So at least four major repos I believe this. This should be something we should add going forward. I'm sorry, the no-go ipfs main repo has some brand contention. I do already, but pretty sure that no, yes, yes, don't go.

B

Ahead give us it like mice. Perfection makes mason mainly in marriage and push protections, but problems of CISC ice like the file randomly project.

B

Jessica APIs also has that but I think it's kind of like lie to ourselves. Saying: oh, just a business has that we are protected it and like GS, mu, P other doesn't have and like it is. It is part of the symmetry right like it is the same code that get shipped like.

B

If we do for one you have to do for all of them and I wonder like I, wonder if we try that and say: okay I, just like do master branch protection like everyone has to review at least once the TI might be bro, as Cooper said, because, like sometimes CI acts, weirdly and changes without us, knowing like we don't control it. If it was our CI to be equal, so we can start with our CI.

B

We could exercise like protection for and preview requirement like if we turn that on today, for all the repos and I did good deal, give it a trial period of like two months. Let's see what would be the pinpoints there like if they was going like if there happens to be a pain point, we might just experience like a really good transition and if it's super painful, if someone is like screams, hang on my god, I cannot compute anymore, because, like we protected a syringe, then we have to solve that situation.

B

If that happens this way, I we get to try it and we get to mess it up. What let's? Let's move it to any share and discuss there like we can we can then we can first thing is: go something and then gain opinion of others, also people that are not in the school. That's good.

A

Alright, so, okay, thank you. David for Polly security, things all right, I anybody have any agenda items that they want to talk about, or we are anybody remembered something well. We.

C

Can plug that there will be a demo of a lot of this data rescue stuff. The data together stuff we've been working on, but it won't be this week, but we have some people on the call today who have been working on it. So you can.

E

Let them say hello, give me guys 60 friend and then Don I will welcome.

F

Also I would add a little bit on the SQL data source stuff. We've been working on some work on the go side of that. The jump in on that issue in forward to contributing cool. Thank.

A

You alright do we have any demos done today. Anybody nobody.

A

Alright I guess we have to we.

C

Have made a nose last week so yeah, it seems like this. The demo is the best time of their like scheduled, like we invite someone to give a demo and they have like a week or two to prepare they like gives or Emma. It seems to go well, I mean having phase for people to show like I hate. Here's something I worked on this week is good I.

A

Hope I'll have a demo next week, but not today, yet yeah.

B

I think there will be two demos next week, because there were two demos scheduled for this week that code before sorry.

D

Next week we will have something for achieving an IPSS uma. Vez has like a ton of work on the resulting of the different objects are actually like.

B

Literally, that was like one of them and I know like a lot. I was working till 2:00 a.m. yesterday very much already Japan.

D

Was presenting today next week, it goes.

B

Exactly nice yeah, like I, think that the demos also like remind people of like previewing all things are integrated, like the last time.

B

I said this way last week and the demo that we were going to add this this week and now got postponed to next week already review like some some issues that people are solving because they're to a demo, so I I, guess that's good, like it's good at like we, we keep the ball rolling and like we keep inviting people, because it helps for our people that don't want to try those those tools to to make sure that, like they can ask for run them run it by themselves.

A

All right, thank you. Everyone believe that concludes our agenda.

A

I'll, stop recording them.