IPFS GUI and Browsers Weekly, 9 Oct 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Browser updates, offline WebUI and lazy cohosting - IPFS GUI and Browsers Weekly, 2019-10-09

Description

Browser updates blogpost: https://blog.ipfs.io/2019-10-08-ipfs-browsers-update/
About IPFS GUI and Browsers Weekly: https://github.com/ipfs/team-mgmt/issues/790
IPFS Mirror: https://ipfs.io/ipfs/bafybeihlchdfctzhahapwwzviyswrk2jayczdmhbirc6gydnf7ax4m2e5e/

A

Welcome to GUI and in web browsers, we click all four 9th of October 2019 I'll be leading this call and we have an agenda if anyone present on the call we got the trip we got Henrique.

A

If anyone wants to join next week, three free to join, we are back so we got an agenda and I'll share the agenda.

A

The first item on the list is pre-caching assets, making web UI work in offline mode, so that one is mine. So I'll give a quick update on this one. So the idea is when user installs ipfs companion the first thing they want to do, especially in the next release. There will be a button when everything is set up. There will be a button open web UI. So that's the very first thing new user is doing opening web UI. The problem is if the web, if it's the very first time user, is opening web UI.

A

The web UI is not in the local repo. The web UI is fetched from ipfs and needs to be loaded from the network itself. That's pretty good for like dogfooding and stuff. However, it's not that good.

A

For user experience, and also not that good for offline use cases, so, for example, if someone goes to to a cafe or if internet cafe or a friend where there's a Wi-Fi and they install ipfs companion and like a Kieffer's desktop and they shut the laptop down and then go someplace without the networking and they want to play with ipfs, they try to open web UI and nothing happens because the note is not able to find those blocks. So it's also not connected to the network and it's not able to open web UI.

A

So that's pretty bad for the offline use case and the way I figured out to mitigate this is to basically precache web UI in a way that's native to ipfs. So the idea is to basically bundle an archive with web web UI assets with the browser extension itself and then on the very first time user is starting I profess companion. We check if web UI is present in local repository.

A

If it's not present in local repository, we check if the tower archive is present in the bundle, because not we may not want to bundle it on every platform. We need to figure it out if it's okay to bundle it Firefox and other stuff. But if the web UI is not in local repository, we have it locally as a file, and we just import that file to local note on the first boat and that's pretty fast. It's like the uncompressed web UI.

A

It is about 22 megabytes, but compressed is 11 megabytes with companion itself, so companion on its own is 4 megabytes. If we add web UI, it's 11 compressed and it's pretty cool. The pretty cool thing is that what we include in the bundle is at our archive, which is an archive which is not compressed itself and companion, and whether you I share a lot of assets such like phones and some, like some images and other assets, so those are basically duplicated.

A

That's why we got this pretty good compression and I feel like 11 megabytes for chromium extension is not that bad, not that good either. But you get this offline first experience so that's implemented and we will land at some point in the future, pretty happy with it, because it's not specific to a DUI. We this way we are able to be. We may even like extract this and create a library for doing that in other apps.

A

I know that in ifs desktop we sort of have web UI shipped locally, because there are different constraints, but for browser, extensions and other applications that wants to ship some data that needs to be available immediately without fetching it from the network. This may be useful and in ipfs companion this means when you open web UI, it will be available instantly, so the first load won't take a long time right now it can be a few seconds of even up to a minute if your network is slow and you are not lucky.

A

So that's a pre-caching assets and the next item is experimenting collaborative website hosting I'll- probably move that after update all ipfs in web browsers, so the trick do you want to take this one sure.

B

So, thank you very much Lytle for helping with the ipfs and web browsers blog post, which I will share here momentarily I, have really a lot of browser windows up. So if this is the right one.

C

B

Ya confessed for advisors update, so we put together a summary of kind of where we've been with browser integrations and where we want get to, but more like a hint where we get to most of its talking where we are today integrations with brave and the work that everyone's been doing. Lydell driving that for the last almost are over a year now, at this point, so really the culmination of really a lot, a lot of work and a lot of help from Gravesend as well.

B

It's so nice to have a browser, they're so willing to work with us and experiment and and exposing capabilities and figuring out what really needs to happen to be able to make distributed network entirely inside of the browser and then also the kind of first public announcement of us working with opera and opera planning on shipping ipfs support with the full addressing spec in Opera for Android by the end of this year.

B

So that should should should be interesting and I talked to to up to Ollie this morning, a little bit about getting the bootstrap servers up and ready for that. At some point.

B

If we has seen boost in traffic from from both of these things happening and better awareness around them as well, you know talking a little bit about the stages like where we want to get to eventually, but there and some of the background history and why it's hard to change browsers, it's very cool to be able to have all of the videos available from the various IP Festivus were where people actually talked about this kind of stuff.

B

All that going back all the way to uh to last summer, when we have Idol and Kyle definitely like I, think that was actually only above, that might admit a month or something after Lindy. Webb was most fully announced, really a scroll back through the history and a hint of where we want to go, and one of the things that we didn't really talk about in this post and then I feel like I'd love to be able to talk about more once we get into to 2020 planning. Are things like out? Co-Hosting is really big.

B

Co-Hosting really speaks to kind of the core use cases that we talked about when talking about why IP fess is good and why we were doing this in the first place and and it's a kind of our first at first serious foray into figure out what those user flows are for. I'm, a regular person and I want to be able to share I wanna, be able to see if something I also want to share it. What does that mean? How do I do it?

B

So I'm really I feel like with browser integration, we're getting things to the point where regular people can start to have access to distributed, networked without having to install extra stuff necessarily, but that next step is even more important and that's really working out the kinks and those types of workflows. If I view Wikipedia.

B

How can I make sure that my neighbor can also be Wikipedia if all of our internet goes out right right, then, so those that I think the talking a little bit about that that next view what comes next, once we have network capability? What comes next steps? We smooth out the bumps for those those really common use cases how to share how to publish how to read what I'm saved and a lot of the intricacies of where's.

B

My stuff I've got multiple repos, multiple devices and web browser extension and web content and mobiles about you know like up other local going, P, fest, node, I'm. Think there's a lot of work to do is smooth out the bumps, but this is a big progress and then so far, the reaction on on the Twitter's been pretty good and I got retweeted by ipfs bot, so that would that means it's official. Let's legit and I'm really getting a lot of positive feedback. People are pretty excited and, interestingly, a bunch of projects talking about mdns.

B

So, like I, think those were its. If you, if you work in the space long enough, you have an understanding that that local connectivity is one of like it's like that. That Holy Grail that cornerstone of two devices be able to talk to each other and and discover each other and be able to interact whether it's IP, fests or any other other distributed network, and especially in offline scenarios, so that kind of, like people, people are excited, be able to see that we're.

B

Looking at that and I think once we have some really good demos and development scenarios like API somewhere close for browser to browser browser, communication, I think we're gonna see even more interest on what we're seeing right now. So the interest that we're seeing right now is people have that kind of longer efficient, but once you make local network pressor browser development a little bit easier in the simple publishing we're close like it's going to be, it's going to be really I'm, not a lot of a lot of abilities for people.

B

So that's pretty much it thick things for again for for helping with that, like you are a a font of history, and you had all these great links to all these videos and resources and all the way issues. Yeah. Thank.

A

You for creating this summary, it's a very good overview and I, really like that's it's so much stuff it the way it balances like over overarching roadmap and where we are today without going to too deep into technical details. That's super useful, because otherwise, if it was too technical, people will just shut off and it was very nice with I I enjoyed it. Thank you we'll.

B

Get there oh I I do have some more I actually have some diet. Some, like visualizations I, did to that kind of model. What the different permutations and configurations of inside brave are that I didn't actually put the post until it forgot what we can save those for a.

C

Future, it's a detailed technical.

B

Post about what you did in brave and on a crazy, crazy, high jinks like no really we're writing a web server inside of a web extension.

A

The next the next one it is q400 quieres. Yes,.

B

So I that kind of that, but that blog post actually is a lot of what we wanted to do like that's. The p0 for kind of for the web browsers group anyway for 4q for the Opera is gonna ship and we're gonna help as much as we can. With that we're gonna be able to communicate those things brave.

B

We talked about like what a brave 1v1 looks like at browser week, which I'll talk about next last last week when we or we prefer last one we're in Malta but um and we we actually did narrow down a set of cases, their LinkedIn me that's what we notice about what that v1 might look like I think, but but ultimately, like the fact that it's available in the way that it is especially now that the sockets are white listed in release rate, which I didn't realize. That's awesome.

B

I we're standing on pretty solid grounds to be able to have like brave, isn't great, I professing brave, isn't thing that exists and, and people can use today. I.

A

I really like the way, this sort of, like a soft launch of experimental, embedded note with a gateway that people can see over time, how like performance, improves and other things and the v1 sort of if I was the end user.

A

For me, the the v1 is when we got like ipfs colon, slash, slash indication bar- that's like probably trivial, but that's like this cherry on top, when you like, say: oh, it's probably done right, but until until we get protocol handler API or like this better ux of address bar, we will iterate on the performance of method. Note which I'm pretty happy is.

B

There anything that is there anything on our end that we can do to to help the development of protocol handler new protocol handler and brave, like is that something that's happening at the brave level or is that required underlying chromium changes on there? And this argument, for maybe us doing more direct browser development? Are that.

A

That would require checking in with the person that will be, but there are two ways: one is sort of like just visual, just tackling the visual side, so basically having HTTP connection to HTTP gateway on the backend.

A

However, on the user interface just change your address bar and that something couldn't be easily done with like a smaller patch, however, like introducing something on par with native protocol handler for browser extension, when you actually have no like HTTP connection, you just inject bytes right from from JavaScript, that's something closer to chromium and also we want it to be closer to chromium past. It's not going that way.

A

It could be reused in other places and having the same web extension API between like Firefox and chromium, would be a dream right and yeah so like right on Oprah, and it's like, like as a segue I'm working on addressing some bugs related to adding files from web UI in brave when using embedded, Jacek TFS. Sorry, it's bit technical, but long story short, is that we there was a bug which I found, but we did not have tests. So that's very fortunate that we have testing with. You is one of our key ours.

B

Yes, so that is the next segue. Thank you very much, I wait! So what we have two acres that are really kind of related to this group for q4 and the first one is the opera brave and the second one is to improve the intent testing scenario, and we talked a little bit about this last quarter. I, you know hoc.

B

You started some of the some of the testing work there and and then Lydell out of this web UI testing matrix and one of the things I actually think that the priority is probably flipped here, like the browser. Stuff is p0 for q4 and the testing is pp1, I think, but I actually think the testing bit.

B

It should be our our p0, the the fact that go has somehow to release in a while javascript is having ready, really regular releases, but neither of those core implementations have their end to end testing fully set up and running makes it even more important that we have our own and and testing set up.

B

So, in you know, as as those core implementations are making big changes, we need to be able to know immediately when web, UI, companion or desktop all break because of a change that happened from integrated core implementation or some module dependency down there in the infinite tree of modules that we depend on. So that's why I think like for me the from a instead of you know we don't want to take you know one step forward and two steps back are getting this test suite set up even with this minimal.

B

You know matrix that that Lytle, you put here, really is probably the most important thing that we need to finish this quarter, so I'd love to be able to know how what what do? What can I do to be able to help this? What do you need from an infrastructure standpoint? How much of this work is already done? It just needs to be put together and put it in the CI. Somehow do we want to have do we need some daily operation? Do we need per commit operation of execution of these tests?

B

How much of this is blocking on and say, infrastructure and test performance improvements like what he does working on? It's a nice. It's a nice small table but I feel like there's a whole lot of stuff wrapped up in there. Yeah.

A

So generally, just like I said there are different dimensions to this. I think that the starting point I, think we sort of agree is that web UI should be a home like the web. Ui repo should be a home for those tests, so basically, whether you I would test itself against chromium, Firefox and electron, and then testing itself against like go ipfs over HTTP api, JS ipfs over HTTP api and then like stuff like embedded JS ipfs in brave. It's something I could add on top of that.

A

But we need this like lower-level infrastructure for having these test matrix, and if we have this test matrix. This test should set up in ipfs web UI. It's why it's important it's important because, for example, recently J's ipfs has an early adopter program and the way it works they enable other projects, maybe I.

D

Or maybe it was named something else.

B

Early tester Iridessa.

D

B

Yeah I saw some people submitted PRS to be able to add yeah.

A

So it's if anyone is interested in joining it's in Guiseppe for three point: Docs, that's a program and basically the idea is, you can join the you can join and your project will be tested against new versions of J's ipfs as a part of CI infrastructure.

A

So that's why going back here? If we have a web UI having self-contained tests for this, then we could just submit to that program and not only having like web UI testing itself. Then, if Jarius ipfs makes a change, it would run web UI tests against a new version of trace ipfs, and that would be a very like a canary in a coal mine. In sum, if something goes wrong and similar I believe the same like process could be then be used for, go ipfs, cuz, that's a pretty pretty elegant solution.

A

When you have the elect project testing itself, and then you just have other project testing itself against that project by just bumping its own version.

A

So that's where my mind is around those tests and I would be happy to to help with setting it up for cost like chromium, Firefox I need that for aquifers companion and like electron and Ricans, that for aquifers desktop so will, but like entire framework. It's a common for all cases. So as soon as I'm, like done with this bug, I'm working on four brave, probably jump into this and help with this.

A

And the thoughts on testing, because I'm very far, I'm very, like laser, focused on just that. Having minimum test of like adding file and I think like a directory. And then we can like yeah.

B

Iii think that's the right approach. We get all the environments covered, we get the CIA running I'm, going out with right cadence for the CIS and then, after that we can add more coverage like getting that getting framework up and running is I. Think the biggest piece.

A

Cool and I think I'm testing, or do we move to what.

B

What what's your big? What's your biggest concern about? What's the biggest risk in this what's gonna be the hardest part, because you've already set up a bunch of desktop testing and stuff before I.

C

When I pick this up, I'd like to do more instrument tests, but I can't because test menus and that's basically what I get has SOPs so for what do I I think the main issue might be uploading files, I'm, not sure if it's really easy to do in twin tests, simulating we are uploading, something by tracking dropping or even clicking on I think it would be easier to click on the key buttons, but I think I search a little bit for about this, and it would not be really easy to do.

C

But I can take a look see if that's true or not yeah.

B

You find, even if you just simulated it right, like I I, would say, spend less time in trying to get the UI bit done, then getting the whole flow done, even if you're actually just like halves, JavaScript injected into the web, UI page that just hey I'm in test mode upload this this file, let's make file and then later we can figure out event simulation and stuff like that.

A

It was like for additional context historically I, remember when stuff broke in the collection. It was not usually not related to like user interface interaction. It was mostly stuff like Oh URL object is kind of different than libera browser, or the fetch event is not really the same as in node, nor like in web browser. All those like intricacies were code specific, not UI specific. So that may be a good point. That's.

C

Also, there is something that, if we test web UI on electrum, when we directories, it won't work on a line. It just works if we type if a desktop, because we have a custom function to call an electric function that opens the open directory dialog, it won't work if we just test it against a row. Electrum version yeah.

A

That might be like a local problem: how to sculpt those tests because, like I know, if we really really want to test it like end to end including user interface on like UNIX and when there's like x11 windows like you, can basically like programmatically produce keyboard and mouse events. But that's basically writing at custom testing framework for your eyes and I sort of wanted to believe something like that exists or could be adapted to our needs, but yeah that that's something we'll look at this quarter.

B

Yeah I think the most minimal case that gets it into CI. Then we can figure out the the rest of the wrinkles later because, like I think you're right, like we're gonna we're gonna, keep. We've already experienced this problem with the fact that electrons and isn't many headed beast.

A

All right what workweek.

A

Deitrick, you gathered us for a week, so maybe you say yeah so.

B

I think I'm, let's see if I can find us I, can find that the right tab come on. I know how to browser.

B

C

B

Yes, we took a couple days to do some planning for planning for 2020, and then we we picked it up at him to hack on it and and try to figure out for complete yeah. The key for planning was basically, we already talked about couldn't with the q4 of cara's. Are some of that was actually like reducing the set?

B

So, instead of trying to go big at 1/4, where we have lab week quarter, we have holidays happening and we also have 2020 planning happening, let's narrow the set of things and see how we can help the other parts of the project as well. So that includes things like helping with the async refactor. If we have time and then doing this prioritizing, this testing work as well.

B

The the next big piece was actually kind of identifying what what they, what the set of I mean. We had a lot of stuff to test. There's a big list here, the next big piece that was kind of like what our priorities for 2020 in browser, specifically understanding what the developer experience should be for people who are developing with ipfs. What didn't need a browser implementation would look like, so we were like. We have a pretty good feel now for what it looks like when companion is running inside of a browser.

B

What the constraints are capabilities are what kind of features we need to be able to. We need to be able to run out full note, but a native browser invitation looks really differently.

B

A native browser application likely is not going to be companion, bundle native implementation is probably gonna have to be in C++ or rust, and what that looks like in chromium is kind of TBD in Firefox they they're already shipping large chunks of rust and replacing bits of the browser with that, but I'm not familiar with what that landscape looks like in chromium, we'll, probably indeed, we have an open higher position for a browser, engineer and I.

B

Think at this point, what we really want, because we're gonna get the most browser coverage for many Daria's, be able to get a experienced, chrome, developer, onboard and joined the team and I made that call out in the in the browser blog post as well and then. Finally, the user experience is part of it. So if we want to be able to I could miss running native browsers and integrated, we need to be able to communicate what the threat model is. What does design of the URL bar needs to be and why?

B

What needs to be communicated when the content is well? One cryptographically, verifiable, but also can come from anywhere, and it has ramifications outside of privacy and security, also in the areas of power, consumption, CPA utilization and other things of then power trade-offs as well. So a lot to figure out there. This table is the prioritized list of 2020 projects where we went through each one of the things that we knew where open issues that we need to be able to fix in browsers to be able to get a full up and running IPS node.

B

Now one thing in this list that is not clear is that that that split between what it looks like when we have companion well, it looks like we have JSOC ufs running and web content and then the third, what happens when we have actual native ipfs running so like things like the some of the connectivity issues that we have here are less of an issue for use cases where we're using IP MS natively, but when ipfs is used in a web application running in web content, and you have native ipfs node, we kind of haven't really like gained out what that topology looks like what the interaction looks like and how that application runs is talk to the local node.

B

Doesn't talk to over h to be the local node just talking to try to get native connectivity to the local node. You just use web RTC to speak to eat built-in, like we haven't, really figured that those bits out. So this is the kind of a stack ranked list of things that we identified the from prior prior, a prioritization perspective that we need to fix in order to address the world of ipfs suppressors as we that's the world is today, so assuming that there's no native implementation.

B

This is talking about what web content JSI PFS running in a web page today under an HTTP loaded web page needs in order to be able to connect and the cid v1 basis ready to work is highest priority, because that affects the contents of the repos that affects everybody, who's, adding things today with a with a cid, that's not base their due compatible believe they need to be migrated or we blow it away like we blow away every post, but content today.

B

Another issue that we here have to figure out at some point, but that becomes be the nature of how the data is stored. Cid v1, based r2, is the highest priority for for 2020 in order to set us up for the next. You know you know about five years of ipfs work without having to worry about the the addressing format.

B

Compatibility of the address in product at the next phase that it was interesting, so we didn't actually do these groupings of category like compatibility, connectivity, UX intentionally, but actually we were talking about the projects themselves and that got grouping really ended up clustering of topics and the topping on its own.

B

So that was kind of fascinating that we were kind of I, didn't add the category stuff until later, so we were actually just looking at the list of projects and me and Hugo and Lionel just like intensely for hours talking about each one of these and slowly doing this sort based on the technical bits behind each one of them and afterwards I went through did the group he goes I go. We have.

B

We have a live, a an emergent clustering, it's very important that connectivity makes sense that connectivity is gonna, be the biggest thing in our mind, because everything else we can fix back after that after the back. So that's that's kind of what came out of this. We still have some next steps around.

B

Reconciling this list with the like I feel, like it'll, probably end up being like us, attacking these things, because the HTTP world will live on for the next prolly forevers. Reconciling this against what a native implication looks like how those two interplay together when you have these mixed hybrid environments is which will probably have forever. So we will have a good story for how that works.

B

What the developer workflows are for that what connectivity options happen when you have JSF PFS nodes running in an HTTP web page, but you also have a 82 bed reading the same same browser, we don't want to do things like say, bundling, happy!

B

That's that's! That's good for now, but that's not at the end of 2020. We want to be able to have a better story for for how we actually handled that type of situation. But the idea of a web server running inside a web browser is something that probably is really powerful and will pilot hot.

B

So if we have an even better story for that and then a crazy module stack that isn't designed for the environment necessarily and that pregnant said the entire d web kind of space itself, since I suspect that will be that's kind of like this simplest p2p, louisville local network with to browse talk to each other scenario, that you can think about phrase and my PFS isn't really a level up on top of that in terms of architecture and complexity and feature set, but as a test case for that it's having mdns and a webserver like you've got a direct browser browser connection at that point.

B

So we prove that out. There will float all boats and we are gonna need that, probably for the long term in some form anyway, so that that's kind of what we talked about in terms of 2020 planning, you know like loose loose, looks at what the timeline might look like.

B

Really just blue sky, putting it up, there will be the big pieces and then spend some time digging into some of these. These deeper problems, like we've, spent a bunch time talking about the wave you I, probably brave, and really now the light is almost done implementing that and then some of the other long-term brothers. Like you know, we have I profess. How many ipfs nodes can we write on one machine?

B

Ideally, it would just be one so and and and really that ends up speaking back to the HTTP model as well. So if we really only want one I picked us snowed running on any machine in any given time you we're gonna, be up. We're gonna have to speak HTTP to that node. Even in that that perfect dish fish world, so lots lots of discussions. All the notes are linked in this in this I can be that one of the fun parts was the next couple days where we actually dug into a technical problem.

B

So we said: okay should we say spent two days hacking on web RTC connectivity should we spend two days hacking on. There are other different routing, optimizations and things like that. Should we spend two days hacking on performance and while doing the Interop test, conversion to async Hugo found that the IP of NGS ipfs tests for basic ads where sometimes pathologically slow? He found it was taking a minute sometimes to be able to add, not really file. That should not take a minute to that.

B

So we said: okay, let's take those two days and kind of dig into two ways that we can solve it and I didn't find where the slow time was coming from and it looked like it was IO bound like there was a there's two issues to two major pieces that he investigating, like just.

B

He implemented the datastore API with just a memory database to see if there is a improvement there and it because we use a level to be wrapper running a web page that talks to index DB which actually in Chrome, is also level 2b in the backend. So we have level DB ception there and on firefox it's kind of similar, but it's SQLite on the back end, that's IDB!

B

Actually, they might have a different datastore at this point on the back end, but either way you have a storage wrappers talking to a storage wrapper talking, just storage wrapper talking to storage talking to the disk.

B

So what happens when we remove some of those wrappers than their early or the results were showing a massive improvement, but then, when he, when he, when he talked to IDB natively weren't, seeing the same results, but one thing that we did notice is that when I pick test ads when you're adding a file, it chunks it all up and makes all those rights individual rights to the datastore, those were not wrapped up in a transaction, so it was.

B

Do you making each one of those rights at the IDB level as an individual transaction and IDB provides transaction semantics to be able to take all this right, split, em up and then make the file system the file, IO, happenin and one large transaction was supposed to a whole bunch of different independent atomic rights. This is a lesson that we learned in browser lands and and really the lesson played out here as well once adding transactions before it got about a 50% increase in performance in both ads and and I.

B

Think there was it gets as well, but I don't remember exactly, but the the rights for sure we had a massive promoters increase, so that has not landed yet as far as I know and we're still waiting on ego to merge those changes. But that will be a significant improvement and I profess and webpages hi, and was it also note as well that we get those the batching improvement? Yeah.

A

You know but sinker both personally.

B

A

Really excited for brave, because in brave everything is in the single-page context, so any performance improvement with you can eyeball it that it's faster, so it could be super cool.

B

So that that was the thing we went our separate ways.

A

It was a fun week. Thank you for this, make it happen.

B

Is it yeah? It's it's a little tense. When you get you know it's actually, the small group can be can be kind of more intense than the big group, sometimes because you can go and next time, I think we need to get a little more organized and happy the ipfs web UI and desk the browser desktop and web UI week push it up to maybe five people. Six people I feel like when we had 14 people in Lisbon. It was like two too much all at once, and three people was super intense about six.

B

That's that we keep you that five! Yes, we we just need to figure out what what Hawks testing schedule is at school and make sure that we schedule around so that you can.

A

Join us, especially like some big overarching UX thing across, like companion like web in general and desktop. That's, would be interesting.

A

Cool thank you to check the last item on the agenda. Experiment in collaborative web site hosting lazy vs., prefetched co-hosting I could like probably write more but I, don't think it would be more like it would be clearer. Now, I apologize I was not sure what to write, but it's basically, this co-hosting spec, which we have been low-key, experimenting.

A

Those are explorations around tolls and conventions for co-hosting websites using hem FS, so we want to see what's possible with existing IP eyes and namely amethyst.I without like adding anything new to the IP first core IP eyes, so that dots does the spec and the spec the first draft of the spec on hundred this week.

A

It's very simple: it's just adding removing and updating. Basically, however, from from the implementation hacked it, he created a simple bar script. So if you want to see how this co-hosting would work, there's a shell script, you can download and it just had become a has comments for adding removing and syncing basically and.

A

From that hack noticed that if he maybe I'll go to the spec that maybe easier easier to show, so when you, when you copy stuff to NFS, so here you can see, we are copying.

A

Ipfs resource from ipfs namespace right turn the first under co-hosting directory. So this operation is interesting because we basically just create a pointer in MF s to this ipfs path.

A

The thing is, it's just a pointer: it's there. This operation does not touch data store. So if this harsh represents entire Wikipedia, you have entire Wikipedia in your MF s. However, it's not physically on your MF s. It's not on your machine. You go to web UI, you go to files in web UI or you use command line to traverse this directory under em FS and with each like level. Each file you open there. It will be dynamically loaded from the network because it's not physically on your machine yet so initially it was a bug.

A

So the idea was to okay. We need to add this step here. To ensure contents are in the local repo. So there's this command, which lists or sub blocks of entire dock, which has a side effect of prefetching everything to your local repository. So that is why, like, if we do this before copying, that ensures everything. That's in MF s, it's physically on your machine.

A

Sorry for this long introduction, but I feel it's necessary to understand, what's happening here, so lazy co-hosting, it's the idea of that. It's not actually a bug.

A

It's a feature right and it's a very cool feature because I, don't you might have a small disk and you're not able to store entire Wikipedia, but thanks to this property of MF s of not being proactively like prefetching stuff, you can store pointers to content that you don't have fully, but you may have a subset of Wikipedia right if I go to Wikipedia right now and start going through some pages.

A

Those pages I visit are cached on my local network on my local repository I, don't have entire Wikipedia I have a small subset of Wikipedia and the idea behind LASIK or hosting. Is that that's actually a feature, so some people may not want to store anti Wikipedia. Some people want to just ensure all the Wikipedia resources they visited are being protected.

A

Being garbage collected are pinned or are basically co-hosted, so that's pretty cool I'm able to like host a lot of websites and I, basically just share and provide content to other people. Only the content I've actually visited. It works like this right now. So the only feature on top of how ipfs works right now is this implicit pinning right now when you go to a page, it's saved and cashed in your local repository.

A

However, at any time, garbage collection may remove it unless it's pin, if it's added to MF s, even as this pointer of an entire website, is how that, as a pointer, if you have any sub resources of that in your local repo and garbage collection is happening that will protect them from being garbage collected. So that's that background behind lazy co-hosting. The second issue I linked it's it's, maybe is explained in shorter terms, because I see it fits on my screen, but I felt this explanation was necessary to understand how powerful this is.

A

You are able to like have the entire Wikipedia as a pointer, and you can traverse it and from a web websites like in web browser or from web UI interface, and you progressively get content fetched. So this lazy, co-hosting idea is, is a proposal to like extend or to change or tweak this co-hosting experiment in a way that, like makes use of this powerful, primitive and some exploration, how the user interface would look like.

A

So the challenges are, how do we communicate difference between full snapshot, full co-hosting and this lazy, where I only share what I actually load that my like on my own and then when user wants to go to a Wikipedia page and wants to like host Wikipedia? What does what does that mean?

A

Because Wikipedia is like 40 gigabytes or 16 gigabytes with images I'm, not sure it's like ridiculous and to fudge that if you like start a click, yeah I want to co-host this, and if you start prefetching entire thing, should that be a blocking cooperation or a background operation, and then we don't have API to track progress of this patch. So it's a very an open question and challenges how to communicate with user and how to build those interfaces. I wrote some notes here.

A

Basically, either we I sort of tend to lean on side where we do lazy co-hosting by default, and only if user, like explicitly said, yeah I, won't entire Wikipedia or I want at this entire website physically on my desk. So that's that's much where I am right now, but it needs a lot more work and I'm really happy. Thank you, hug for like crazy, low scripts and working on spec and to helping us pushing this hosting experiment further cause.

A

This is exactly why I started this experiment to like discover those hidden gems in our API and what user experiences could we build around them, because this m FS property of being a pointer, a laser pointer? Was there always? We just did not think in a way how to productize that or how to expose that to user and I really want to remove the spinning interface from ipfs companion. Make it more like seamless and I feel like if compile would do this, like lazy, pinning with like optional switch to fall?

A

That would be much better experience even for for companion, but it could be much more. So there's the discussion with more words, I'll shut up now. Sorry.

B

That's awesome: hey we have in late to ipfs Carlos si. You said that a PRG has coalesced as well. What is the difference between the script that you did it? My.

C

Pfs co-host would be a more beautiful, you know, y-you know. Yet that's something only did before with pin API, but I appeared to change the to match the spec. You.

A

Had also a very good idea of basically turning it into a library like co-hosting, librarian.

B

A

Can embed in their applications and basically in activist companion and in ipfs desktop we would. We would just use this library. So that's also.

A

The shell script is just a super, simple shell script and it just to quickly play with it or add it to cron job, so you can like put it on your server and run it. But, honestly, if you have note there, you could run this IP fresco host as well. It's just easier to iterate. Why.

C

Do we probably you end through this one, your comment five hours ago, but what about the idea of getting new snapshots every X time like twelve hours? If it's a lazy, co-host thing what would happen? That's.

A

That's one of open questions. The way I've been thinking if it's lazy I still want to know when, like the route like the the website got updated and the pointer in my MF s would still get updated. So if I want to go to Anna, fest I would see those snapshots, so it would still be lazy right. We could just the open question is how do we distinguish distinguish yeah? So you what how this prefetching would work? Would you just I want to preferably fetch every website I have co-hosted?

A

No, probably not, but I want prefetch everything apart from Wikipedia or I want to add Wikipedia as a lazy. However, I want to add this other smaller website as a full snapshot, because it's my website I want to co-host it fully. So that's a good question: how do we distinguish it? At the MF s level, we could like add the suffix to a directory name or something I'm, not sure.

A

It's an open, open question both on the like low level, how to represent that difference between lazy and fall on the MF s site, especially and also like in in UX. So if anyone got ideas and.

C

A really simple idea: instead of a suffix or prefix, or something like that, a simple file inside the main name in the king, if it's lazy or full or having one by default and just indicate in other ways.

A

Or we could have a separate, separate sub directories like like yeah, it's probably probably harder to answer is how to represent that in, like companion UI this choice when you want to decide when you want to decide, is it lazy or full and like how to give this UI to a user? Who may not have the all the technical context? So probably we should tell Oh Wikipedia is like 60 gigabytes in size.

A

We need to really.

C

Want do you really want to do yeah.

A

Stuff like that yeah, so that's like where we are with this lexical house thing it's in the discussion phase right now, but it's pretty powerful 25 you and I had discussion with Michelle Hartsville last week on the Kinning generally, how pinning in ipfs ecosystem works and I mentioned this em FS experimentation. We have so probably I hope. We also also bounce some ideas with her to see how we can refine this UX.

D

Yeah, it's interesting.

B

I'm super excited about I think there are higher-level workflows in republishing, sharing, saving archiving and republishing that'll get unlocked by this initial low level, work and experimentation. It is really exciting. Yeah.

A

And also, we sort of start seeing there are problematic or missing KPIs for things like when is this block or is this CID in local repository? Is it fully in this local repository or only a part of it or how many percent of this thing is local? We don't either don't have api's or will you need to chain multiple api's through to get dots? So that's also like super powerful, forcing function on like thinking about those needs and what those things could unlock so expect more on co-hosting in the future. I guess guys.

A

We are over time unless there's anything more to add, I I'm really happy. We are back on track with this call and see you next week. Bye.