Filecoin Filecoin Liftoff - the Main Event, 22 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Filecoin Protocol Implementations: What’s Next?

Description

The Filecoin Distributed Storage Network is an open specification with numerous implementations. Today we’ll hear about the status and development of three of these implementations: Fuhon, Venus, and Forest.

Join us for Filecoin Liftoff Week, an action-packed series of talks, workshops, and panels curated by the web3 community to celebrate the Filecoin mainnet launch and chart the network’s future.
https://liftoff.filecoin.io/

Events take place all week, October 19-23, 2020. #FilecoinLiftoff

For more information on Filecoin
- visit the project website: https://filecoin.io/
- or follow Filecoin on Twitter: https://twitter.com/Filecoin

Get Filecoin community news and announcements in your inbox, monthly: http://eepurl.com/gbfn1n

Chapters
00:00 Introducing Ales + Project Fuhon
07:31 Introducing Steven Li, IPFS-Force + Venus
17:02 Introducing Austin Abell + Forest
26:14 Lotus Update from Aayush Rajasekaran

A

Hello, everyone, uh my name is uh alex, I'm a city at saramitso, I'm replacing makoto takimiya, he's a ceo at suramitso. Unfortunately, he was not able to join at the very last minute.

A

I will be describing project fujon, which is a c plus plus implementation of the protocol.

A

So before we start, maybe just uh okay having some problems with changing the slides, uh a few things about sara mitsu, ceramics, japanese company, we uh initially developed hyper luxury well blockchain platform, eroja that is now known under the hype, legendary roja. We are still uh main contributors to that. Then we also developed the first cbdc solution for national bank in cambodia.

A

That is also in production uh and build a platform uh d3 ledger uh that supports uh digital assets and uh recently we are also working obviously for the file coin, also for polka dot and building the centralized exchange and the wallet. That's named fearless.

A

uh Regarding the fujon, as I said, it's a c plus c plus plus implementation of the falcon protocol, it started in 2019. uh It was completed in q3 2020, so uh not not far ago, but now we are, as you will see later on. uh We are finalizing some things. We are working on testing, improving certain things and focusing also what will be in the future.

A

um The idea is that this is more mac, os and unix oriented, but of course you can try to use this c plus plus implementation on any other platform and try to run a node on the right side of this slide. You can also see our aha board that is showing different phases after we started.

A

Initially we were researching the uh specification and then we were developing um c plus plus implementation uh over uh that period, and we were having with protocol labs. We were having regular meetings every uh two weeks, uh reporting about the progress and uh getting their instructions and also they're very helpful, all the time uh having additional meetings to explain technical questions before we continue with the implementation.

A

So what is the current status of the foohan project? So we have a node in minor compatible with lotus version 0.5. So now we are catching up with the latest version that is available. I think it's 0.9 and a storage in the retrieval market are uh completed.

A

uh We also received, which was a very, very helpful uh uh conformance test suite so which has a number of tests, as you can see here, uh all together 577 tests, and we are very happy to announce now that, uh just from this week, uh all the tests are are passing. So it seems that our uh implementation is uh without any uh visible problems.

A

um Also, all required transfer layers or security protocols uh are implemented. So this is a tls noise and security. I o that we are also supporting in c plus plus, um so what is still in progress, as I mentioned before, we are working on some compatibility uh issues with mainnet, because the mainnet is a little bit upfront of our version that I mentioned before 0.5. So we are catching up with this um and uh then we will also verify whether the blocks that we are producing is also compatible with the latest lotus version.

A

So everything works together without any problems, and then we need to integrate and also perform security testing to make sure that there are no no problems in this particular implementation.

A

That means in c plus, plus implementation.

A

So, what's next, uh one of the ideas, of course, having an alternative implementation of filecoin in c plus plus, is also to potentially make it faster uh in c, plus plus, at least under certain conditions. So we will be working on performance improvements and then um certain owners of the nodes might choose c, plus plus in the future or some other implementations.

A

We will see how that will go and everyone will be able to test this in the future and decide whether this is uh in a certain case is more uh appropriate than uh some other uh implementations of the the firecon protocol.

A

And then the next improvements that we are working on is uh improvements of the command line, interface, uh adding some documentation and providing uh some examples of how to use and run c plus, plus uh implementation of the file coin.

A

uh After we complete all this, which we believe it will be quite soon so, probably this month um or beginning of the next month, uh then we will be following uh all the changes, uh all the upgrades and future versions, uh also in c plus plus obviously- and we were already exploring some new ideas. uh I would just like to say that this was not discussed with the protocol labs or with the community, but we would like to do this in the future, how we can expand what is offered now.

A

So one of the ideas is to also support nfts, uh for example, for polka dot. We were already discussing some uh use cases when, where you can integrate uh filecoin with some other networks, I will not mention the name, but it was one of the very uh well-known and major uh blockchain networks that are existing, uh but since we did not get approval for this or on both sides, uh I I will not mention the network, but it's very interesting for the future.

A

um So I guess that's more or less all about the fujon about uh our work. If there are any questions, uh we are happy to answer them.

B

A

B

You everyone, my name is stephen, I'm from ipve force, um I'm the co-founder and cto of the activist. As you all know, the fpv sports located in shanghai, china and we have been working on the ipvs and falcon for about three years and focusing on to build up the ecosystem, and uh in these two years we are focusing on file code mining and to build up some other services and applications.

B

um So now uh we're taking over the new falcon to build up um another implementations, like notice so um yeah before I show our next step, and you want to talk something about how we um yeah transfer the name to falcon to venus. um So we live it as well, as is because available is the first style we can say every morning on earth right and as good falcon is. The first uh is the first implementation of um falcon protocol.

B

um So we want to keep that uh and also um we think that uh the goof um okay, the cool falcon and the system is actually for the um yeah for uh interplanetary uh yeah system is like, um for example, like the yeah uh yeah, like the solar system, and the vlas is one of them. Yeah one planet. Planet of the whole system, so yeah, it's a part of the ecosystem. So that's why we use this name venus.

B

We pick up this project from mid of september, which is last month and okay in the last month passed. um We have done something and yeah. Firstly, we want to say that um the falcon cool falcon actually the development was was post in june.

B

um They stopped upgrading and the whole falcon team actually focusing on the lotus and also the speak and upgraded.

B

So after that um and actually in need of june, I think the good falcon hot west to be able to interoperate with the lotus.

B

But after that the builders had been um involved a lot, and there are many many kind of cool change and the speak also changed and and some others, for example, the gas and gas mechanism and the speed after and change, and also data, yeah data structure, change and all this.

B

So when we pick up the good falcon, we we have done something um to make it um um be able to interoperate with doctors, and what we have done is like yeah, for example, all of the package dependency, upgrading and also the data structure change, and to follow up the the perfect change and the message handling the gas mechanism.

B

All this, and also we need to have the folks um so based on that, um I would like to yeah I'm very glad to say that um very nice right now can sync with notice to the height twenty thousand okay yeah. There is also a long way to go, but we we have done something to make it, and you know to this head this height is: um we are continuing to work on this. We currently we did. We we. Currently we do not use the test vector. We we just use the the.

C

B

The the space swiss chain to directly and to sync with the chain and to find any problem and yeah and to fix them um yeah the team here my team thinks that yeah, that is more direct and perhaps more efficient,.

B

To implementations and yeah, inter an operating wheel and we'll continue to this, we hope uh in two weeks we could have uh the two impressive two implementations could work together um so yeah.

B

If we look forward, we want and is because uh venus and lotus actually are all um uh all written in google and right the same language. So we want to have some difference between them and to make the venus to be suitable. For some people to use we define venus is like a mining portfolio.

B

Implementation and the big feature want to support is to support disability, demanding pool and yeah for achieving this. We will have um kind of different, an architecture and then notice, so we want to decouple um many things we want, um which can make this whole system uh very flexible and, for example, we yeah. We. We have the chain sinker as a independent component, and we could have a chain sinker and yeah.

B

We could have a cluster chest anchor yeah and, as you can know, that uh well in our um is yeah many many pool we we cannot only have one um changing load is because we need to have some redundancy.

B

While we have this redundancy, we could have multiple changing load to do the same replication, and actually they can also do no balance right. So we want to do that and to have the chance and team more efficiency, and we also want to separate the messenger and to be independent to make sure that all the messages can be unchained and we could have the secret manager and we'll have the seating process separated out of the yeah, the chainsaw king, and we have the proposed um separate and we could have the gpu outsourcing implemented.

B

So we could have different components and to which enable miners to build up their different kind of clusters or disputed many pool, and also um with current lotus implantation. I think that they say I, inter and yes, the common line.

B

Interface. Actually, is that so um yeah, because uh the camera interface uh separated the notice on lotus minor and some other components. We we want to actually have this kind of um falcon client uh interface, um yeah, very, very small, and an added one to have all this, to link to the synchrony messenger, to make all this kind of command or message clemson out. um Yeah and- and another thing was thinking about- is about the storage we want to the. Currently.

B

We have the file system storage uh to build in the notice, um but for the falcon system we will think that object. Storage will be more appropriate or more suitable for the falcon emulations, so um yeah. That is also one option.

B

We want to do yeah there is that, and naturally it's a long way to go so the firstly we're focusing on the chain center and while the chain center is okay, we may use some part of the delta's components to um interpret with the yeah with venus system, and we can build all the components one by one.

B

D

B

That's all yeah. Thank you. um Any questions and yeah very welcome.

C

My name is uh austin, I'm uh the tech lead of the forest project, which is basically a rust implementation of the uh falcoin uh protocol.

C

um As you said, um chain save works on a few different ecosystems, falcoin ethereum, one ethereum, two polka dot, cosmos, building, ethernet, uh etc, and also we are hiring so specifically also uh rust developers improve that mode, actually, uh specifically rust developers um so yeah. I just wanted to give a quick note about that. So yeah, why are we building rust? So um the reason for rust is primarily for safety.

C

Rust has a lot of unique features in a very robust type system. So the main the main things that kind of provide benefits to the our implementation are pointers and immutability are checked at compile time, which means that there's uh no null pointers, no double freeze, no dangling pointers or use after free, which may not mean a lot to someone who isn't a developer, but just know these are very kind of common problems and kind of opens up to security vulnerabilities.

C

um Also, there's no data races, even with concurrent programs. um The type system in russ kind of ensures that, if anything is sent over any thread boundaries uh that it has like atomic counting for any references or uh any sync primitives that are necessary, um other is no seg faults and no buffer overflows.

C

um Again, all these, like kind of memory, unsafe issues, may not seem like a big deal for a lot of people, but, for example, chrome and microsoft. Kind of did an analysis on the on the security vulnerabilities that they encountered, and it was around. 70 of them were memory on safe issues.

C

So this kind of just like avoids that all together um and also the benefit for us writing this in rus was that we were able to find um you know bugs with you know existing implementation and kind of find a lot of these issues at compile time, instead of finding them at runtime, which is kind of you know strange for alternative implementation.

C

uh So the next thing so also we we want russ because of performance. It's a compiled language. It compiles the machine code through llvm, uh there's no garbage collection or run or run time, which means there's no gc pauses, no memory overhead, um there's zero cost abstraction, which is kind of a buzzword, but it just basically means it's all going to optimize any unnecessary code uh efficient generics, which means um it's basically going to optimize if there was no generics and avoids dynamic dispatch and then also the main benefit.

C

I guess in this use case is that uh we interact directly to the rust filled proofs and we don't. We don't have to go through an ffi uh which is kind of you know a benefit um for sure, so, basically to kind of explain what we're what we're building um this is kind of like an extremely simplified version of the kind of architecture. That's set up um where you have the you know, the lotus node the markets built in and then the api which the lotus miner connects to.

C

Basically, what we're doing is we're basically swapping out the node for a force node uh implementing our own apc rpc api separating the markets, uh implement implementation and then also, basically, just allowing you to use the existing lotus miner, which will in future be replaced with uh our own native implementation.

C

So basically, what have we done so far? So basically, what we have done so far is all the ipld implementations which includes the hampton implementations, which are used all throughout actors. um Basically, uh our vm and actors, rust implementation up to network version, four, um currently we're the uh right. Now, all the other implementations are kind of relying on the uh go implementation of spec actors, so it kind of provides some.

C

You know network resilience by having this in another implementation um and through this implementation we are almost back in format, we're about we're more than 95 passing uh there's about a thousand testable vectors for us right now. uh There is more but right now, because we're testing only specific, because we can't test uh network version four and five. uh It's limited to about a thousand test vectors um and the next things we kind of have is the syncing and networking. So we have all the sync protocols implemented and we have full verification of blocks.

C

um The only thing we kind of the only reason. That's the only thing, that's kind of stopping us from doing a full sync of mainnet is just like a small state discrepancy, um but we're solving that right now um and obviously the json rpc api, which is intended to be compatible completely with uh lotus in the in the future. Right now we're only implementing what we need, but yeah. This will be expanded.

C

All right, so, basically, what are our goals right now? So our goals, at least for the near fut like for the past I'd, say about a month or so has been generally interop. um There is a lot of code within the vm and actors that has to be like very meticulously matched. um So our goal for this is passing all conformance vectors, we're getting extremely close right now. They're, just very small discrepancies. um Next is obviously syncing.

C

Mainnet tip sets doing the upgrade uh the v2 actors upgrade and then also being able to like sync mainnet completely. uh This is by far our primary goal, uh we're definitely getting close um and we'll obviously update uh whenever we make. You know big progress in this in this um and then our kind of secondary goals, but definitely still in progress, um are the lotus minor integration.

C

So we should have this soon as well, um basically allows you to produce blocks using our the forest node as as, like the uh the base node, um and then allows us to like run internal test nets and actually like verify the functionality, um but we are completely interrupting with lotus as of now. So um next is markets integration.

C

So uh the integration we're doing is through go fill markets interface, it's a it's another project that chainsafe kind of did, which separates the market's implementation outside of uh outside of lotus, to be able to be used by any implementation, um and obviously the goal for this is having uh storage and retrieval conformance um and then finally, our like, I guess other goal to this- is json rpc compatibility, so basically allowing it to having our api set up such that.

C

If you swap out a lotus node for our node, you won't be able to tell the difference between the two. That is definitely our goal and hopefully we'll get to that soon.

C

And uh just to kind of talk about future work, uh it's nothing's def like nothing's, really set in stone because it's very up in the air- and we have been kind of very focused on um very focused on interop for the past uh I'd say a month or so, but just to kind of give an idea of what we're kind of thinking about.

C

We obviously do want to have a full rust implementation of the storage minor markets just to be able to have like optimizations and then have the ux improvement of not having to you, know, build and then run other other code bases, um and then the next is obviously researching and refactoring architecture based on client needs just to be able to do kind of more intricate or unique um architectures, for maybe different client usages, this one's kind of, if anyone does have anything that they do want supported like we do, want to kind of obviously take in node operators, um opinions and kind of wishes for this.

C

So if anyone kind of has any suggestions, we would love to hear it, because we will be going through this kind of refactor once we hit interrupt and then kind of another thing, that's kind of more of an interesting point. uh I guess more personally interesting to me is- is kind of compiling actors to wasm, so uh rust does have great wasm compilation support, um and this would be very useful in the case of having, as I said before, all every other implementation is relying on the same code for the vm state transition.

C

um So it would be nice to kind of have an alternative. So if you do still want to use a lotus node but want to use, say the rust implementation of actors, you could swap it out if there is any kind of security concern or you know anything else like that and then uh yeah. I would also just like allow other implementations to kind of share this logic instead of everyone either having to re-implement it themselves or go through an ffi or c go, and that's pretty much it.

C

um I I don't know if we're taking questions right now, but I'm kind of happy to answer any, uh but my you know get out there as well.

D

Obviously, lotus is what's running uh by all of the nodes on the network uh right now at least uh except for folks who have written their own private implementations.

D

uh So I'd like to talk about uh about what we have coming up next um in in lotusland uh yeah version 1.0.0 is out now and that really just flags the code that we launched mainnet with um the code that we that was running for our liftoff epoch, uh we actually just put out 1.1.0, which is the first uh kind of mandatory release of lotus um it'll it'll, be implementing uh a fip that a lot of miners have been very, very excited about, um and uh the current network upgrade.

D

The network upgrady park has been set for thursday evening pst time so maybe early friday morning um in some parts of the world. uh So that's uh that's kind of been our immediate focus. uh Apart from that kind of work, splits into two categories, there's there's work that we're doing in the core protocol itself.

D

um Obviously, given that we've just launched, uh we think there's there's a lot of attention on filecoin right now and we are kind of in a uh in a state where we're like really monitoring the network, making sure everything is working as intended, making sure no rules are being violated and so on. So if there are any issues we want to be, we want to be. We want to act on them as quickly as possible with uh and fix them in in releases of lotus.

D

um We're also always talking about uh improvements to the protocol improvements to the specs actors code. The actors code is kind of uh where all of the uh pre-compiled contracts in file coin world live um as well as uh sips, which are five one improvement.

D

Proposals uh that we may want to to include the cadence that we're looking at is around one network upgrade release a month, but uh especially in these early days, we might, we might have to move a little bit faster than that, and I just want to say we really appreciate uh the cooperation from miners and our various other integration partners uh to to move as quickly as we need them to um it's.

D

It's critical to the network to the security of the network, uh an outside of protocol stuff, there's also a whole bunch of improvements that we're planning for kind of core lotus itself. um uh One of our engineers, magic who's, apparently faced with a lot of y'all, has been working really hard on a major refactor and improvement to the core mining logic. um Some of this has already landed and we're going to be kind of carving it up and landing it in lotus.

D

Over the next couple of weeks, we have improvements to the produce libraries coming that will hopefully speed things up and make those uh work a lot uh better that we're hoping to land soon uh and then, as anyone who's been on the lotus, github repo will notice. We have a lot of open issues uh submitted by users, miners community members. uh We welcome all of them. Please keep them coming and we're excited to kind of work on some of the maybe smaller bugs or feature requests that we've had from there.