Filecoin Filecoin Orbit, 23 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IPLD in WASM for Techies

Description

Where we'll explore what IPLD can bring to Wasm from content addressed Wasm modules to typed APIs over structured, merkle-linked data. A tech-heavy session!

A

Hi, uh so I'm here today to actually talk about ipld and wasm and how we're going to use this to power the fbm at least that's the current plan, uh and also how you can use ipad, ld and volusion together uh in your own systems, and why you might want to do that. uh So this is really going to be about either lean wasm on a blockchain. uh I couldn't resist using the transition there haven't used years, so you have to get ready for that, um but yeah.

A

So if you don't know what ipld is I'm going to give you a short introduction here, this isn't going to be too much, so you might enjoy just more research by yourself.

A

Okay, what is ipld ibld really is abstraction over arbitrary merkle trees, so git is ipld. uh Ipfs is ipld. Filecoin is ipld. It's really any uh mercury data structure uh you can understand as ipld. uh Really I just want you to think trees or hash linked trees of data. um So whenever you think, ideally you just think look. You have a sort of a reference to an object that object. References, other objects when you have these trees of objects.

A

Okay, uh really it's all in the cid that is sort of the magic sauce of ipld. A cid is a content id or a self-describing content id.

A

Or like, ideally, you think of it as a content pointer, um so a cid is actually it's composed of a kodak that describes the cid itself. That says this is a cit, probably a version one cid. uh Then it has a another number. Basically, that says hey uh this: the data that this cid is referring to is of this type and then finally, it has something called multihash, which is just a self-describing hash function or hash. um Now before we can really understand this, uh we have to understand what is kodak in this case.

A

Codex is actually multi-kodak. It's a separate thing. We have um it's just a number: that's all it is it's just a number in a table that is unique uh that we use uniquely identify things. um Okay, what is a multicast because remember yeah, I talked about how this has a it described. Basically, the format is a the codec that describes it. This is the id then something that says what the describes the actual data it's pointing to and then finally, the multi-method, a molding hash is just a hash, but it's again a self-describing hash.

A

uh It's a hash digest. That also includes information telling you what hash functions used to produce the digest. This just means we can upgrade the hash digest later. If we need to, we can also swap out the underlying form of the data later for youtube, but that's the the basics of my pld. If you want to understand more about this, I highly recommend you read the docs or take a tutorial. This is a very short introduction.

A

um Thank you, finally, or second of all, why ipld uh so ipld is a or basically the the primary reason to use it is it's content addressed uh content address means that, uh instead of pointing at it by location, uh you refer to ipl the objects by the content itself. You describe the object in this case using hash function.

A

This means the data is verifiable. uh If you, if you're giving an activity object, I can verify uh that it matches the address of the data they have with the cid uh without having to trust anyone, uh it's immutable uh which makes it really cash friendly. So if you fetch something, you don't have to fetch it again and again and again, just wonder if it's changed, because it can't change.

A

It's also copy on write, which means that if I have some large iq data structure- and I change it- you don't need to transfer the entire data structure again and again, every time small change is made. You only need to transfer these small pieces that have actually changed. uh It's also structured and self-describing.

A

um This uh yeah so like the the structures of the nature, means it won't become tooling. We can have. I build these schemas, which are another form we have for describing data types and stuff like that. We can have standard protocols for transferring this data. We have bitswap and graphsync, and you could have standard languages for querying over this type of data. For example, we have selectors, and these these different things can actually operate over any type of iplp data, so they can operate over data.

A

They can operate over cbor, they can operate over ethereum data. Even it also removes ad-hoc parsing, because again it's all structured. So you just you, don't have to think about parsing, your language and finally, uh the self-describing uh codex. Actually, let's upgrade and change since later like there are always advances in formats people decide they want to use something new in the future.

A

If you just say well, everything's going to be json, then the second something new comes along you're kind of out of luck uh with something like ipld the the code act that describes the format you used is in the cid, so you can just add a new codec define a new format and move on okay. What is wasm? This is very fast, but wasm is a format for portable secure modules.

A

If you're watching this there's a fairly good chance, you guys have introduced to this concept. Really it's a state-of-the-art vm that draws from both the jvm and the lvm. It's also an instruction set, it's also a module format that describes imports and exports of functions, global variables and memories, and all this kind of stuff.

A

It was originally designed for the web with security isolation in mind, um but it was heavily influenced by the needs of blockchains. uh Ethereum is thinking about using the next version. There are a lot of other blockchains that are currently looking into it or actively using it, uh because it's deterministic and a lot of predictions have like built-in gas accounting and that kind of stuff. um Okay, what is ability plus wasm? uh Really? This is just uh wasn't modules encoded as ipld.

A

What this means is, you can have a wasa module that can import other wasp modules by cid content id. um It also means that you have ipld, or you have functions that operate on ipld data that is instead of just taking like an arbitrary set of of um of like numbers or arbitrary data. It would take an input I could retreat and spit out not going to trade um so why? I could see this well one the constant, addressing um uh or contrast code, which means you can easily load code.

A

You can also securely uh use libraries and you have secure code reviews. uh You have content just data. This means that you can lazily load data, your your data, your basically, your memory is network transparent because it's all, ideally these you can just transfer it over our standard, actively transfer verticals. um You could you can get uh easy distributed computations where you only need to load uh removable data, that's changed and you actually need you can do this lazily, and you also have general purpose tooling for actually moving the data around working with it.

A

Finally, because it's all content addressed and immutable, you can memoize results. So if I have the the cid of the content id of the code, the content view the inputs and I've already computed something I can just cache the outputs.

A

Also, you can use data by reference, uh so this means that I can pass a large immutable uh tree to some wasm module or some awesome function um without actually sharing my memory with that. uh With that function, uh copying all this data back and forth, and that was a module, can actually just load the data it needs dynamically as necessary and return a new cid for its new data structure. This means that modules can operate on large data structures without trusting each other.

A

It's also typed and structured, which is just really nice when working with other languages. Okay, why? On a blockchain? Well uh again, it's constant addressed uh code can be loaded and deployed uh many times um uh without having to like re-upload the entire code. You just refer to by cid uh libraries and new contracts are, can be cheaper to deploy um uh because you don't actually have to like re-upload all of your code.

A

You just need to have upload like a little bit of code for your contract, but you can use libraries in our system chain yeah. It increases runtime costs a bit, but this can be alleviated in various ways. I'm not too concerned about that. um uh Second of all, um actually because you can pass around data by reference, uh it a secure delegation.

A

So currently, when you delegate from the contract, you usually say: okay, you can use my memory, do whatever you want um with with a ipld, you can give another function for the contract, forwarder access to a specific id tree, let them compute over it and then get back access or get back the like the new cid, and you can actually verify that they correctly did the operation on this. On this data, um you can also have bulk access.

A

So if someone asks you for some receipt, you can just give them reference to basically an entire subset of your state and they can sort of write through it as they need to, rather than having to just call back and then ask for more and more data every single time, so we can reduce gas costs, um it's also again, typed and structured.

A

This means you can have really nice descriptions of your inputs and your outputs for your. Your your smart contract, um uh uh apis uh and uh actually type information can actually be encoded. As I go to the object, so you can upload to the chain or keep it offline.

A

If you don't want to pay for that, um it also means that uh the the actors will directly manipulate the same tree um uh so in like ethereum actors or sorry after school, in the fbm um in ethereum contracts, sort of like they just read and write different locations in memory, but um uh this means that, like implicitly behind the scenes, everything gets uh serializes. Mercury in ipld, uh like the user, would directly manipulate this tree itself. So when the contract is done, everything is done.

A

There's no implicit operation, which means the gas cost, is very accurate, and actors can also optimize their own data layouts for their use cases. Okay, why not ipld! I always like to mention this just in case or just to be clear, like one, there is the cost for serializing deserializing. um uh So if you're trying to pass data between different modules, you're going to pay that cost second of all, hashing uh hashing can't be expensive. We've actually found that it's like modern cpus are pretty good at this.

A

It's not that bad um and you can also try to wait as much as possible. 20 libraries ideally is still growing, uh there's a lot of work and finally abstractions, I believe, is instruction and you will pay for it in terms of complexity, but you get tooling and get all the standards out of it, which is really nice.

A