Ethereum Cat Herders PEEPanEIP, 25 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: PEEPanEIP #91: EIP-1153: Transient storage opcodes

Description

Topics covered
----------------------------
0:51 - Welcome to the episode
1:25 - Guest’s introduction
3:43 - Presentation
3:50 - Goal & purpose of the EIP
4:20 - What is Transient storage?
5:30 - Re-entrancy locks
6:00 - L2 point of view
6:30 - Today’s use cases
7:25 - Why EIP-1153, what is the problem it is solving?
9:45 - Solution EIP-1153
10:25 - More efficient use cases
11:05 - unlocks new patterns
11:30 - A marketplace contract
14:35 - Future protocol improvements
16:30 - The “Till” pattern
17:02 - Current specs
17:10 - Some links
17:40 - Advantages of transient storage over memory slot for app dev
18:55 - What is Interframe communication, how is the EIP helpful
20:20 - Helpful for future storage designs like Verkle Tree.
23:15 - Why this proposal can’t be implemented on Ethereum clients using soft fork
26:10 - Security consideration for client and app devs
29:12 - Client implementation
30:45 - When decided to champion the proposal
32:40 - General feedback on EIP process

Resources:
-----------------
Proposal - https://eips.ethereum.org/EIPS/eip-1153

Discussion - https://ethereum-magicians.org/t/eip-transient-storage-opcodes/553

Shanghai Playlist - https://www.youtube.com/playlist?list=PL4cwHXAawZxpok0smGmq-dFGVHQzW84a2

PEEPanEIP - https://www.youtube.com/playlist?list=PL4cwHXAawZxqu0PKKyMzG_3BJV_xZTi1F

Questions for Guests - https://docs.google.com/forms/d/e/1FAIpQLSdsmujnH_ncIlSEJ23s9jzXAKwfpTGNeJqjREWPloBbkJVQTA/viewform

Check out upcoming EIPs in Peep an EIP series at https://github.com/ethereum-cat-herders/PM/projects/2

Follow at Twitter
-----------------------------------------------
Moody @sendmoodz | Will @wschwab_ | Pooja Ranjan @poojaranjan19

Check out upcoming EIPs in Peep an EIP series at https://github.com/ethereum-cat-herders/PM/projects/2

Contact Ethereum Cat Herders
---------------------------------------------------
Discord: https://discord.io/ethereumcatherders
Twitter: https://twitter.com/EthCatHerders
Medium: https://medium.com/ethereum-cat-herders
Website: https://www.ethereumcatherders.com/

A

Reducing excessive gas cost and unwarranted situation and enabling new use cases for contract storage will be helpful to Smart contract developers till date. The use of transient storage is severely limited. If you want to learn about proposed transient storage, art codes and what it brings to the ethereum mainnet continue watching.

A

Welcome to Peep and eat episode, 91 a series dedicated for overview of proposals and protocol upgrade features I'm your host Puja ranjan. In this special episode on Shanghai EIT, we will learn about eip1153 transient storage, op code with Modi Salim Modi is a smart contract developer and is a co-author to the eib.

A

-1153 was created on 15th June 2018 by Alexi achhono lead developer at Aragon client The Proposal is currently in review status and CFI for Shanghai upgrade. We will have an overview of The Proposal in next few minutes with Modi Saleem. Thank you for joining us on the show today. Yeah.

B

Thanks for having me.

A

So this proposal has been under discussion for over three years, I'm looking forward to learn more about the proposal and what it has for ethereum clients and smart contract developers. But before we get there, maybe we can start with a quick introduction and when did you discover ethereum, maybe about your journey in this ecosystem?.

B

For sure um I actually was introduced to ethereum uh by just being a coinbase customer uh back in like 2017 or 2018 I, think they added uh ether, as is probably I, guess one of the first assets after Bitcoin um and so I really didn't know anything I just bought ether and then I saw from like ten dollars to like 40.

B

um I I had used coinbase I used Bitcoin in the past, uh just for like small payments, but never uh any other blockchain. So um I started reading about the ethereum protocol on the white paper and um as a software developer. It just sounded really interesting that to have like a possibility of Unstoppable applications that felt pretty novel to me, I think it was at the time pretty hyped up and, like you know, trying to be applied to a lot of things that wasn't the right fit for.

B

But you know, as a software engineer at the time like that, like managing servers and managing infrastructure was a huge pain, so I just really connected on that point um started going to hackathons uh I spent a few years working at like Amazon I worked on the manage blockchain service for AWS and um and I. Just really didn't see like a perfect fit until I found uniswap and I worked there for a couple years worked a lot on V3 um and yeah. It's it's been a lot of fun being in ethereum.

A

Indeed, starting from consumer to a contributor, it's quite a journey. Thank you for all the contribution and yeah without further Ado, let's peep in cool.

B

So yeah today, I'm talking about Transit storage, um so I guess the goal uh when I talk about this is that uh anyone who's listening understands that it's not just a gas, optimization um and I.

B

Think if you understand that, like you understand like why why it needs to happen and uh the purpose of it um and because it's not just a gas optimization, it just makes all these new patterns possible um that are much easier to work with in terms of like smart, smart contract development and I expect a lot of protocols to to make use of these new patterns and I'll talk more about that later um and yeah.

B

So, just to briefly, like recap, the current state of things, um transient storage is already like it's a concept that already exists in the evm um and really it's just when you write to a storage slot that has a zero value, a non-zero value and then within the same transaction. You write a zero back to the slot so effectively you've uh you you've taken a slot that wasn't used uh and you just temporarily use it, and then you cleared it um so there's no actual uh disk rate.

B

That's involved with transient storage and today, how that's supported is when you clear the slot. You get a large gas refund for the initial S store, which wrote the wrote the first value um so because we have accounting rules for it. It's already a concept, that's uh enshrined in the evm, and we use it today for uh most most commonly uh re-entrancy locks, so uh re-entrancy logs are required to prevent re-entrants, which oftentimes uh can be the cause of a vulnerability. So they're used all over the place in a lot of contracts.

B

There's a lot of Base contracts that just prevent re-entrancy uh across the different soluti libraries like opens up doing contracts um and.

A

Then we also use it in some other.

B

Creative cases like uh reading construction, Constructor Arguments for message sender, so we do this in unit swap V3 to make it cheaper to compute a pool address on chain um and I know some other uh protocols probably uses um I know in other protocol uses transient storage for uh their L2 to L1 inbox so like it's also applied for the smart contracts that power l2s um and- and this is these- these contracts just handle uh making State making uh L2 State available to L1 contracts.

B

um So so it's used in a lot of places already it already exists and um and yeah so here's the most common usage I just mentioned uh reagency locks. uh Typically, you just have some storage variable which you'll set to a non-zero value such as true uh you'll. Do your your contract logic and then you'll set it back to false, um so you'll see this everywhere.

B

um This is just a copy from unosault B2. This is the The Constructor arguments thing from I'm actually standard, it's pretty pretty complicated or convoluted. While we why we actually do this, so I won't get into it too much. um But it's another use case. You can see here we're setting parameters and then we're deleting the parameters.

B

Here's another use case from arbitrum's code base, so they will set uh the context of the L2 block um when they're, when they're sending an LPS use an L1 message and then they'll send that message.

B

um So if it already exists, what's the problem, why why eip1153 um so there's a lot of problems? Firstly, uh this storage refund I mentioned earlier is Cap to one-fifth of the transaction gas, and that was done uh in the London, hard Fork to prevent gas token.

B

um Another issue is storage. Refunds aren't given. If you revert so like. Let's say you set this re-entrance lock and then something happens that you need to revert in the transaction. The user won't get the gas refund and there's an EIP to fix that. But you know it's just sort of a um sort of a smell I'd say with the current state um and and then there's this other issue where uh your lock may always be zero.

B

If, if the user successfully interacts with your contract, um however, the client ethereum client still has to read from disk to know that it is zero. So.

A

B

Read from disk of the zero value can't be avoided, even though we always know by the logic of the contract that it will be zero at the beginning of a transaction um and then also like it's just very complicated. So if you ask a slowly developer, how much some right is going to cost? It's they're, probably not going to be able to tell you, because uh if you want to figure out uh what you you have these contribute to it.

B

It's there's probably like five or six of them, and it's changed every time um and it depends on the current state whether the slot is dirty, whether it's set to the original value. So um so like the branching logic is at least two layers deep and it's hard to understand, and then one other important. Possibly underemphasized point.

B

Is that? Because you have to read from Storage uh transient, storage does block parallelism, so you can't have you can't run two transactions in parallel that touch the same contract. If that contract uses a re-entrancy log, which is in storage um and and the reason is, the client doesn't know that both transactions fit around parallel because they touch the same storage slot and they have to read from the slot and and it's blocking parallelism but parallelism. But the other thing that blocks it is self-destruct.

B

um So that's just a little caveat.

A

B

The solution is the ip153 which just provides these two new dedicated op codes uh for this transient region, effectively transient memory region, but we're calling it transient storage and it's the same as storage, but at the end of every transaction it just gets discarded. So it's it's more like memories and storage and here's a diagram that was posted on the magician's thread, which I found pretty useful and I think other people have mentioned they liked it too.

B

um So yeah credit to Pascal. For this um it you can think of Transit storage as more like closer to machine State than uh than like the actual persisted storage region and I. Think that's what this illustrates. So, let's say you have the ip153 like what happens uh you? You can just immediately save billions of gas and I'm, not even with regards to like how much of the refund you're actually getting now, uh but just with regards to the disk reads: you're: saving billions of gas for anything that was previously regular storage.

B

If you just Market transient you're, now saving a ton of gas for your users, um and so this this isn't the final solidity syntax. It's not even under development right now, but this is how it might look in the future. Just adding a transient keyword to your storage variables that you know are Trends yet and.

A

B

I said in the beginning: it's not just a gas optimization, it unlocks new patterns, and this is why I think it's really powerful.

B

um So, let's, let's talk about um designing a Marketplace contract um and, let's just say it's some contract, you have items that people can buy items that they can sell to a store contract for fixed prices. You might think this of something like openc uh C Port, like as a corollary so.

B

A

Talk about how you might design this today.

B

um A customer comes to your contract and they want to buy a few items and sell one item. So the way that looks in the smart contract is they might call bye for item one and then within the buy. They have to send payment for item one and then they call buy for item two and then send payment right into uh solve for item three. That's for the item and then receive payment for item three.

B

um So um it's this is a little bit awkward for integrators. It's kind of like um you go to a grocery store and you have to pay for each item separately. Every time you scan an item, you have to pay for it.

B

um It's not ideal because you're doing multiple transfers, multiple calls of value, um really you just want to like, buy and sell them all together and just pay at the end, whatever you're owed.

B

um So uh let's just improve this a little bit using today's method.

B

uh Let's say now: you can do batch buys where uh you say you want to buy item one and two and you send payment for the both of them and then you uh you say you want to sell three and four and then you send the both the items and you get the total payment for both of those.

B

um So we've we've improved it a little bit. We've reduced token transfers. um We reduced uh payments item transfers, um but it's also not great. You might want to exchange items three and four for uh items one and two. So it's still not ideal because you have to have those funds uh to buy one before you sell, sell two Etc, um so um you can create this new method called execute orders which supports this use case, which uh just combines all the buys and sells uh and just requires the Delta payment, the Delta items owed.

B

um So uh so this works, but like only for the simple case like let's, let's imagine that uh pricing is dynamic and when you buy one or buy two you're, not gonna, know up front what the price is uh and when you sell three, maybe maybe you don't want to sell four. If you have enough money after you sell three so like you may want to do something more complicated than executing this batch order.

B

um So it's it's still it's better, but it's still a little clunky for integrators um because they can't operate on Nets and they can't do anything uh in between these like buys and cells. Just they can't answer their own logic, like let's say only cell three, if uh one and two are cheaper than Etc, um so it's better, but but not quite um easy to work with.

B

um So let's talk about how this can be better in future protocols so, instead of like, instead of uh enforcing these invariants, for example, when a user buys item one, you need to immediately collect payment for item one um and so, instead of enforcing that variant that you've collected payment for item one uh as part of each operation.

B

You can create this overarching operation called lock or uh open or enter other of just other names for for this operation and then and then this Marketplace contract calls back into the customer uh sort of with its open, open uh register.

B

um And now, while the customer has the register open, they can call buy as much as they want. They can call any other contracts, they can call sell as much as they want and then uh only at the end do they have to pay the difference and transfer the items. So they may not even have the item yet they may be selling other things and then buying from a different Marketplace, and then they transfer the item in um so like.

B

What's what's really happening is, is this accounting is happening uh in transient storage with this new kind of contract, um but it does require a lot of transient storage to keep track of uh the state. So like the refund is a huge blocker to this pattern, uh because it's limited to only one-fifth of the gas that's used in the transaction.

A

um And so at the.

B

End you, uh after after the user, has done all these operations.

B

You check that the balance is settled um and if not, then you revert otherwise you continue and um and the user is happy with their order and so uh I'm calling this the till pattern in conversation um it because the contract sort of acts like a cash register or a till um and um you're just you're, just checking you're just running the variants on the on the total value with the till um and you can you can sort of defer all the validation and token transfers um until the user is done, their activated contracts, it's just another way to remember it until so yeah and uh the current state, as was mentioned earlier, it's C5 for Shanghai, um it's tracked into the Shanghai spec and on a unisoft GitHub project board.

B

There's some links um as well and yeah. That's all I got if you have any questions.

A

Awesome, first of all, congratulations for the proposal into CFI for Shanghai and thank you for sharing the overview. I think Marketplace, as the use case will Intrigue many adaptive and the release of this episode will help the community to set expectation with this upcoming upgrade and learn. What's in it.

A

um Moving on to the next section q a about this proposal in simple words, can you maybe explain the advantage of using transient storage over memory and storage lot for application developers, especially yeah.

B

So uh the advantage over using memory is pretty clear. It's that uh the uh memory is only accessible within the current call frame, and so, if you want to communicate with the current contract uh call frames deep.

B

um Let's say you want to call into another contract and you want to remember something that happened before you called into the contract later, um then you, you actually need to use stranding storage. You can't you can't use memory for that, because it doesn't there's no way to communicate between call frames for for memory.

B

Besides call data and these contracts aren't trusted in most cases, so yeah like a lot of the times where you're using in our entrancy locks or anything like that, um it should already just be transient um and you kind of are already using transient storage. It's just now. You can use these new up codes. That's.

A

Good to know so talking about the communication, maybe if you can explain a little bit more about the problem of inter-frame communication and how is eip1153 be helpful. Yeah.

B

So so, when, uh when a call comes into a contract, you sort of get this uh like clean memory region, um and so you can write things to memory and you can read things from memory but um but like if you call into another contract, they have no access to that memory region and if you call into another contract and that contract pulls back into the original one you you also don't have access to that. First memory region.

A

B

So, there's no way to communicate even just with your own contract uh for like a future call besides um storage used transiently. So that's the story. That's the op codes or uh or call data, and the issue with call data is that if you're calling an untrusted contract, they they have the ability to to intercept the call data and send something else that you didn't intend. So you can't trust, call data and that just leaves uh storage used transiently for this.

B

For this interframe communication, which has a lot of those problems uh shared in that earlier slide,.

A

Awesome uh in The Proposal I read somewhere like it could be.

A

Could be like vocal tree um if you would like to take a few minutes talk about like how this is going to be helpful. I know it's going to save a lot of gas. You have mentioned it and yeah. If there is anything specific you want to add here, yeah.

B

So this is just uh for my conversations on Discord and the awkward as chat um I think it I think it has implications uh for the future storage designs.

B

um I know, for example, that if you write to a vertical tree, you never actually removed the leaf, um so that doesn't necessarily mean that you have to write to the vertical tree if you're using storage transiently.

B

But but there are implications with you know: refunds, for example, if you, uh if you have a slot with a value- and you clear it now- you have to you- have to think about how you're going to change the refunds um and so having this like dedicated uh transient storage region May mean that we can greatly simplify the refund logic and as well as the caching logic, if it if it appears that that's uh the correct design decision um something I tweeted about, for example, is um right.

B

Now all the caching Logic for storage lives in the client. So if you do 1s load of a slot and uh it's a cold s load, meaning it hasn't been read in that transaction, it's going to cost you 2100 gas and then, if you do a subsequent s load of the same slides, 100 guests, um so so that caching logic is directly written in the protocol, which makes the protocol a little more complicated.

B

It also means that every s load has to be cash or else that 100 gas may be underpriced, um so uh this could just be pushed up to the application layer. So, instead of um the client caching, every s load, uh the the smart contract may do an s-look for the first time. It's needed, uh t-store it and then for future times it's needed t-load it.

B

um It also may just uh s-load it once because it's only used once in the transaction and then the client doesn't have to uh cache it at all. So it basically just opens up the design space for simplifying uh storage, accounting um and possibly making clients more efficient by uh you know. Saving these, like caching operations that.

A

Is good to know, we know that it is considered for upgrade and if this is going to be helpful even for future upgrades uh when we will have it for our culture and all so yeah, that's a good one. uh Talking about the consideration for the Shanghai upgrade I'm trying to understand like for this uh proposal. Specifically, it was proposed in 2018 and then was uh under discussion for some time proposal did not move forward. It was moved to stagnant status in 2021 and brought back into active discussion in 2022..

A

um Why do you think it is important for ethereum climbs to implement in the first place and why we cannot get it done with soft for getting it implemented at like um individual uh time frame instead of having it considered for Shanghai yeah.

B

So I I think the reason uh it's coming up more in recent discussion is because uh the evm has changed in a way that detriments uh the transit usage of storage today, for example, uh before the London hard Fork, uh the refund was capped at 50, not 20 percent and and so there's a very good reason that it was Caps or the cap has changed to 20 um and- and that was to sort of internalize from my perspective, to internalize this like block space elasticity.

B

um That previously was at the application layer with a gas token, um and so since there's a very good reason for that change.

B

uh It should be there, um however, it it does affect uh existing use cases of transient storage in a way that makes it unusable for these use cases um and I think it sort of brought up some of these, like technical debt issues um with transient, storage um and uh I think. The reason it should be fixed earlier rather than later is that people have to rewrite their contracts to make use of it.

B

um Transient storage has been around for a long time and the way you've used. It is a store and so contracts need to migrate. They need to be Rewritten um before the uh the transient use case of storage can be uh de-prioritized by the client Developers. For example, uh there's a lot of usage on you're talking to a lot of users on here's button. Three.

B

uh If, if one day the transit usage of storage refund is deprecated or removed with like the vertical tree, upgrade that may make unisoft BT B2 much more expensive, it may make clients all sufficient, so so just having more time for contract and app developers to to move to the new.

B

um So the new outcomes is very valuable. Yeah.

A

That'd make a lot of sense um so, as we are clearly seeing that, obviously it's going to be helpful for smart contract developers. Is there any security consideration for client developers implementing this inclined and maybe for smart contract developers while implementing this yeah.

B

That's a good question, I think for client Developers um their the security considerations are um that this Transit storage region can be used to allocate uh more memory than the current m-store op codes, um and so we ran some numbers for that and uh and that sort of goes into the gas pricing, consideration um and uh and I think I think it's pretty safe to say that it's it's not an issue for clients, because it, the maximum you can allocate, is on the order of 10 megabytes, um which is less than you can do with a malicious uh m-store worst case use case.

B

um So as far as client developers, uh I, don't think, there's any there's I think the security considerations are well known um at the application layer. I think some slave developers have brought up that it may make uh contracts harder to audit um as well as formally verify um and statically analyze and decompile I. Think my opinion is that uh you know the contracts. As far as you know, formal verification goes.

B

For example, you know the contracts that people are formally verifying are usually already stateless um and- uh and so this this doesn't make that case any worse, like formal verification is very rigorous and challenging, and you know we try to do some of that with uh uniswap uh V3 and even just doing it for a square root function was quite difficult, um so I think um the idea that it's gonna make formal verification, harder, isn't very convincing to me as an application developer um and uh and then, as as far as like you know, the risk of um risk of smart contract developers writing more bugs.

B

um It's also the smart contract developers who who mostly want this um so I, think we acknowledge that there is a risk that we write bugs with it. But uh it's worth it to us. Yeah.

A

Right I mean obviously people need to be careful about, but then they know what are the areas there where we can pay more attention is helpful, so for a smart contract developer make sure that we are not adding buds to it. I'm talking about implementation, I know um when this is in CFI. It probably would be considered for devnet client team test net for any upgrade. Are you by any chance aware of any client implementation, yeah.

B

um On this tracking board, maybe I can just pull it up. um There are a few, so it's implemented in a PR with gas, a PR with best you um and it's merged another mind in ethereum, JS um Aragon said they could take from gaps implementation. So hopefully that comes shortly after the getth one is merged um and there's also uh an EIP 1153 test net. That was started for a capture the flag, um challenged by Vox from optimism.

B

um I can find this Twitter and send it to you later, but uh um but yeah it's uh there's. Definitely some work. That's already been done and been happening over the past year. Awesome.

A

And uh hearing about testnet for 1153 makes it even more interesting and shows your like interest in helping out with this proposal. I know the proposal was originally uh written by Alexi and uh you joined to Champion and co-authored the proposal. I'll be curious to hear your motivation behind pushing this proposal. How and when you decided to join as a co-author and Champion for this proposal.

B

Yeah so I've been writing smart contracts, for you know, for a couple of years, um and um I was focused on some potential future versions um of uniswap um and uh one of them really stood out, which I think takes advantage of the till patter um as an option, um so that that made it very clear to me how useful changing storage could be.

B

One of the things I tried early on with that um prototype was uh just instead of running it on the London, hard work running on the Berlin hard Fork, just to see the gas difference, and it was massive, so I could see the impact of the evm change that went into the London hardcore that capped the refunds, um and so that brought my attention to all this technical debt, which was I, didn't have a way to do enter frame communication um and so that I think it.

B

It seemed to me like it was blocking what was very clearly.

A

um A better design.

B

For uh uniswap and protocols like uniswap, um so that was that was what really motivated me to pursue it. That.

A

Is good to know I know many more things came in between 2018 to 2022, which took priority uh over uh this proposal, so thank you for jumping in and helping out push this proposal towards final and maybe implemented in the Shanghai upgrade.

A

um One last question on this topic is like um the general feedback. Now that you have contributed to the EIP repository I'll, be uh interested to hear your thoughts on the EIP process for core proposals in general, like this, like any Improvement suggestions, yeah.

B

I can I can say you know. One thing I've mentioned to Tim before is that uh I think application developers should have um a big see at the table big voice as far as uh the the core protocol development goes at least for the evm stuff.

B

um You know, I, don't know a lot about the consensus stuff, but I've been working with the evm for a long time um and I I have an idea of like what things are hard or what things are easy I think uh you know I've seen in the past cases where uh some eips are pulling to the Wayside like three zero, seven four, that was another big one um and I think they can really have a huge impact on the user experience, um but they're not prioritized because I think the people who really uh want it or maybe a little bit closer to the end users through their their surveying, um and so it's hard to to represent that opinion without um including these like big protocols, a little bit more I.

A

Think that's a very valuable piece of feedback. I know. 3074 is also up for discussion, not sure if it would be considered or not, but I would agree to the point that uh there are a few core proposals which are actually helpful to application developers. So yeah having their thoughts heard, is also important, while considering any proposal for future upgrade. Well, um uh that's all from our end um and it's almost time to wrap up. Is there anything that you would like to share with the ethereum community about the proposal or in general.

B

um Yeah I mean I think, despite that, like it's, it's been pretty encouraging to see that uh you know the client teams appreciate the contributions and um and I I think it's gaining momentum because of all the hard work that you know, people on the unusual team has have done. um I.

A

Don't want to say.

B

I, don't want to come off as gloomy just because yeah I.

A

B

I Wish app developers had a bigger voice, but you know um it's definitely it's it's a very hard problem to like to move at this scale. So I understand that I think it's it's cool that it actually happens. Yeah.

A

Right and no I think uh sharing is also very important. It's good that you have mentioned that to them earlier. uh He is managing all core Dev meetings, so probably that will be discussed in some of the future meetings, but yeah, it's good to know, appreciate you taking out time to talk about EIP, 115c, transient storage up course, which may be expected on the ethereum magnet in 2023.

A

On this note, thanks to our Channel subscriber for watching and listening to this episode on Shanghai eips check out descriptions for Shanghai playlist to follow other eips and this proposal related links. Should you have any question on this or any other EIP? Let us know at e-cat holders Discord next week we will be back with another interesting talk till then keep watching keep sharing your love with. It is cheers.