NEAR Lunch and Learn, 18 Oct 2019

Previous Meeting

⏯

youtube image

►

From YouTube: NEAR Lunch and Learn Ep. 01: Cross Shard Transactions with One Block Delay

Description

Here, Alex discusses cross shard transactions with one block delay.

~~~ABOUT Lunch and Learns~~~
This is the first of a new series of videos exploring a concept on the NEAR protocol blockchain discussed in the NEAR office at lunch. Grab a sandwich and settle in!

Follow the latest from NEAR Protocol on,

Discord: https://near.ai/discord
Medium: https://near.ai/medium
Twitter: https://near.ai/twitter
GitHub: https://near.ai/github

A

Okay, so we will talk today about receipts in Charlotte blockchain, specifically how to make receipts arrive from one blockchain from one short-chain to another search in in exactly one block.

A

So, first, let's see, let's see what the problem is and I will use nightshade, which is near sharding, designed for the food example, so in in nightshade.

A

There's a main chain which produces blocks every well, that's a few seconds in each of those blocks for every short contains 1 or 0.

A

We call them chunks. So, for example, it said, is exactly 4 shards.

A

And every every block will contain 1 or 0 chunks.

A

For each shirt, those chunks they build on top of the last chunk that that was produced for the chart and so affected those chunks, they kinda represent a short chain right. So this chunk in the in the in the first chart- and the third block is that is building on top of the chunk. That was included last time for the same shirt.

A

Yeah, okay and so for simplicity of this of this presentation, we will not look at the and dimension blocks. We will just look at the the short chain at the chunks.

A

And so the so the short chains they produce the shorts they produce chunks at some time intervals and so we're gonna, select time interval, 0 time interval, 1 time interval, 2, etc.

A

So what we want to happen is in an optimistic case where two sharks do not skip chunks, which in reality is going to be the case most of the time, assuming that the the uptime is sufficient of the validators and they're not skipping skipping bits.

A

We want to achieve the following that, if a transaction that touches multiple shirts arrives at certain heights would set height zero in one short, so that's transaction, then the receipt of the transaction which effectively executed the second part of the transaction, arrives to the sec to the next chart at the very next block that sounds like it should be very simple and doable, but there is a challenge.

A

There's a problem and the problem is the following: the the to understand the problem: let's first see how how blocks are processed and produced and and what is the actual timeline of producing a block right?

A

So if we if withdraw, if this X is this time- and you look at the blocks and what happens and there's a particular moment in time, what are the checks right, not blocks, there's a particular moment in time when the chunk for for shards ero was produced, and then the the question is whether the block producer or chunk producer who produced the chunk, validated the transactions in the chunk before sending it out, and so, for example, in bit in Bitcoin.

A

If you produce a block, you do validate transactions before you send them send the block it has. It has a problem. It has a problem that if, if the, if the block producer of chunk 0 validated transactions before and computed the new state before sending the chunk, then it means that any other chunk producer. So, let's say sham producer 1.

A

They cannot start validating the same chunk before chunk producer or offer of this chunk finished validating because it wasn't sent before it was validated right, and so these two events, the chime producer, validating a chunk and any other recipient in the system. Any other validator, for example, chunk producer for the next height validating the chunk. They they end up being serialized one after another, but naturally the chunk producer of shard. One cannot produce chunk at height. One and you'll be validated the chunk of height, zero.

B

There are two sharks right: oh those.

A

Are that's one shortened to chunk producer story so that strong producer one jump register for height, zero and chunk producer for height one, but it's within the same shard right. So if you do validate transactions before you send out the chunk, what ends up happening? Is you create a chunk you validate it? Then you send it out. The next chunk producer has to receive it validated before they can produce the next chunk and then the same happens again right. The zeroth chunk produced receives the chunk, validates it before producing the next of their own.

A

I tried to sorry. This is one, and so therefore the willit validation happens. Serialize twice and validation is actually pretty expensive. It may be that the biggest bottleneck in the system, but it does take time and so that ideally should be removed, and so instead of that near and many other block chains, but near specifically, which is going to be relevant.

A

What Peretz, in a different way where, when you produce a chunk, you do a very basic validation of transactions, specifically when the validate that the transaction issuer has money to cover the gas price, accumulate transactions, new chunk and immediately brought broadcasted, and that chunk is considered valid, even if it hasn't valid transaction. Valid transaction just charges the gas, but but remains in the chunk right and why it is important. It is important because the chunk producer when the broadcast the chunk- they don't know the post state route.

A

They don't know the state truth as of after applying the transactions, and so therefore that posted cannot be stored in the chunk, and that also means that the receipts that are going to be produced as a result of applying this chunk, they are not included on the chunk as well. Right. 30 sheets will only be known when everybody else receives the cheque and processes it right, and so this is where the challenge happens is that by the time now, let's consider another shot right.

A

So these another chunk produced or let's call engine producer and they're producing in a different chunks, are so so that so this is charge 0, that's good as 0. This is s 1 over 0, and this is a short one. So when they receive the chunk.

A

Or like some information about chunk that was produced for the shard 0, they do not know yet they don't know the digits, but also they cannot apply it. Why? Because they don't see the full shard the full chunk, in a sense that if there is 100 shirts, it is infeasible for every chunk produced and every shorter send the full chunk to every other shirt right so effectively. The chunk produced in short, one will not see the full time from charge 0.

A

They cannot see the full full chunk from every shirt, so all the receive is they receive some header and some information with it, and so in particular they cannot. They cannot receive the receipts from Charlotte from this chap from this shirt, but also they can not reproduce the receipts because they cannot apply the Qiang. They don't see the full jack.

A

They don't even know if it will have any they do know if the shank is relevant when the block is produced like when the mention Church mentioned block is produced.

A

So this is the main chain block another main chain block, but the time by the time they receive the main chain block, they know all the chance which include it. So at least they know that the tank is actually is included on the main chain and assuming heavy load. You probably expect most of the shorts ending list, it's the most of the shirts, so the no track is included.

A

They they sort of know that probably there are you see it's coming from that from the chunk, but they don't know I didn't know what what they are right and so coming back to this picture now that we discussed that problem. So what happens is in this chart? Whomever is going to be producing this chunk by the time they produced chunk. They have no way of knowing these sheets.

A

As of applying this chunk right and that's the biggest problem, there is no way for us to do that. If, if we did go with the approach where the chunk is fully applied before before it is sent, that problem would be solved because then the chunk producer would have known the receipts as of applying the chunk, and so whatever message they sending to the other shirt. That message would just include these sheets, but again that that becomes significantly slower right. Okay, so what what can we do?

A

We definitely cannot without some magic with, as we just discussed, what definitely cannot know the disease by the time we creating the chunk, but also we don't know it. We don't need to know this. It's by the time we creating the chunk. We need to know the disease by the time we executing it right. So in reality- and so let's say this- this chanc produces produces chunk one for the short one right. They don't need and so later down the road. Someone else is applying it in reality, we don't need to know receive.

A

This is at this point when we create the chunk. We do need to know the issues by the time we apply the chunk right and so, let's make sense, cool, and so the way it is done today is the following when, when a chunk producers in this case one one of the chunk registers which are zero produces the chunk, we already have to distribute small parts of the chunk to every block producer in every shot.

A

This is called date availability unless it works right, so when they do that they also actually, let's use one as an example. One is a better example, so, by the time chain producer created the chunk one, the very the very last thing they did before that is. They applied chunk, 0 right, so they just they received chunk 0, they had the state, they applied it. They know, they know they knew post title and they know all the new receipts. So when they create a new chunk chunk a tight one, they do the following.

A

First of all, the header of this chunk will contain pre state route, which is the post state route for the previous chanc, but it will also include the receipt the receipt. A miracle morkul root of all the receipts produced as of after applying 0 right, so we don't know yet the easiest which will be produced after applying one. But we do know this is the two produced after playing zero. So there is a miracle root of all those receipts and also those this is all sorted by the destination.

A

Sharda G, and so you can prove any subset of those receipts. Go into this to a single shard in a single miracle proof, and so when they, when they, when they like, cut this little chunk into small pieces to send to every validator for data availability together with those little pieces, they also send all the receipts relevant to the particular block producer for their chart right. So when this little little thing, it goes to this chunk producer in the short one, all these sheets that go from short zero to. Third one.

A

As of after applying this chunk, they will be included in the message.

B

Going to chunk number look.

A

B

A

In time in time, so, okay, so it will not get yeah. It will not get include it into this chunk right, because this shank and this string they produce concurrent yes right so yeah so you're right. So it goes like in time it. It will end around here. Right now. What's important is that when you receive a block, a particular block- and you see chunk headers in it, that block is considered orphaned, I'm kill for average for every shirt and for every child concluded. You either have the food chunk.

A

If that's some short you validate or you have this little one part. If it's not a short you validate, and there are protections in place which protect against lazy behavior. So you cannot pretend you know it. If you pretend you know it is a good chance, you will be slashed. There is a protection in place, so you will not unorthodox.

A

It will not consider the block valid until you have those little one parts for every chunk that is included, and that also means that you have all these sheets for every shirt and the receipts are provable because there's a moral route here right in the header. That header is on the block and- and you have the Merkel proof against that.

B

A

Send all the relevant receipts for the visa for the destination person for with it, with the with the part that goes to that person. Yes right and so now by the time I need to apply chanc one I would only apply chunk one if I receive the block which has shrunk one right. So if, if we draw these chunks well, unfortunately I don't use the short idea right. So so, let's use so it's as 0 as 0 as 0 0, and this is all this one right, so this 0 1 and this 1 1 right.

A

So this is the chunk 1 for char 0. This is junk 1. For short, one I will only apply this chunk when I processed. This block I only processed this block. If it is not an orphan, it is only not an orphan. If I have one part for every chunk included, including the chunk chunk, 1 for short 0, and so those by the time I'm playing this chunk. I, don't know the receipt right, so we guarantee that by the time you are playing chunk at height 1.

A

You have all the receipts from all the chunks that included in the same height.

B

A

So every every block producer- this is a subset of shorts. They care about as the subset of shorts that they validate, but there's another sub sort of short way, larger that you don't validate, and so you need the full chunk for every shot you fully validate, and you need just one part with the receipts for the shirts that you do not validate either way. You will have the receipt. If you have one part that receipt was sent to you with the Merkel proof.

A

If you have the full chunk and you validate the shirt, then naturally you applied the previous chunk. So you you have this locally because you computed them right. So either way you have all the receipts and so that guarantees a that that that gives you one block cross card transactions right. Oh sorry, this is 0 because the the receipt which was created as a result of applying this chunk will be processed already.

A

As of the time this chunk is applied, so this is exactly 1 block between the first and the second hope of the transaction, and the same applies if it sends a further cross card transaction. It will also be applied on the very next block and, moreover, we guarantee that pieces don't get lost. This is another problem that many block chains face today that the Charlotte block chance is that they don't have any guarantee. The digits will be applied here if the receipt is not sent.

A

If it's not part of the moral proof, that's a /hreul behavior, because every other participant in the same short chain. They will also apply this chunk. They will also get the issues. They will also the merkel proof. The merkel proof doesn't match the miracle proof included in the header, that's easily provable cryptographically and that's a slash whole behavior, so everybody will arrive to the same proof, every recipient and every other char.

A

They they will check against the prove they will know they have all the receipts so uses cannot get lost, risk of guaranteed delivery and on the last problem, which we're not going to discuss very deeply today is that there is a problem that, if all the shorts and uses to a single chart, that will spend the short ride. So we're not going to go very deep into that, but this is also partially solved because you can accumulate the receipts, but you don't have to apply all of them at once.

A

You can put them into the queue and then you just process the first 103 sheets that got into the queue the the earliest and then the only problem is that how you prevent the queue from growing indefinitely, and I think the only way to do that is to fail transaction civic, useful right. So by the time you apply the chunk, the queue the destination queue for shard 1 is full. You will fail in a transaction that tries to go to third one cool, so that covers the receipts.

A

You have more questions, awesome.

B

This your guarantee right I, mean it basically allows you to say when we have the chance that we care about or.

A

B

Have like one part of whatever we not care about yeah, but it includes receipts. Yes, that's correct supposed to be like like this. What allows chardon, because you care about all the smoke, all the shards right and everything else, you kind of don't here. So it's like a little bit with the warhead, so I have a question one. It's actually a little bit of the overhead, because if we have a system where we have many shards and most of the transactions across our transactions, it would like.

B

Let's say we have financial system people just any money between each other, like on your charts. Right I mean most of the transactions are going to be cross. Our transactions, which means like the content of the blog of the chunk, is going to be half receipts, hard, half transaction to just drive right. The.

A

Question is: do they do all of the dishes go to the same chart, because this one part it doesn't contain all the receipts. It only contains receipts which go to the charts that they just receiving person cares about right. So.

B

Basically, care about one charge, then, on average, all the receipts accumulated from all one part says: I, do not care about approval, gonna, be on the same volume as yes.

A

Yes, so if the system is not skewed, if it's well balanced and then you will be receiving as many issues as you see, of transactions, which is definitely viable, it is still a problem, though, that if you're assured that the super popular you might be getting way too many receipts that can be solved in many ways that we did not discuss like you. Can you can limit how many issues can go to a particular shirt to some constant? Like you know, team 3x.

A

Something like one over a number of chars x to you, something like that: yeah cool, more questions, no okay, so that covers our first lunch. Serious thanks. Everyone for listening.