Everything EOS EOS Node Operator Round Table, 26 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Node Operator Roundtable - April 26th, 2023

Description

In this Antelope developers call, participants discuss the new proposal surrounding Get Table Rows and Get Block, exploring their value for developers and potential issues. They also touch on the possibility of enhancing the Trace API plugin for better functionality. Additionally, there's a debate on the hard limit for rows and the option to override it. The group also expresses excitement about the HTTP public API and threading capabilities. Don't miss these insightful discussions!

00:00 Introduction
04:32 New Proposal: Get Table Rows & Get Block
16:58 Enhancing Trace API Plugin
27:10 Hard Limit for Rows & Override Option
37:42 HTTP Public API & Threading
49:10 Discussion Takeaways

The EOS Network Foundation coordinates financial and non-financial support to encourage the growth and development of the EOS Network. We’re harnessing the power of decentralization to chart a coordinated future for the EOS Network as a force for positive global change.

// EOS Network Foundation //
https://www.eosnetwork.com
https://twitter.com/EosNFoundation
https://www.linkedin.com/company/eos-network-foundation
https://t.me/EOSNetworkFoundation
https://discord.gg/eos-network
info@eosnetwork.com
#EOS #Web3 #Antelope #DeveloperCall #GetTableRows #GetBlock #TraceAPI #API

A

Welcome everybody., it is april 26th.. We just calculated the number of, number of meeting this is,, or this is number 64,. If our count is correct.

A

Uh, so congrats everybody. leap, 4.0, just released.

A

uh,, I'm gonna hand it over to brian, talk about the, the leap status, and, uh,, get into the agenda for me.

B

Yes. what,, what viewers who are watching this after the fact missed was that there was a whole cons, consensus protocol involved in calculating that figure.

B

So, um, yeah,, so I,. I do have a couple of announcements: on, um, release, things, uh, in the antelope world.

B

So first of all,, um, antelope leap, v, 4.00, um, you know,, is officially released as of yesterday.

B

And, uh, some new operators, are,, are already upgrading to that.

B

Um, so, uh, please,, you know,. If you,. If you want to help sort of spread, the word, uh, go out to the social channels, and, uh, retweet, those, uh,. You know, give your.

B

Comments on it, that sort of thing, just so that we can get the word out there.

B

Uh, but yes, and I,, I wanna, you know, thank all the core developers who have been involved in that.

B

Lot of really good work went into that. and also everybody who's been actively engaged in this call, uh,. It's been a, a very important source of feedback, and so we appreciate that and continuing on that theme.

B

Um, I have another announcement before we do this,, but continuing on that theme, we do have a, um, uh, a proposal we wanna review, uh, with folks on this call today, uh, after this next announcement.

B

Which is that, um,, the c d, t, um, version 4.00 release candidate. One will hopefully be released before close of week.

B

Um, I'm sorry, uh, update to that.

B

uh,. My notes contain both of these statements, so, uh, you're, getting it live.

B

No,, I'm just kidding. uh, c d t is actually targeted. For, uh, early next week.

B

That's the release candidate one.

B

of version, four, c d, t.

B

Uh, that's,, that's it on the announcements.

B

um, and with that, the, the thing that we wanted to sort of review: today, um, we,. We brought greg on, um,, who has written a proposal.

B

We haven't,, we haven't posted this proposal publicly yet.

B

we'll be doing that shortly. After, um, after this.

B

Discussion for when we're seeking, basically initial feedback, on, um,, I suppose I'll, it might be most effective to just let greg, take it from the top and introduce it.

B

I'm good if I may, right quick.

C

Um, we're,. I think this is the first time greg's been on the call.

C

So greg's, uh, uh, core developer, uh, e n f, um, our,, one of our new rock stars, uh,, doing excellent work over here.

C

He's somewhat new to the ecosystem,, but east.

C

go right: in. and we've had, uh, we've had a great time actually working together, and, and discussing this proposal.

C

So with that, thanks kevin.

D

Thanks kevin. um,, I'm gonna try my best.

D

no,. Don't hesitate to jump in, kevin, if I,, she won't say anything.

D

Um,, so basically what we've done? Is, um, kevin started data that've, I've worked on it as well.

D

We, we have moved,, we used,. We used to basically process a p. I request,, such as get block.

D

Completely on, on the single thread, on the main thread of the application.

D

So long requests were a big issue: because, um,. They basically would freeze the application for every other process,, any other processing.

D

Uh, so we, we had very strict, uh, time requirements in place to avoid freezing the main straight,. The main thread for too long.

D

So what we've done now is: we've moved.

D

big part of the processing,, the main part of the processing of the main thread onto the htp threat, pool,, which is a mo where we can run multiple threads.

D

So, so now the, the re time requirements we had that sometimes would cause requests to fail, such as get block,. You know, would fail because of a timeout for some blocks that were.

D

Especially big or required, or had lots of actions, requiring, uh,, timely,, serious ization.

D

So now that we've moved that to the h d, p threat, pool, we,, we thought of a better way to basically specify the time. Delays that a p I requests, uh, are constrained. With.

D

And we, for some, for some requests, such as get block, for example,. If you run a a api node,, you basically want your get block, request to actually go through, right?.

D

Because if it fails,, because it's a very large block that has lots of actions.

D

There's not much thing a user can do except retrying it, and it's gonna fail again because it still has lot of action.

D

So,, so we wanted this to, to not be constrained by the h t, t p, uh, max request, time.

D

I forgot,, I forgot the name. max restaurants, time. and,, um, we,. So now what we're gonna do is we, uh,. We are going to, for such requests. Such as,, such as get block, or some transactions, which,, which are bounded.

D

It's, uh, and most of the processing is done on the http threat pool anyways,. We are not gonna enforce any delay, except for this realization of some special abis, because we want to guard against,.

D

Uh, especially crafted a api that would basically explode the time, take, take to ize something.

D

So, so we,, so what we're gonna do is like a get block would actually run as soon as it takes,, but every single a b I ization, would be bounded by the max ization time.

D

And if it fails,, we just gonna still return the results.

D

But in a non serialized form.

D

in the binary, form., so the,, so basically you, if,. If a,, if there is a block that has some.

D

Some apis that we especially crafted to try to trick the node into taking a very too much time.

D

Those won't be ized,, but we'll still return a valid block where everything that can be digitalized in, in a reasonable time will be ized.

E

So in that way, it work like get through block.

E

if it goes over the the max, uh, ization time.

D

Yeah, so get, of course, get roadblock would also be the same.

D

so, so, so,, so now,. Now there is, there would be less need for get roadblock except if,. If you do it,, if you did want to just get a roadblock.

D

Without anything, they serialized,, you can still use that, that, that api.

D

But now they won't be,, I think get roadblock was basically introduced, because some visualization was taking so long that sometimes, uh, you,. You would not be able to do it within the max response. Time.

D

But now that, with that new proposal, you,, you could actually just always use get block, and there won't, shouldn't be any problem with it.

D

It would always return the block.

D

and, um, some,, especially long abis, will not be visualized, but in, in some cases,, which were really in intended to basically hurt us, right?.

D

It's not like a valid use case where in, in, in any regular, any regular a b. I would be dis realized within the, the time limit.

D

The abi,, the abi, uh,, did um, abi max time milliseconds,. So thank you.

D

Grab that., that's yeah,, go ahead.

C

If I might write quick, actually there, there's another reason, we provided the gi well block,, which is the fact that when we deserialize a gi block call,, we use current avis on chain.

C

And if that's a old block and not a current block or even for a current block,, the avis that's being used to deserialize. That block are not.

C

Necessarily the avis that were in play when that block was created.

C

And so there is potential. problems with the a block and using those, those ob apis,. That particular issue has not been fixed.

C

We still just use the, the current abi., so if you wanna do, uh, abi,, I use serialization, you know, correctly in all cases, uh,, then you can do the give raw block, uh, cash to the abis itself.

C

And use the correct abis that, uh, that are appropriate for the actions.

C

Lieutenant net law.

D

And, and so, so the,, the last part of the proposal is that we still have this ht max response time. That is now used for some request that.

D

Can request a variable number of items,, like first all get table roll.

D

So if you run a node,, you can still decide that.

D

The maximum time you want to spend on the main thread for a specific api that requires a viable number of items is so many milliseconds.

D

And what's gonna happen is that we're gonna retrieve as many ice stem as we can within that max response time on the main thread.

D

And then we're gonna pause that to be ized on the htp threat pool.

D

and they will all, all,. The one that were retrieved will be dematerialized.

D

So, and that, that way, the user will, can actually request if,. If they request a really large number,, they will still get something.

D

And they will,, they will be able to request more if they won't.

D

So,, so that's basically the, the meat of the proposal.

D

D

I'm, I'm, I'm happy to answer the question. Or.

C

So a couple other things there we might add.

C

um, we're also proposing a hard coded 1000 limit on get table. Rows.

C

Um, uh, because of the fact that, uh, eel people might have max response. Time set high already, or, or they might.

C

Know it's possible benet at something as small as you know, or, let's say 30, milliseconds or something, and it,. That could be a rather large number of rows, like that, might be 10,000 or a.

C

Hundred thousand rows of table data that you would then need to do. Share life,, which could take, you know,, who knows,, depending on how the, the abis are for, for those particular could take.

C

In certain amount time and so,, we thought at beth to maybe just limit that to, to a thousand.

C

Um,, knowing that, of course, that,, you know, people can make, um, local calls there.

C

Now we did say even,, and so we can discuss that thousand higher, lower and maybe not there at all.

C

That's something we certainly look for feedback on.

C

The other thing in response to that,, if you're running a local note,, you can actually fit hvp mattress once time to negative one.

C

And then you could make a call to say, just gimme, all the roads.

C

and I'd say there's,, you know, a million rows and it would just take, you know, an hour to do or something.

C

Well,, you can just sat there and let you, you know,. Let you do that. On, on your local node.

C

Uh, but then it won't.

E

Sorry. uh,, if it does that, like get cable rolls on, say like all the tables,, then you have over a hundred thousand and uh, whatnot.

E

Are you guaranteed, then those, uh,, all these row,, even if it takes an hour,, are actually all coming from the same kind of snapshot. Stake at that point in time?.

C

Yeah. okay. yes,, which would be, you know,. One reason why you might wanna run a local node.

C

Just have to make that simpler. uh, otherwise,, you know,, if you're,, if you're,, if you don't have that setting or you're getting a public node,, you know, you're, not quite.

C

Sure, you know, between the gaps,, whether something have changed.

C

For example, you know, block has come in and chamber of state.

C

E

So that currently, the only way, I think, there's just through either a history solution, like diffuse.

E

And at certain blocks you can get,, you can specify the block number to be retrieve the state you want.

E

Or like the old school way,, I guess, is you load,, you know, a snapshot and you just set it not to peer and you keep like, you know, retrieving, uh,, the deck as say, uh, from it.

E

I was wondering if there's any plan,, this might be kind of like, uh, on the side, but diffuse used to have a tool as part of the diffuse binary that allows you to,.

E

Um,, it was part of the bootstrapping a chain,, but they had this thing called migrate.

E

And what it does is it just takes a simple snapshot file and it kind of deconstructs it into like a folder hierarchy for every single,. You know,, like,, um,, uh, contract.

E

Or our account and whether it had a d v I and all the different peoples it had.

E

But that got broken., or at least like, you know, diffuses, not maintained., so that doesn't work anymore after I think version, 2.0 0.6 or something to that effect.

E

Are there any plans to do something like that? cause? I think that's super useful.

C

We have something and we, you know,. We welcome feedback on it.

C

It's already available., you can try it out,, but you can convert a, a snapshot to j s. Ot.

C

And that will expand out the, the snapshot in j o form, and you can open that up and, and look at it and, and do whatever you want, to, or potentially even break that down into the smaller pieces.

C

And if one gigantic j o n file a as it currently, currently is, is implemented,, but there are plenty of editors that are able to open it and work with,. You know, large files like that.

C

Um, yeah, just programmatically max, you'd wanna, you know,, handle that programmatically.

E

Uh,, that's perfect.

C

Can I have the name of that tool?, it's available., I think it's in, uh, three,, two, uh, actually does anybody off top of their head?.

C

It's just the,, it's the leap, util, um, and snapshot..

C

Yeah. yeah. and I think in, in three, one,. I think it was part of notia, um,. I have to look exactly when it's added,, but it shiv in whatever, uh, version you're using.

C

That, I believe, is everyone, hopefully done at least three, two at this point.

C

So you, you should be available to you.

F

B

All right. any, any other questions on this proposal. Or.

B

E

Um, just a, just a quick, one. um,, you said it's a different thread, right?.

E

So is that like, offload some of the weight off the main notius thread?.

C

Yeah, mo most of the processing will be done in the h. P p thread pool, um,, which would be one reason,, like in, in, in five oh, to increase that even in four.

C

Oh. you probably would, would want to consider increasing that third pool size because even in oral, we moved a number of things off of the main thread into that thread. Pool.

C

So that thread pool's getting a lot more use in 4.0 than it did in, in any three x, version.

C

So, uh, she see, see benefit in increasing that thread, pool size.

C

The default,, I believe, is two., I believe, all of our t pulls deepal alter, are vice two.

C

Um, but yeah, you may want to consider increasing that particularly on, on a public a b I note.

C

Uh,, you may see some, some benefit there.

E

Nice. thanks tim. goad. and that's the ization, uh, that occurs on like read, only, uh, or like get calls for example.

E

Does that take an effect if you're making say, one of those big calls under the new proposal for get people rose?.

E

Or it's more like it can just handle, like each request, might go to a different thread and scale that way,, but one call will never kind of go on multiple threads,. Even if a big call got.

C

That is correct., so I,, I believe what you're referring to in 4.0,. We have the new read, only thread pool, uh,, that you can specify and then you can specify a rewind or right window.

C

Right? and so during the, the rewind portion of that,, it will use the number of threads that you specified in the, in the read only thread pool.

C

And, and so when you're in the right window,, you have one main thread: that's doing and everything, including like get table roll call,, right?.

C

And then, when you enter the, the read window,, it stands out,. If you will, into a number of, into the number of threads that specified in the read only bread, pool.

C

During that, that time, window, uh, calls like get table. Roads will be, you know, allocated across those threads and run, uh, they're not like moved during.

C

The processing,, but you can process two or more, uh,, depending on how many of those df configured get table. Row calls in parallel during that reap window.

E

Although, yeah,, but that happens on multiple calls,, but not on like one call.

E

We'll never use multiple thread. that is correct, colonel.

C

Uh,, that's the, the way that it's, uh, for that out.

D

So it's garvin., so it's gonna be interesting, um, that now we have basically two separate processing.

D

We have the, the processing of the,, a api itself, like the, the,, like the processing of transaction or,, or, or the retrieval.

D

Like you, run, get table rules., we retrieve the binary data and that could actually be, again, split into multiple thread like kevin was just saying.

D

Then, once we have retrieved the, the data or process the transaction,, then we need to return the result to the user,, including the ization, with order into jason.

D

And so that's that again,. That second party is far to the htp threat pool to be, to be done on separate threats.

D

And again, each request will be on one thread..

D

So if you to are running a get block that has lots of actions,, you know, he,, there's a lot of,. It takes a lot of time to create the response.

D

You,, the jason response., then that will occur on one thread..

D

So if you,, if you have a htb thread, pool with eight threads,, only one thread would be busy with that request.

D

And you cf, seven other threads being able to serve, serve other api requests.

D

And one thing you may want to do: besides increasing the size of your hb thread pool is maybe red.

D

Use the htp max response time, because now, uh, request will be, will spend much less time than before.

D

On the main thread, and that's the h, hp max response time will be only limiting that part.

D

So, so before,, if you,, if you had the max response time that allowed you to serve a table of 1000 rows and you leave it the same,, then that.

D

Same time would now potentially be able to serve a request of much ha.

D

Much more than that,, except that we've limited it to now: 1000, but, so,, so you, uh, for serving the same, same size of requests, you,. You probably.

D

Will have to reduce your max h, should b max response time, significantly.

D

And, and before there was this issue, that, uh, because of some specific request like get block, that could be very,, could take a really, really long. Time.

D

No operators would be, were attempted to set a really,, really large, h, v max response time to make sure that any block went through.

D

So now that that's not delete anymore, to be cautious, like that,, you can actually, because get block will not be subject. The h, hp max response, time.

D

So, so you, you can actually set the actual max response time. You reasonably want for any request to, to be something like I will 10 milliseconds or something like that.

C

And again,: that's for the proposal we're talking about here in port point.

C

Oh, you'll need to continue to set that large to get the, the behavior that you want for the, for the block,, which involve, I think we discussed that,, but the end result, sorry.

E

But the end result: there, like,, say,, doing a thousand request calls or your, your old max response. Allow the thousand request called, uh,, to get cable roads, for example.

E

Uh,, a thousand pla, uh, road., sorry for get cable, uh, rose, then.

E

Even in the new paradigm,, even though those same milliseconds now can get you much more, uh, due to that hard coded 1000,, you're, still limited by that.

C

Yeah. so, and that's not a big question, I guess. yes. so,, and that we're seeking feedback, right?, that's the whole purpose. Here. is the,. The 1000, um, steris is kind of almost a safeguard of like,. We know people have that setting high, and so we limit it to a thousand.

C

You know, that should, that should keep people from like,. You know, cheating themselves in the, but,, but I mean, we could remove that thousand if,. If.

C

That sounds, um, uh, restrictive or we could increase it to something larger.

C

Um, welcome to feedback there for sure.

E

Personally,, I think like having it like a thousand by default as they config and then allowing an api,, especially like a local api note,. You know,, like that. My family exposed to, to the public to.

E

Handle whatever else, you know,, whatever the admin or the administrator office, wants to provide.

E

Uh, okay,, so that is in the proposal, elisa.

C

So let, let,- let's know if this satisfies that request.

C

So currently in the proposal is, if you set htd max response time to negative one,, then it ignores that a thousand and you can basically hit it at for as.

C

Many as you want, and then it will honor, what's in, in the request, right?.

C

When you make the it get, real, um, get table. Rows, uh, request, the, your, the,. The hpp request has within it.

C

Two limits the time limit, a and a uh, and a number of row limits.

C

If you set, you know, number of lows to fe maxient and time to,, you know, whatever from gigantic value, or, or max time,, then um,, then it would,.

C

Then it would basically take as long as it takes to return all that data.

E

The use case I was, but concerned about..

E

This was great to know. thank you., yeah. alright.

B

So just some questions, um,, you know, for the, for the sake of stimulating conversation here, and, and fully exploring this and getting feedback.

B

Is it,, so is it in the case where the serialization takes too long?.

B

Right. um, would it.

B

Would it potentially return partially dec serialized objects?.

B

Is that how it would work?, or would it no. so it would just, what exactly happens in that situation?.

B

So, so, so for every specific,, um, action, for example, that you want to be dec, serialized, um,. If we, we try to dec serialize one action, to, and, and it takes, it takes longer than abi.

D

Max, max api max time, ms.

D

Then what we do is we return the, the action in binary form only for that action.

D

So you may have a block with 1000 action, and maybe one of them was especially crafted to triggers into spending a an ordinate amount of time.

D

So that one will not be ized., all the other one will be ized correctly., so you would have a. one action return in binary and all the others return in correct form.

B

So is there any,, I guess the question I'd have for note: operator, api sort of hosts is, uh,. Are there any concerns around the,? What's the word, I'm.

B

Looking for, uh, the non-determinism of what might actually get returned?.

B

Um, with regards to, well, it's, it's,, so I think,, I think we already had that behavior before,, like we already would,, would basically bail out.

D

If any,, if any, action, and we may be more tolerant than before in the sunset.

D

Before we might, when any action was taken longer than a b solar is a max time, ms.

D

We would just abort the api request, and, and fail it.

D

So now, so, so,. So now we are behaving., so we are, now, we're just trying to say. If there's an a b, I that's, that cannot be visualized,. We won't visualize it,, but we will not fail. The request.

D

So like, if you,, if you run the get block request,, you will always get a result.

D

And same. another thing also is before we,. If you,, if you do get table rows, same thing,, you will always get a result.

F

D

Which, before sometimes it would fail, and you would get nothing.

D

so here you would get at, regardless of the time,. You would always get at least one row and you get as many as you can within the specified h, p max response, time.

D

And same thing for forget: table rules., so e everything that can be ized will be materialized.

D

If one a b, fails, it'll be returned in by the reform.

C

Yeah. and we,, and we can't even really relax that on a local node because I, I just google zip barg,, it's the same for the beijing.

C

And I'm, and I'm, I, I,- I can't be certain on this, but I,. I think it's the case that we actually have some of these on chain.

C

And over e f like if you were trying to get blocked from genesis all the way, up, you've actually hit one of these, um, abi bombs that if you will, where, if you were to give it infinite.

C

Amount of time, it would be happy to take that infinite amount of time to try to ize that, that abi or at least close enough to infinite that it doesn't make any difference, right?.

C

I mean, measured in months or years to, to,, to piece their lives.

C

Uh, some of. uh,, if you're, clever enough to, uh, create an a b. I that, that is malicious,. If you will,, like that there, um,, is,, um,, yeah.,.

C

So you have to guard against that no matter what. and so we're, we're doing that. With, uh, greg,. Talking about the a b I max serialization time.

C

And so, uh, uh, uh,, currently in 4.0 and, and three and two, that abi uh, sterilization time was more,, was more of a hard limit over the whole thing.

C

That would, um,, even though it was kind of applied individually, once it was hit, often it would just draw an exception and you'd be done..

C

And that,, and so you were required to kind of like keep, uh,, making that larger and then you'd either get like a whole block, deser de er, alive,, or,, or, or not.

C

So in the proposal here, we're talking about applying that individually for each action.

C

And so that that setting that you set,, you may want to do, uh, in, in 5.0, when this comes out.

C

Uh,, you may want to also consider setting that lower..

C

I think a lot of people have that at 2000 or 3000, um,, which is probably still fine, unless somebody would actually.

C

Take the time to craft an a, b,, a bomb for, say,, every action within a block for say,, a thousand transactions.

C

Then, you know,. It would be a thousand times that that setting..

C

So if you have that setting at two seconds, it'd be,, you know, two seconds, times a thousand would, uh, for it.

C

So it was a thousand transactions in that block., and so you might actually consider setting that one lower as well.

C

Although the, the chance of someone actually creating a one like that,, I can.

E

Is there a way it's like,? I I, I kind of agree with like the issue, brian, a bit raised about this,, you know, in deterministic, uh,, stuff.,.

E

Cuz generally you're not going on cleos and doing get block.

E

you're,, you know, you're, doing it part of some code and then you, you're gonna have to handle these weird exceptions and there's no kind of built in way..

E

Is there a way where you can, as part of that, like get block,, you can say like, you know,, something where if a certain, you know, contract or a certain abi is taking too long,, can you request that.

E

E b, I as well as part of the return, so you can continue dematerializing it, for example, locally.

E

So you always have a path to keep continuing versus, not knowing kind of really the stitch you're in and how you're gonna manage it.

C

Uh, yeah,, we could potentially do something like that.

C

But again, like you,, if you were to do that, local and you get that avi and you tried to de serialize it, like some of these, can be drafted to take effectively infinite amount, amount of time.

C

Right. you know, so you're. trying to do it local.. You can just pause as well. like you had sat there for years, trying to decentralize this one row in, in a table or one actuator.

E

Yeah., I guess counter argument, would be that you can, you know, once you,, it's your own code,! You can set your own time. Limits.

E

Right. and mr. abor a call. so, and then like, you know,, ban that contract from index or, or whatever else like you might do.

E

But in reality,, like I, don't think, there's really use cases that will, you know, require that there might be better solutions for that kind of stuff. As history solutions, increment.

C

Yeah, I would,, I would think anything along those lines that we might talk about. Doing, which,, you know,. We certainly could talk about doing, uh,. You know,, should also include the.

C

The fix in terms of like currently to get block air serialization uses the current avi on chain,, which is not necessarily the avi I that's,. That's.

C

Associated with a given action at the time of the execution of that action.

C

So that issue,, you know, if,, if we provided some sort of thing like that, we,, I think we'd wanna address that as well.

B

So I think this is an interesting, it,. It in some ways relates to, um,, matthew, darwin in, in the chat.

B

Brought up a bigger picture question,, which is he said,? Do we really need a gi block?.

B

A p, I call that somewhat works, or should we just leave this to history, solutions?.

E

Uh, is it history solution, the ones that use gi, block?.

C

So that's a good question. like who uses gi block and for what?.

F

I don't know if somebody uses the hell out of it. heart. yeah,, they can tell you that.

B

If I'm not mistaken,, I don't,, I don't mean to speak for aaron who,, who can't be here for time, zone reasons or whatnot,, but I think he said that their solution is using gi. Block, like under the hood.

B

Anybody,, can anybody check me on that?.

E

I believe they use, uh, no. the, uh,. As far as I know, they're, like talking about their robo or like their internal history, solution.

E

It uses actually,, um, uh, the trace api and then they add additional decoration,, like, or information to that is not usually provided by trace. Api.

E

Uh, and then they provide that as part of their,, you know, like front facing api.

E

But I think the main reliance is only trace p. I, as far as I remember.

C

Yeah,, I think that's right.. I think they've have a, a, a long um, idea of actually providing a pr. That added adds the additional stuff that they've added in trace api,, but I don't think they've ever gotten. Around.

C

They actually grieving request., so yeah,. I think it's their own packed up version of trace, api.

E

Matt,, as far as I know,, they needed to add additional things, because they ever really incorporated the new action return value in their stuff.

E

So, you know, you'd, have to calculate the action digest. Differently.

E

The action receipt digest and all that stuff has gonna differ., so I think they were planning to do that. Work before potentially like creating that merge, or, or proposal.

B

So matthew and chat also added that yeah, we would probably still need gi raw block,, but not necessarily gi block for if,. If we left this to history, solutions.

C

I,, I think it's definitely in the realm of, of a history solution to do it. Correctly.

C

Right? because, because again, you know, we're not necessarily using the correct avis.

C

And dematerializing that block. and so to do it correctly need some sort of of history type solution, because an notius as it is today, doesn't have the data it needs to actually do that. Correctly.

E

Uh, we use gi block info on heads up.

E

We use it so that we can, uh, work out uh,. What's the, the last block that a bp produced.

E

Our bp, that we are monitoring and it'll, you know,, give the block it wise.

E

But that's, I actually,, I was thinking about it..

E

It might need to go into the, version. Five wishlist.

E

Some way to easily see this producer., what was the last pro block? Produced?

E

get block, doesn't use ebi.

E

like just really the header or like even a smaller version of the header.

F

E

But you, uh, extension. yeah, yeah, yeah., but that's what we were just saying. get block.. I say we use give block info quite a lot.

B

So could you re repeat that wishlist item for me, so I can jot it down?.

B

What,, what was it? an easy mechanism to see what the last block, a block producer or block producer count produced.

E

And, you know, and then from there,. I suppose we could derive, um, you know, when it was done.

E

But, you know, any kind of information, like if I,. If I did your filming, I, I, yeah.,.

E

What,, whatever it is., I mean,, we all,, we all do, like,. This isn't just a quick little api..

E

It's always like a little bit of skunk works. okay. we gonna monitor,, or these mini blocks,, see when we lost produced, check those blocks,. You know, mean us.

E

Something along the lines of like, yep, yep.

C

Block producer,, schedule, or whatever that call is.

C

E

Sunday. sunday. like that. yeah., but then it will say that this was the last block and this was the time.

E

Right., you wanna know whether they're, you know, 11 block and pit 12, for example.

E

Well,, that's the next,, that's the next step.!

E

But how about you get voted, outta, schedule,, um, and you don't know, or whatever it is.

E

You just wanna know when the last time you produced or your, your beef, bp went down.

E

When was the last block out produced? you know, like,, you can just expand, get producer jason, to include for example, like an additional like field,, which is tells you like last block would use.

E

Uh, yeah., I'm not sure, that's the best way,, but that's like one way.

E

C

But that, that's the type of information you're, you're seeking howie.

C

Yeah, yeah, yeah, yeah. you're, looking and, and or for, uh,, a given number of, of maybe the top 30 block producers,. Let's say when they last produced, how many and how many blocks they produced during that route.

E

Oh, yeah. yeah. it'll,, it'll be,, I mean, I understand where I'm coming, from.

E

I'm coming from a purely real time. Monitoring.

E

I wanna know, um, what has, like, if,. If I know it has gone down,. When was the last block that I produced?.

E

Right, now? um, our heads up platform, is, uh,, building a little.

E

Uh, history database of,, you know, when it's produced and, and it's not clearing the,. I I, I'm trying to get away from scraping a block log.

E

I want to be able to just build that information straight from an p note,. You know what I mean?.

E

So, um, well,, so maybe, um, is the promeus something you might look at there if it included that information?.

E

Absolutely. but there needs to be prometheus on the back of, uh,, like a,, a normal api node that supports it.

E

Not,, I don't want to rely on a something else..

E

I don't want to rely on a history solution to know that if I produced a block,, I mean, this is like base bone stuff.

E

It's like foundational stuff., I just wanna know like. Has it produced,? How can I check what the last block I produced?.

E

I know that I am in the schedule because that's easy to see.

E

You know, but,, but didn't I produce why?.

E

What is the last block? I produced?, I just, that's kind of nice and I think that'll help a lot of our monitoring. Solutions.

E

And the only reason why it's like top of mind is because you were speaking about get block and I said, oh yeah,. Well, we use get block info cuz, it's a little bit. Lighter.

E

And, um, and I,, and I know that we use that specifically to check when the last block we produced was.

E

C

So yeah, I mean, you know,, currently that type of information is not stored and that's not,. Of course, a large storage requirement to, to add that to, to a node.

C

If,, if we were to add something like that, if,, if transient tracking of that, uh, sufficient in terms of like, uh, as long as the node has been running long enough, to, to yeah, absolutely.,.

C

Question. uh,, then yeah,, then we could turn it., otherwise, the middle of the light,, just not know.

C

like if you shut it down and restarted it wouldn't necessarily know without like,, jumping through a whole lot of hoop, try to figure out absolutely something. Lightweight, super easy.

E

That just happens to be available. Everywhere, you know?.

C

E

And even if it,, and even if it's just,, you know, short term,, you know, something like that.

E

C

Just trying to back into that information, while the not yeah.

E

Okay. yeah. sorry,, I didn't mean to sidetrack where we were going there,, but we just said he might not get block,, obviously get block.

C

I mean, that's the reason for these calls, so that you can huff through things to brian.

C

He. scribbling it all down,, so yeah.

C

yeah.. Then we could no. coming, some coming back. Doing.

B

Yeah., coming back to the current proposal, uh, one,, one other piece like question for operators: is, um,, whether,, uh, folks, are running into.

B

These issues currently related on,, like in, in the real world, right.

B

Related to a b, I to serialization time, causing timeouts on, on these requests.

E

I mean, I've, I've seen,. Do we have longs? That would tell us this.

E

Theios nation validator?.

F

It tells us the sterilization is sick.

F

too low. yeah. you get those heavy blocks,, they don't work.. That's true., that's probably the most consistent, to be honest with you,, but yeah.

A

F

I mean, you decipher it. In, in the.

F

Complaints about your api being broken when yeah,, but I mean, in general,, run it out of potato.

F

I can tell you potatoes,: it causes all sorts of problems.

C

Yeah. not without running like hvp and debug mode.

C

Um, you know, in 4.0, you could run it debug mode and actually see, uh, that information, I believe.

C

But, uh, otherwise no, it would just have,. You would usually would just have to report like what michael micro, I've then say. A.

C

This call keeps failing on why, you know, sort of thing,, which is, I think, how you discovered the issue in three, two, when,, when you started enforcing the hcp max, uh, response.

C

Time, you know, had that discussion. About, uh, happened to increase that to be at least the size of the a b. I max relation time for your kip blocks, calls to, to, to, to work.

F

I remember when either the default changed or the flag got added,, it really rippled, because it was like 30 milliseconds or something crazy. Low.

F

I just remember the, and that's why it's set all across.

F

But I could also tell you, you know, yes,. We run faster, nicer, servers,, you know,, obviously for, for bigger chains,, but you wanna try to get by on something slower.

F

Maybe it doesn't have to be speed. Demon.

F

yeah. did he oh, of that stuff. can, can start timing out pretty quick.

C

Yeah., I don't think the default would work on and gi block, holocaust,, all of them on any hardware. That's known to man, currently.

C

Uh, so, you know, the, the,. Some of those are rather expensive blocks to dematerialize the default.

C

I, yeah,, I think you're right. 30 or whatever it is.

B

What I'm trying to dig at.

B

yeah.? What I'm trying to dig at here is just to understand: that, uh,, whether like for the sake of argument,. Let's just say that we did end up deciding,, you know, as a community that.

B

This, this is,. This is better handled. Through, um,, you know, history, solutions,, but that's coming in,. You know, a couple versions or something, again, sake of argument, right?.

B

This is still worth doing now, for, for 5.0. If we're going in that direction.

E

I guess what I'm getting at is you're asking a question for which you didn't provide us any instrumentation for us to give you that data, right?.

E

I,, I think it's the question, right?. If you want us to tell you how our nodes are operating,, you need us to give us tools, so we can measure how our nodes are operating,, and then we can give you that feedback.

E

But yeah, I mean,. If we're gonna redesign, something in a few versions, then,, you know,. Is it worth doing an interim solution?.

E

I don't know., maybe not.

B

Yeah, just, just for clarity., that's a,, that's a hypothetical.! I don't,, I don't know that, to, to re-clarify the question,. I think right, is,. Is there a need for this hybrid hit, block?.

F

It sounds like that's what we're.

F

no, that's a,. I asked that question a few weeks ago in previous known operator, table.

E

Do we want that blocked it all?.

F

Well, if,, if the end result, is, well,, it's gonna have to do its best case effort and then it'll throw out trash if it needs to..

F

Yeah. and if it doesn't have the right a api,, it doesn't work. Either.

E

So,, you know,, if, like there's a lot of conditions on this thing, where it doesn't work.

F

The,, the ab, the,, the serialization and the timeouts and all of that,, I think, isn't just constrained to the get blocks.

F

Like I say. I mean that,, that's been all sorts of different, you know, whatever., so it's kind of a little separate issue..

F

I would say if the targeted question is about this hybrid, uh, you know, halfway working get block,, it does sound, like maybe that's,. Maybe it is time to say nine. If you want.

F

Here's the block, do your own work.

F

or if you want to go, get the old version of it,. Do your thing or use a history. Solution.

F

That'll be: how could you do it for it?

F

layer, two,, some kinda layer, two songs.

C

Yes., I think that's a good summary there, because yeah, the proposal- is, you know,, applies to get table, rose and, and really it get table. Rose.

C

I think probably the most important one to discuss in, in terms of this.

C

Proposal and yeah, we might talk about even get table rows eventually only be interested in a history. Solution.

C

But, um,, I think short term that's,. That's not going to be the case.

C

So I,, I think our proposal still stand something yeah.

F

For the lock- and I think is, is out of there, but get table row,. I do think that ability for it to snag a snapshot and say, fine,. He gonna rip for 30 minutes and.

F

Get one result of the pristine static, data.

F

Cause I mean,, I spend five minutes smashing a table to get eight, 9 million rows that state's changing.

F

So, uh,, I mean,, if that's included in this proposal,, that's two pieces and one thumbs up and one maybe not.

F

The get table, but keep in mind, shaq is second..

F

E

I think the get block is useful for a lot. Of, also, keep in mind,, you know, new developers coming into the space, and it is very easy to visualize for them, like to do a get block.

E

And actually see all that data simply for them without running apis and history. Solution.

E

Yeah. yeah. I get it., not the question, rely on it in code,. You know, like,, like that halfway.

F

You know,, I mean,. That's. The concern is, if the only way to truly get it working and not crippling it,, it's gonna halfway, return, results.

F

I mean, that's why I'm like, you're right. it, it's,, it's returning, inaccurate,, you know, a b, I information and potentially throwing things out. That, that might not be.

E

Well,, if we look at the viewpoint from a developer,, his like, or a new developer who's,, you know, exploring,, he is gonna, do get block generally on, like modern blocks or like,. You know,.

E

Recent blocks that are more likely than not, he is gonna, get all the information he needs.

E

Or somebody writing a smart contract and wants to check like how effective the block and his, you know, code is obviously not gonna run into an a b. I for realization, cuz you know, in starting.

E

With simple code,, I still see a value for it versus him being like,, oh, get block and cuz. He can't get the transaction right,. He can't get the transaction without a history. Solution.

E

So I think there is still value for the proposal for get block,, even if it cannot really be relied on a grammatic if there's value, and then it will trip up the developer.

E

When its like, get me the previous old block and whoops,, it has the wrong api.

E

Right. so first, yeah, for that simple use. Case, it's valid.

E

F

So then there might be a agenda item here too for the, for the.

B

Antelope developers call that steven has started running just around like use cases for gi block for, for developers, versus, um,, you know,, so I,. I think there's,. There.

B

Are likely very valid use cases,, but my naive, uh, assumption is that, like 99% of what they're gonna want to do is more get table, rows type, stuff,, not,, not block level stuff.

C

And I would wonder there, like if we shouldn't be, uh,, encouraging developers to use to trace api plugin, um, or,, you know,, and if that doesn't have what they need,.

C

You know,: maybe we need to enhance that so, so that it does, um, as kind of the.

C

The way appointing them to know carly, it,, it doesn't do a b. I certification at all.

C

Unless you actually manually provided the a b I I was about say,, I think that's the problem.

F

It's so clunky and manual and,, I mean, oops,. Another contract got added and oh,. I gotta go back and get it.

F

I mean, right now they certainly are making it more friendly.

C

Yeah,, we could certainly fix that. right?. I mean, it wouldn't be a heavy lift to have it cashed, a apis at the point of time when they're created, so that it always has to correct adis to when.

C

Serializing, uh, any call,, whether it be get transaction, or,, or, or gi block, you know, the, the whole what, one of the use cases for the trace api plugin?.

C

Um, well kinda after the back, but, um,, currently one's caseload,, the trace api plugin placement for the big one, history.

C

Um, plugin, right., which kind of what developers used a lot because it was not,. It was quickly not appropriate to be used on the main chain.

C

Right., just because of the gigantic memory requirement for the thing.

C

Right. but developers,, I think, continued to.

C

for, for long after its nominal use on you in on a cha.

C

On the on the light chain, was, was anywhere near missile.

C

And so you know, the trade api, again,. I think in some regard tackles that,, at least in terms of like all of our integration, tests.

C

That's what we replaced, you know, before all of our integration tests used to history, aide.

C

Now, all of our integration, jeff sheen's, media trace api, columbia, and.

C

It seems like load could enhance that fort developer.

F

That's also one important fact.. I know I heard from a lot. Of, and exchanges were typically the ones,, but anybody that's financial and really whatever,. They really don't like having layer. Two solutions: invite number one.

F

Obviously it's more complicated and clunky,, but you have to trust somebody else's set of code.

F

Is that when they lost that v1 native aspect and it was, well,, you've got my periodic fuse or all, you know,. All these other ones.

F

You now have to trust the rio team or the, you know, substrate.,.

F

And I would really didn't like, you know,. You need to have in the native code base of the node at least enough.

F

Maybe get raw block is enough where they can then retrieve their abis and or trace api. Whenever you statically.

F

But the more we strip off those, the cores and go ve the layer, tunes the,, there's a lot of people that don't.

F

Consider those options.

E

I think something that maybe like just the idea for future is like an option to either, you know, retain,, you know, the last x abis per contract or retain all abis for certain,.

E

Uh, you know, whitelisted or whenever contracts that might fit a lot of these use. Cases.

E

Cause exchanges are monitoring only a few,. You know, potential contracts, uh, like to satisfy this use. Case.

E

And then they can just really have that contract, save all this abi in that later, one in node dios, and then be able to either use trace api with this serialization of that.

E

Or, you know, get block would be guaranteed to be, uh, working for these contracts.

E

Um, matt be like on side queue.

F

Trace api., I mean,. If the trace api were a little better,, it's great at individual contracts.

F

It's the monstrosity of everything, that's really painful.,.

C

See, I,, I think so. you know,, the original, you know,. Originally it was developed word an exchange, right?.

C

That direct the whole point of why trace a plugin was created.

C

And, and that's obvious in its design.

C

like,, you know,, it didn't need the cash a guys,, because the only a b I that it,, that the, the.

C

Targeted user cared about was sio token, and that hasn't changed since day. One.

C

And you can easily just give it to it. and that's what it does.. That's a matter of fact.. That's in the, the code example., it's like,, here,, here's the one a b. I that, that you probably care about and you can load it and it'll work.

C

And so that's why,, you know, and it's more efficient because it only looks at the one, right?.

C

So that's,, that's why it was designed when,, that's why that has that capabil.

C

But yeah, it could be. actually, uh, enhanced to to, to certainly do more than that.

C

Uh, another option here, depending on people's use cases,, which is available today for people,, is the new, uh, gi transaction status. That.

C

We've added in in v1 that actually does use the correct a b I in all cases.

C

He uses the avi that that was in effect when the transaction was executed, and it gives you that transaction trace as it was, uh, as it is in block correctly.

C

So that's another option for an individual transaction..

C

It doesn't give it to you block,, but does give it to you correctly for a given transaction.

B

So just a point of facilitation, here.

B

we, uh,, it is 9 58, where I am,, which means we've got two minutes left in the scheduled time.

B

So lots of good discussion. um,, but what I do wanna do now is sort of stop any, uh, new points and try to synthesize and simplify, like what are the takeaways from this discussion.

B

So if anybody has kind of the,, uh, sort of bullet point level takeaways that we should track, throw 'em out for me.

E

I guess the main things we talked about was the new proposal.

E

Which everybody who commented on,? Maybe they didn't like a certain part of it,, but they definitely found value in other parts like getti, rose.,.

E

I don't anybody objects to the value it adds, and from my perspective,, even get luck. Has that, uh, you know, additional value.

E

My only thing would be that, uh,, that thousand hard quoted, uh, limit.

E

Uh,, if you can kind of override it,- and it seems you could at least if you want it unlimited with setting,, you know, de serialization, minus one,,.

E

That will also override the max road,, which just should be documented.

E

But I think, like for me, that works for, for our ut.

F

And, of course, we're all thrill about the h c t p,, you know, at the public,, a p I.

F

Threading., that was where we started. bing built for.

C

Yeah., even a 4.0 gi block is way faster.

C

B

All right. any other high level takeaways?.

B

Very good. uh,. I think that concludes our discussion. I'll hand. It back to daniel to close us out. thank you.

F

B

Worries. so I will, I, I lost my mute,. My mute button was hiding.

A

I'm back. thank you, guys. um,, so yeah,. Thank you brian.. Thank you. Everyone for your feedback and we'll meet again.

A

E

Take care, everyone. cheers.