Status Core Developer, 2 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Core Dev Call #39

Description

Core dev call #39

A

Sure hi everyone, my name is: I live in new delhi, india, I've been a bitcoin and crypto enthusiast for a very long time. I know status for almost two years now I've always been a fan, and today is my first day I've joined in as a core team developer and I'll be helping build the closure application.

A

I'm very excited to be here. Thank you.

B

Oh well, I probably- and they have more information about specifically rpc call stuff, but the problem is that we are exceeding um our limits on infra and we need to optimize the way how we fetch uh transactions history.

B

uh The main problem right now is that, uh in order to like fetch, live and new transactions, we execute get block each 10 seconds, which causes huge amounts of this requests.

B

And we actually don't need to do this in many cases and specifically for users who don't want to use the wallet, because many users are just joining status for for chat not for for wallet.

B

What we can do here is to stop fetching this blocks each 10 seconds, and the first thing we can out there is to fetch transactions by user's request, so when he actually want to update they want to update history, another thing would be if we sent transaction from the application, we can like uh resume fetching of each block for some short period of time until we uh have transaction confirmed on chain and also when we receive a message, this transaction, we also can start listening for new blocks and watching for that transaction and then stop doing that.

B

That should save a lot for us. um There are another thing which we can do is actually a different way to watch new transactions, so we we can instead of fetching.

B

Then we get block by number request. We can execute several uh other requests like check, balance and check transaction account and that's how we can actually figure out if there were incoming upcoming transactions.

B

uh The problem with that is, it actually will produce even more requests than get block.

B

It will cost less traffic, but it will cost more requests, and what can be done is that we can uh instead of doing this every 10 seconds. We can do this like once per 10 minutes or so uh and again. The the problem here will be that it won't be actually live. Yeah it will. New transactions will appear with some delay, so we kind of can combine a few different approaches here and, in the end of the day, reduce the number of requests.

B

Probably someone else has other ideas how to solve it, but that's uh probably the best we can do for now, because we don't need to add too many changes on status quo side. Probably we don't need to do any changes there. We have like start stop methods for transaction history, uh which we can use on client side and all we need to uh is is to like use them properly and uh do not watch for new blocks when this is not really necessary.

B

That's it yeah from my side.

C

I have a question on that roman uh written mostly just to get the the requirements clear. What, if you can pinpoint, is the problem with fetching too many transactions, because I would like to understand: is this an.

B

C

B

No, no, that's not about fetching too many transactions, but about making too many rpc requests because yeah we have some limits on infra and we for some time I I'm not sure, probably last month or so probably and andre knows better about specifically this side uh yep. So we need to solve this because if we will have like two times more users, we will even we will exceed the test limits even worse. So.

C

B

C

It's a subscription so because I I have some context around this, but I'm not sure to what extent this is purely a subscription issue or us dealing with infura or if the problem would be the same if we were to use another gateway, because I do want to make sure we don't compromise on the experience of something so core as the wallet and making sure your balance is up to date or your transactions are up to date because of.

B

The subscription issue- yes so well, um as I said uh it's only about infra right now, but I don't think that uh switching to a different uh like service uh or well, our own notes will be much better because uh we'll still make tons of requests. Specifically, these get block requests which are heavy and another issue which we actually have. But it's not noticeable, uh probably for some users that by making those get block requests, we actually require a lot of traffic, because each block is huge.

B

We request each block with all transactions in that block uh and well that's costly for some users, and they also. I I'm sure that this has some impact on battery as well on the ground and stuff yep.

C

Awesome. Thank you.

B

Not well, most of those requests do not require call not so mostly all requests which we are executing right now do not require a call request. That's like uh for last five blocks, or so uh this get blocked by number. We do not execute it for uh archival yeah. We only execute it for recent blocks, so we actually need some requests for historical data, but that's, usually you get balance and get transactions count and sometimes it itself they get blocked.

B

But I assume that I, I think that, uh like 95 percent of requests are get blocked by number requests we have now are for recent blocks, maybe even more it's hard to.

B

D

D

B

We actually can uh unload uh non-historical requests to a different service right now. That's kind of a separate issue here, but anyway, if we will optimize the the way how we fetch this new blocks or how we check for new incoming transactions, we will save a lot of traffic and also we will save a lot of requests. Even if you choose another service, because I'm not sure that their capacity will be like unlimited for all these requests.

B

Anyway, we don't have too many users right now, but we execute uh like millions of requests anyway and it won't scale at all as it works now.

B

Well, that's like two separate issues uh still for in my opinion, because the one is the way how we actually fetch history and another one is uh where we wrote those requests and whether we actually can execute them at all uh and if either user has to pay for them. That's like a different topic.

E

One question: uh have you kinda evaluated to um move to a different protocol, basically in a in a push way, let's say that for especially for status to status, transactions might cut off some of the requests that we have to do. If there's a welcome notification with the transaction hash, so we can look up that directly.

E

So this is basically would maybe improve performance or cut down on requests for status to status uh transactions.

B

Well, well, if we have some api, which will uh just uh instead of sending requests, each 10 seconds will like receive notification when we have a transaction, that's nice, but but I'm not aware of them. The assumption was that we use like a standard ethereum rpc for all this stuff, uh at least at the beginning when it was implemented, and I'm not sure if there is uh like a better way to watch the history than.

E

We I think we could also like make this sort of like a um a standard, that, if you want to send the transaction to a status wallet um you are, you should uh send a wacom message informing the receiver about uh uh income and transaction. I think that that would for some.

B

Cases not all transactions will be sent from status. No.

E

Of course not, but you could make it like an open standard that other wallets might also use to to kind of improve the whole thing, because the issue that we're seeing probably a lot of wallets have- and you don't need to fetch the whole block and a lot of history to kind of see. Okay, there has been transactions sent to my.

B

Wallet yeah! That's nice thing, though, though, I'm not sure this is like a great thing to to to fix to to make it work.

E

Yeah, I just want to bounce a gear on.

B

Well, that's it from my side, I'm not sure if someone wants to add something.

D

B

What do you mean I created like right? I don't remember even about this issue. I need to check it that was that was years ago uh or a year ago, or something like that. I.

D

B

I I will check it uh after the call right, uh yeah yeah. I will update it if it's.

D

F

The main issue is personal sign, no yeah yeah. So it's uh actually it's not the first time that metamask implements something without specs and we have issues in steroids.

F

We had this before a few times and now so where there is a spec for personal sign and actually I'm not sure there is no specific. There is no specification about the parameters, but there are on the. There is only example, but most dubs they I I mean.

F

Probably it's um it's expected. When you interact with web free, you send parameter as hex values and uh most dubs do this and they sign and send message as a hex value and we convert it to utf-8 and show it and sign as hex message as hex, but at some point with a mask. They implemented that it's possible to sign stream and they just check if it's string, they convert converted to hex and sign, but there are no. There is no spec for that and most dubs they work fine. They sign the hex value.

F

So I don't know if we should like just implement the same thing, how it works in metamask or I don't know. I actually, I know only one dot, it's I think I don't know who is working on this dot and they send it's like a string or it's because they use a library, and this library probably is.

F

Probably more dubs used, I don't know so there is no yeah. There is no spec for that. So the only thing they can do is like to implement support for strings.

F

D

um Yeah, that's a good question, because the on best of the way we are doing right now is determine if the string starts by 0x and if it contains hexadecimal string, meaning characters from 0 to 9 plus a to f. And if that's the case, we assume that the value received is a hex. Otherwise it is a normal string.

D

So I think that unless there is a domain that starts by 0x and hexadecimal value, we shouldn't have any problem at all. By converting the the value we receive to hexadecimal string on otherwise assume that it is a hexadecimal string.

D

I I I don't think there will be an issue at.

D

D

D