Ethereum Cat Herders PEEPanEIP, 31 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Peep an EIP #4: EIP-1571: EthereumStratum/2.0.0 with Andrea Lanfranchi

Description

EIP-1571: https://eips.ethereum.org/EIPS/eip-1571
Slides: https://drive.google.com/drive/folders/1apZrP9vyr5yQrFxBlXkCsD1KgaevBieW

Follow Andrea Lanfranchi at Github @AndreaLanfranchi & Twitter @Lanfra68

Contact Ethereum Cat Herders
Discord: https://discord.gg/sgdnxZe
Twitter: https://twitter.com/EthCatHerders
Medium: https://medium.com/ethereum-cat-herders
GitHub: https://github.com/ethereum-cat-herde...

A

A

I'm fine! Thank you. What about you great so it's early morning for.

B

You 2 30 in the afternoon. No, it's uh eight 30 minutes past 8 pm almost dinner time for me. Oh, are you? Are you feeling reasonably healthy again? Finally, yes, I had a bad august. Unfortunately, and uh well you know I yeah we, besides being the the center of the infection of kovid here in italy, I also had some personal issues that completely ruined my holidays, yeah. Well, I've stayed healthy for quite a while now, but I haven't been out to expose myself to anything.

B

Okay, I'm afraid my immune system is just gonna, go to sleep.

B

Good to hear both of you are healthy. We want everybody to stay safe. It's like dangerous time.

B

Well, these are interesting times beside being dangerous, because everything here at least here is changing quite rapidly.

B

uh Also, all the economic paradigms are breaking down and also the social habits are being quite affected right now, because um you, you see looking out of the window, how many few people are running and how many are, can I say frightened, frightened by by by contact by uh persons who do not respect social, distancing and so and so on, so on and so forth,.

B

Yeah people are maintaining social distancing, it's important to be safe, but yes, of course, we miss like that community vibe around yes, and also all the official information is coming from the government and so on are quite confusing because uh on behalf of the moment, if the economy is important, then.

B

The the infection has gone away and every everyone must reopen their activities and so on. If social health is more important to them, be warned that the infection is still here and we need to take care of social distancing and so on and so forth.

B

In a month we will have regional elections here and uh all the candidates are saying: no, there is no problem, we can go everywhere and it's always the same story worldwide right, it's everywhere, even in united states. I think this year we are gonna, see elections in the november, so yeah yeah we've got. The colleges are all opening up like this week, a lot of them, and next week, last week and they're coming up with hundreds of cases.

B

You know right away, and many of them are just shutting down or going to shut down again because they don't you know they don't have enough testing um to to make sure that this many people can can work and study together safely. um Yes, we also have some fancy ideas, because in first grade schools, our minister of instructors have came just came out with the idea to fit wheels on under the desks, so uh the kids can self this distance to them in the better way possible.

B

Well, if they don't have decent ventilation in the room, it doesn't matter. Of course,.

B

Yeah, we are exploring new ways, new ways, I'm a musician and what I miss the most is to be able to go out, and you know just to open mics and things and perform and all of that shut down, and even if I could go, you know you're close to other people, you're sweating, you're sharing, microphones.

B

You know, when will this happen again? Yeah? Yes, um oh very soon, so before we go ahead and start with this um andrea, I would request you to uh check. If your screen is like you know, we will be able to share your screen because sometimes june's come and create some issues. I will try to uh okay, just a second.

B

Can you see it? Yes, we can, of course well. This is not the point uh where I wanted to start from, uh but uh which is this one.

B

Okay, uh so before you uh start, let me uh start recording and introduce you for this episode, and then you can go ahead with your presentation and talk about it. Just let me know if you see the initial slides.

B

Like right now, what we can also see here is your eip screen.

B

B

You should see a slight introduction right. That's right! It's basic mechanics in crypto, okay, okay! So we'll start in just few seconds.

B

Hello: everyone I, with another episode of pete and e. Today we are going to talk about eip 1571, ethereum, stratton 2.0, with andrea lafranchi. We are joined by william schwab and alita moore from our own ethereum catalyst group and greg colvin. An experienced eip editor, who happened to be the champion of another mining related proposal.

B

Eip1571 is a standard track interface eip, which is in draft status. The.

A

B

The eip is provided in the description below welcome to the show, andrea, tell us something about yourself, and then we will dive into the proposal followed by a question answer session from the community. Okay.

B

Thank you very much pooja thanks for having me, um I am a former developer of heath miner, which was which is actually the so-called official miner provided by for as an open source product project led by um paul bilisa, which is one of the core developers and I've been working on youth minor for almost two years, and I had the opportunity to drag very deeply into the stratum protocol issues and from that point on, we got me power and marius van der veeren with the idea to try to set a default as a standard for the stratum protocol which is so used by miners worldwide.

B

And this drops to the genesis of the iep we're talking about today.

B

I would like to spend a few words about uh what is the strutting protocol and how and why it is so important for the mining activities.

B

um If you look at mainly the twitter threads about mining, you always read that miners can do this and can do that. This can lead a great misconception because actually miners, as they are working nowadays, can we do really few things that basically, they only can run and hashing algorithm, which for ethereum is called if hash and nothing else.

B

When we refer to miners, we have to make a distinction amongst the mining nodes, which are a huge component of the ethereum ecosystem, but it's a complex component and very resource intensive component and hashing miners, which are basically the the persons who run a mining rig or some kind of machinery which can produce hashes so which can produce the work is needed to find the nonce, which eventually will allow the node to seal a block.

B

These two categories are completely distinct and completely separated they more often than not find.

A

B

In what are the so-called pools of mining, where multiple miners, multiple owners of this kind of machinery, can connect and receive the work packages uh they on which they will try to find the right solution to the ether hustle?

B

The stratum protocol is the protocol, the miner and the pool providers or the miner and the mining node generally implemented to exchange information, about which kind of work has to be done and which kind of a response the provider of the work, expects back to allow the ceiling of the block.

B

So basically, uh we distinguish the the two categories talking about them as work provider, so the mining node, which needs to seal a block, provides some work to one or more work consumers which are the machines which provide the work and actually do the task of filing the nonce solving the hash problem.

B

uh This protocol was at the very beginning um of the bitcoin era, implemented over http protocol or http transport, and nowadays uh I'm skipping, the the slides very very fast about I think puja will have, will post the links to the to the uh to the documentation.

B

In the resume of the of the call, um I told the http transport transport that payload, which is.

B

In the form of json object, json is an acronym, which means javascript object, notation, which is a notation which can basically any kind of data into simple human, readable messages.

B

The problem of the http http transporter is that requires the work consumer to continuously call the work provider for new information, because for every http request, no kind of state is maintained and the word provider which receives the call doesn't know if the next call will come from the same ip address.

B

So the stratum protocol has been slightly modified to is a slight modification of the initial http plus json messages to have the json message on the whole json message on a single line and transmitted over simple tcp sockets.

B

The advantage of such approach is to have a bidirectional communication which keeps the state of the session during all the conversation and.

B

Have the and keeps all the simplicity of the strength of the json object notation, even though all the json objects are not not any longer beautified, because the single message has to re to lie on a single line.

B

This kind of communication is actually the most used communication uh amongst miners and their providers. Miners meant as world consumers, so pull mining tools implement json protocol the stratum protocol, even some flavors of ethereum nodes, implement some rough versions of the json protocol of the stratum.

B

Protocol problem is that we lack completely the standard definition of this kind of protocol and during my years at the youth minor development team, I have to deal with uh tons of fancy implementations of this kind of communication, which varies from pool to pool from node to node and which cause a lot of problems for minor mining. Miner developers, minor software developers that have to deal with very, very, very different implementations and within uh the stratum protocol. Also, the json rpc definition and constraints are more often than not violated.

B

Another problem of the json rpc over stratum is that the whole communication is very verbal because it has, as you see in the diagram, every member of the communication. Every member of the object has to be defined with the label um and with bound with the values it carries.

B

So even though the tcp socket has advantages and helps um reduce the overall traffic because, as you recall, I told you, um the http headers are removed and the http polls are less frequent. It has some cons concert in the implementation.

B

Because we have some limits due to the nature of the tcp ip transport, but above all, the very uh huge problem of the of the structural protocol. As I told you is uh categorizable in two different parts, the first one is the verbosity and the second one is the so-called man-in-the-middle attack.

B

So if I can sniff the tcp conversation, I can insert on the conversation and, for instance, I can intercept when a miner finds the solution and replace the crediting address of the reward of the slice of the share that has been found with my own address, without having the need to have any mining machinery, and so not spending any single, a single diamond in electricity or in capital expenses to buy gpus or whatever mining equipment is needed.

B

um The mine, the the man in the middle of that is, may be immediately mitigated using uh secured connections using uh ssl or tls implementation, but nowadays still nowadays I see many many many many pools uh small pools, mainly uh not using this kind of protection.

B

So um just let me change the share of.

B

The screen to another.

B

Topic. Okay, so here we come to the.

A

B

C

Which is uh not directly related to the core of the ethereum ecosystem, because.

D

It's it's an attempt to define a standard to interconnect all the peripherals uh which actually carry out the mining tasks.

B

For the ethereum ecosystem um and which tries to amend some design flaws and which, among them the the most important, is about the verbosity of the of the protocol.

B

Here, I um I'm selecting the the main sample um I've been witnessing for well actually a few months ago, because the eip is quite old and making some calculations only regarding the messages which are needed to notify my connected miners.

D

For the race of new jobs,.

C

um To be mined, if a pool has to sustain the connection of roughly 15 50 000 miners per month, it has to spend only to send one message per block: roughly 1.8 terabyte of data.

C

This, not counting all the replies that the mining pool has to receive back and all the initial handshaking for all connections and disconnections of new miner mining and.

B

Mining machines and thus mining sockets and many mining accounts.

B

So the specification tries to minimize and compact as much as possible uh the message which are which are sent among minor when among providers and work consumers trying to eliminate that some actually, let me call them stupid informations, which are actually not really needed for the mining. Just to make an example to mine, a mining machine which receives a work package receives three informations: the job id, which is unique for pull the block header, the candidate block header and the seed.

B

The seed is nothing else than the hash of the current epoch of the ethereum blockchain, hey, which does not make sense to send this kind of information on every single block, because if I send it when the miner connects, I won't send it the the same information or another 30k blocks.

B

So basically, we could reduce the 256 bytes bits, sorry to to the to the mining, to the mining machine, so speeding up all the information, because we also have to remind that miners meant as work consumers so again as mining rig owners and their rigs are in a constant rush. One against the other, uh so every millisecond of network latency, which is same in communication, helped them achieve better results and also helps widening the audience of mining machines to even regions where network connectivity does not sustain so cannot sustain a large bandwidth.

B

So the whole specification tries to.

B

Reduce at most the number of informations, which are exchanged among work providers and more consumers, and also making some violations of the standard, json rpc rpc protocol and just in the.

E

Trying to achieve the purpose of having the most.

C

The smallest messages possible and to keep the the whole state of the session um within within the two uh within the two parties exchange exchanging conversation. So we have introduced new modes, new methods of.

B

Of information setting- and we have also designed- uh we have tried to design a protocol flow, so every kind of mining activity can find a really strict specification of how can it be carried out.

B

So basically, this is what the iep does and parts of the eips could be well uh implemented within the um the implementation of nodes of clients of all nodes which actually are ceiling nodes to dispatch work packages to their to their connected clients.

B

Of course, uh part of the eap takes into account. How pools should account the rewards of slices of share of shares to their miners, which is a part which can completely skipped, be skipped by.

B

Node implementations, but uh I think that uh this pretty much stands for a good standard which could have been implemented for the ethereum ecosystem. uh Actually, nowadays, uh it's worth to mention that the only so-called standard implemented.

A

For the ethereum.

B

Ecosystem is the nice hash mode, which is very basic and very specific for clients to connect to the nice hash marketplace. So that's pretty much. It.

C

And if you have any question, I would be glad to answer.

C

Sure, um maybe first what I'll do is I'll, try and open the floor up to.

B

Greg, if you have any specific comments or questions that you'd like to make um backing way up, um what are the technical and economic advantages to minors forming pools as opposed to just expanding their own operation?

B

Well, as I told the advantages are for sure, a mitigation of the uh man, the man in the middle attack, besides the implementation of tls, 1.2, secure connections and the second one is, of course, an economic advantage for mining pools that will save costs in bandwidth per month and by implementing a more concise and less verbose protocol could help by consequence, reduce the mining fees they charge on their clients.

B

Thanks andre you're welcome, maybe if I could continue it not exactly in the same line but close to it. um You talked in the beginning about how, in your time with f minor, you would get questions about alternative stratem protocols or different different protocols. Why is it important for there to be a unified protocol across across the different mining pools? So I understand that it would definitely make it easier to advise or help um other implementations if everybody has the same standard, but could it be okay if.

C

Different mining pools have their own custom implementations and they have to support their own model. Why have a standard.

A

Well, this is a good question.

A

We have to note that the eip 1571 is last almost two years old and.

A

Was conceived in the moment when the youth minor project was very lively uh actually, at that moment we developers.

B

Of the miner project, we're still hoping that the ethereum ecosystem could adopt an open source miner as a standard uh as as a reference implementation of mining activities for other closed source miners.

B

If miner project is now mostly abandoned,.

B

And I've experienced not much interest about having an open source reference miner in the ethereum ecosystem.

B

Maybe is this is due to the roadmap to proof of stake, which makes it not very interesting anymore, but at that time we proponents of the eap, still believed that there was the need of the standard definition nowadays. To be very honest, all the mining activities well, not all, but uh roughly 95 of the mining software's running on customers gpus are close to closed source softwares, which- and this is my personal note- more often than not violate or any kind of gpl license, because I have found a huge part of the of gpl code.

B

Even the original comments in the in the of the code in the binaries of the most well-known.

B

Mining softwares.

E

And but actually heath miner.

D

Which was which is the only open source, serious open source project which could compete.

B

D

Names like claymore or something else is mainly abandoned. We were a team of roughly 10 people and no one actually is committing any more on the project. So I agree this. The this proposal does not make any sense anymore if there.

A

Isn't a will to define a standard for the mining, the mining operations and if, in the future, the the proof of work mining activity becomes less and less relevant?.

A

A little bit um do you think.

D

That there is a relevance in terms of um framing it as a reference for other proof-of-work chains that are unlikely to make any kind of transition that are still gpu.

B

Friendly well, I think yes, because.

B

Don't shoot at me if I name the protocol proper, but if, uh for instance, uh uh this this kind of protocol fits perfectly uh within the boundaries of both hip hash and uh pro pal coin, mind coins.

B

I know there are several implementations of the ether algorithm for other chains. Some are alive. Some are dead recently. um Another project in the top 100 of the listings from uh the coins from the coin tickers, which is named raven coin, has endorsed and reinforced a switch to proppal algorithm, which also needs some definition of the mining set method, uh which is basically the setting of the epoch of the dag.

B

Of the deck memory area, plus the definition of the period, so I think it could help uh many other. My many other coins, some maybe other chains, I better say uh even private ones. uh If, uh if the standard is defined.

B

C

For the the specific purpose of the ethereum met as a public chain, I.

D

Don't know if it's if it's worth implementing.

D

C

Mean that you're happy with the status of the eips is, are you looking to have this eiep push to an accepted.

D

Standard at this point well, I think, uh on a personal level, uh that every father would be happy to see their children grow. um It would be a pleasant surprise to see the uh eap 1571 push forward and I would gladly contribute to it, but I think that the the possible implementation.

A

Of the possible switch to an accepted state.

A

Needs to go to undergrow.

A

A scrutiny of the relevance of proof-of-work money in future developments of the uh of the ethereum public care chain.

B

Which nowadays.

A

Seems to get to an end within uh I don't know if it will be a few years or or less, but uh in in the end will we will become uh with a relevance very, very, very near to zero. If, on the other hand, the purpose of the eap is also to try to define a standard for any kind of chain which is built on the peridolum of the ethereum public blockchain, which is also a possible market for actual ethereum 1x node implementations.

A

I think this could help.

A

I understand it's a very that's, a very good answer, so um one question I have for you andrea. It's about the implementation generally, an.

B

Eip template requires an implementation section, which I don't see here for this particular proposal, so I was wondering about what is the real implementation is. If mine are doing, I mean using this protocol, and if so, do you do you by any chance happen to know if any other mining pool are using like how we want to push it to get into the accepted state?

B

Well, um we have carried out several tests with the support of big flight, the owners of uh um ethermind.org.

B

um I I had uh some very important feedback from peter pratcher and they actually had um a private connection endpoint, where we could test the implementation.

B

As far as I know, they still have it, and even though it's not like to for a public connection, I had some developer connecting me trying to contact me and trying to implement their this kind of specific aspects within their mining pools within the development of their own mining pools.

B

But I have not knowledge of any public mind was implementing it yet for the for what concerns all the part of mining software's actually implementing it. Only uh if miner only finder has in its latest release, has the uh the ep 1571 implemented in the code. Nothing else implementation by besides doesn't need any pseudo code, um any pseudo code, description and or design, because it follows basically the json rpc specifications.

B

So so it's only a matter of uh parsing the json rpc message correctly. I was I was going to say this. This is an interface standard, not a core standard yeah. So what's important is you have you describe the interface in a standard way that anyone implementing can can implement an interface in whatever language?

B

um So I don't think an implementation section is nearly as important as for as for a core proposal.

B

That's good that you already have answered, because I am coming to you. I was about to come to you with this question as an editor. What do you think? Is it important for this proposal to move to the next state like right? Now it is in draft. uh So what is it that is missing for this proposal to move into next step? If I have a as for my understanding, it depends upon author uh if they would like to pull it into the last call or to the final step.

B

So is there anything that andrea has to do or anything that you would recommend?

B

I don't think anything is left except andrea, one wanting to push it. The last call and allocate some time to deal with that yeah. uh As I told um I would be glad to spend some time to have it step into the next stage, if possible.

B

um As I, I reiterate, the concept, um I think uh I I would not like to have um the this kind of discussion about the eip 1571 to get into a philosophic discussion about the very relevance of proof-of-work importance within the ethereum ecosystem for the next few years.

B

Because, due to my recent experience with the profound matter.

B

I don't want to create some sort of chaos within the community um about whether uh the proof of work is still extremely live and extremely important for the ethereum ecosystem. This said as it as the greg said, this is uh an interface standard.

B

I think and as I think that it could help open the market of private chains based on the ethereum paradigm, I will be glad to attend a call that might set the ceip.

B

So that would mean maybe going on to all core devs greg like what what would be the necessary to put it in I'd have to check, but I don't I don't think the court does need to have anything to do with this, uh since, since it's not a client implementation, it's it's independent of that.

B

Okay, it's an interface! Nobody has to talk this interface if they don't want to agree my mistake and it it's useful. Just in that sense you push it through it's there. It's a standard.

B

People can pick it up or not I mean it sounds like it would be useful to even within the ethereum chain to if you're setting up pools to have a standard you can follow and not be making it up yourself. Maybe I'm maybe I'm wrong about that.

B

Well, it looks like, along with the community, I am going to learn a lot from this series myself today. Today's talk literally took me to a wonderland of how mining takes place on remote climb that I would have hardly imagined. Thank you, andrea for sharing that with us. Thank you with that. We have reached the end of the episode today.

B

I thank you for your contribution to the ethereum community, also to greg william and alita to join us today. Adding more perspective to this proposal for people are watching. This show on youtube. Please subscribe to the ethereum cat headers youtube channel. If you have not done it already, um ech and guest contact along with the presentation slide that was shared in this meeting today, will be added to the description section below and there's an announcement. We are going to have greg talking about the onboarding eip editors on our show very soon so stay tuned.

B

We hope to see you soon greg again. I would like to thank you all and we'll see you with another proposal in peep and eep till then cheers stay safe. Thank you. Everyone. Thank you goodbye. Thank you.