Ethereum Foundation DevCon 6 - ZKPs: Privacy, Identity, Infrastructure & More Track, 15 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Zero Knowledge Machine Learning

Description

Visit the https://archive.devcon.org/ to gain access to the entire library of Devcon talks with the ease of filtering, playlists, personalized suggestions, decentralized access on IPFS and more.
https://archive.devcon.org/archive/watch/6/zero-knowledge-machine-learning/index

ZKML could be the key to giving the blockchain eyes, letting smart contracts exercise judgment, one-person oracles, and generally getting data on-chain in a scalable way. I'll discuss applications, how to quickly convert PyTorch models to zero-knowledge proofs, and the roadmap.

Speaker(s): Jason Morton
Skill level: Beginner
Track: ZKPs: Privacy, Identity, Infrastructure, & More
Keywords: zk

Follow us: https://twitter.com/efdevcon, https://twitter.com/ethereum
Learn more about devcon: https://www.devcon.org/
Learn more about ethereum: https://ethereum.org/

Devcon is the Ethereum conference for developers, researchers, thinkers, and makers.
Devcon 6 was held in Bogotá, Colombia on Oct 11 - 14, 2022.
Devcon is organized and presented by the Ethereum Foundation, with the support of our sponsors. To find out more, please visit https://ethereum.foundation/

A

A

For those of you that aren't as familiar I like to say that zero knowledge proof started digital signatures as ethereum is to bitcoin we're replacing a digital signature with a any program right and the thesis of sort of zkml is that the most interesting program actually to put in a zero knowledge proof is a machine learning model and the reason for that is that it makes it as though the model the AI is actually running on chain, so we're giving intelligence to ethereum or whatever chain it's running on um and it's kind of like we're, giving it Eyes Ears.

A

You know sensory organs for which with which to perceive the physical world um with which to make decisions about physical reality and the most important bits of physical reality to us, which are the humans right. What our intent is, um so it makes it possible for humans not just field elements to uh control their destination, to control digital assets.

A

So imagining that in the future we'll be able to say just like we talk to our smart speakers, you know hey ethereum, you know please transfer 10 F to Alfia, and that will work right, because some Ensemble of machine learning models will be able to combine all the signals of my intent. All the data exhausts that I've given off in my life and decide whether that is true or not, for a human that exercise is Trivial if we've talked for 10 minutes um and I.

A

Do that I make that request two years from now, you'd be able to authenticate and send that message. The fact we can't do it yet with uh with ethereum is just a just a technical problem um and, finally, you can think about it as a way to let smart contracts exercise judgment right deal with any kind of ambiguous situation. Decide if a contract is satisfied, decide if a news story says that a hurricane has hit the coast.

A

That thing- and this little chart here is just a reminder that both the input and the parameters we can choose either private or public and all four of those squares are interesting, everything's public. We can think about the news model, scalable Oracle. If everything is private, maybe we're thinking about decentralized, kaggle or some kind of um medical situation where, on the one hand, the model has to be kept secret for reasons in the patient data has to be kept secret.

A

um So what we've built is a program called Ezekiel. It is a tool to build turn Onyx model, which is an export that you can make from pytorch or tensorflow. So a baked machine learning model that's been trained into a zero knowledge proof. We can prove and verify at the command line, export to a binary or a contract, smart contract or wasm we're adding new layers to it daily and it's enough for sort of small production models.

A

The scale is increasing very quickly sort of two to eight times a month, depending on how much time we choose to spend on optimization, as opposed to adding features and I expect that we'll be able to do some really exciting kind of identity stuff within maybe six to 12 months.

A

Maybe sooner um and we've been building in the open since July, it's Apache 2.0 license and uh that's the repo. We would love any contributed contributions from you if you find this interesting, um so here's an example of the kind of thing we can do right now. uh If you like python, this is very relaxing. As opposed to writing circuits. You can define a forward model. X, Y and Z are tensors they're, with a shape.

A

That's determined at runtime and I can just sort of Define somewhat arbitrary functions of those tensors, Matrix multiplications, Powers, non-linearities, compose them, and then the tool will translate that into determine a quantization strategy, figure out the runtime shape of the tensors and translate that into a something that can be run as a zero knowledge proof.

A

um So the pieces we can do right now are sort of the describe. What we're going to do do a mock or a full proof is the back end is all Halo 2..

A

um We can do a complete, a proof and we can verify a proof um and then there's some sort of technical parameters and this applications talk. So I want to kind of give you a sense of what's possible and then we'll talk about what applications are cool um right. So this is an example of computing, the table from a very simple model that just has an input, a weight and a bias, a matrix multiplication and a relu.

A

um We can do a proof. uh Looking at different uh back ends proving back ends, and we can do a verification. The the proof will construct a file, a little Json file that can that contains the data of the proof that you can upload to whatever the verification thing is, and then you can run also the command line, to verifier, to check that proof um and okay. So now we go back to Applications I always like to show a little bit of code so that we can think about kind of what is the big?

A

What is it, what stimulate imagination? But I like to think about this is kind of allowing scalable, automated oracles right. So there's three stages to that. The first stage is ingesting some kind of signed signed data. Now it works fine with not signed data, but there's sort of an adversarial problem. So we need to somehow rate limit the amount of data that's coming in or or make sure that it's been signed off on by, like say a news organization or camera, or something like that. Then we run a model.

A

Maybe it's a text model, maybe it's image classification. It makes a decision about what the data was and finally there's a unchain verification that then feeds back into the sort of attestation Loop and can be used in the next model, um so interesting things in the sign ingestion. This is this thing. This space is really an interesting stage, so there are solutions, there's something called a sxg s XG, which is a signed HTTP standard promulgated by Google kind of replacement to amp.

A

um There's an nginx plug-in. There's a one click button on cloudflare, a lot of people promise to do it. Almost no one has there's email. There's a few other people from Xerox Park that have worked on the email that you can hear about from ayush.

A

um There's images at the publisher, so the news organization, New, York, Times Etc- might sign off on their images so that other people can say. Oh New, York Times said this bad thing happened um and then they're third-party notaries, like TRS notary lip protocol that you can use to um create signatures uh as sort of a semi-trusted third party, and they all use things that we can now verify. um But I think we need we're going to need an industry at a push like https to push everyone to sign their data. It's technically easy!

A

It lets us compose things in zero knowledge proofs where people just don't bother to do it right now, although they've all promised to uh so. The second thing is the text model classification? What kind of models can we build and that's sort of a scaling process?

A

um I describe our roadmap as kind of ontology recapitulates phylogeny, which is to say we look at all the models that have been built over the history of uh of machine learning and then you can download them. You try to run them to your tool on.

A

It doesn't quite work because they're scaling problems or quantitation problems or because you haven't implemented all the nodes and then you fix it right and then you repeat so, as I said, open source project would love you to join me on this rotation, and then we have to worry about scaling and there's kind of five ions that we use for scaling. So optimization, I won't say much about that.

A

I think you will talk about that a little bit in his talk, aggregation, which is a tool for combining multiple proofs into one proof and checking it checking it once recursion, which is being able to verify the last snark inside the new one and that kind of lets you do something like uh doing a separate proof for each layer right that gives you another scalability tool and between occurred, aggregation and recursion.

A

What you're going to see is the is instead of memory constraint, which is what usually the problem is in for the size of the models we can handle. um It's just money constraint right how how much, how much computation we like to spend? It's not, it has a lot of overhead, but it kind of eliminates the limits on how big a model.

A

That's why I can make grandiose claims about the size of the model that we're going to get to um and then finally, Fusion is kind of the strategy of once we have a higher level understanding of the intent of the programmer, the computational circuit that they're expressing in python or whatever, it's easy to swap out more sophisticated zero knowledge arguments, instead of looking at the level of constraints, I, look at the level of matrix multiplication or convolution, and you can use those arguments and make it invisible to the person who's created, the um the the machine learning model, the machine learning model doesn't change at all, but it kind of gets faster and faster in the back end um right.

A

So, finally, on chain verification, um uh this is, as I think, you've heard uh you know severely constrained by the pre-copiles we have available in ethereum and a lot of the the strategies just getting it to fit into a bn128.

A

um So the way that it can be done now is you have, and probably the future honestly, just because of the nature of of zero knowledge proofs is you have a first stage, which is you know, some sort of input, and then you have easy to make yet hard to verify proofs right sort of wide proofs.

A

Those proofs are aggregated uh using an aggregation strategy which might require a fairly Hefty machine. So one example: I did use 450 gigabyte machine RAM and then that produces it hard to make but easy to verify proof that you can then be checked on ethereum and right now it costs about 600 000 gas to do that, but there's some things that we should hopefully improve that over time um right so putting all this together. What do we get? We get kind of this scalable on-chain data feeds you can sort of think of it.

A

As oh ethereum can read the news, or instead of having a network of nodes that make a decision like in a chain link about something, that's happened and then vote on it and there's a complex, crypto economic process by which they penalize people who, who tested wrongly you just have one person they download the signed data from Bloomberg. They produce the zero knowledge proof they upload it.

A

And now everyone can trust that information, so that enables a much more scalable fire hose of data coming from off chain from web 2 to on-chain, um and it's my belief that uh zkml will actually be table Stakes for chains over the next sort of five to ten years that, while delivering on This Promise of blockchain to a mass audience means that we really need this kind of a human. Not a field element owns all my assets right. There's lots of solutions.

A

I've heard at this at this meeting towards that, and this is part of that you can imagine. For example, when the account abstraction starts working that part of the account abstraction check, that you're doing is to submit a zero knowledge proof of your identity right using this kind of 10 000 Factor. Everything about me um as one of the pieces, um so zkml oracles will be simpler, faster and more scalable um to put arbitrary on off chain data on chain and just I think really opens the fire hose to what we can do.

A

um And finally, as I said before a zkm model, you can think of as a smart judge that interprets ambiguous events and in the remaining two and a half minutes. Let me just talk about a couple of examples, ideas of things that I hope people will build with this.

A

um So one that's sort of timely is zkyc right, so we can take a person and an ID and prove that they match and that the ID number is not in some sanction database right. Okay, of course, that's technically something we can do very soon. However, Regulators won't accept it right. Banks have a kyc rule. It has that it's. It says specifically that they have to know the customer not that they have to know the customer is not on a sanctioned list, so it seems like it won't work.

A

However, maybe if we had done that, it would have prevented the tornado stations right, it does if you're a D5 developer or your mixer developer. You add that to your pipeline, it sort of makes you less of a Target. You know it lets. You run faster than your friend when the bear is chasing you, um or at least let it uh does something to prevent. You know unwanted actors from interacting with your contract.

A

So even though it's not perfect, it's interesting, um of course. I talked about prediction markets. You could imagine setting up a a contract that pays if a news story classifies to a particular thing. Someone won an election, a hurricane of a certain intensity to hit a coast, a car received a lot of damage and then a small classification model, because it's relatively few classes can be used to to decide whether that happened and anyone can you know download the science story run the model submit the proof.

A

Another thing is kind of fraud checks, so gut checks for smart contracts. You could imagine uh the abstracted account or the smart contract just has another little zkml check, that is a rate limiter and a fraud check and makes it harder for people to scam that contract right.

A

um A proof of humanity, for example, would work but lots of other kind of fraud, checks or checks of the state of the network, and you could imagine getting baked in a little bit like civil protection, but it's okay, if it's weak, because it's it's just a layer of security um I think it's exciting to think about this as putting the a in Dao so really making taking a situation where humans make a judgment over what happened, vote and then some multi-sig signatories uh actually execute that judgment and actually putting that all with an effectively on-chain AI right.

A

So, instead of working for a doubt again, we all work for the the all-knowing AI in the cloud. um It's gonna be good or bad, but interesting, uh and you know they can use, for example, to test whether someone who promised to do some work for the Dow did a good job did the work right. You can imagine making a classifier that did that um you can think about using it for genetic screening situations where, for example, the model has to be secret because it's been trained on on non-consented data.

A

The data has to be secret. You know just like when someone wants to get an STD test or something they would like to choose, whether or not to reveal it to anyone and maintain anonymity.

A

um So that's another application, um and finally, it composes well with not just NPC but also things like differential privacy. You can imagine a data owner. uh This has nothing to do with blockchain necessarily, but you can imagine a data owner that wants to protect their data.

A

They can release noisy summaries in a differential privacy sense and then receive the the the query or can run the model and torture the data and send a final query, which can then be run in a proof to show the real model matches the commitment it was used to create the noisy marginal summary and when it was run on the data It produced the real result. Okay, so income out. Thank you.

B

Thanks Jason, um if you want to take one or two questions um well, his son gets set up Assuming he's he's here here. Hello um uh Eastern is going to talk about another aspect of DK machine learning, specifically in neural networks, but did I see a question over seven questions. Yep.

C

Machine learning models are often kind of already black boxes. How do you verify that the right machine learning model did the prediction or the verification yeah.

A

You you can you commit to the model, so that's particularly easy when the model is presented as an onyx file. You just take the hash of the file and that's your model, for example, or you can commit to the parameters themselves, so you make you make some kind of basically a hash of the parameters or hash of the whole thing, and that's how you know it's the model. You want it to be.

B

Do we have one more quick one.

B

Everybody else can take them offline. Sorry, really.

D

Cool stuff um has the CK kyc sort of being worked on? Is that a theoretical possible application of something like this or like? What's the state of yeah.

A

I'd say it's: it's absolutely being worked on. I think it will be a little while before it's feasible, um but you know it's definitely something that I expect to see. Probably multiple people create in the next year.