Ethereum Foundation DevCon 6 - Developer Infrastructure Track, 15 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: How to Integrate Diverse Sources of Reputational Evidence: Learnings from the Lens Protocol Repu

Description

Visit the https://archive.devcon.org/ to gain access to the entire library of Devcon talks with the ease of filtering, playlists, personalized suggestions, decentralized access on Swarm, IPFS and more.
https://archive.devcon.org/archive/watch/6/how-to-integrate-diverse-sources-of-reputational-evidence-learnings-from-the-lens-protocol-reputation-system/

Many people are working on reputation systems. Many are computing reputation scores as linear models, which is less than ideal. I propose a new way of combining sources of reputational evidence based on Bayesian updating.

Speaker(s): Pedro Alcocer
Skill level: Intermediate
Track: Developer Infrastructure
Keywords: machine learning,data,lens

Follow us: https://twitter.com/efdevcon, https://twitter.com/ethereum
Learn more about devcon: https://www.devcon.org/
Learn more about ethereum: https://ethereum.org/

Devcon is the Ethereum conference for developers, researchers, thinkers, and makers.
Devcon 6 was held in Bogotá, Colombia on Oct 11 - 14, 2022.
Devcon is organized and presented by the Ethereum Foundation, with the support of our sponsors. To find out more, please visit https://ethereum.foundation/

A

A

Coaster, I am a staff data scientist at the Ave companies and I'm the lead data scientist on the lens protocol and uh my talk today is going to answer this question. How do I integrate different sources of reputational evidence? Reputation is something that lots and lots and lots of teams are working on right now and I'm. Seeing some patterns that I think are less than ideal and I want to sort of address that uh my whole talk really is like in this slide right.

A

So the answer to the question I just asked is what I want you to do is represent your beliefs about your reputational signals as beta distributions and then do Bayesian updating to combine them and I'll sort of explain what all these things mean. Okay, so this is what you should not do right, so lots of teams. What they're doing is that they're, taking a weight vector and a signal, Vector they're, doing the dot product and out comes your reputation right.

A

Another way to think about that is like this, so you know you have your weights, you have your signals, you you multiply them up and then you sum them up and then out comes a reputation.

A

Number I think this is less than ideal and I'll explain why so um information about uh the variance of your signals is lost when you do what I just showed and variance is important to know, because it's a measure of quality, it's how you represent the quality of the signal and I'll explain a little bit more about what I mean by quality in a minute, so what it means, if you were throwing away, the variance is that you're throwing away quality information.

A

So now you don't know if your final reputation is coming from lots of like sort of low quality signals or maybe a combination of high and low or you don't know anymore. This is what waiting looks like without variance right. It's just one number. So in this case it's maybe like 0.8, and that's really all you get you get this like one weight. But what, if you did it like this right?

A

So you still have this uh 0.8 sort of mean uh Point estimate, uh but now you have like confidence intervals around it right, so the the underlying distribution behind and this weight is actually something kind of wide right that this is what I would call a low quality signal versus something like this, which again is mean 0.8, but has much narrower confidence intervals, so we're much more confident about the value of this weight than the previous one. This is a high quality signal, so you know yeah.

A

Now, if you have these two signals, one with uh sort of a high value, but maybe low quality, and maybe one with like a low value but high quality. You know what are you doing to combine these? How do you do that um and uh part of the answer is that you have to represent these as beta distributions, so a beta distribution is just a kind of statistical distribution which is bound between 0 and 1 and takes two parameters: Alpha and beta and I'll- explain sort of what these parameters mean in a moment.

A

So um look like just do some concrete examples of some potential signals, so a really high a high quality signal is like. Do you have a pull app from Rave right? So this is a very difficult to Sybil uh signal that maybe I would say. Maybe one out of every thousand uh users uh who have this Robbie poapp are sybils, they somehow sibled it. So that's pretty high quality versus something like um having an ens address which is harder to sip. It's like medium hard disable because you still have to pay for it.

A

uh But it's you know, that's anybody can sort of get many uh ens addresses. So you know the beta distribution comes in where uh you know the the way to think about the alpha parameter. Is that it's the number of non-sibles that you have in like your sample and then you sort you.

A

You fix the uh beta uh parameter to one and you sort of adjust the alpha parameter accordingly to sort of match this like real intuition about like one in a thousand, our sybils or one in a hundred or one in ten, are sybils so then to combine them. uh We use Bayesian updating and for the beta distribution. Bayesian updating is really easy, so all you have to do is just add up the alphas and betas.

A

So to give you an example, if uh I was combining the three signals, I just showed uh I'd add up, you know the alphas and the betas I'm going to get this outcome like posterior distribution. That better represents uh my belief about the true reputation and sort of how confident I am in the like the final value. But here's like a little bit of a more wild example.

A

So um say you had 50 signals and they were all low quality, uh so beta three one, it's very like one in four are sybils uh so, but if you you know, do basing updating you combine all these beta distributions. The outcome distribution is beta, 150, uh comma 50, which is actually a pretty high quality distribution, because the uh the confidence interval is really small on it. So uh to sort of motivate this whole thing. uh So let's say you have these two outcome.

A

Reputations um again, like let's say this, one is like that that has that mean this one is like this and as this mean the means are identical, but uh this one we're much more likely to trust than this user, who might actually be kind of low, potentially or maybe really high, but maybe low right. So uh what we're asking this guy?

A

We really know that that's sort of where reputationally uh this user lives, uh which hopefully maybe has a little bit more contacts now, and maybe it's a little bit easier to understand what I want you to do- is to uh represent your beliefs about reputational signals as beta distributions and then combine them using Bayesian updating, uh so that is essentially Bayesian inference in like seven minutes. uh So thank you for listening.

A

um I am still out of breath from this this altitude. uh My name is Pedro coaster. These are my uh contact details, I'm going to be releasing a lot more information about this kind of approach, which I think is um kind of a very useful approach. It's the one we're going to be using for the lens uh protocol reputation system, so you can find me on the leaf apps payalco.lens. uh The Bird app at Palco and then on telegram, calco XYZ. Thank you.