Filecoin Ecosystem, 6 Apr 2020

Previous Meeting

⏯

youtube image

►

From YouTube: Gossipsub v1.1 - A scalable, extensible & hardened P2P PubSub Router protocol with David Dias

Description

You can now use a permissionless P2P PubSub that scales as the network grows and that is hardened against a multitude of sybil and eclipse attacks.

Featuring David Dias (Protocol Labs).

Originally recorded May 4-6th, 2020 at Ready Layer One.

Want to be a part of the next event?

Follow @ReadyLayerOne https://twitter.com/readylayerone on Twitter for updates!
Sign up to receive e-mail updates: www.readylayer.one

A

Hi everyone I am the Vidya and I'm part of the protocol. Apps team I work on ipfs. We peer-to-peer pork labs research and, more recently, I've been building a research group inside protocol apps with the focus on brazilian network research. The the talk I have for you today is about gossip cell scalable. It's extensible in Heartland, peer-to-peer, pub/sub router protocol I want to start by really taking the opportunity to be part here of the ready layer. One event so far, the organization has been amazing.

A

The set up has been working flawlessly, I'm really impressed like I I'm, always like inspired and motivated to participate in virtual events, but I'm always a little bit skeptical but I. This one is working so far. Thank you so much for putting it together. So what is the agenda for this presentation? I'll start with a motivation for peer-to-peer pub/sub and then I'll go into the evolution of lis Pierre Pierre pub/sub from flood sub to run themself to gossip self. So you can kind of like fall like what all the idea got developed.

A

Then I will introduce you to the security extensions. There are the name of gossip cell and last, but not the least I'll share with you. What is available for you to use today, and so, if you have been following with Peter Peter pops up for a while, probably the last two sections are the ones that are most interesting for you. So what is the motivation for peer-to-peer pops up so when we build ipfs and when we build with peer-to-peer, we wanted to enable real-time applications to work on the network.

A

We wanted to enable developers to build the rich experience that users got used to in the web to another world and we wanted to do it without relying on any piece of centralized infrastructure. So we wanted to make sure that people could connect to each other propagate updates at real time without relying on any piece of centralized infrastructure, and so our experiment and our first product was really orbit chat, and this is back in 2016 at DEFCON 2. We showcase orbit shot the first user of with peer-to-peer pops out an IRC, a Lac application.

A

That's worked on the browser, so just using jess, ipfs and jess leepu peer connected to other nodes and was able to create a chat platform without relying on any central point of communication. This was really great. It was a really great achievement. We were really proud and we kept building on it. We built collaborative text editors, then we started seeing applications occurring in the community like distributors and commerce, applications, social networks, even virtual reality and verge of mental reality.

A

Applications photo-sharing apps that then god repurpose as building blocks for building many many more apps and now the ipfs and we PP eko systems are kind of like exploding in terms of usage and as ipfs uses only papaer. Everything that built on top of my PFS is building on top of we peer. So why is pub/sub so powerful? Well, the first thing to note is that message: oriented communications is very easy to understand. It's very easy to build.

A

On top, you subscribe to a topic, you publish on messages on that topic and it has support for different topologies, for multiple types of interaction patterns, from blogs, to forums, to chats to social networks, and so on, like the the topology that you set and the pub sub router that you use really can benefit you, depending on your use case as very loose coupling right.

A

So we can separate the producers of content from the subscribers of content in and make them work in synchrony, and it's very scalable because, as the network grows, their network kind of like helps itself, so it adapts as a network grows to provide the the scalability properties that it needs to serve more and more users. But there are also many many many challenges, and so one of the challenges, especially we like we face- and we peer-to-peer- is that we are operating on a permissionless network.

A

We cannot control, joins or waves, so we cannot control or block someone. That is a malicious actor that is trying to damage the network. We cannot control the network topology, we don't have any comment Center, so we don't have the notion of like a broker inside a peer-to-peer permission, wise network, which adds to the challenge. Then there is a network churn, so the topology, the network itself, the mesh is rapidly changing because nodes come and go very rapidly and optimizing for latency bandwidth, usage or deliver guarantees is really a conversation about trade-offs.

A

Depending on the usage, depending on the interaction pattern, the developer asked to pick what is best for their users and be willing to make some trade-offs among these properties and of course, there are malicious actors that can come to attack your network because they have bad intentions. So, with these challenges we took them add-on and we have been like evolving with peer-to-peer upset for a while in and I just want to make a quick point to clarify.

A

So you hear me say we peer-to-peer pub, 7 and I, say gossip sub and now I'm going to even introduce one more concept, flood sub and so to explain this in a simple way. Think of leap, peer-to-peer pops up as an interface. So this is the interface that apps built on top of will appear, appear, consume to subscribe and the publish, and then things like flat, sub and gossip sound our routers.

A

It's basically the this sub system that teaches liquid appear and teaches the network how to set up its own topology, how to propagate messages, how to add notes to a specific mesh and so on, and so because we prepare pub/sub is an interface. We can have multiple pub sub routers that give us different kind of kinds of guarantees. Today we have two robust implementations for itself and gossip sub, but you can also bring your own.

A

You can also experiment in the way that we Peter Peter works, and you might have heard this in another telexes, it's very modular. So even the router is not the pin on a transport. You can have gossip sub over TCP. You can have flats up over quick. You basically pick your building blocks in and you develop on top, so the first router that we implemented was flood cell and the reason why why it was like really the simplest protocol that we could come up with.

A

That's at a really high level of resiliency: it used ambient peer discovery, and so what this means is like it uses is used singles, like my PFS main Network HD, to find other peers and the routing was achieved by flooding and the name flood sub. So you might have seen kind of like these summoning circles before in your networks, research exploration and I I pictured one here to kind of like explain out what sub works so in foot. Sir.

A

Imagine like these bull circles are the nodes that they have all of these connections, pitted in pink and on one nail wants to propagate a message to other than notes, so that node will use the channels that it has open to propagate the message to the other nodes. So in one round it reaches four nodes in a second round. It reaches all the nodes. So so far, so good right, like it, looks like a very good protocol. 100% message: delivery.

A

The problem with flood cell is that nodes will try to help each other by forwarding the messages as well, and here is the flood, because other nodes will try to help each other by forwarding your messages.

A

Suddenly, we are using too much bandwidth by sending messages that the nodes already had received and duplicating effort and ends wasting bandwidth that is precious what's up is very simple: to implement it's very robust, so it's very hard to attack it's very hard to miss a message given that everyone in the network is just like sending the messages to each other and really gives us this property of like minimum latency, because the like the the router doesn't pick any latency path, doesn't prefer any any path to minimum Melissa, just Peaks all the paths, so one of them must be the minimum wait to see, but like the problem is really bandwidth, inefficient and so to visualize.

A

This I have like this simulation, which exactly it's playing right now, and so this is the network of a hundred peers with five messages per second run for two seconds and and I've expanded that the time here by 10x, so that we could see it's it's going and, as you can see, there's like a lot of chattering in network there's.

A

A lot of connections happening a lot of messages, but at the same time we'll see that, like it's actually quite efficient, it will finish quite soon and this might sound very useful because if it finishes quite soon, then we achieve our goal, which is delivering all the messages but like we have to remember that we are always limited in bandwidth and once the we start like abusing the bandwidth that we have available, we might start choking in ourselves which then creates problems. So we started exploring and in the next pups are.

A

Rather let me implemented that we don't talk a lot about. It was really an experiment, was random sub, so random sub takes inspiration from flood sub, but rather than picking every single path to send the message, it picks some paddled random and it was less bandwidth intensive but like, as you may be guessing. It creates dark spots where, because now you're picking some paths at random, there might be a chance where there are some notes in which they never received the message because their path, so that note never gets picked.

A

So we knew like we needed something better, something that picked the best from flood sub, but that didn't use as much bandwidth, and this is where we get into gossip, sir. So gossip sub is a protocol that trades the latency for bandwidth efficiency, and this is really important- no to cooperate with each other on a self-stabilizing routing algorithm to make sure that the network rebalances itself to make to provide the best delivery to every single node. That is part of the network and also important it supports for protocol extensions.

A

So classic sub is not like a final state. You can always add more things to website. If you want so gossip sub is actually a hybrid of two networks, no to construct meshes for message, propagation and then notes use the remaining connections to spread up metadata, which we call the gossip, hence the name, gossip sub. So on the first network, which we call the full message network or the data plane nodes establish reciprocal viewing agreements that basically say hey, like all the messages I see. I will send to you all the messages that you see.

A

Please send to me, and so these agreements are really important, because it's all these meshes between nodes get set up to make sure that the the nodes get a propagation channel to all the other nodes and that they can trust those propagation channels by default. These nodes connect and establish at minimum 6 reciprocal peering agreements, but it's just a parameter. You could like change this value at any time for your configuration.

A

Then there is the other network, which is the metadata network or the control plane, and this is like a very dense network, so we basically take on all the other connections that we happen to create, and rather than sending the messages which are more expensive in size and in the number of messages appear, we send only gossip in this network and- and we use this gossip to inform other nodes of what we have received in what we are missing.

A

As notes join the network, they announce to their peers, the subscriptions of the topics that are interested in. So this is where the process starts, of, like establishing meshes as nodes will tell others the topics that they are interested.

A

So they will join meshes of the same topic so that they get the message faster notes, keep a partial view of the network so in if they learn about some note that has been publishing on a topic and if they decide to subscribe to the topic, they know which node to contact and as the network evolves like these subscriptions can be updated. So you can like subscribe and subscribe over the lifetime of a node.

A

Then there are really two key messages that I introduced with a Kazakh cell protocol, which is the graft message and the prune message graft is to establish those receiver whose reciprocal peering agreements that I told you about in pruning is essentially when you was on to no longer want where that agreement you prune a channel to dissolve the debt agreement at you ad will appear. Additionally, we also have support for message evaluators.

A

So if you have a use case in your application, where you want to have a custom message, evaluator to drop messages beforehand before they get forward into the network, you can also add that and when nodes send gossip, they send gossip about multiple types of information from topics that they have subscribed, messages that they have seen peers that are connected, subscription changes and and messages that they want that they see that someone as received, but they haven't, and therefore they are asking a network to get that message for them.

A

This gossip gets sent at a regular epoch by default is 1. Second, but again, this is just parameters you can always select whatever you want for your use case and again by default. We select 6 peers to send this metadata, so we also don't send the metadata to everyone at at same time. Otherwise we would be flooding at the network at every epoch.

A

We send this gossip and from that gossip, then peers decide which new reciprocal agreements to establish in case they if they detect they are failing to receive some messages or which peers to prune from their agreements from their previous mesh. If they see the fact they appeared only contributes to a topic that they are interested and just like for you to be able to compare. Let's see now the visualization for gossip, some, and so you can see now, comparing back to the previous one in fluid sub. This one is way more light.

A

You can see the Reds here are the messages like the the previous case. The yellows are the gossip messages. You can see that the network is way more balanced. We don't see nodes abusing as much of the channels, but we can also observe that, like it's going to take a little bit more time for the full propagation to get completed so again, trading bandwidth for latency and to make the drive the point home, let's see them in comparison side-by-side. So you can see now that on the left.

A

Well, you can even guess right, so you can see like who is float so and who is like gossip sab, not only because one is completely abusing the bandwidth that is available on the network, but also because it already has finished well, gossip stuff is still going what what our simulations demonstrated is if bandwidth is limited and as far as I know, we still haven't figure out to beat the speed of light, so we'll always be emitted by the bandwidth.

A

Floods appears will eventually choke on themselves if they have too many peers of too many messages to send, and so they start failing to deliver messages and gossip sub comes in. We better awaited see results in those scenarios where, basically, the flood cell network is constrained by by depend with so cost, except even even be faster in larger networks, which is really interesting, property, so kind of a quick recap. We repeat web stop and cost observe we get like the real-time updates.

A

If we wanted in the beginning, we get resilience to network churn, because nodes can like graft and prune as new notes join and like self sub. You lies the mesh for propagating messages you get to adjust this trade-off between by the bandwidth and latency and I can keep scaling as a network grows, but this is not all so. The this is gossip. So what we have been like really working hard recently is hardening gossips up and we call this the version 1.1 of gossip stem.

A

So when we were ordering gossip table, we study several classes of attacks in here. I talked about three. That kind of like encapsulate the majority of the talks. Anyway, it's just like to give you a notion of what we are dealing with so there's the cost of attack, which is the civil attack and the civil attack is when someone creates fake identities, basically multiple, malicious peers to either squad areas of the network or potentially put the peers in specific locations to degrade the the other Network.

A

Then another type of attack is the Eclipse attack, where a node puts itself in a specific position to eclipse to put on front of all of the nodes that he wants to attack so that it can control the messages that those nodes see and get. Then a third class is the spam attacks, where malicious nodes will just abuse their network to attack a specific node until it chokes on itself causing a denial of service.

A

So what about okay? So these are the tags we wanted to mitigate them, what our mitigation strategies, and so before, like going into the tail of each one of them, I just want to say, I'm, really proud of the team. They worked really hard on crafting a spec. That is actually really like. It's five pages long super quick to read, really easy to read with some examples.

A

If you are interested on like understanding the details, better I really recommend going through this FAQ I'll do my best here to try to explain these details, but please bear with me as I, try to explain this because it gets a little bit more complex from here. So the mitigation one. The first mitigation that we came up with is peer scoring.

A

So if you have been on peer to peer research for a while, you know that, like often one of the common solutions is really the reputation game, and so that's what we did, we have a peer scoring, so the peers can evaluate other peers for their contribution in the network from time in a mesh, the the latency to deliver the first message, they valid messages that they send forth topic, application, specific scores and so on, and so with the scoring.

A

We have multiple thresholds from sending all the messages to appear to not sending all the messages to appear to potentially just like remove that peer from our mesh and not consider that fear anymore. To contribute to the network, then on our mitigation is flat publishing.

A

So when we see that the the message that we are sending are not getting to the peers that we want, we can enable an optional flag to enable flood publishing, and so this kind of, like over loads, the mesh rather than just sending to the six peers that are on our mesh, we'll send to everyone that we are connected in, and so this is kind of like doing.

A

Floods of you know, localized manner, where we send our message to so many peers that then we get that guarantee that, like we eat some honest node along the way that will deliver the message that we have published, then we have also peer exchange on prudence, so before gossips have relied on ambient peer discovery. To find your notes now, when some connection gets lost, when some reciprocal peering agreement gets dissolved, the peer that is asking to resolve that peer agreement can offer, in exchange some new peers for that peer to connect.

A

So this way, rather than having to go to the HT, to find new peers, the network can help each other by providing a peer. This creep discovery service. Then there are spent protections essentially rather than doing a lot of work to avoid to responding to wrong messages. We just now ignore them and, like we free, observe spam being generated by some peers, we will drop their scores to the point where we can consider two to discard them from our mesh. Then last, but not the least, there is the explicit peering agreement.

A

So this is very useful for networks Federation's, for example, where there are multiple parties cooperating together, they can just say, keep a connection open between these machines. These data centers these laptops, these peer IDs that will never get dropped out of the mesh, so there will always be appearing agreement between those places so that organizations can collaborate until again. This is like a really high level overview of all these medications. I really recommend checking out the spec.

A

What I have for you is while wrapping up is saying hey, you can try this today, like gossip serve our reference. Implementation amigo is ready to go and other implementations are coming, so we have gossip, seven JavaScript in rust, infighting and in Java, and the themes are also working to get the version 1.1 input for that language ecosystem. We didn't stop only at the documentation. We actually did a pretty extensive evaluation plan where we developed multiple test cases and test plans to test our gossip selves, and all these medications behaves in large networks.

A

We used a new platform that, if you are interested, you can watch Raul session tomorrow. The platform is called test problem and it really enables us to run large simulations and get 25 all results of what are the performance and properties of the network that we are building. We, as the gossip sub team, have an extension to test ground, which is an integration with Jupiter notebooks into what you are able to do is grab any of those test plans that I previously mentioned open the test plan.

A

You can adjust the parameters, the number of nodes, the latency, etc, and then you can run it to visualize the results, and this is extremely powerful because he nevile's everyone- that is a researcher and enthusiast, a hacker to kind of like tinker with different parameters and find new optimizations for the multiple interaction patterns that exist. That users can like benefit from and I have one more thing, which is.

A

We produce 60 page reports out of like this very extensive and very comprehensive evaluation that we will be sharing soon expect in June to see this report published in the we pure website and the research topic all the way I website right now, it's in the review stage, well I'm, really getting to the end here. I wanted to make sure so I'm a slide just a beak and a lot of love to everyone that has participated in creating the gossip cell 101. But also everyone that has contributed to the peer-to-peer pub/sub over time.

A

Many people have contributed with code, go to reviews, opening issues, testing, writing tests, writing the spec, or even just for the research ideas where we take our inspiration from to view these kind of protocols to share with everyone. So this is all I asked for today. Thank you so much for your attention I'm happy to take questions on the time we have left here. Also, if you want to join the community, like always passed by our lip, you'll appear IRC channel on freenode.

A

You can also access it with a matrix with matrix or go to our forum at discussed with peer-to-peer that I own. That is it. Thank you and now I'll stop my screen share so that I can't see what people are asking. What comes after gossip self, so we have in the works a well in the next portion phase episode, so epidemic broadcaster ease. This is like more research on like scalable peer-to-peer pub/sub.

A

We have a speck in progress to discuss his ideas that pick inspiration from the main paper and yeah, that's like where we want to explore. Next, we also ask the resident lab: have an open problem in an RFP open for scalable pub/sub for peer-to-peer networks. So if you are really interested on observe and peer-to-peer pub/sub I recommend checking on Geeta Geeta comm /li peer-to-peer, slash notes.

A

There is a folder there that says open problems and then inside there is the scalable pub/sub open problem, which will it's a really good document to welcome you into what pop surveys? What are the challenges? What is the current state of the art? What has been tried in repeat appear such ffs.

A

What has been tried in the whole research ecosystem, from what we know and kind of like kind of defines a baseline of like what would be the ideal goal to inspire people, and if you check our RFP in research or protocol that I, you can apply for a grant to work on this stuff, which is very exciting others good.

A

The gossip see know when it reached its end, so there the gossiping so notes cooperate with each other, but they limit themselves to just say: here's what I know of like the peers are unconnected right, and so it's not that we have a gossip that kept Smike hopping from peer to peer and I'm building up it's more here's. What I know now you make your decision without what I know and I also get everything that everyone that I'm connected to what they know and I kids.

A

With that information that then I can make a decision to see to say. Oh, it seems like you have been receiving some messages that I have haven't so maybe now I should connect to some other peers. That can get me closer to the publisher to to get the message well, either 100% delivery, or sooner or if I'm, missing some message. I can also say right away to you, hey I. Can you give me like? It seems like you received something that I haven't?

A

Can you just give that message to me so but like it's really localized, so every no that's a partial view of the network and not the entire network? Okay, we had a question here. One just excited view: can you comment on optimization of messages assimilation, so we have seen like really great results in and it really goes back to the network sizes.

A

We tested we networks in simulation of like 10,000 nodes, and we managed to make sure that the delivery was under a second and like we in a simulation, we could get all the messages to be delivered with a latency, equal or less than a second, and even like in cases of attack. We could get that delivered. We always be below that two-second threshold, so in terms of current amount of data retention replication in the latest version of ipfs definitely more of an IP first question, but also relevant, probably peer-to-peer.

A

We I profess, the protocol itself doesn't control, who stores what, and so it really is that for each user to this side, what that user pins? What that note pins, the the network messaging itself could be used either to propagate updates about the specific piece of content.

A

So we imagine, like you, have a website like a forum or like a ball or something you could use this messaging where to like updates of that website very quickly to users or potentially, you could use the top sub absolutely should with blocks of over over file, rather than using the same. The system that is today, which is bit swap that's like something we have been discussing and exploring. We haven't really tried.

A

So I can't comment anything else on that, yet protection, it's been going to remain and Korean and a petition for notes, direct, intelligent, twenty two conditions, that's a great question. So, ideally, ideally so right now we have a network with a lot of parameters right but like humans can only do so much on tuning up parameters. So, ideally, what we want to get at is a stage where we have captured a lot of interaction patterns from other networks from other applications, and then we can feed that into some model that can figure out.

A

What is the ideal optimization? What are the ideal type parameters? So that's the then we find what is the optimal point like so the the parameters that will give us the minimum latency possible and for the minimum bandwidth, but also that parameters that can adapt for specific network changes, and so the the way we think about it is very similar to what happened with TCP.

A

So TCP for many years was considered like a dried field, because well there was no more human that could come up with a TCP congestion flow algorithm that was best than the ones that were already invented and there was like always a question of trade-offs, and then someone basically created a new version of TCP that use machine learning, to develop a passer model and and I totally recommend everyone to check it out, because it's called TCP Remy and the FDA chart that shows all TCP port forms and, like you, have all the TCP previous versions developed by humans kind of like very close to the reference axes.

A

And then you have TCP Remy, which completely blows them away just because well, the the model was able to figure out a way, a better way to to optimize it that that's a great question. Sonia yeah, like the I, don't have like a top of my head answer. I all need to like just basically run a simulation, and for that case and give you like the answer for the network size that we would run the simulation with essentially but like something that you you will be able to do.

A

So if you want to drop by by one of our channels, I clip it appear on three node or discussed about protocol, I we can even help you get that set up so that you can play with it so node based ml, so yeah like so something that would grab basically like a lot of history from multiple users, multiple use cases more than users, and that could like figure out what would have been the ideal parameters to resist attacks to to optimize for delivery rate to minimize latency and so on. No one is posting here.

A

The links and I was referencing for boom filters. That's a great question. I'm sure we have, we have I, don't have a top of my head like what is the current statement right now? I need to check double check that well, our current rule, map as to month horizon and in one of the months, is just like getting the the report previewed and share with the community so that we can welcome everyone to read the report understand our results.

A

Comment contest the results and and also run the test plans, and- and it's like like show us what what you can do it with it and like see if you cannot find new optimizations, we are also now we invited an external auditor to all its the spec, the the code based implementation, so so that we can have like a an additional like degree of confidence and additional certificates that, like the thing, is ready to be used by the masses, and so once that is done, we'll release it to the world, then, as part of that roadmap is also getting all the other implementations to upgrade.

A

So that, like you, can use gossip serve on its prime states in other languages beyond that is again like open problem, RFP, exploring epidemic, broadcast trees and yeah. Whatever ideas people have. Thank you all, like yeah, I hope this was fun. Yeah we reach out to tell me what you think. Try it out send us feedback. Everything is on get up. Everything is open. Source yeah have fun you're all the best. Thank you so much.