Status SWARM Orange Summit, 1 Aug 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: PSST-Is anyone there?

Description

In this presentation from Day 2 of the #SwarmOrangeSummit, Mainframe’s Shane Howley gives a talk on PSST. With the limitations in PSS when it comes to sending messages, in this presetation Howley proposes a PSS sockeT (PSST) that would alleviate these shortcomings (these include unreliability to deliver messages over several relays, size limitations and others). The socket takes inspiration from current internet protocols and acts in a similar way. The protocol’s design is done, now implementation, simulation and formal specification await.

A

A

Sorry about that everyone, so, as you can see, we have a very complex setup now involving two slide shows, but uh so I'd introduce myself I'm Shane, Haley, I work with mainframe, and one of the things that we're interested in is building cool things on top of PSS I know. This is actually what I only looked like when our designer edits my slides, but normally I. Look like this I'm github sure joined, he's the handheld or other stand closer.

A

So what do we already know about PSS at the network level? We know that PSS can be used to send encrypted data payloads to an address or partial address in the swarm overlay Network. We know that the overlay network is routed by nodes, sending encrypted dev p2p payloads to their code. Emily appears, and we know that the underlay network is routed by node, sending TCP payloads across the internet.

A

So it's some of some useful properties that we already have TCP gives us reliable, payload delivery, but only between pairs of connected peers. Tcp also gives us some information about connection liveness. So, if we're connected to someone else, we know if this connection is alive or dead. But again this only works between connected peers.

A

Def p2p allows us to send like pretty large payloads significantly larger than TCP ice and PSS gives us encryption and integrity to our payloads, but maybe, as I have alluded to by pointing out that some of these properties are only held between connected peers, there are some limitations to using PSS.

A

For example, when a message is sent into the network, you can wait for a response, but wasn't received is anyone there? You might just have to pray, there's also no guaranteed message ordering. So, even though TCP ensures that messages are sent in order between any two connected peers, this guarantee doesn't extend across multiple hubs. For example, when redundant routes are used to deliver a message, some routes may deliver a message quicker than others.

A

These two aspects can cause problems for large payloads, also, as the sender could have transmitted large amount of data before knowing whether the receiver was even ready to receive it.

A

These issues can be solved already by dealing with ordering acknowledgments and retransmissions over the application layer, but this is quite a burden to leave at the feet of front-end developers, building on top of PSS.

A

So another way we can solve these issues is, to just add a layer on top of PSS to make it behave similarly to something developers are already familiar with sockets and given the swarm projects fund, this for acronyms I'm calling this PSS socket.

A

And just to summarize, the go.

A

But now we have it.

A

Okay, maybe we can talk about it more after so just to summarize the goals that I'm setting it to achieve with this protocol. We want to set up connections between nodes to indicate a willingness and readiness to receive data. We want it to be reliable, meaning that we want to guarantee that all the data we sent is received.

A

At the other side, we want to send arbitrarily sized data payloads and have them reconstituted on the receiver side before they're processed, and we want data to be ordered so that segments are received and processed in the same order that they're sent, and so just some of the some of the inspiration I got while while researching this, when considering how reliability can be built on top of a unreliable protocol, a good place to start is looking at TCP at a high level of PSS shares some similarities with UDP.

A

So any existing protocols that attempt to build reliability on top of that or IP directly are also interesting and I was also interested in data framing to allow large payloads to span multiple PSS messages, so I also took inspiration from WebSockets.

A

So this is what the segment format looks like for messages. The most important detail we have here is the compare the control flags which are in the first byte, and these are used to control the connection space.

A

So there is synchronize which is used during handshaking to agree on the connection parameters acknowledged used to inform the pier that segments have been received in sequence, extended acknowledged, used to inform the pier about out of sequence of segments, received reset used to tear down the connection and know which is an empty segment used public if the connection is still open and when used to transmit data. The overhead per PSS message of the header is 8 bytes and so we're gonna.

A

Look at some protocol examples now and in these there is already an implicit assumption that Alice and Bob have already exchanged keys, addresses and topics, and these being used at the PSS lair, so uses a three-way handshake similar to TCP to agree the connection parameters and one of the peers needs to begin in the listening state.

A

Now Alice can attempt to open a connection by sending a syn segment containing an initial sequence number and some connection parameters. Bob acts, this segment, sending his own initial sequence, number in connection parameters and then finally, Alice acts Bob segment and the connection is opened.

A

One of the aims for the reliability logic is to minimize unnecessary retransmissions by acknowledging out of order segments in this example, alice is sending a payload spanning multiple segments, so she sends a segment 101 and Bob axis. She sends segment 102, but it never arrives. Then she sends some more segments, 103 and 104. They received by Bob and I Bob detects that some of these segments are at a sequence, so he sends an extended ACK.

A

So to do this, he acknowledges the last in sequence, segment, which was when I won and he acts 103 and 104.

A

Now Alice has the knowledge that segment, 1 or 2 didn't arrive and can read retransmitted and then finally, Bob acts 104 as the last inorder segment he received under normal operation, though the peers may not be as eager to a quarry transmit as in this example.

A

So, for example, second 102 may just have been delayed and received after 104 here, meaning a transmission. A retransmission would not be required during handshaking. Some of the parameters that are agreed are time. It's for this and act frequency.

A

When a connection is idle periodically, a null segment is sent to test the connection liveness. So if Alice sends a null Bob to respond with an ACK for the most recent in sequence, data segment. But if there's no response Alice as soon as the connection has failed, so will send a reset segment mark the connection and inform the application layer of this happening.

A

One of the parameters that can be configured as the maximum message size so I believe currently deaf Peter pls matches up to 16 megabytes, but sending a pail of this large and a single PSS message could be wasteful if the recipient is not prepared or drops out. Mid transmission. I haven't investigated this quite yet, but I wanted a test if aligning the max size such that a PSS message could be encapsulated inside a single TCP packet and check whether that has any difference in performance. Efficiency versus just like sending real large PSS payloads.

A

But, as you see here, the way that which payloads are framed is pretty simple. The size is encoded at the beginning of the frame using le b12 8, which is a base 1 to 8 encoding for variable sized integers, so, for example, up to a 1 to 7, byte payload will use a 1 byte size field up to 16, kilobytes would use a 2 bite, size field and so on, and once the recipient reaches the end of the frame, that is when they receive that number of bytes across segments.

A

The messages handed up to the application layer, so some of the applications they think could be useful for our games, collaborative editing, real-time messaging or really anything that requires any p2p application. That requires some kind of shared stage and syncing that stays so maybe like a CR DT.

A

A

It's it's kind of like message-based, so I consider UDP like Datagram base, where it's just sending like individual, like payloads of data and you're handling them as they come where and then TCP, which is stream based where you're just getting bytes and eventually you need to decide where the cutoff is. But with this it's message based. So you like on the sender side, you send like fully framed messages and they're received as fully framed messages in the application layer, but I mean at the end of the day, everything's bytes, but like as a developer.

A

This is the way you'd work with it right.

A

Yeah yeah the PSS layer is, is like the closest analog would be datagrams, so the current status of the project is that the protocol design is basically complete, and this is kind of what I've talked about here. The next step is to make a prototype of this and have it integrated with PSS. So we can play around and see what kind of things we can build with it.

A

Another thing I'd like to do is use the network simulation framework to to like simulate different Network conditions and see how the protocol performs there and also produce a formal specifications so that there can be like competing or alternate implementations that can coexist.

A

Some some potential improvements, I'm already thinking about, could be added. I mean at the moment. I just try to keep it like relatively simple, but you know I'm still thinking of ways we can kind of make it better. One thing is adding no more sub protocols, so I know so. Pss is already kind of a sub protocol of swarm and Lewis's talked earlier was about building some protocols on top of PSS, and you know the protocols everywhere.

A

It'll be great yeah, so WebSockets actually do this when they're like setting up the connection, so I think you something that you can do during handshaking to say, like I, want to talk this protocol and then the handshaking can decide whether the connection is successful or not. Based on this another thing, which I also noticed in WebSockets, is allowing unknown length length payloads.

A

So when I spoke about the frame, I said always the first, the start of the frame told you the size of the payload, but this that could also be applications where you don't know the size of the payload up ahead of time, but I couldn't really think of a great use case. So I kind of didn't want to make it overly complex, and maybe these are the great ideas that, like the community has and I'd be.

A

You know interested in talking to people about that, where you will eventually be able to get updates is probably on our company github mainframe HQ. There is no repo there yet. So this is why I have not told you that, but you know, if you keep watching there, eventually one will appear and that's it yeah. Thank you.