IPFS / non regularly scheduled

Michael discusses his work on IPFS providers strategies (how an IPFS node tells the network the content it is serving) and where the next improvement opportunities are.

Yiannis synthesized and presented research to IPFS and libp2p engineers on 3 systems in the wider academic landscape with novel practices that might be of use in evolving the systems to increase performance and scalability.

Speaker:
Yiannis Psaras

Abstract:
In parallel to the great work being produced within the IPFS and libp2p projects, there has been a parallel stream of work mainly driven by the academic and research community to shift the focus from host-based networking to content- or information-centric networking (CCN/ICN). Most of the work has been kicked off around 2006, mainly from Van Jacobson (at Xerox PARC at the time) and in the form of talks - see [1] for a pretty inspiring talk by Van Jacobson. This later materialised in several papers and is today still going on under the umbrella of the Named-Data Networking project (see below).

A number of projects have since emerged (mainly in the US and Europe) to investigate the properties of a content-centric network, where content itself is the addressable and routable primitive and to build a scalable, efficient and secure Future Internet Architecture. The work has (mostly) assumed that this future Internet architecture will build on top of IP and will involve in-network routing and forwarding entities (i.e., network routers), which will be able to “talk” ICN language.

In this workshop, we will start off with a general introduction to the properties and differences of name-based routing and name-resolution-based routing in Information-Centric Networks, as well as the strength of applying application-level semantics into network-layer names. We will go through some of the most prominent architectural proposals, which are also conceptually close to IPFS and libp2p. The purpose of this interactive workshop will be to:
Identify interesting aspects or features of previously proposed architectures that are worth looking into closer.
Identify (both for the short term and for the long term) what are the desired features and the overall architectural structure of IPFS and what does that mean for libp2p.
Assuming those changes take place in the near future in IPFS and libp2p, what are the features gained and what are those lost.

Below is a brief description of the architectures that will be discussed together with pointers to reading material. The overall length of the workshop is estimated to be 2hrs. Each of the sessions (the three below plus the introduction) will be ~20mins each allowing 10mins for discussion and Q&A in each session).

1) Named Data Networking (NDN)

NDN is a US-driven project, which however is attracting attention worldwide. NDN follows a “name-based routing” approach, where names are hierarchical and routers do longest-prefix matching to find the next node to forward the request to. This is the most radical of the ICN architectures, which assumes that routers in the network (or at least some of them) understand the NDN stack and support in-network content caching. Although it comes with several challenges (e.g., scaling the routing table size), its expected benefits are significant, being able to natively support client mobility, in-network caching and multicast.

Material:
Website (includes a ton of material): named-data.net
Original 2009 paper: https://named-data.net/publications/networkingnamedcontent/
Follow up 2014 paper: https://named-data.net/publications/named_data_networking_ccr/

2) DONA: Data-Oriented (and beyond) Network Architecture

DONA is a 2007 ACM SIGCOMM paper, which advocates a hierarchical data resolution structure, where there is at least one “resolution handler” in each ISP domain. Names are structured according to the form P:L, where P is the cryptographic hash of the principal’s (publisher’s) public key and L is a label given by the principal/publisher which needs to ensure that names are unique. Given the hierarchical structure of the architecture, a challenge for DONA is the stress on the top-level resolution-handler (assuming we’re addressing the entire Internet).

Material:
Paper: http://ccr.sigcomm.org/online/files/fp177-koponen1.pdf

3) NetInf: Network of Information

NetInf is the product of an EU project (which ended around 2015 or so) which resembles a lot the IPFS structure. NetInf is building on name-resolution-based routing (through a Name-Resolution System - NRS). The NRS is a multi-level DHT and results have reported that the architecture can scale to several millions of nodes with lookup latencies of less than 100ms.

Material:
Overall Architecture paper: https://www.sciencedirect.com/science/article/pii/S0140366413000364
Measurement/Simulation study: https://www.sciencedirect.com/science/article/pii/S0140366413000418

For more information on IPFS
- visit the project website: https://ipfs.io
- or follow IPFS on Twitter: https://twitter.com/IPFS

## Testing Census Meeting Notes
Benchmarks of js-ipfs up again!
All benchmark data relatively level - no observed performance hit
Running this within network testing pipeline could trigger this api, and spawn this job and input into unified metrics database (in addition to here)
Want more control in our own UI to display relative info in a comparative fashion (maybe that allows you to drill down in grafana)
Grafana good for time series, not for compare against baseline we choose
Meta objective: every group should have an MVP with ZERO dependencies on other groups - the stage 2 or 3 can dovetail and merge with a shared vision - however the first step that will block development should be separable from everyone else’s work
Doesn’t give nice overall view - but can be tweaked
Nice thing this *doesn’t* have is go-nightly or something like that to use in tests
Test infra - want specific test cases being run to extract metrics of interest
Want to see greens, yellow, and reds to make it clear where we should put attention to
Want it to be that any community engineer can see health on various go and js commits against other implementations (hard with grafana)
Alan will put a baseline of work into js benchmarks to make it useful for go, and then go from there
What does everybody actually need?
- project operations and package mangers need benchmarks they can use for their specific testing / feedback loops (and need to be able to start running this prior to test infra work completing
- Gateway team wants to be responsible for spinning up nightly mirrors and similar
- Testing pipeline that will hopefully be an orchestrator and aggregator
Using, adapting, and adopting benchmarks using js test suite to cover all the things we want
Alan get go tests running, steven add go tests
Kubernetes repo also has a whole series of go tests written in yaml-based dsl
Package manager tests will likely be shell scripts to import real world data
Current benchmark format looks good for v1 is good, also ability to run locally so can do it on branch

This is the IPLD call from Friday. See https://github.com/ipfs/pm/issues/267.