ETA: 2023-02-19

Create a service that emulates the ElasticSearch API that Lotus uses to publish traces and write traces to the probelab postgres database

ETA: 2023-02-19

eta: 2022-12-05

description: This Milestone focuses on the Replication Factor (k), the Re-Publish and Expiration Intervals in IPFS. More details can be found here: @Provider Record Liveness and here. The (pending) spec PR is here and the new values identified for the Re-Publish and Expiration Intervals are here and here, respectively.

ETA: 2023-09-30

Description: Firm up the current Composable DHT design into a precise specification to be shared and discussed with community.

Children:

• probe-lab/go-kademlia#14
• probe-lab/go-kademlia#11

ETA: 2023-09-30

Description: We want to have all of our measurement tools running continuously to have an as complete as possible view of the performance of the network.

Children:

• #15
• #16

ETA: 2023-05-30

Description: Adding privacy guarantees to libp2p has been a long-standing request by the community. During the last couple of months we have been working to do the background research, define the spec and do the implementation of the most elegant solution. The approach we’re following is based on Double-Hashing of requests to the IPFS DHT, which addresses Reader Privacy (and not Writer Privacy at this point). The design and implementation are already underway (in Q4 2022). This Milestone will be complete when the solution is integrated in the IPFS network, subject to negotiation and approval from the IPFS Stewards team. You can find more information in this Notion page: Double Hashing for Privacy.

children:

• #19
• #20
• #21
• #22

eta: 2023Q2

description: This Milestone focuses on the performance evaluation of the Double Hash DHT implementation. We want to make sure that any potential overhead (Time-to-First-Provider-Record (TTFPR), CPU, network load etc.) is acceptable before rolling out the DHT upgrade.

ETA: 2022-12-31

Description: Hydras were deployed in the IPFS network at a time when the network was much more unstable and before the Content Routing v0.5 upgrade. It has not been investigated whether and to which extent Hydras, which also incur a substantial monetary cost, are still useful today and what performance improvement they’re providing. With this milestone we aim to quantify the performance improvement brought by Hydras and have a solid justification and recommendation for whether Hydras are needed in the IPFS network. More details can be found in this Notion page: Hydras: Reliability and Effectiveness

ETA: 2023-11-30

Description: Tracking issue for the design, specification and implementation of Reprovide Sweep, as per the docs and discussions here:

• libp2p issue: libp2p/go-libp2p-kad-dht#824
• Design doc link
• Video explainer [link](https://youtu.be/bXaL64fp55c\)

Children:

• #33
• #34

ETA: 2023-03-30

Description: TBD
https://github.com/protocol/prodeng

Children:

• #18
• #10
• #11

Description

This is the tentative roadmap with which the ProbeLab team will be fundraising for H1 of 2024. A fourth item might be added, subject to sponsorship and effort needed.

• The roadmap will be finalised once the fundraising period is over.
• This issue exists as a place to foster discussion around the selected roadmap items. Anyone should feel free to comment on these items and/or suggest new ones.
• The roadmap items are not stack ranked, meaning that they won't necessarily be dealt with in this order.
• A document will be shared here in the next few days with more details on the fundraising process.

IPNS Performance Measurement and Protocol Improvement

• Effort Estimate: 3 dev months

• Summary: Investigate the protocol design choices of the InterPlanetary Name System and identify space for performance improvement. We will investigate whether the InterPlanetary Name System is as fast as can be and ideate towards an IPNS-v2 design.

• Description: We hypothesize that some design choices of IPNS are slowing down performance (e.g., quorum size). We also expect that alternative, smart designs, such as Optimistic Provide, which is now integrated into kubo can improve IPNS performance significantly. We will develop tools and orchestrate the right infrastructure to investigate those alternative directions and propose improvements as needed.

  Tasks for this milestone include:

  • Impact of quorum on performance - measure the performance of getting IPNS records from the network with different quorum values.
  • Quantify usage of IPNS in the ENS ecosystem and improvement directions in terms of performance and adoption by the ENS ecosystem.
  • Investigate option to use proximity estimation (like in Optimistic Provide) to terminate the query early.
  • Use Optimistic Provide to put IPNS records into the network (as it’s currently done for provider records).
  • Beyond performance: how would IPNS-v2 look like, what would it be able to solve that cannot be solved today? Which early design decisions could be improved with the knowledge of today?

• Outcomes: Report, Continuous Performance Measurement at https://probelab.io/ and Spec Draft (not implementation) for IPNS-v2

• Impact: IPNS is very central in the operation of IPFS, but also critical for other platforms and ecosystems, such as ENS. It is the sole enabler protocol for a decentralised Web and Naming infrastructure. IPFS will only be able to host the next billion websites if IPNS operates efficiently.

Gossipsub Performance and Security Guarantees

• Effort Estimate: 4 dev months

• Summary: Verify that Gossipsub performs according to its spec when it comes to transporting messages in the Filecoin and Ethereum blockchains.

• Description: Gossipsub’s design is complex and based on a score function in order to maintain connections to well-behaving peers and disconnect from peers that look malicious. We will investigate the stability (or instability) that this score function provides to the network, i.e., how often do nodes alter their peerings). We will also investigate the scalability of the protocol when a large number of topics is required, such as the situation expected for Ethereum Danksharding developments.

  Tasks for this milestone include:

  • Stability (or lack thereof) of peer connections due to the frequency of GRAFTs and PRUNEs.
  • Message propagation latency estimation through passive node participation. Active participation (through extension of an existing node’s software) will be investigated and planned, but not executed as part of this milestone.
  • (Tentative) Impact of large numbers of topics on performance and protocol scalability.

• Outcomes: Report and Continuous Performance Measurement at: https://probelab.io/

• Impact: Gossipsub is critical for the performance of blockchains such as Filecoin and Ethereum. Malfunctioning of the protocol is likely to cost millions of dollars. A thorough investigation on its performance and operating principles will increase confidence of thousands of applications and businesses.

libp2p bandwidth baseline in different networks

• Effort Estimate: 2 dev months

• Summary: We will quantify what are the bandwidth requirements for an idle node when participating in networks such as IPFS, Polkadot, Ethereum and Filecoin. We will identify improvement directions in case of excess bandwidth consumption.

• Description: We will investigate if libp2p is operating according to expectations on several different networks and across different implementations. We will identify the potential sources of overhead in those different networks and propose improvements, as and where needed.

  Tasks for this milestone include:

  • Measure bandwidth requirements of an idle node in the IPFS, Filecoin, Ethereum and Polkadot network and compare the outcomes.
  • Compare different implementations of libp2p when operating on those networks.
  • Identify and stack rank the sources of overhead (per libp2p protocol involved in the handshake)
  • How does smart dialing affect the handshake overhead?
  • How does the handshake process affect other components/protocols of the system, such as a DHT

• Outcomes: Report and (if applicable) Continuous Performance Measurement at: https://probelab.io/

• Impact: libp2p is the core building block of several billion dollar networks. It is responsible for starting, maintaining and terminating connections to millions of nodes on a daily basis. Efficient operation of libp2p’s core properties, e.g., in terms of overhead can save significant amounts of bandwidth and expedite performance of peer connections.

eta: 2023-03-30

description: This Milestone focuses on various impactful DHT improvement, as requested and described in this Notion page: DHT Response Times

children:

• libp2p/go-libp2p-kad-dht#811
• probe-lab/network-measurements#35

eta: 2022Q4
description: AAP (ex Ansible Tower) replace the flaky Circle CI playbooks. Bring a robust and stable process for deploying our changes fully. AAP’s RBAC feature enables developers to deploy custom Kubo or ipfs-cluster builds to select servers as self-service.

ETA: 2023-01-31

eta: 2023Q2

description: This Milestone focuses on implementing the Double Hash DHT, as a fork of go-libp2p-kad-dht. We need to commit to the Double Hash DHT Spec before the implementation is finalized. This Milestone also contains Protocol Testing of the implementation, to make sure that it exactly matches the Spec. WIP implementation by ChainSafe.

ETA: 2023-03-30

Description: We have identified that there is a number of fixed and set parameters in the IPFS and libp2p codebase, but no rigorous investigation has taken place as to whether the pre-set value is the right one. The theme is composed of three milestones:

Children:

• #12
• #13
• #14

ETA: 2023-06-30

Description: Gossipsub is the protocol that transfers blocks and messages in the Filecoin blockchain. It includes a number of smart features to achieve low propagation latency without overloading the network and the nodes, but also some security-related mechanisms to detect and automatically mitigate attacks. We currently have very little (if any!) visibility into the performance of Gossipsub. This Milestone aims to find: how fast are blocks propagating in the Filecoin blockchain, if there is virtual fragmentation (or unfair advantage) of some Storage Providers due to their geographic location, how resilient is the network against attacks, are there duplicate messages travelling in the network and overloading links and peers.

Children:

• #29
• #27
• #28

ETA: 2023-02-24

Updates on Notion page

ETA: 2022-12-31

eta: 2023-01-31

Children:

• Reduce the idle costs of Thunderdome infrastructure
• Reduce ingress costs of running an experiment
• Allow experiments to use smaller, cheaper instances

eta: 2023-02-28

description: As a first step we want to have Nebula running and reporting results at https://stats.ipfs.network/ in a more convenient manner than is done today.

Children:

• #23
• #24
• #25
• #26

ETA: 2023-10-30

Description: Finalise the design of the the Reprovide Sweep based on the design doc.

• libp2p issue: libp2p/go-libp2p-kad-dht#824
• Design doc link
• Video explainer [link](https://youtu.be/bXaL64fp55c\)

eta: 2023-02-28

description: Thunderdome is developed so that it automatically runs comparisons of release candidates with previous versions to uncover potential regressions.

See Notion page for more information and task tracking.

eta: 2023-09-30

description: The initial version of the Continuous Measurement Infrastructure developed earlier to have the Nebula crawler running continuously will be enhanced to include more features and services:

• the [CID Hoarder](https://github.com/cortze/ipfs-cid-hoarder), developed previously to monitor Provider Record Liveness,
• the CID Eclipse Attack Detector (pending),
• the [Antares tool](https://github.com/dennis-tra/antares) to detect Gateways/Pinning Services,
• the Hydra PeerID coverage tool to constantly measure the effectiveness of Hydra peers,
• an API interfacing with all our tools in the background
  • Gateway Logs exporter (configurable time frame)
  • Nebula crawl information (configurable time frame)
• the DHT Lookup Latency tool, developed previously to reflect the quality of service that end users (in different geographic areas) are experiencing.

When the above services are integrated, the Milestone will be complete with a periodic newsletter to include a summary of the IPFS network’s health and statistics.

eta: 2022-11-30

description: This Milestone focuses on one of these parameters, namely, the Bitswap Provider Delay. The Milestone aims to answer the question of whether it’s worth asking immediately connected peers for CIDs (and waiting for 1s) before defaulting to the DHT. In other words, we aim to quantify the Bitswap Discovery Success Rate. More details can be found in this Notion page: Effectiveness of Bitswap Discovery Process

eta: 2023Q3

description: This Milestone focuses on the Migration associated with the network upgrade to the Double Hash DHT. First we need to define a migration strategy and timeline. Then, we need to reach out to the community to advertise that we will perform a DHT network upgrade, and notify them if they must take action. After we are satisfied with the performance of the Double Hash DHT implementation, we need to make changes to kubo to make sure that it publishes and looks content up in both the old and the new DHTs. The changes to kubo must be tested e.g using a small test network. This milestone will be considered as complete once the Double Hash DHT is live.

ETA: 2023-01-31

ETA: 2023-09-30

Description: Prepare the way for the Composable DHT by refactoring the current DHT implementation to make it easier to extend and by firming up the Composable DHT specification.

Children:

• #32
• #31

eta: 2023Q1

description: This Milestone focuses on getting the Double Hash DHT Spec public and approved by the community. We will finalize the implementation once we have committed to the spec, and made sure it won't change. PR: ipfs/specs#373

ETA: 2023-11-30

Description: Reprovide Sweep is implemented, tested and ready to go into a release.

eta: 2023-03-30

description: We develop Thunderdome to bring it to a stage where anyone can define and run an experiment by themselves. More details can be found in this GH issue. The Milestone includes documentation, blogposts, guides and HOWTOs on how us and others within and outside PL can use it for their own experiments.

See Notion page for more information and task tracking.

ETA: 2023-09-30

Description: Refactor the current Go DHT implementation to make it easier to maintain, more reliiable and provides a solid foundation for future extension and the Composable DHT.

Deliverable: a working but unoptimised DHT that can be used in Kubo without error. Performance and resource consumption should be no more than 10% (?) worse than current DHT implementation, with the expectation that better results will come with optimization.

Motivation:

1. The codebase is very hard to understand and to modify. While trying to implement ipfs/specs#373, we figured that it would be more time-efficient to refactor/rewrite the implementation rather than trying to plug the changes in the existing implementation.
2. Flaky tests due to concurrency issues. Unit tests evaluating if the implementation is working as expected are impossible to implement without a major refactor due to massive parallelization. Having a sequential Kademlia implementation would enable proper testing.
3. Performance evaluation. For the reason above, it is hard to conduct performance evaluation tests. The current implementation is far from optimized from a resources POV, and it is likely the cause of ipfs/kubo#8807, ipfs/kubo#9530, [Thunderdome Experiment Summary: provdelayrouting](https://www.notion.so/Thunderdome-Experiment-Summary-provdelayrouting-41b5807982c74938a9e6e15a3e38a7ad?pvs=21)). We cannot efficiently debug the current DHT implementation to find where the problem comes from. The outcome of ipfs/kubo#9530 is expected to have a high impact, as it could lead to a massive reduction in the Bitswap broadcast, but is depending on solving bugs in the DHT.
4. Hard (or even impossible?) to test the protocol. IIUC Testground was built to help with testing the DHT, but hasn’t been (very) successful so far. Having a sequential DHT, using a fake clock would make protocol testing, evaluation and simulations much easier. So research and enhancements of the Kademlia protocol would become easier.
5. Implementation mistakes. The current implementation contains multiple serious flaws that have been introduced over time. For instance, Kademlia should only handle Kademlia identifiers (256-bits bitstrings) internally, but it is currently using strings. A new start helps correcting all these fundamental issues, and allows us to apply the learnings gained over time.

More information in Notion

Children:

• libp2p/go-libp2p-kad-dht#855
• probe-lab/zikade#17
• ipfs/boxo#408
• probe-lab/zikade#18

ETA: 2023-12-20

Description: Implementation of the Composable DHT is complete and tested, as per the specification in: #31

ETA: 2023-02-28

ETA: 2024-01-30

Description: Following from the Composable DHT design and specification, this Milestone focuses on the implementation of the Composable DHT and the plumbing that needs to be done in order to land it in a kubo release.

Children:

• #36
• #37

ETA: 2024-01-30

Description: All the required plumbing and extra testing is done so that the Composable DHT goes out to a kubo release.

ETA: 2023-04-30

Description: The libp2p team has shipped a new module to deal with NAT Hole Punching. Our team has taken up the task of measuring the success of the new module by building the corresponding infrastructure and recruiting clients to measure hole punches. This Milestone covers both the development of the infrastructure, but also the analysis of results. If refinement of the module itself and a new measurement campaign is needed, then that will constitute a new Milestone further down the line. More information can be found in this Notion page: NAT Hole-punching Success Rate.

ETA: 2023-02-24

Description: The Provide process in the IPFS network is painfully slow, regularly in the order of tens of seconds and sometimes in the order of hundreds of seconds. Our new design decreases this time by an order of magnitude and as a first step we plan to ship Optimistic Provide in an experimental kubo release. We plan to roll this out firstly as an experimental release to stress-test it in the network and then in the main release. This work is being carried out together with the IPFS Stewards team. You can find more information and results in this Notion page: Optimistic Provide.

Title: ProbeLab 2023 Roadmap

Children:

• #2 - 2022-12-31
• #3 - 2023-02-24
• #4 - 2023-03-30
• #5 - 2023-03-30
• #6 - 2023-04-30
• #7 - 2023-05-30
• #8 - 2023-06-30
• #9 - 2023-09-30
• #30 - 2023-09-30