As we close out 2024, we’d like to reflect on the development of Tari’s P2Pool implementation. P2Pool has always been a focus for Tari (ever since we announced Tari’s integration into the current P2Pool project alongside other merge-mined coins, and we started implementing P2Pool mining with SHA3x). It has taken on even greater significance with its incorporation with Tari Universe and has proven to be one of the core features of the app.

P2Pool is highly challenging to set up and tricky to diagnose when things go wrong. Tari Universe has simplified this process tremendously, and we’ve been fortunate to have incredible testers, developers, and community members providing valuable feedback on its implementation every step of the way. It’s not perfect yet, but we’ve made meaningful progress in improving the experience over the past several months.

But first, why P2Pool?

P2Pool is integral to Tari contributor’s vision of creating a fully decentralized, miner-driven ecosystem. At its core, P2Pool operates as a blockchain within a blockchain — a network where miners work together to mine “shares” rather than full blocks. These shares are easier to win, even for ordinary PCs, which makes P2Pool accessible to a wide range of miners. The mining rewards are distributed based on the number of shares each miner contributes to the pool.

However, decentralization brings extraordinary challenges, particularly regarding network synchronization, peer discovery, and reward distribution. Let’s dive into the key improvements and hurdles we’ve encountered over the last few months.

September - Laying the Groundwork

In September, our focus was primarily on improving the connection handling within the P2Pool network. Long read/write locks were causing UI controls to become unresponsive, making it hard for users to interact with the application. We implemented improvements that ensured a smoother user experience, particularly for home users connecting with the network.

October - Overcoming Sync Issues and Improving Connectivity

October saw a series of updates aimed at stabilizing P2Pool and improving its reliability. The introduction of v0.5.39 included under-the-hood changes to address the persistent CPU mining issue caused by failing node infrastructure. Although this didn’t immediately resolve the problem, it was a step in the right direction.

Another important release focused on enhancing P2Pool’s network connectivity. The challenge of synchronizing the P2Pool network and forming consensus among miners was still a significant hurdle. By the end of the month, we rolled out v0.5.42, which introduced a hard fork to P2Pool — a breaking change designed to improve pool sync and address some of the intermittent mining issues.

November - Building Momentum and Fixing the Bugs

As we moved into November, P2Pool continued to evolve. We implemented network optimization improvements that reduced redundant data transfers and allowed for better synchronization between peers. We also introduced a more robust peer management system, including greylisting and auto-blocking disruptive peers.

Alpha testing of the new pool sync methods yielded positive results, with miners achieving an 80%+ daily win rate in smaller test groups. However, when the changes were rolled out more widely, new issues emerged, including excessive peer network traffic and fragmentation of pools. To address this, we focused on refining our peer discovery strategy, adding more relays to improve connectivity between miners.

December - Tackling Fragmentation and Reward Distribution

Heading into December, we’ve seen both progress and new challenges. We’ve made significant strides in network optimization, achieving over 70% daily wins for networks of up to 200 miners. However, as miner counts exceed this threshold, we’ve experienced network fragmentation — with orphan pools and inconsistent performance.

Our telemetry has provided valuable insights into the issue, and we’ve identified that libp2p communication reliability is a key factor. Miners experiencing timeouts and chain desynchronization contribute to the problem, particularly with low hash rate miners receiving inflated rewards.

In response, we’ve increased the sharechain block interval from 10 seconds to 30 seconds, which has improved synchronization and reduced fragmentation. We’ve also introduced new mining stats, including tip distance tracking and mining history improvements, giving miners more visibility into their performance.

Going forward, there are several issues we still need to address. Inconsistent chain-tip alignment among miners results in two groups of miners operating on two different chain levels, leading to latent shares and orphaned pools.

We also need to tackle the issue of scaling P2Pool. We’ve noticed that as more people participate, P2Pool performance and stability degrades. P2Pool scaling is a major technical hurdle that our contributors are actively grappling with and need to conquer for Tari Universe to reach its full potential.

A Big Thanks to the Tari Community

As this is the last update for the year, we’d like to thank our community for all of their efforts, hard work, and contribution to making P2Pool work over the past several months. You are helping shape the future of Tari - a future that ensures the democratization promised through blockchain and cryptocurrency is one that everyone, regardless of their resources or wealth, can participate in equally.