News

Decentralized Web Facing Natural Pressures toward ReCentralization

• By John K. Waters
• 12/06/2022

Since Elon Musk concluded his acquisition of Twitter in at the end of October, there has been something of a clarion call to abandon the platform and join one of the decentralized microblogging networks, such as Mastodon, Hubzilla or Diaspora. But the growth of the decentralized Web may be inhibited by what a group of researchers believes are "natural pressures" toward re-centralization.

Using Mastodon, researchers from London-based King's College, University College and Queen Mary University conducted a large-scale measurement study of the decentralized Web. They published their results in a paper ("Challenges in the Decentralized Web: The Mastodon Study"), in which they identify two innovations these decentralized platforms rely on, and inherent properties of the Web itself that are creating a kind of natural pressure toward re-centralization.

In their report, the researchers identified the decentralized Web as "an evolving concept, which encompasses technologies broadly aimed at providing greater transparency, openness, and democracy on the Web."

Mastodon is a decentralized, open source microblogging service launched in 2016. It has features similar to Twitter, but unlike that platform, there is no central ownership of the service. Individual users set up independently operated servers (called "instances"), each with their own codes of conduct, terms of service, privacy policies, privacy options and moderation policies.  Users post "toots" that can be seen by members of their home instances, as well as others, depending on the codes of conduct established by each. It's comparable to the way Gmail users can send and receive messages from Yahoo! Mail and Outlook, and vice versa. The result is a federation of independently operated, interconnected servers—and the coining of the portmanteau, "fediverse."

In November, Mastodon reported (ironically, with a tweet) that the platform had just passed the 2 million active user mark, bolstering its claim to be the largest decentralized social network on the Internet.

These two innovations—the ability to support independent instances that anyone can easily bootstrap, and the ability to use decentralized protocols to let instances interact with each other—are the keys to the growth of this aspect of the evolving decentralized Web the researchers identified. But they questioned the long-term viability of a truly decentralized Web in the face of "inherent challenges that are difficult to avoid."

These challenges include user-driven, infrastructure-driven and content-driven pressures toward centralization.

The user-driven pressure the researchers cite in the case of Mastodon is the result of the popularity of the platform being heavily skewed toward a few instances, which drives an implicit form of centralization. According to the researchers, 10 percent of Mastodon instances host almost half of the users. Consequently, a small subset of administrators has a disproportionate impact on the federated system.

The infrastructure-driven pressures are the due to the simplicity, low cost and colocation of instances within a small set of hosting providers. Individual failures in these autonomous systems have the potential to create a ripple effect that fragments the federation. "We observed six cases of these autonomous-system-wide outages within our measurement period. We also observe regular outages by individual instances (likely due to the voluntary nature of many administrators). Again, this has a notable impact: 11% of instances are unavailable for half of our measurement period."

And then there are content-driven pressures resulting from the differing popularities of toots. The researchers found that outages in just 10 instances could remove 62.69 percent of global toots.

In their conclusion, the researchers suggest that these pressures are currently largely ignored, and may risk convergence toward a "semi-centralized system."

For this study, the researchers relied on three primary datasets: regular snapshots of instance metadata and availability, historical user posts (toots) available on each instance, and follower and federation graphs, which are included in the paper.

"Our analysis reveals a highly dynamic system and, thus, our measurements are capturing a 'moving target,'" the researchers emphasized. "Similarly, we highlight that the timelines of each dataset differ slightly due to the progressive deployment of our measurement infrastructure. This, however, does not impact our subsequent analysis. It is also worth noting that we do not have comprehensive data on all toots, due to some instances preventing crawling (we have 62% of toots). Thus, we have avoided performing deeper semantic analysis on toot content and, instead, focus on instance-level activities."

The researchers plan to follow up this paper with a longitudinal analysis of Mastodon focused on the issues identified in this paper, as well as a study of the effect of hate speech of alt-right communities, such as Gab, which are starting to use Mastodon forks.