The Altmetric Top 100 Is a Farce

Overly reliant on black boxes, biased sources, and bots, the score should be ignored

The Altmetric Top 100 Is a Farce

A few weeks ago, I wrote about what’s good and what’s broken about Altmetric. Today, the most broken aspect — the Top 100 — is being promulgated, even celebrated, as if it means something. Here’s a quick overview of why you might want to think twice before promoting something I contend is far more broken than the impact factor:

  • Social sharing doesn’t equate with attention. 

  • Altmetric is too reliant on a black box. 

  • This black box (Twitter) has become infested with bots and bad actors. 

  • The Altmetric weighting system doesn’t make sense. 

  • Altmetric does not generate a useful index. 

    If you look at the 2017 Top 100 list, you see a score of 5,876 for the #1 article. What does “5,876” mean? Here’s the first level of detail:

    As you can see, the score is mainly driven by Twitter. When the weighting is factored in, Twitter accounts for 84% of the score, with “News” only accounting for 14%, Facebook for 1%, “Blogs” for 1%, and everything else for less than 1% each.

    With Twitter driven by an algorithm designed to optimize advertising dollars, and relying on traffic that is increasingly non-human, what does this mean for Altmetric? We don’t know, and neither can or do they. Twitter isn’t telling.

    So we have a score that is 84% mysterious at its base, a weighting system that makes little sense, and an index built on shifting sands moved about by bots and trolls. What does being #1 mean in this scenario? It’s hard to know, which is why Altmetric is a poor index of attention. It’s largely an index of Twitter, which itself is a faulty index of attention.

Ironically, the #2 article in the 2018 list is fatally flawed, as I wrote about shortly after the study was published. I even predicted it would finish high in the Altmetric Top 100:

Sure to become an Altmetric darling because it deals with Twitter, the engine of Altmetric, the study has been generally viewed as a major step toward a better understanding of how social media can infect our society by exploiting human nature. Unfortunately, the authors did not gather the data needed to analyze a potentially major contributory factor, perpetuating a blindspot that underscores the “unsafe at any speed” dangers of social media as it currently manifests in our culture.

The Altmetric Top 100 is 80-90% a Twitter measurement system which weights sources in ways that make little sense. Twitter itself is not a measure of attention but rather a black box compromised by bots, bad actors, and an algorithm we don’t understand. Together, these factors alone render the Altmetric Top 100 worse than useless — it’s downright misleading.

Subscribe now

Subscribe to The Geyser

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe