The Altmetric Top 100 Is a Farce
Overly reliant on black boxes, biased sources, and bots, the score should be ignored
A few weeks ago, I wrote about what’s good and what’s broken about Altmetric. Today, the most broken aspect — the Top 100 — is being promulgated, even celebrated, as if it means something. Here’s a quick overview of why you might want to think twice before promoting something I contend is far more broken than the impact factor:
Social sharing doesn’t equate with attention.
Altmetric is too reliant on a black box.
This black box (Twitter) has become infested with bots and bad actors.
The Altmetric weighting system doesn’t make sense.
Altmetric does not generate a useful index.
If you look at the 2017 Top 100 list, you see a score of 5,876 for the #1 article. What does “5,876” mean? Here’s the first level of detail:
As you can see, the score is mainly driven by Twitter. When the weighting is factored in, Twitter accounts for 84% of the score, with “News” only accounting for 14%, Facebook for 1%, “Blogs” for 1%, and everything else for less than 1% each.
With Twitter driven by an algorithm designed to optimize advertising dollars, and relying on traffic that is increasingly non-human, what does this mean for Altmetric? We don’t know, and neither can or do they. Twitter isn’t telling.
So we have a score that is 84% mysterious at its base, a weighting system that makes little sense, and an index built on shifting sands moved about by bots and trolls. What does being #1 mean in this scenario? It’s hard to know, which is why Altmetric is a poor index of attention. It’s largely an index of Twitter, which itself is a faulty index of attention.
Ironically, the #2 article in the 2018 list is fatally flawed, as I wrote about shortly after the study was published. I even predicted it would finish high in the Altmetric Top 100:
Sure to become an Altmetric darling because it deals with Twitter, the engine of Altmetric, the study has been generally viewed as a major step toward a better understanding of how social media can infect our society by exploiting human nature. Unfortunately, the authors did not gather the data needed to analyze a potentially major contributory factor, perpetuating a blindspot that underscores the “unsafe at any speed” dangers of social media as it currently manifests in our culture.
The Altmetric Top 100 is 80-90% a Twitter measurement system which weights sources in ways that make little sense. Twitter itself is not a measure of attention but rather a black box compromised by bots, bad actors, and an algorithm we don’t understand. Together, these factors alone render the Altmetric Top 100 worse than useless — it’s downright misleading.