The “Gaming of Citations” Paper
Leaps of logic ignore more helpful facts and concepts that may guide us toward improvements
People like to get worked up about citations and the Journal Impact Factor (JIF). The playbook is pretty established — they portray the JIF as a primitive approach to assessing journal quality, skewer a culture of mindless prestige-seeking, and ignore all the other sources of intellectual corruption (grant money, tenure) they really prize.
When it comes to the JIF, I’d argue it’s a relatively simple, understandable, and replicable way of assessing journal centrality within a field for a given period of time. Quality may be similar to centrality by some lights, but quality is a subjective value judgment, while centrality is more measurable. As I wrote in 2018:
. . . the higher the Impact Factor, the better the journal is as a connector among relevant outside resources. It is important as a hub, as a hint within a discovery tool about where might be a good place to start looking for the information many people with expertise are talking about.
That doesn’t mean a journal with a high JIF is better — it means it’s a good starting point for discovering what a field has been talking about recently.
A recent paper by Stuart Macdonald from the School of Management, University of Leicester, entitled, “The Gaming of Citation and Authorship in Academic Journals: A Warning from Medicine,” has been lighting up the socials. It indulges in many playbook tropes about citations, makes misinformed conceptual leaps, and ignores factors that are actually damaging to the reliability of scientific outputs generally and citations as a measure of temporal centrality specifically.
For instance, Macdonald writes toward the beginning of the paper, “the higher the JIF and the better the journal,” himself citing as a source of this idea not a scientist or bibliometrician but a journalist writing in 2007. He then starts down the mistaken path of conflating paper citations with journal citations, lacking the discipline to stick to the “J” in “JIF,” which by definition limits the metric to the journal — again, as a measure of centrality for a given period of time.