Peer review: Shifting mindsets and the promise of blockchain
Peer review has always been the gold standard for quality and rigour in research, but, as is frequently discussed, the current system of review has its flaws. Reviewers are frequently overworked, while, at the same time, small reviewer pools can perpetuate biases and the lack of diverse voices contributing to the assessment of research. While important to many models of review, anonymity can provide the opportunity for fraud or poor-quality reviews, as well as making it more difficult to award recognition to good reviewers. Beyond this, despite highly useful guidelines provided by organisations like the Committee on Publication Ethics (COPE), Sense About Science, and many publishers, there is a lack of consensus on what a good review should contain, even within disciplines. This is especially true as research publication expands beyond the article and into the publication of other research outputs like data, code and results-only reporting.
Throughout Peer Review Week 2018 and beyond, you’re likely to hear about a whole range of initiatives to tackle these problems: reviewer recognition using systems like Publons, training and mentoring to increase the diversity and size of reviewer pools, and build review skills in the next generation of researchers, as well as experiments in open review and the review of research outputs beyond the article. These developments are crucial, and are changing the face of peer review in 2018 for the better. Broadly, they aim to improve the review process by shifting the mindsets of editors, authors and reviewers, and encouraging greater engagement with ideas around what constitutes a quality review.
But alongside these developments, what if we could change the fundamental structure of peer review, the way that it is recorded, tracked and verified across the whole scholarly publishing ecosystem? Many of the issues that arise in review are the result of non-standardised processes that are owned by individual publishers, and the resultant lack of transparency and verifiable review data. Blockchain technology offers us the chance to rethink this paradigm, providing an exciting opportunity to create a transparent, de-centralised review system owned by all.
Broadly, a system based on blockchain would be much less susceptible to fraud, and would allow for review activity to be independently verified and recognized, as well as increasing interoperability between multiple publisher systems. We, along with other founding partners, are currently exploring this possibility through the Blockchain for Peer Review initiative. The pilot aims to capitalize on the main features of blockchain technology:
- Blockchain data are stored in a decentralized or distributed structure, with a copy of the database stored by each ‘node’ or actor in the system. The data is not ‘owned’ or ‘controlled’ by any one entity. This allows the opportunity for better industry collaboration and interoperability as it removes the barrier of competing interests: in blockchain, the data are not, and can never be, owned or controlled by a competitor.
- The system does not rely on trust and is based on consensus, meaning that all participants define whether a particular change can be made via the ‘consensus algorithm’. Changes can only be made to the data if a change request satisfies the consensus algorithm designed by the participants, achieving a level of consistency and validity across the system that is impossible in the current system of individually-owned publisher peer review systems.
- Blockchain data is immutable – changes can be made to the data but these changes are stored only as additional data; old data are not deleted or replaced. This means that the blockchain would store all data relating to a particular review, even as it changes over time, logging a complete history of activity.
- Despite the decentralized, distributed structure of the blockchain, data can still be encrypted in a range of ways according to the needs of the system. For peer review, this means that blockchain could support the confidentiality needs of different peer review models even while making the whole review process transparent. Actors in the review process can be pseudonymized so that identities and personal data are kept private, even while it is still possible to see and validate all review activities that have taken place.
- Blockchains can be private or public. In the case of a peer review blockchain, we can keep the data private between participating organisations until we are sure that the system is working well, and then make this information publicly available, so that any interested party can see information about the peer review related to an article.
As of Peer Review Week 2018, the Blockchain for Peer Review initiative is still in its early days; many of the details of how the technology can be used support better review are yet to be ironed out. Additionally, the technology has potential implications for complex areas of scholarly publishing beyond this focused peer review initiative – for example, measuring and keeping track of research impact. Alongside continuing efforts to increase the quality and diversity of review through education, recognition and experimentation with new forms of review, we hope to seize upon the opportunities blockchain technology provides: to create a fairer, more transparent foundational platform for review, this most crucial of research activities.