Welcome to Peer Review Week (September 25-29, 2023)! This week celebrates the scholarly peer review process, explores its significance in maintaining the quality of research publications, and addresses questions and challenges faced by researchers, academics, and librarians. The week is coordinated and sponsored by a number of publishers and affiliates.

What is Peer Review?

Peer review is the process of having your research paper independently assessed by experts in your field. The purpose of peer review is to evaluate the paper’s quality, academic rigor, and suitability for publication. In a subsequent post on our Digital Scholarship blog, we will take a deeper dive into understanding the peer review process, the different types of peer review, and the common misconceptions held by new scholars.

Peer Review and the Future of Publishing: AI and Machine Learning

Since the theme this year is peer review and the future of publishing, looking at artificial intelligence (AI) and machine learning to see what effect they have on publishing seems like a natural move. Of course, AI has been used in the publishing industry for the last several years. Streamlining the peer review process by cutting down on the length of time has been a priority for most publishers.

However, as AI and machine learning advances at an unprecedented rate, they can be powerful tools that have the potential to reshape the traditional peer review process. Here are four ways that AI and machine learning are changing the landscape of peer review.

• Automating Peer Review Tasks
  One of the most significant contributions of AI and machine learning to peer review is the automation of various tasks. From identifying suitable reviewers for manuscripts to detecting potential plagiarism, AI-driven systems can significantly reduce the administrative burden on editors and reviewers. This not only expedites the peer review process but also ensures a more efficient allocation of resources, allowing researchers to focus more on the actual evaluation of scholarly content. In 2022, Zhao and Zhang examined the success of the automation of the reviewer assignment algorithms in addressing assigning peer reviewers.
• Enhancing Reviewer Recommendations
  AI can assist editors in making more informed decisions when selecting reviewers for a manuscript. By analyzing the research interests, expertise, and past reviewing history of potential reviewers, AI algorithms can provide tailored recommendations. For example, Web of Science has their Reviewer Locator, an algorithm that searches the Web of Science database for the most suitable reviewers. This not only ensures that the most qualified experts are chosen but also promotes diversity and inclusion in the peer review process by identifying reviewers from underrepresented groups.
• Quality Control and Fraud Detection
  Maintaining the integrity of the peer review process is paramount. AI-powered systems can aid in identifying fraudulent activities, such as fabricated data, manipulated images, or unethical practices. These algorithms can detect anomalies in manuscripts, prompting further investigation and ensuring that only high-quality research is published. Publishers have been proactive in outlining best practices and policies for authors if they use AI to generate content.
• Streamlining Manuscript Preparation
  Publishers are slowly rolling out AI-driven tools to assist reviewers and editors in their evaluation process. The content of a manuscript, checking for clarity, coherence, and language quality, can be evaluated using algorithms. This not only supports reviewers in their assessments but also helps authors improve the overall quality of their submissions before peer review even begins.

Limits of AI in Peer Review

Of course, there are limits in how (and if) one should use AI in the peer review process. Some publishers have prohibited the use of AI by peer reviewers as a violation of the confidentiality inherent in peer review (open review does not apply). Funding agencies such as the National Institutes of Health have prohibited the use of AI in analyzing and drafting peer review critiques, citing the need for confidentiality. The NIH even requires that peer reviewers sign a confidentiality and non-disclosure agreement. The heavily discussed issue of bias is also a factor that editors need to be aware of. Last, lack of context, expertise, feedback loops, and ‘peer‘ are possible drawbacks for peer review. Only time will tell.

Final thoughts

For an in-depth look at the challenges facing the future of peer review, check out the Scholarly Kitchen‘s recent post, “Ask the Chefs: What is the Single Most Pressing Issue for the Future of Peer Review?” Scholarly Kitchen is the official blog of the Society for Scholarly Publishing, the sponsor of Peer Review Week.

Last, subscribe to the Catholic University Libraries Digital Scholarship blog for updates.

 

Further reading

Hosseini, M., and Horbach, S.P.J.M. 2023. Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review. Research Integrity Peer Review 8, 4 (2023).

Kousha, K. and Thelwall M. 2023. “Artificial intelligence to support publishing and peer review: A summary and review.” Learned Publishing. 2023.

Leung T.I., de Azevedo Cardoso T., Mavragani A., Eysenbach G. 2023. Best Practices for Using AI Tools as an Author, Peer Reviewer, or Editor. Journal of Medical Internet Research 2023.

Willis, Michael. 2022. How Can Technology Aid Research Integrity? Wiley blog. November 22.

Wulf, Karin et al. 2023. Ask the Chefs: What is the Single Most Pressing Issue for the Future of Peer Review? Scholarly Kitchen, September 22, 2023.