Digital Scholar Bytes: Peer Review Week 2024: Innovation and Technology in Peer Review

Peer Review Week
Peer Review Week, September 23-27, 2024

Peer Review Week (September 23-27, 2024) is a global, community-driven event that celebrates the critical role peer review plays in maintaining the quality and integrity of academic research. The event brings together institutions, individuals, and organizations that are committed to upholding the high standards of peer review and advancing scholarly communication.

This year’s theme is “Innovation and Technology in Peer Review,” with more than 35 organizations around the globe participating. This year’s event aims to engage stakeholders in discussions about how innovation and technological advancements are reshaping the peer review process. The theme was chosen through a global poll, which saw 494 votes from members of the scholarly community. Other competing themes included IDEAS (Inclusion, Diversity, Equity, and Access) in peer review, ethical integrity in peer review, and optimizing the researcher experience in peer review.

Challenges for Peer Review

The challenges listed:

  1. Increasing focus on research integrity.
  2. Growing demand for greater transparency in the peer review process.
  3. Rising demand for efficiency in the peer review process.
  4. A surge in the number of publications.
  5. A shortage of peer reviewers to keep pace with the volume of publications
  6. Diversity, Equity, Inclusion, and Accessibility concerns (see The United Nations Sustainable Development Goals).

These problems are forcing innovations in the peer review process. One possible solution is the building of AI initiatives which may streamline publishing workflows and address these problems.

AI and the Limitations for Peer Review

Strengths, Weaknesses, Opportunities, and Threats: A Comprehensive SWOT Analysis of AI and Human Expertise in Peer Review. The Scholarly Kitchen.

AI is impacting the entire scholarly communication in various ways. As we wrote last year for Peer Review Week, AI is automating peer review tasks, enhancing reviewer recommendations, maintaining quality control and exposing fraud detection, and streamlining manuscript production. While AI-driven innovations hold great promise for streamlining peer review, concerns around algorithmic bias, data privacy, and the displacement of human expertise must be addressed. AI can help with tasks like keyword extraction and plain language summaries, but its role in improving review quality remains debated. Ultimately, AI must be integrated thoughtfully to maintain transparency, accountability, and ethical standards, so that the integrity of the peer review process is protected.

The diversity of presentations and discussions surrounding Peer Review Week demonstrate the variety and complexity of the peer review process. A sampling of events include:

  • Eliza, an AI-driven peer review tool that uses NLP technology, including LLM and semantic matching) for analyzing papers and peer review drafts.
  • The Public Knowledge Project will explore how Open Journal Systems (OJS) is evolving to meet the demands for transparency, efficiency, and collaboration in peer review.
  • A discussion of a possible new hybrid model of peer review that envisions a collaborative environment where human expertise and AI complement each other.
  • The American Society for Microbiology discusses the ethics of AI’s role in scholarly publishing.
  • VeriXiv is a preprint server built by F1000 and with assistance from the Gates Foundation (Gates Open Research). This year, VeriXiv added a post-publication open peer review model.

Final thoughts

For an in-depth look at the challenges facing the future of peer review, check out the Scholarly Kitchen‘s recent posts below. The Scholarly Kitchen is the official blog of the Society for Scholarly Publishing, the sponsor of Peer Review Week. Tweet on X (@PeerRevWeek) and follow #PeerReviewWeek.

Last, subscribe to the Catholic University Libraries Digital Scholarship blog for updates.

Kevin Gunn is the Coordinator of Digital Scholarship at The Catholic University of America Libraries and past editor-in-chief of the journal College and Undergraduate Libraries.

Further Reading

Cuomo, Christina, Colin Parrish and Diana Proctor. 2023. Peering Into Peer Review. American Society of Microbiology.

Dyke, Gareth, and Shivendra Naidoo. 2023. Best Practices in Peer Review. Peer Review Week 2023.

Ease. 2024. Peer Review Toolkit.

Ghosh, Roohi. 2023. “The Peer Review Renaissance: An Urgent Call for Transformation.” The Scholarly Kitchen. October 12, 2023.

Ghosh, Roohi. 2024. “Strengths, Weaknesses, Opportunities, and Threats: A Comprehensive SWOT Analysis of AI and Human Expertise in Peer Review.” The Scholarly Kitchen. September 12, 2024.

Ghosh, Roohi, Maria Machado, Gareth Dyke, and Maryam Sayab. 2024. “Peer Review Week 2024: “Innovation and Technology in Peer Review”.” The Scholarly Kitchen. May 29, 2024.

Meadows, Alice, Jasmine Wallace, and Karin Wulf. 2024. “Some Thoughts on the Promise and Pitfalls of Innovation and Technology in Peer Review.” The Scholarly Kitchen. September 23, 2024.

Sutter, Paul. 2024. “Peer Review is Essential for Science. Unfortunately, it’s Broken.” Ars Technica. July 12, 2024.

Share this:

Leave a Reply