David C. Mallinson, PhD; Corlin Jewell, MD; Fahad Aziz, MD
WMJ. 2024;123(2):70-73
The main purpose of a scientific journal is to share research findings, progress in the field, and general information with the public. This means that journals have a responsibility to vet manuscripts for their accuracy, thoroughness, and significance. Consequently, academic peer reviewers play a vital role in this process by evaluating the suitability of manuscripts for publication.
Being a peer reviewer is a shared responsibility in the academic community. However, there is often a lack of formal guidance or training on how to review a manuscript.1,2 Sometimes, reviewers are unsure how to evaluate a manuscript or write their feedback. This can lead to authors receiving unhelpful or biased feedback, and journal editors may struggle with deciding whether to publish a manuscript with inadequate or unreliable reviews. This means that a manuscript might be published despite significant shortcomings that undermine the manuscript’s integrity. On the other hand, high quality papers may be unjustly rejected for insufficient reasons. Thus, it is important to give reviewers guidance on evaluating manuscripts for publication.
This editorial is a brief manual on peer-reviewing a manuscript for scientific journals. It consists of 5 sections: (1) making the decision to review a manuscript, (2) evaluating the manuscript, (3) composing the review report, (4) handling revisions, and (5) additional essential considerations. While we designed this guide explicitly for reviewing original research articles and brief reports for the Wisconsin Medical Journal (WMJ), the principles can be broadly applied.
HOW TO DECIDE IF YOU SHOULD REVIEW A MANUSCRIPT
The initial stage of the peer review process may seem obvious, but it is still crucial: determining whether to review a manuscript. Before you commit to reviewing a manuscript, there are 3 criteria that you should meet (Figure).
First, it is critical to assess whether you have a conflict of interest, which can compromise the objectivity of your evaluation. The International Committee of Medical Journal Editors defines a conflict of interest as a situation where your professional judgment on a primary interest, like reviewing a manuscript, might be swayed by a secondary interest.3 This could be a financial interest, such as owning stock in a company whose product is being studied, or a nonfinancial one, like a close association with one of the authors. Remember, even the perception of a conflict is significant, as it can compromise the reviewer’s impartiality.
Next, ensure that you possess the expertise to critique the manuscript. A peer reviewer should be able to understand, evaluate, and provide well-founded feedback on a manuscript. Someone not versed in the specific topic or the broader academic context is unlikely to perform effectively in this role. However, being an expert in every aspect of the research isn’t required. For instance, some reviewers bring deep knowledge of the subject matter, while others contribute as methodologists or statisticians.
Lastly, you should have the time and attention to devote to review a manuscript. Reviewing thoroughly and providing constructive feedback is a time-consuming task. The WMJ allows a generous 21-day period for completing a review. This ensures that reviewers can be meticulous without rushing, and authors receive timely feedback on their submissions.
Suppose that you are invited to review a manuscript and meet the criteria: no conflicts of interest, the right expertise, and sufficient time. In that case, you are well-positioned to be a peer reviewer, and we encourage you to accept the task. If not, it is best to decline the invitation. Remember that your suitability as a reviewer may differ from one case to another. If you decline, the editors will appreciate suggestions for other potential reviewers and their contact information.
HOW TO EVALUATE A MANUSCRIPT
After agreeing to review a manuscript, you will receive the entire manuscript, including any additional materials. Reading and evaluating the manuscript is a demanding but crucial process. We offer guidelines to help you prepare for a detailed and practical review.
Begin by reading the manuscript multiple times, paying close attention to every section, from the title and abstract to the appendices. Academic journals like the WMJ often feature complex studies that require in-depth reading for complete comprehension. As you read, take notes on critical aspects of the paper: the research question’s clarity, the thoroughness and relevance of the literature review, the description of data acquisition and analytical methods, and the interpretation of results within the broader context of research, practice, and policy. Additionally, consider attributes that are missing from the paper (eg, an undetailed description of the methods that would preclude a replication of the study). Your insights into these omissions will contribute to a more constructive review.
Next, classify your notes as either “major” or “minor” comments. Major comments are essential queries or critiques that address the manuscript’s readiness for publication or point out fundamental flaws that might prevent publication as it currently stands. These could relate to unclear elements that could be clarified within a short timeframe or to deep-seated issues requiring substantial modification to the study. On the other hand, minor comments suggest improvements that, while helpful, are not critical to the manuscript’s readiness for publication and its contribution to the scientific literature. These might include expanding on a study procedure or adding a pertinent reference. Ensure your feedback aligns with the manuscript’s goals, avoiding suggestions that would alter the research question or fundamental methodology. Your comments should enhance the authors’ work rather than reflect a different vision for the study.
The distinction between “major” and “minor” comments and the significance of each relies on the expert opinion of the reviewers. These assessments determine how much a comment impacts the manuscript’s overall integrity and suitability for publication. For illustration, Table 1 lists examples of major and minor comments using a hypothetical study investigating the effects of a pregnancy care coordination program on birth weight (see Table 1 in full-text pdf).
HOW TO WRITE THE REVIEW
You can compose your review after carefully reading the manuscript and preparing your feedback. Start with a summary paragraph: first, describe the manuscript and its key findings or messages in 1 to 3 sentences. Then, in another 1 to 3 sentences, outline your general assessment and any major concerns, including a publication recommendation if the journal’s policy allows it—as the WMJ does.
Structure the main part of your review with 2 clearly defined sections: major comments and minor comments. Use headers for clarity and present each point separately, possibly as bullet points or individual paragraphs, to enhance readability. When referencing specific parts of the manuscript, include page and line numbers. A clear and concise rationale should accompany each comment.
Consider a hypothetical trial on the medication Drug X for reducing blood pressure among adults with chronic hypertension. Let us assume that the manuscript omits the inclusion and exclusion criteria for participants. This is a significant oversight and should be highlighted as a major comment. A weak comment would simply identify the omission. A good comment would explain how this lack of detail hampers the study’s external validity and replication, for example:
“Page 4, Lines 11-17: The authors did not list their inclusion and exclusion criteria. This makes it difficult to evaluate the generalizability of the findings, as we do not know the characteristics of the study sample, and the estimated effect of Drug X on blood pressure may not apply to other patient populations. Additionally, this omission hinders study replication, so other researchers cannot build upon these results.”
If you suggest a revision, your recommendation should be actionable and specific. Reconsidering the hypothetical study on Drug X and blood pressure, an example of an actionable and specific recommendation follows:
“The authors should outline their inclusion and exclusion criteria sequentially, detailing the number of potential participants excluded at each step. Additionally, the authors should justify each inclusion or exclusion criterion. A flow diagram could effectively illustrate the selection process.”
Table 2 provides example recommendations of varying quality alongside explanations of what makes them (or does not make them) specific or actionable (see Table 2 in full-text pdf). For confidential concerns such as ethical considerations or suggestions for accompanying commentary, use the confidential report to the editors, which remains unseen by the authors. This section allows you to communicate sensitive issues or ideas that are best handled between reviewers and editors.
HOW TO RESPOND TO REVISIONS
After you submit your review, you might be asked to evaluate a revised version of the manuscript. This will come with a response letter from the authors that addresses your feedback and outlines their changes. As you review the revised manuscript, ask the following questions:
- Did the authors adequately address each of my comments through their revisions, or did they provide reasonable explanations for any comments that did not elicit a revision?
- Can I find the changes mentioned in the authors’ letter in the revised manuscript?
The depth of your second review depends on the extent of the revisions. If the manuscript has been significantly altered, reviewing it as thoroughly as before is advisable, reassessing both major and minor points. For minor revisions, focus on the sections that have been updated. However, reevaluating the entire manuscript will best ensure that nothing is missed.
As you assess the revised manuscript, clearly state whether the authors have successfully addressed your concerns. If specific comments have not been sufficiently resolved, highlight this in your review, explaining its significance for your overall evaluation, especially if it influences a recommendation against publication.
OTHER IMPORTANT CONSIDERATIONS
This editorial aims to be a helpful resource, but you may still have questions during the peer review process. Here are additional tips and considerations to keep in mind.
- Focus your review on the research’s substance and methodology rather than the manuscript’s grammar and formatting. While it is your role to assess the study’s validity, not to proofread, do mention any grammatical or formatting issues that hinder your evaluation as minor points. However, if such errors are pervasive, making assessing the study’s content difficult, you should report these to the editors confidentially.
- While a manuscript’s literature review should be comprehensive, it does not need to be exhaustive. Authors should include enough references to provide context but need not cite every related source. Only recommend additional references if they add significant value. Refrain from suggesting your work unless it’s directly relevant; insisting on citing it without necessity can be inappropriate and unethical.
- If you suspect plagiarism or other ethical concerns, promptly communicate these to the journal’s managing editor and detail your concerns in your confidential report to the editors.
- Be selective with your comments and recommendations. Each should offer substantial help or point out critical issues. Avoid overloading your review with minor criticisms—remember, the goal is to enhance the manuscript’s informative value, not to nitpick for the sake of it. No manuscript is without flaws, but we can guide authors towards meaningful improvements.
CONCLUSION
Our aim with this editorial was to offer a concise and valuable manual for peer review in academic journals, such as the WMJ. While we could not cover every aspect of the process, we hope our instructions and advice are helpful for both new and experienced reviewers. Our guide is designed to enhance your experience with peer reviewing and the overall publication of scientific manuscripts.
REFERENCES
- Galipeau J, Moher D, Skidmore B, et al. Systematic review of the effectiveness of training programs in writing for scholarly publication, journal editing, and manuscript peer review (protocol). Syst Rev. 2013;2(41):1-7. doi:10.1186/2046-4053-2-41
- Patel J. Why training and specialization is needed for peer review: a case study of peer review for randomized controlled trials. BMC Med. 2014;12:128. doi:10.1186/s12916-014-0128-z
- Disclosure of Financial and Non-Financial Relationships and Activities, and Conflicts of Interest. International Committee of Medical Journal Editors. Accessed February 1, 2024. https://www.icmje.org/recommendations/browse/roles-and-responsibilities/author-responsibilities–conflicts-of-interest.html
- Kleinbaum DG, Morgenstern H, Kupper LL. Selection bias in epidemiologic studies. Am J Epidemiol. 1981;113(4):452-463. doi:10.1093/oxfordjournals.aje.a113113.