Scientific Peer Review

Rigorously evaluate scientific work for quality, validity, and reproducibility.

When to Use

• Self-reviewing manuscript before submission (REVIEW phase)

• Evaluating methodology and experimental design

• Checking statistical analyses and reporting

• Assessing reproducibility and data availability

• Reviewing others' manuscripts for journals

• Evaluating grant proposals

• Quality checking your own work during ANALYSIS phase

Review Workflow

1. INITIAL SCAN → Overall impression, scope, significance
2. SECTION REVIEW → Detailed evaluation of each section
3. METHODOLOGY → Rigor, assumptions, controls
4. STATISTICS → Appropriate tests, effect sizes, reporting
5. REPRODUCIBILITY → Data, code, materials availability
6. FIGURES/TABLES → Clarity, integrity, accessibility
7. ETHICS → Approvals, consent, conflicts
8. WRITING → Clarity, organization, accuracy
9. SYNTHESIZE → Major/minor issues, recommendation

Stage 1: Initial Assessment

Quick Questions (5 minutes)

• What is the central research question?

• What are the main findings?

• Is the work scientifically sound?

• Are there any immediate major flaws?

• Is it appropriate for the intended venue?

Initial Summary Template

Initial Assessment

Research Question: [One sentence summary]

Main Findings: [2-3 key results]

Initial Impression: [Sound/Concerning/Major issues]

Significance: [Novel contribution to field?]

Stage 2: Section-by-Section Review

Abstract & Title

Check Question Status

Accuracy Does abstract reflect the actual study? ☐

Clarity Is the title specific and informative? ☐

Completeness Are key findings summarized? ☐

Accessibility Understandable to broad audience? ☐

Introduction

Check Question Status

Context Is background adequate and current? ☐

Rationale Is the research question justified? ☐

Novelty Is originality clearly stated? ☐

Literature Are relevant papers cited? ☐

Objectives Are aims/hypotheses clear? ☐

Methods

Check Question Status

Reproducibility Can another researcher replicate this? ☐

Rigor Are methods appropriate for the question? ☐

Detail Protocols, reagents, parameters described? ☐

Ethics Approvals and consent documented? ☐

Statistics Methods described and justified? ☐

Controls Appropriate controls included? ☐

Critical Details to Verify:

• Sample sizes and power calculations

• Randomization and blinding

• Inclusion/exclusion criteria

• Software versions

• Statistical tests and corrections

Results

Check Question Status

Presentation Logical and clear? ☐

Figures Appropriate, clear, labeled? ☐

Statistics Effect sizes, CIs, p-values? ☐

Objectivity Results without interpretation? ☐

Completeness Negative results included? ☐

Common Issues:

• Selective reporting

• Inappropriate statistical tests

• Missing error bars

• Over-fitting

• Batch effects or confounders

• Missing controls

Discussion

Check Question Status

Interpretation Conclusions supported by data? ☐

Limitations Acknowledged and discussed? ☐

Context Placed appropriately in literature? ☐

Speculation Distinguished from data-supported claims? ☐

Significance Implications clearly stated? ☐

Red Flags:

• Overstated conclusions

• Ignoring contradictory evidence

• Causal claims from correlational data

• Mechanistic claims without evidence

Stage 3: Methodological Rigor

Statistical Assessment

Check Question Status

Assumptions Are statistical assumptions met? ☐

Effect sizes Reported alongside p-values? ☐

Multiple testing Correction applied? ☐

Confidence intervals Provided? ☐

Sample size Justified with power analysis? ☐

Missing data Handled appropriately? ☐

Exploratory vs confirmatory Clearly distinguished? ☐

Experimental Design

Check Question Status

Controls Appropriate and adequate? ☐

Replication Biological and technical? ☐

Confounders Identified and controlled? ☐

Randomization Properly implemented? ☐

Blinding Adequate for the study? ☐

Stage 4: Reproducibility Assessment

Data Availability

Check Question Status

Raw data Deposited in repository? ☐

Accession numbers Provided for databases? ☐

Restrictions Justified (e.g., privacy)? ☐

Formats Standard and accessible? ☐

Code and Materials

Check Question Status

Analysis code Available (GitHub, Zenodo)? ☐

Protocols Detailed enough to reproduce? ☐

Materials Available or recreatable? ☐

Reporting Standards

Check adherence to discipline-specific guidelines:

Study Type Guideline Status

Randomized trial CONSORT ☐

Observational STROBE ☐

Systematic review PRISMA ☐

Diagnostic study STARD ☐

Animal research ARRIVE ☐

Case report CARE ☐

Stage 5: Figure and Table Review

Quality Checks

Check Question Status

Resolution High quality? ☐

Labels All axes/columns labeled with units? ☐

Error bars Defined (SD, SEM, CI)? ☐

Statistics Significance markers explained? ☐

Color Colorblind-friendly? ☐

Scale bars Included for images? ☐

Integrity Checks

Check Question Status

Manipulation Any signs of image manipulation? ☐

Splicing Gels/blots appropriately presented? ☐

Representative Images truly representative? ☐

Complete All conditions shown? ☐

Stage 6: Writing Quality

Structure and Organization

Check Question Status

Logic Manuscript logically organized? ☐

Flow Sections flow coherently? ☐

Transitions Clear between ideas? ☐

Narrative Compelling and clear? ☐

Writing Quality

Check Question Status

Clarity Language clear and precise? ☐

Jargon Minimized and defined? ☐

Grammar Correct throughout? ☐

Concise No unnecessary complexity? ☐

Structuring the Review Report

Summary Statement (1-2 paragraphs)

Summary

[Brief synopsis of the research]

Recommendation: [Accept / Minor revisions / Major revisions / Reject]

Key Strengths:

1. [Strength 1]
2. [Strength 2]
3. [Strength 3]

Key Weaknesses:

1. [Weakness 1]
2. [Weakness 2]

Bottom Line: [Overall assessment of significance and soundness]

Major Comments

Issues that significantly impact validity or interpretability:

Major Comments

1. [Issue Title]

   • Problem: [Clear statement of the issue]
   • Why it matters: [Impact on conclusions]
   • Suggestion: [How to address it]

2. [Issue Title] ...

Major issues typically include:

• Fundamental methodological flaws

• Inappropriate statistical analyses

• Unsupported conclusions

• Missing critical controls

• Reproducibility concerns

Minor Comments

Less critical issues that would improve the manuscript:

Minor Comments

1. [Page/Figure X]: [Issue and suggestion]
2. [Methods section]: [Missing detail]
3. [Figure 2]: [Clarity improvement]

Review Tone Guidelines

Do ✓

• Be constructive and specific

• Acknowledge strengths

• Provide actionable suggestions

• Focus on the science

• Be thorough but proportionate

Don't ✗

• Use dismissive language

• Make personal attacks

• Be vague or sarcastic

• Request unnecessary experiments

• Impose personal preferences as requirements

Self-Review Checklist (Before Submission)

Use this during your REVIEW phase:

Methodology

• Methods are reproducible

• Controls are appropriate and documented

• Statistical methods are justified

• Sample sizes are adequate

Results

• All results support conclusions

• Effect sizes are reported

• Negative results are included

• Figures are clear and accessible

Reproducibility

• Data will be available

• Code is documented and available

• Protocols are detailed

• Reporting guidelines followed

Writing

• Abstract accurately summarizes the work

• Conclusions are supported by data

• Limitations are acknowledged

• References are current and complete

Integration with RA Workflow

REVIEW Phase Activities

• Run self-review using this checklist

• Document issues in tasks.md

• Address each issue systematically

• Re-review until checklist passes

• Update .research/logs/activity.md

Pre-Submission Verification

Before calling a manuscript complete:

• Self-review completed

• All major issues addressed

• Figures meet journal requirements

• Data/code deposited

• Reporting checklist complete

• Cover letter prepared