Skip to content

Latest commit

 

History

History
116 lines (70 loc) · 4.42 KB

HOWTO_review_a_paper.md

File metadata and controls

116 lines (70 loc) · 4.42 KB

Golden rule

** Review unto others, as you would like your own papers to be reviewed. **

Reviews can be needlessly harsh. Remember Dennett's 4 rules for successful critical feedback:

  • You should attempt to re-express your target’s position so clearly, vividly, and fairly that your target says, "Thanks, I wish I’d thought of putting it that way."
  • You should list any points of agreement (especially if they are not matters of general or widespread agreement).
  • You should mention anything you have learned from your target.
  • Only then are you permitted to say so much as a word of rebuttal or criticism.

Suggested Review Form

1. Overall recommendation

Overall recommendation: accept or reject? Give 1-3 main reasons for your decision.

2. Summary of Strengths

List out at least 2 strengths of the paper (even if you didn't like it). 1 phrase or sentence each.

  • S1: ....
  • S2: ....

These strengths should be referred to (via the monikers S1, S2, ...) in sections 5-9 below, when you elaborate on the issue more.

3. Summary of Weaknesses

List out at least 2 weaknesses of the paper (even if you liked it) 1 phrase or sentence each.

  • W1: ....
  • W2: ....

These weaknesses should be referred to (via the monikers W1, W2, ...) in sections 5-9 below, when you elaborate on the issue more.

4. Summarize the paper

In your own words, what does the paper do? You should summarize:

  • The problem being solved
  • The proposed solution
  • How the proposed solution is evaluated

This section should have facts only, not your opinions. Your goal should be that the authors of the paper agree with every statement here.

5. Contribution relative to Previous Work

Clarify your evaluation of what already known (with specific citations), and what is added by this paper. Evaluate how well the authors summarized the literature. Point out any gaps. Evaluate the novelty of the contribution in context of the literature.

5. Significance

Describe how significant the paper's findings are. What impact will it have? What kinds of researchers are likely to want to read this or learn from it? How likely is current practice to change?

6. Soundness and Rigor

Describe the quality of the study. Have the claims been justified with appropriate evidence (theory or empirical)? Are scientific best practices followed? Are experiments reproducible by another member of the field using the paper alone?

7. Presentation Quality

Evaluate the organization of the paper as well as the clarity and conciseness of the writing.

8. Limitations

Describe the key assumptions or limitations of the approach. Are they adequately described in the paper so future readers would know these?

9. Detailed Feedback for Authors

Anything else the author would like to know (typos, minor suggestions, etc). This section could be entirely ignored by the other reviewers/AC in decision making.

Suggested review process

First, write the paper summary (#4 above)

Next, summarize the contributions (#5)

Next, summarize the significance (#6)

Next, force yourself to list out strengths as well as weaknesses (#2 and #3 above). Write out detailed comments, covering strengths and weaknesses, for #7-9 above).

Finally, write your overall top-level summary (#1). Should this be accepted? Would you recommend this paper to a colleague?

Before you submit it, read over the whole review again... are you being fair? is this the kind of review you'd like to receive? Is most of the review addressing the truly important strengths/weaknesses?

Useful resources

Other resources

Remember the 7 W's, from https://mvaldenegro.github.io/files/LXCV-CVPR-2021-review-mentoring.pdf#page=13

Less vetted, but still could be useful