The Peer Review Process

Big Picture: The point of peer review is not to help each other feel better! It is to help each other understand and improve the quality of our work.

In my courses students correct and evaluate each other's writing according to these guidelines.

When writing and editing, be familiar with my suggestions for essay writing. Use standard editing marks. Look for the common errors described by standard proofreading abbreviations. (If you prefer, you may use full descriptions rather than these abbreviations.)

Step 1: The writer finishes the initial version of the exercise. (Students want to call this a "rough draft," but it is not. It is a finished piece of your best work.) I use these initial versions to make sure the others in your group have adequate time to review your work and you have adequate time to revise it if necessary. If your initial version is radically incomplete, I will count it late even if you submit it on time.

Step 2: The writer prints the initial version, attaches two copies of this review form (here is an HTML version if you cannot print the Acrobat version), and brings it to class on the initial due date. It is the writer's responsibility to submit work that meets the parameters of the assignment and the requirements listed in this guide.

Step 3: Your group meets together within two days of the submission date. Meeting later than that will not allow you enough time if you need to revise your work. There each reviewer reads the exercise, marks the assignment with editing marks and marginal comments, and adds evaluations and summary comments on one copy of the review form, along with his or her name and total score. These comments are directed first at evaluating the quality of the exercise, and second at offering suggestions for improvement. It is each reviewer's responsibility to catch the flaws and note the strengths mentioned in this guide. Then the writer takes the paper back. This is a good time for the group to discuss each paper.

Step 4: The writer can make changes and resubmit both the original and the final version. If the submission is unacceptable (that is, below college level), the writer must make required changes, and should resubmit to the reviewers for another peer review cycle. (After one or two shocks, most writers will learn to pay close attention to the questions and sources from the outset of an assignment.) As a rule you have a week from your original submission to submit your revised work. Otherwise additional late penalties will normally apply.

Step 5: Finally I, the coach, look over everything, and may adjust the score and add comments of my own. I also grade each reviewer based on the quality and accuracy of the review. Then I record the writer's and reviewers' grades and hand the exercise back. I promise to have your exercises graded and available to you again within twenty years of the original submission date.

Before you raise questions, complaints, or criticisms, I request that you consult my peer review FAQ.

Warning: I have found students to be habitually more affirming and less critical of each other's work than they should be. As a reviewer, your job is not just to find nice things to say. Your job is to test arguments for their strength and identify problems to correct. In your reviewing, consider encouragement optional and specific correction required.

Here, in more detail, is how each reviewer should evaluate each section:


Character

An essay meets the expectations of team membership.

A writer owes it to reviewers and readers to have his or her 'act together': to be available, to be on time, to submit work that is complete and properly written, and in general not to put undue burdens on audiences.

Character Standards:

Reviewing this exercise was convenient because the writer acted professionally throughout the process, or the coach has excused the issues below (for instance, due to athletic events off campus or unforeseeable health problems). -0
The writer was unavailable to meet at a time convenient to the others in the group within two days of the original submission, or was late to your scheduled meeting, or did not provide a completed and reasonably proofread copy of the exercise. -2
Two or more of the above issues. -5

Question

An appropriate essay addresses the whole question put to it.

Both the writer and the reviewer should review the whole assignment, identifying all of the little questions within it. If the entire question is not explicitly addressed, then the submission is unacceptable. (If the answer is flawed or wrong, evaluate it accordingly in the reasoning section, but not here.)

Question Standards:

All aspects of the essay question are addressed explicitly and thoroughly -0

The question is addressed incompletely, so subtly that a typical reader misses it, or not at all

U

Organization

A well organized essay is structured so that it communicates clearly.

A reader should not have to work hard to learn an essay's main ideas. To help you do that, an essay should have

a brief, specific introduction that clearly guides the reader,
an appropriate progression of thought, and
ideas adequately paragraphed (with topic sentences as appropriate).

Unless the assignment says otherwise, these are required even of submissions that do not have a typical essay form. However, a conclusion is optional.

Introduction Standards:

The beginning of the essay introduces the argument specifically, directly, and briefly in a way that clearly guides the reader. -0
While the beginning introduces the basic argument in a way obvious to the reader, it does so incidentally or vaguely. -2

There is no introduction, or else the lead paragraph is just a "hook" that gets the reader's attention but fails to guide the reader through the rest of the essay.

U

 

Progression Standards:

The argument progresses in a clear and orderly way without meandering. Its points are set apart in coherent, appropriately structured paragraphs. -0
While the argument follows a basically orderly progression, it occasionally gets sidetracked, or else it paragraphs confusingly, misleadingly, or incoherently. -2

Sources

A serious essay depends upon course materials and documents its sources.

I craft assignments explicitly designed to draw you back into sources such as lectures and readings. Essays that fall short of that expectation are greatly weakened. Essays that simply fail to meet that expectation are unacceptable.

Readers can tell whether source dependence is 'structural' (part of the basic argument in a way that makes the source indispensable to the argument, so that removing the source material would substantively weaken or change the argument) or 'ornamental' (not actually bearing much or any weight, so that removing it would not really change the argument).

Source Dependence Standards:

All sources relevant to answering the questions (lectures and readings) are explicitly and substantively consulted, so that they are built into the structure of the argument itself. -0
All relevant sources are consulted, but one or more only superficially or ornamentally. -3
One or more required source is missing. U

When the writer draws directly or indirectly on some other source, a citation should follow. Mentioning a source in the text ("Work argues that ...") but without a page number is often inadequate. A citation may cover any number of consecutive sentences it follows. Cite

direct quotes,
indirect quotes,
authorities, and
ideas.

Citations should be in parentheses in the body of the essay. Punctuation can go either before or after the reference; just be consistent. The following formats are acceptable: For books, (Work 1) or (Work, 1) or (Work 2001, 1). For lectures, (Work, 1/1/2002) or (Work, "Lecture Title", 1/1/2002). For Bible references, (Gen. 1:1).

Footnote only when drawing on texts beyond the course materials.

Reproducing the exact words of any source without including them in quotation marks, or reproducing thoughts of outside sources without documenting them, goes beyond inadequate documentation to plagiarism. My draconian warnings about plagiarism are here. If the editor identifies possible or even probable plagiarism, he or she should mark it and notify the writer so the writer can revise. Once I have received the essay, I do not accept excuses, and I punish academic dishonesty by failing the student from the course.

This score is not the place to judge how well or how thoroughly a writer uses sources. If an essay does not cite because it does not draw explicitly on course materials in the first place, then mark it unacceptable under the "sources" section.

Citation Standards:

The submission uses proper citation style with no missing references. -0
The citation style is incomplete (e.g., no page numbers or lecture dates), but there are no missing references. -2

There is one or more missing reference.

U

Style

A well written essay conforms to standard written English.

Watch for the errors described by standard proofreading abbreviations as well as your style guides. I am especially concerned that the essay has

correct spelling,
proper commas,
proper apostrophes,
complete, straightforward sentences (no sentence fragments, run-on sentences, and so on),
noun/pronoun/verb agreement, and
consistent verb tenses.

Style Standards:

There are two errors or fewer. -0
There are 3-4 errors. -1
There are 5-6 errors. -2

There are seven or more errors.

U

Reasoning

A well reasoned essay answers its question thoroughly and logically.

If an essay is actually acceptable as college-level work, then this is finally the place for the reader to determine how well the essay answers the question. (If the work is unacceptable and must be resubmitted, then the reader may want to wait until the submission is acceptable before making this determination.)

Evaluation is unusual here in that even when the work is acceptable the reviewer is required to defend his or her assessment. Why specifically is this judgment warranted? What specific points are unusually helpful, or somewhat flawed, or basic answers?

Readers and writers should be on the lookout for both inadequate reasoning and tangential or even fallacious reasoning. One fine online guide to logic is at http://datanation.com/fallacies/, and another is at http://www.nizkor.org/features/fallacies/. If an essay commits a logical fallacy, it is yours to judge whether that fallacy only weakens the argument in a minor way, or catastrophically. You should assign a score accordingly and explain your judgment on the peer review form.

Reminder: Again, when appraising others' work, I have found students to overvalue each other's reasoning much more often than they undervalue it. Do not be needlessly critical, but do not be affirming for the sake of comfort either. Be specific, rigorous, helpful, and fair.

Grading:

Outstanding The paper argues with quality that rivals that of the assigned readings for the course. 30 pts
Excellent It answers the question with rare insight. 28-29
Very Good It thoroughly and solidly establishes an unusually helpful answer. 27
Very Good

It answers the question thoroughly.

Or, it is helpful, thorough, solid work, but with minor flaws.

26
Good It answers the basic question well with only minor flaws. 25
Good

It answers the basic question with only minor flaws.

Or, it is somewhat flawed but otherwise superior work.

24
Adequate It answers the basic question. Its flaws do not discredit the answer itself. 23
Adequate It answers the basic question, but only weakly. Its flaws do not discredit the answer itself. 22
Barely Adequate Its flaws obscure but do not entirely discredit a basic answer to the question. 21
Problematic Serious flaws allow only partial progress in answering the basic question. 18-20
Inadequate

It basically fails to answer the question.

Or, its fatal flaws in reasoning discredit the fundamental answer.

0-17

Total

A college-level essay 'puts it all together.'

The overall score adds all these numbers.

Consider an essay whose basic score is 25: it answers the basic question well with no more than minor flaws. Its writer acts professionally throughout the writing and reviewing process. It thoroughly addresses the question, includes a somewhat vague or tangential introduction, progresses clearly and paragraphs coherently, consults all relevant sources (but one only shallowly), cites all its sources in an appropriate format, and has five grammatical or spelling errors. The reviewer would adjust the basic score as follows:

25 - 0 - 0 - 2 - 0 - 3 - 0 - 2 = 18 points

In other words, this B paper just became a D paper. My example shows you that a few deficiencies in an otherwise solid paper can easily knock it down one or two grade levels. It also shows you that surviving these assignments (and the course) is really just a simple matter of following directions — figuring out the whole question, using the course materials, writing clearly, introducing your argument, and staying on point. Students who get into these habits quickly find that writing exercises becomes both faster and easier than they imagined it could be at the beginning of the semester.

Your Actual Grades

I will be grading both writers and reviewers in a much less precise way:

Coach's Grade:

+ Excellent The work stands out as exceptional.
check  Strong It fulfills the requirements of the assignment.
- Weak It barely meets the requirements of the assignment.
0

Failure

It fails to meet the requirements of the assignment.

I reserve the right to correlate the scale to traditional letter grades however I like if it produces more appropriate results at the end of the course. However, as a working assumption I think of a 'plus' as an A, a 'check' as a B-, a 'minus' as a C-, and combinations of these scores to fall somewhere between these grade levels.

After requiring such a detailed peer review, why do I 'round off' my own assessment so radically? For several reasons:

First, I want to focus you on writing to learn instead of writing to get a certain grade. (Do you want a good grade? Then do well, improve, and do both consistently.)

Second, I want to keep peer reviewing and group meetings constructive. You are not grading each other. You are helping each other write, argue, and think well. I am the one who has to grade you. Moreover, I make my judgments before I read your reviews. However, I do so according to the same criteria as your reviewers, so their scores are good leading indicators of what mine might be.

Third, I want to reduce the amount of time I spend reviewing these assignments. I have better things to do than remind you again and again how to use apostrophes. With a scale like this, I can spot-check everyone's work, keep everyone honest, get assignments back to you promptly, and keep obsessions and disagreements over grades to a minimum.