For JOE Reviewers—Processes
This is a description and explanation of the various processes associated with being a JOE reviewer.
Click a link to jump to a section below:
Help for Reviewers
Reviewer Mass Mailings
You will occasionally receive messages the editor sends in a mass mailing to all reviewers via <joereviewers@joe.org> that contain information she hopes will be helpful.
Other Helpful Material on the JOE Site
The Help for JOE Authors page, although directed at JOE authors, should help you as you review the articles those authors have written. Needless to say, the other material linked on the For Reviewers page should also help.
The companion document to this one, For JOE Reviewers—Principles, should help you most of all.
Blind Copies of Review Results & Reviews
A process has been added to the array of JOE processes that will enable you, as reviewers, to both learn the ultimate fates of the submissions you review AND see how your fellow reviewers have reviewed the submissions you've reviewed (which will help you develop "JOE reviewer best practices").
The new process requires the editor to blind copy both the corresponding author and the reviewers when she sends authors their review-result syntheses and copies of their reviews, to send the mail to herself, and to address the messages to "Dear JOE Author." Seems strange, but it means we maintain the integrity of JOE's double-blind review process while also learning from each other.
Thus, you'll be getting messages from the editor addressed to "Dear JOE Author" that concern articles that you did not write. Don't panic. Remember that they probably concern articles you have reviewed.
JOE's Two-Tier Review System
JOE employs a two-tier review system. That is, the editor first reviews all submissions to determine whether or not they are suitable for double-blind review or publication.
If the submission is not suitable for review as written, the editor either rejects the submission, including an explanation, or returns the submission to the author with (often substantive) revision suggestions. (See the April 2001 Editor's Page for some of the things the editor looks for.)
In 2008, the editor returned 163 submissions (58%) for revision before accepting them as suitable for double-blind review or publication and rejected 34 (12%) as unsuitable for JOE.
This two-tier review system allows JOE to make the legitimate claim that it offers "a unique combination of professional development and academic rigor."
Receiving Submissions for Review
Submissions Sent in Batches
The editor sends submissions to reviewers in batches or "rounds" as Microsoft Word attachments when there are submissions to "cover" just about every reviewer. (Doing anything else, given that there are some 70+ reviewers at any given time, would be a tracking NIGHTMARE.) The editor gives you 4 to 5 weeks to review the one or two submissions she sends you.
Notify Editor Ahead of Time
Because of the above, if you know that you will be unable to review for a particular span of time, it's very important that you notify the editor ahead of time—BEFORE she has sent you submissions you are unable to review.
If You Don't Receive Submissions to Review
If you don't receive submissions to review, that means there are none in your areas of expertise or that you have conceived of/described your areas of expertise too narrowly.
RSVP When You Receive Submissions
When you receive submissions to review, you are asked to RSVP the editor so that she knows you've received your submissions. It is important that you RSVP as soon as you receive the submissions. A lot of time gets spent sending inquiries to reviewers who have received their submissions but failed to acknowledge that fact.
Check Submissions ASAP
And please at least "eyeball" the submissions as soon as you can upon receiving them. Waiting until right before the due date to discover that you have a conflict of interest regarding a particular submission or that the editor has mistakenly sent you a submission outside your area of expertise means further delays for authors anxious to receive their review results.
If You Have a Conflict of Interest
If you do have a conflict or the editor has sent you a submission outside your area of expertise, please contact her and explain.
Read the Messages Accompanying Submissions
Also, be sure and read the messages that accompany the submissions you receive. They all contain new information, and it's information the editor wants you to have.
Returning Reviews
Remove Personal Information
Before you return review files to the editor <joe-ed@joe.org>, remove personal information from them, as per the instructions you'll find in the December 2008 Editor's Page. This is necessary in order to preserve the integrity of JOE's double-blind review system.
"CC" Cindy Verhey
When you return your reviews to the editor, remember to "CC" Cindy Verhey <clverhey@purdue.edu>. She tracks reviews for the editor and maintains the JOE database. If you do not "CC" her, the editor has to send her your reviews—if she notices that you have not. If she hasn't noticed your omission, there's a chance your reviews will languish in her overfull Inbox.
Return Reviews as Microsoft Word Attachments
The editor sends submissions and their accompanying review forms to you as Microsoft Word attachments, and that is how you should return them. DO NOT embed the reviews in an email message or turn them into PDF files (or something). The steps involved in ensuring reviewer anonymity for Microsoft Word files are numerous, complex, and time-consuming enough without also having to worry about covering other bases.
Late Reviews
Late reviews are, of course, not good. But this is Extension, so occasionally you will be absolutely unable to review the submissions you receive in the 4 to 5 weeks the editor gives you. Do not assume that, when your reviews are late, the editor will simply send the submissions to other reviewers. As explained above, submissions are sent for review when enough of them have accumulated that all reviewers are "covered." In other words, there are no "other reviewers" to whom to send "your" submissions until the next batch of submissions is sent for review.
And being late with reviews does not mean you will not receive more submissions in the next batch of submissions sent for review. That would not be fair to your more timely colleagues.
Submission Identification
Because JOE uses a double-blind review system, the submissions are identified by a 5-digit, 3-character "string." The five digits indicate the year of the submission and the order in which it came in. So 09156 would be the 156th submission of 2009. The three characters following indicate the article category of the submission (FEA = Feature, RIB = Research in Brief, IAW = Ideas at Work). So article 09156FEA would be a Feature article.
You'll sometimes receive submissions with quite different "order numbers" because the editor reviews all submissions initially to determine whether they are suitable for review or publication (Commentaries and Tools of the Trade) and sends many back to authors for revision before she accepts them for publication or puts them in the waiting-for-review-queue. So there'll be earlier numbers amidst the later ones.
Security & Anonymity
Regularly check your junk mail, and "teach" your system to accept mail from <joe-ed@joe.org>. Some reviewers have found submissions for review in their junk mail. Unfortunately, given the abuses of spammers and the resulting increasingly formidable barriers IT folks have erected to protect users, checking junk mail has become pretty much a daily necessity.
Again, please also see the December 2008 Editor's Page. It contains instructions for how to remove personal information from your review files, a necessity given JOE's double-blind review process.
Sending Review Results to Authors
Review Result Syntheses
When the editor receives all reviews of an article (3 for Feature and Research in Brief and 1 for Ideas at Work), she sends the corresponding author an email synthesis of their review results in which she offers further revision guidance and reconciles discordant review recommendations if that's necessary. She attaches the review or reviews to that message and blind copies the reviewer/reviewers.
Rejected Articles
When the final disposition of an article is that it is rejected for publication, she does the same thing and ends the message with the following: "I hope that the attached reviews will help you should you decide to send future submissions to JOE or prepare and submit a new article on the topic of this article."
The editor rejects Features and Research in Brief articles for publication if they receive a major revision and resubmission recommendation for disposition and a reject recommendation for disposition from their reviewers or two reject recommendations for disposition.
Reviewing Post-Reviewer Revisions
When the editor sends authors their review-result syntheses and reviews, she asks them the following: "In the email message accompanying your revised submission, please include a detailed explanation of how you have revised your article to respond to your reviewers' comments and suggestions. (Note: HOW you have revised, not simply THAT you have.)"
This practice serves two purposes.
- It encourages authors to take their reviews and their revisions more seriously than they might otherwise, to carefully consider their reviews and to "own up" to their revisions.
- Their explanations allow the editor to review the revisions herself instead of sending them back to their reviewers for "re-review." (Sometimes, the editor returns revisions for further work if the authors have not sufficiently addressed their reviewers' concerns or provided satisfactory explanations of their revisions.)
The editor consults reviewers about revisions only on rare occasions.