CyberChair(PRO) and the Review Process

CyberChair's development was started in 1996, and has (mainly) been used for conferences in the Computer Science domain. Prof. Oscar Nierstrasz (University of Berne, Switzerland) published a paper called Identify the Champion, which discusses the peer review process of Computer Science conferences: The first sentence of his paper is "The peer review process for technical contributions to conferences in computing sciences is very thorough, and can be as stringent as the review process for journal publications in other domains."

Classification

Prof. Nierstrasz describes a number of 'patterns' (proven practical techniques) in his paper that help the program committee to find the best submissions for the conference.

CyberChair(PRO) incorporates (most of) these patterns in its tools. The most important question for reviewers to fill in on the review form (of which both an online and an off-line version is available) is the following: "How do you classify this submission? 4 answers are possible: 

A: I will champion this submission at the PC meeting
   (advocate/accept).
B: I can accept this submission, but I will not champion it
   (accept, but could reject).
C: This submission should be rejected, though I will not fight strongly against it
   (reject, but could accept).
D: Serious problems. I will argue to reject this submission
   (detractor).

The second question on the review form is used to collect the reviewers' overall expertise on the submission's topics: X (expert), Y (knowledgeable), Z (informed outsider).

Other fields of the reviews form are 'open', and can be used by reviewers to explain their classification to the other members of the program committee, and to provide (detailed) comments to the authors, such as comments about the spelling, the style, novelty, etc. With other reviewing systems, the latter are often asked to be graded. These grades are then used in formulas to calculate rankings. CyberChair(PRO), however, does not work that way. Instead, it uses the classifications given to submission to categorize the submissions, as well as to indicate, during the review process, the level of agreement between reviewers. This way, reviewers (and the Chair) can prepare optimally for the PC meeting. Conflicts could even be solved prior to the PC meeting, possibly eliminating the need to discuss submission. The result could be a very efficient PC meeting.

The classification and overall expertise listed above are used for all kinds of overviews that help to find the best submission among those submitted. The review form can be changed, but this could imply rewriting large portions of CyberChair(PRO), so this is not encouraged.

Abstract, Bidding & Assignment proposal

Authors are asked to submit an abstract along with their submission. This abstract is used by the reviewers to indicate their eagerness to review certain submissions (also called bidding - see the instructions). The reviewers also indicate their expertise on the conference topics. The bids and expertise of the reviewers are then used to make an submission/reviewer assignment proposal for the Program Chair, who may use that as a first step in the assignment (typically a very time-consuming activity of a Program Chair when not supported by any tools). Using bidding is optional. If you, being the Program Chair, would like to do the submission assignment without this information, just don't let reviewers bid.

Back to Borbala