Suggested Guidelines for WITS Review
First, thank you for your contribution to the review process. We realize the time commitment involved in preparing a detailed review, and very much appreciate your willingness to contribute to the review process.
Please follow the guidelines below when preparing your review.
Guidelines for writing your review:
Elements to build your evaluation on
- Summarize the main claimed contributions.
- Relevance: What is the relevance of this paper to an Information Systems audience?
- Significance: We strive for WITS to present and bring into the community’s awareness novel, interesting, and potentially impactful ideas. Even if the work presented leaves much to be desired, if you find the problem and ideas to be novel and stimulating please consider such work favorably. Please consider the following questions: Are the results important? Is it likely that other researchers and/or practitioners will use these ideas or build on them? Does the paper address a challenging problem in a better way than previous research? Does the paper offer fairly novel knowledge or a unique theoretical or pragmatic approach? Note that a single paper need not offer all the above to be of significance.
- Technical soundness: Is the paper technically correct? Are the concepts correct and accurate? Are the measures used relevant for demonstrating the value and are the evaluations done correctly and sufficient details are provided to establish the contributions?
- Novelty: Are the problems or approaches sufficiently novel? Note that a novel combination of existing techniques is valuable if the combination is non-trivial (e.g., it is not a straightforward implementation or execution of existing solutions) and solves an important problem. Do the authors do a good job at describing clearly how this work differs from previous contributions? Is related work adequately described and referenced?
- Quality of Evaluation: Are the main claims well-supported by theoretical analysis or experimental results? Note that for a workshop paper, there may be unexplored strengths and weaknesses of the key ideas. Thus, the primary goal is to evaluate if reasonable and reliable evidence is provided to suggest the main idea is promising.
- Clarity: Is the paper clearly written? Is it well-organized? (If not, feel free to make suggestions that the authors may quickly undertake).
Guidelines on how to write your comments on the above elements.
Overall, we strive for WITS reviews to be respectful, conscientious, and constructive. When preparing your comments, please also consider the space limitation and scope that a workshop paper can undertake.
Papers should demonstrate solid and meaningful scientific contributions. Whether or not you find the paper constitutes such a contribution because of weaknesses along one or more of the elements above, please provide the authors and review committee what contributions/limitations of the work are key in your assessment, and outline your rationale for why you find these contributions/limitations make the contribution (un)suitable for being accepted to WITS.
It is important that you provide sufficient details to support your assessment. For example, if you find that the contributions have already been made elsewhere, please (a) provide relevant references, and (b) explain how the work you cite have already established the specific claimed contributions in the paper under review making clear that the similarity is not superficial but significantly limits the current paper’s contributions. If you find that the analytical/empirical evidence provided does not suggest that the main claims are promising, then point out why so, and include any relevant references if such are needed. Similarly, if you find that the problem the authors consider is not important and is unlikely to be built on, please offer the rationale for this assessment. If you find that the results/evidence provided is not satisfying, state what research questions/claims the evidence did not support.
Feel free to comment on how any issues may be corrected, when possible.
In general, please avoid asking the authors questions or suggest work that you are curious to see the result of without offering the reasoning underlying these suggestions, particularly if these comments underlie your primary criticism.
Finally, please review your comments to ensure they are clear, well-thought of, polite and respectful of the authors and their effort. Be read your report at least once carefully before submitting it.
We thank you and are grateful for your conscientious service to the WITS community, and hope that the community will reciprocate and facilitate your work as much as you do for others.