Reviewer Guidelines

Reviewing for the Programming Historian is a great way to learn new technical skills and engage with the digital humanities community. We go out of our way to make sure our reviewers get credit and recognition for their work. Because reviewers directly contribute to significantly improving lessons, you can take pride in that your work helps thousands of readers.

These guidelines are to help reviewers understand their role in the editorial process and to answer common questions about how to be most efficient and effective with your reviews.

Contents

• Reviewer Guidelines
  • Review Philosophy
    • Constructive Criticism
    • Transparency
    • Openness and Inclusivity
  • What To Comment On
    • Audience
    • Getting Ready
    • Skimmability
    • Payoff
    • Workflow
    • Sustainability
    • Integrating with the Programming Historian
  • Submitting your review

Review Philosophy

We consider the review process an integral component of a collaborative, productive, and sustainable effort for scholars to create useful technical resources for each other. Once a tutorial enters our editorial workflow, we work closely with the author and reviewers to maximize its potential and publish it within a reasonable amount of time.

Constructive Criticism

To review a tutorial is not simply to assess its merits and flaws (though it is important to do these as well), but to help improve the tutorial by virtue of an outside perspective. We ask that reviewers provide fair and constructive criticism (and of course praise) that will improve the tutorial for a wide range of readers.

Transparency

We highly value transparency in our lesson production and review process. Our review process happens on Github, where our lessons are hosted. Namely, we use Github’s commenting and issue tracking capabilities to keep track of suggestions and how they are handled and discussed by the authors and reviewers. This helps avoid important conversations becoming buried and lost in email threads.

Therefore, your work as a reviewer–and your identity–will be fully visible to the author. Comments should engage with the author and lesson directly, rather than the review editor. If at any point you are unsure of your role or what to do next, feel free to post a question to clarify and an editor will respond as soon as they can.

In keeping with the ideas of public scholarship and open peer review, we generally encourage discussions to stay on GitHub as outlined in our editorial workflow. However, we also want everyone to feel comfortable. In some cases a private word may be more appropriate. If you feel the need to discuss a matter related to a tutorial or a matter related to the review, please feel free to email the assigned editor directly, or to contact one of our dedicated ombudsperson (Dr Ian Milligan - i2milligan@uwaterloo.ca).

Unless you instruct us otherwise, your name will be indicated as a reviewer on the lesson’s page when it is officially published; you’ll also be listed on our contributors page.

Openness and Inclusivity

The Programming Historian is dedicated to providing an open scholarly environment that offers community participants the freedom to thoroughly scrutinize ideas, to ask questions, make suggestions, or to requests for clarification. We insist on a harassment-free space for all contributors to the project, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age or religion, or technical experience. We do not tolerate harassment or ad hominem attacks of community participants in any form. Participants violating these rules may be expelled from the community at the discretion of the editorial board. If anyone witnesses or feels they have been the victim of the above described activity, please contact our ombudsperson (Dr Ian Milligan - i2milligan@uwaterloo.ca). Thank you for helping us to create a safe space.

What To Comment On

The informal style of Programming Historian lessons can make the lessons appear deceptively simple to write. In fact, writing a good tutorial is every bit as demanding, if not more so, than any other kind of scholarly writing. Below are a few common questions to keep in mind as you review a lesson. Some will be more relevant than others, depending on the topic, intended audience, and the difficulty of the tutorial. Needless to say, this is neither a restrictive or comprehensive list; we do not ask that reviewers respond to each and every question, but hopefully they can provide some general guidance.

Audience

The Programming Historian speaks to a variety of audiences and reader skill levels. Some lessons are for complete beginners, others are for those considerably more comfortable with technical concepts and methods, and yet others are directed toward experienced digital historians looking to solve difficult technical challenges. We welcome such variety!

While we like each of our lessons to take the unique voice of its author, we also want each individual lesson to remain clear and consistent in its tone. More specifically, we want the technical explanations (and difficulty) to remain as constant as possible throughout the lesson. As a reviewer, it is quite useful to note any sections of a tutorial directed at more technically advanced users that seem to over-explain relatively simple concepts; the converse is true, too: we want to avoid sections of tutorials aimed at beginners that don’t adequately explain fundamental concepts central to the tutorial. Some questions to keep in mind:

• Does the author address a consistent model reader throughout the lesson?
• Are some concepts or steps over-explained while other are under-explained?
• Does the audience seem to match at least vaguely with other Programming Historian lessons? How is it new?

Getting Ready

• What software / programming languages are required?
• What prerequisite skills are needed?
• What familiarity or experience is needed?
• What data are needed? Is the dataset readily available?

Skimmability

• Are there clearly defined learning objectives or sets of skills to be learned listed near the top of the lesson?
• Are there useful secondary skills to be gained / practiced from the lesson?
• Do screenshots and other diagrams illustrate crucial steps / points of the lesson?
• Do sections and section headings provide clear signage to the reader?

Payoff

• Does the tutorial suggest why the explained tools or techniques are useful in a general way?
• Does the tutorial suggest how a reader could apply the concepts (if not concrete steps) of the lesson to their own work?

Workflow

• Should a long lesson be divided into smaller lessons?
• Are there logical stopping points throughout the lesson?
• If datasets are required, are they available to download at various points throughout the lesson (or different versions of them as the tutorial may require)?

Sustainability

To increase the lifespan of our lessons, Programming Historian reviewers should keep in mind the following questions about sustainability. Every submission is different and some of these areas may not be applicable to all submissions. Keeping in mind the difficulty level of each lesson and its intended audience, reviewers should use these questions as guidelines to ensure that lessons are as sustainable as possible from the date of publication.

• Are all software versions and dependencies listed in the submission? Are these assets the most recent versions? If the lesson uses older software versions, does the author note why?
• If you have expertise in the specific methodology or tool(s) for the lesson, is the methodology generally up-to-date?
• What are the data sources for the submission? Are they included in a way that does not heavily on third-party hosting?
• What kinds of other external links does the submission use? Are these current or are there other, more recent or appropriate, resources that could be linked to?

Integrating with the Programming Historian

• Does the lesson build upon an existing lesson and explain how?
• Does the lesson tie into existing lessons and have appropriate links?

Submitting your review

We manage all of our peer review comments through GitHub. When a new lesson is ready for review, editor will provide you with a link where you can read the lesson, and a link to a discussion board where you can submit your constructive feedback. This discussion is held on Github, a free social coding environment. You will need to sign up for a free GitHub account to post your review. We encourage discussion to stay on Github but you are free to email the editor privately or to contact one of our ombudsperson (Dr Ian Milligan - i2milligan@uwaterloo.ca) if you would like a private word.