Distributed Peer Review - User Guide

Overview

The PeerReviewPlugin is a plugin for Trac that facilitates distributed peer review of source code. Normally peer review happens in a central location with paper copies of code, or works in an unstructured fashion involving email or other electronic means to communicate. However, these systems are inefficient for multiple code reviews and can become unmanageable, because of the need for tracking and synchronising the suggestions for improvement. Hence, this plugin assists developers in managing and performing code reviews in a distributed and centralised way.

Manual

Main Page

The main page is displayed when the Peer Review button on the navigation bar is clicked.

The first table is the list of code reviews where you are the reviewer.

Clicking on the review id or name in a particular row will take you to the View Code Review page for that review. Clicking any file name brings you to the Perform Review page for that file. Also, this table displays the current status of the review, the creation date, the total number of comments made for any file and the number of comments not marked as read by you.

The second table is the list of code reviews where you are the author. It shows a similar amount of information.

Managers will see an additional table at the top of the page above the two tables that are typically there for a developer. These are the code reviews that need to be reviewed by a manager. After approval by a manager, the manager should add the changes in the code review to the development trunk.

Finally, just below the navigation bar in the upper right corner are some navigation links for moving through the plugin. The links will take you to the Main Page, New Code Review Page, and Search Code Review Page, respectively.

New Code Review

A new code review is created by the following steps:

1. Choose a name
2. Select files and section to be reviewed
3. Add reviewers
4. Provide additional notes and instructions

Choose a name

This will be the name that appears on the main page after the code review is created. A name cannot be longer than 50 characters.

Add files

The user is presented with the Subversion repository tied to Trac.

Navigation through this repository is identical to navigation through the Trac repository. Clicking on folders navigates through the file structure. Clicking on files will open that file if an HTML preview is available. To navigate back up, click on the corresponding folder above the repository, for example, where it says "root".

Once a file has been opened, the repository browser will look similar to this:

There are two ways to select a file to be added into the peer review:

1. Select one or more files in the repository browser and click Add files. This adds the whole files to a review.
2. Click the corresponding lines of code on the left that you would like to add. For example, here the user clicked on 82 and clicked on 87 to select lines 82-87. Alternatively, the user could type those numbers into the corresponding text boxes below the repository browser.

Once the correct lines of code have been selected, click Add File to add the code to the code review. The location will appear in the table below the repository browser, along with the line numbers and revision number.

Add reviewers

Select reviewers from the dropdown list and click "Add user" to add them to the table of reviewers. This is the list of people that will vote on whether or not the code for the current code review is acceptable. Reviewers may be removed by clicking their names in the reviewers table.

Only users with permission CODE_REVIEW_DEV will be shown in the dropdown list.

Provide notes and instructions

General comments can be added to the code review. These comments can be seen by anyone viewing the code review.

After adding general comments, the user is ready to add the code review. Click "Add Code Review" at the bottom of the page to do this. The user will be taken to the View Code Review page for this code review.

Peer Review Search

This page helps search for code reviews, given relevant information to find the correct code reviews. First, search constraints are selected, and then a search is performed by clicking on the search button. When search constraints are selected and a search is performed, code reviews that still satisfy the constraints are listed. If no constraints are given, all reviews, both active and inactive (closed), are listed.

Combinations of fields narrow the search results. For example, searching for reviews after one month ago and by a certain author will likely return less results than searching by one of the constraints alone. It is possible to constrain the search such that no results are returned. Such a condition is indicated after searching.

To find a particular code review name or code review names that contain certain characters, enter it in the "Code Review Name" field. Further, to find reviews after a certain date, enter the date in the second set of input boxes. Likewise, to constrain the returned results of the search to a certain status, change the "Status" field. Finally, to find all code reviews matching a certain author, fill in the final field. After specifying the constraints for the search, click on the "Search" button.

As seen above, the results include information on the review ID, author name, current status, and review name. For each review listed, links are given to jump directly to the view code review page for the review. Alternately, the constraints can be increased or relaxed by choosing different values for the fields and searching again.

Perform Code Review

This page allows commentary on code reviews, and viewing others' comments. All users can see the comments posted on this page.

The page shows the file that is being commented on, as well as the revision number and starting and ending lines. Initially, the page shows the section under review with three lines of code on either side of the section. To expand the number of buffer lines shown at the beginning and end of a section, edit:

Note that the number of lines will never exceed the beginning or end of the document. In the case that an invalid number of lines is provided, the page will restore the last valid value entered.

Comments can be associated with each line in the range of lines the author wants others to examine. If a line already has some comments, it is denoted with a 'pin' symbol. The line numbers are links that will open a comment editor:

After the comments box is opened, the "Refresh" link updates the comments box to the most updated comments, which is automatically done whenever the box is initially loaded. Also, users can reply directly to previous posts by clicking the "Reply" link under the existing comment.

Alternately, users can add a new comment by clicking the "Add New Comment" button:

This will open up a new comment box, as shown above, where comments can be inserted into the text box. A code sample can be attached as well by clicking on the "Attach Code Sample" box. Then, type the pathname to the desired file on your computer or click the "Browse..." button to pull up a file selection dialog which will populate the filename. Finally, click on the "Add Comment" button to add the comment so others can view it, or click on the 'X' in the upper-right hand corner to cancel at any time.

View Code Review

This page allows users to view the details of a given code review.

At the top of the page the name, status, author, and creation date of the code review are listed:

Next, the files that this code review encompasses are listed:

The file ID, filename, revision number, and line numbers of the files are prominently displayed in this table. Upon clicking on any of the links given in the table, the user will be taken to the perform code review page for the file in question. Note that this file is unique for each code review, regardless of duplication across separate code reviews. Also, the filename can be identical, with different line numbers. This is so that two different portions of the file that the author modified can be shown without all of the lines in between. In the example above, about 140 lines would have been needed to be shown if only one file was used.

Next, the users assigned to this code review and their vote status are visible.

Managers can see whether each user has voted, and what their vote was if they have voted:

The author of the review can only see whether each user has voted or not:

Users that are neither author nor manager of the review can only see the users that are assigned to the code review:

Next, the author's notes and instructions are visible, which are general comments that the author added about the code review:

Below this is the vote summary, which summarizes the votes for insertion into the main trunk of the project, votes against insertion into the main trunk of the project, and votes that have not been cast:

This summary is only visible if the user is a manager or author or is a reviewer that has voted.

Voting members for the code review can vote if they have not voted, or view their votes and modify them if they have voted:

The author of the review can submit the code review for inclusion, resubmit for review, or close the code review:

Submitting the review for inclusion can only be performed when the percentage of the reviewers that have voted 'Yes' exceeds the threshold set by the manager.

There is no way to change a code review once it has been created. However, resubmitting for review closes a code review and creates a new code review with the same parameters. A developer might want to resubmit for review if changes were proposed and the changes have been implemented. In this case, there is a different set of code for reviewers to look at, so a new review should be created to reflect this. However, creating another review with the same parameters would normally be painstaking.

Authors can also close their code reviews, removing it from reviewers' main pages. It can still be viewed when searching.

Finally, if you are the manager of a code review, you can change the status or resubmit for review at any time:

Manager Options

This page allows a manager to set manager options.

The voting threshold is currently the only manager option that can be set. The voting threshold can be changed by altering the value in the field to an integer value between 0 and 100 and clicking on the "Set Threshold" button.

Glossary

Some terms that are exclusive to the peer review plugin (as opposed to being in Trac or Subversion) are defined here.

Author

The user that created a code review. Has special privileges, which are overridden if the user is also a manager.

Code review

A collection of reviewers and a collection of files to be reviewed.

Developer

A normal user that reads and writes source code. Can create code reviews and be assigned as a reviewer to them. Can search for code reviews, see reviews they created, and see reviews they are assigned to.

Manager

User with highest privileges. Can do everything that a developer can do. Can also change the status of a review at any time. When reviews have "Ready for inclusion" status, these appear on the manager's main page in a table. Can also view the Manager Options page.

Resubmit

Resubmitting for review takes an existing code review and creates a new code review based on it. The fields of the new review are filled in as they were when the original code review was created, and then the author can edit the review details. After creating this new review, the old one is closed.

Reviewer

A developer or manager that is assigned to review a code review, make comments, and vote on the review.

Status

The current state of the code review. The status dictates whether or not certain actions are possible. A manager can change the state at any time, and the author can close the review when it is open or reviewed. States include:

• Open for review - Initial state of the review, where reviewers can vote on the review and add comments.
• Reviewed - Enough people have voted yes to push the review over the threshold limit. At this point the review can be submitted for inclusion by the author.
• Ready for inclusion - The author feels, and his/her reviewers agree, that the review is ready for inclusion into the main trunk. At this point it is the manager's responsibility to verify the review, merge the changes with the main trunk, and close the review when finished. No voting is possible during this stage.
• Closed - The review can be viewed, but no more voting is possible. The review is not visible from anyone's main page, but can be accessed by directly typing the review id into the URL of the PeerReviewView page or successfully searching for the file.

Threshold

This setting defines the number of positive votes versus total votes necessary for an author to submit a code review for a manager's approval.

User

A generic term for someone using this plugin, and not necessarily someone as an end-user of the product. A user can be either a developer, reviewer, manager or author.

Vote

"Approve" or "Not Approve", which signal that the reviewer either accepts the changes made by the review author, or feels that they should not be inserted into the main trunk unless some important changes are made.

License/Disclaimer

This plugin is licensed under the BSD license.