Assigning scorers to scoring projects

Once a scoring project has been created and one or more deliveries have been linked to it, scorers need to be assigned to it before manual scoring can begin.

Assigning scorers (users) to a particular scoring project is called ENROLLMENT in TAO Grader. To enrol users on a project, follow the steps below.

1. Click on the ENROLLMENT button inside your scoring project.

This will bring up the list of scorers and reviewers who are enrolled to work on a given scoring project, and shows the type of tests they can mark (scoring criteria), along with their workload (see the image below).

 

Note: In the example, two scorers and two reviewers have been enrolled, but in a new project, the list will be empty, so the scorers (and reviewers) who you, as Scoring Project Manager, wish to undertake the scoring on this project need to be enrolled.

2. Click on UPLOAD ENROLLMENTS on the right.

You will need to upload a CSV file containing the username (the same as in the user’s CSV file), the role and the assignment_criterion, as shown below. The following is the corresponding entry in the assignment file for the jdoe user you created earlier, which assigns them to the x-tao-scoring-english category:

username,role,assignment_criterion

jdoe,scorer,x-tao-scoring-english

Note: The assignment_criterion must follow the x-tao-scoring-<category> format.

The two possible options for role are scorer and reviewer. A user can be enrolled as both a scorer and a reviewer for the same scoring project, though they cannot score the same responses twice, of course.

If the Assignment strategy has been configured as Item group, and no scorer or reviewer has been enrolled for a given subject, an ‘!’ appears next to the ENROLLMENTS tab as a warning. If the Assignment strategy for scoring has been configured as Test-taker group, scorers and reviewers need to be assigned to particular classrooms (for example ‘Classroom A’ or ‘Classroom B’, as in the image below), rather than subject categories.

 

When a Scoring Project has been set up with the necessary delivery information and has scorers assigned to it, the manual scoring environment is ready, and scoring can begin as soon as the responses start to arrive when the test-takers complete the test.