Set the Steps for a Standard Setting Session

A standard setting session consists of multiple steps, in which Panelists place proficiency standard bookmarks in the ordered item booklet An ordered item booklet (OIB) is a set of test items arranged from easiest to most difficult, based on actual student performance data. and review feedback in order to arrive at a consensus for the official cut values Cut values are the thresholds between two proficiency standards in the ordered item booklet (OIB). For example, if items 1-10 in a booklet require a different proficiency level than items 11-20, then the cut value is placed at item 11.. There are typically multiple rounds of review for each item booklet. There may also be steps in which Room Leads provide an overview of the booklet being reviewed and Panelists practice placing bookmarks. There are also pauses between each step of the workflow.

As a Room Lead, you are responsible for monitoring the progress of the standard setting session. The Standard Setting Tool allows you to control when your room moves to each step of the process. Upon moving to the next step, you can also choose to exclude any Panelists from the data calculations if they were unable to complete the previous step.

For an overview of the tasks involved in each step, refer to Overview of Steps in an SSBM Standard Setting Session.

Set the Steps for a Room

In order for Panelists to complete the standard setting process, you must manually progress the room through each step of the workflow.

Figure 3: Set Step Window

1. To move the room to the next step, click in the top-right corner.
2. In the Set Step window, select the appropriate step from the list and click Validate & Go To Next Step.
   • You should move through steps in the order they appear, without skipping any. However, if you need to return to a previous step or jump ahead to a future step for any reason, you can do the following:
     1. Mark the Enable more steps checkbox.
     2. Select the appropriate step from the list and click Validate & Go To Selected Step.
3. A summary appears on the right, indicating whether the Panelists in the room are ready to move to the next step. 
   • If any Panelists in the room were unable to complete the previous step, their names appear in this window. If you wish to exclude a Panelist's incomplete data from the room's calculations, mark the checkbox for that Panelist and click Submit.
     • You can re-include a Panelist's data in the calculations later on from the Room Progress tab.