Overview of Steps in an SSBM Standard Setting Session

The workflow of a standard setting session involves multiple steps. The table below provides information about the steps that may occur in a session, including which roles and tabs of the Review Panel are involved in each one.

Note: The actual steps that occur in a standard setting session can vary by session. Your session may not include some of the steps in this table. Some of the steps may be repeated, as each booklet typically undergoes multiple rounds of review.

Table 1: Steps Involved in a Standard Setting Session
Step	Description	Roles Involved	Review Panel Tabs
Pause	A pause typically occurs between each step of the workflow. While a pause is in effect, Panelists cannot perform any actions in the Standard Setting Tool. When a pause occurs after an official round, this time is used by Psychometricians to review the Panelists' work.	N/A	N/A
Operational Test	During this step, Panelists review the items from the ordered item booklet as they appeared to students in the original fixed form test. Unlike the OIB, the operational test does not list items in order from easiest to most difficult.	Panelists	N/A
Review of Ordered Item Booklet	During this step, panelists review the OIB, with items listed from easiest to most difficult. The answer key for each item is available to Panelists during this step. The impact data for each item may also be available. Panelists can use the Notes tab of the Review Panel to answer questions about the difficulty of each item in the booklet.	Panelists	Additional Info, Notes, and Impact tabs
Practice Bookmark Placement	During this step, Panelists enter bookmarks for items in a practice booklet in order to familiarize themselves with the standard setting process. None of the bookmarks that Panelists place in this step will be recorded.	Panelists	Marks tab
Bookmark Placement Round X	During this step, each Panelist reviews the official item booklet and places bookmarks to indicate its proficiency standard cut values. Multiple Bookmark Placement rounds may occur during a standard setting session. A Psychometrician Review step and a Results step follow each round. During subsequent Bookmark Placement rounds, Panelists should take the results and feedback from the previous round into consideration as they place their marks.	Panelists	Marks tab
Results of Round X	During this step, the Room Lead and Panelists receive feedback from the Psychometricians based on the results of the previous Bookmark Placement round. Any discrepancies or outliers in the data may be discussed in the room before moving to the next step. The standard setting session includes a Results step for each Bookmark Placement round that occurs.	Room Lead Panelists	Feedback/Prior Feedback tab
Final Moderation	Once the room has reached a consensus regarding the appropriate bookmark placements, the official proficiency standard cut scores are set. A Panelist who the Room Lead assigns to act as the Room Scribe may adjust these values, if necessary.	Room Lead Panelist—Room Scribe	Moderation Chart tab