Overview of Steps in an SSAM Standard Setting Session

The workflow of a standard setting session involves multiple steps. The table below provides information about the steps that may occur in a session, including which roles and tabs of the Review Panel are involved in each one.

Note: The actual steps that occur in a standard setting session can vary by session. Your session may not include some of the steps in this table. Some of the steps may be repeated, as each booklet typically undergoes multiple rounds of review.

Table 8: Steps Involved in a Standard Setting Session
Step	Description	Roles Involved	Review Panel Tabs
Pause	A pause typically occurs between each step of the workflow. While a pause is in effect, Panelists cannot perform any actions in the Standard Setting Tool. When a pause occurs after an official round, this time is used by Psychometricians to review the Panelists' work.	N/A	N/A
Operational Test	During this step, Panelists review the assertions from the ordered scoring assertion booklet. Panelists can view the assertions by clicking Item Score to score each item. Unlike the OSAB, the operational test does not list assertions in order from easiest to most difficult.	Panelists	N/A
Review of Ordered Scoring Assertion Booklet	During this step, panelists review the OSAB, with assertions listed from easiest to most difficult within each item. The answer key is available to Panelists during this step. The contextual data for each assertion may also be available. Panelists can use the Notes tab of the Review Panel to answer questions about the difficulty of each assertion in the booklet.	Panelists	Additional Info, Notes, and Context tabs
Practice Assertion Mapping	During this step, Panelists map assertions to performance levels in a practice booklet in order to familiarize themselves with the standard setting process. None of the Panelists’ work from this step is used.	Panelists	Set Levels tab
Assertion Mapping Round X	During this step, each Panelist maps the item assertions to performance levels, resulting in cut values. Multiple Assertion Mapping Rounds may occur during a standard setting session. A Psychometrician Review step and a Results step follow each round. During subsequent Assertion Mapping Rounds, Panelists should take the results and feedback from the previous round into consideration as they work.	Panelists	Set Levels tab
Results of Round X	During this step, the Room Lead and Panelists receive feedback from the Psychometricians based on the results of the previous Assertion Mapping Round. Any discrepancies or outliers in the data may be discussed in the room before moving to the next step. The standard setting session includes a Results step for each Assertion Mapping Round that occurs.	Room Lead Panelists	Feedback/Prior Feedback tab
Final Moderation	Once the room has reached a consensus regarding the appropriate assertion mappings, the official proficiency standard cut values are set. A Panelist who the Room Lead assigns to act as the Room Scribe may adjust these values, if necessary.	Room Lead Panelist—Room Scribe	Moderation Chart tab