Setting Technical Constraints

You can set three types of technical constraints: theta, single value, and proficiency level.

Theta Constraints

  1. Click Add Theta Constraint. The Add\Edit Theta Constraint window appears.

  2. Select either TCC Simple, TCC Split Range, or Each Theta.

    • If you selected TCC Simple, enter the maximum TCC difference that will be tolerated for any theta.

      This is a constraint on the allowed absolute difference in Test Characteristic Curve (TCC) of the reference form and current form for a range of theta values. The same tolerance is applied throughout the entire range of data.

    • If you selected TCC Split Range, drag one of the handles on the Split at Theta range tool. The other handle automatically moves to the position with the same absolute value and the range is split into three sections. Then enter the maximum TCC difference for each section in the fields. The upper and lower sections always have the same value.

      As with the TCC Simple option, this is a constraint on the allowed absolute difference in Test Characteristic Curve (TCC) of the reference form and current form for a range of theta values. Unlike with TCC Simple, different tolerances are applied for different ranges of data.

    • If you selected Each Theta, follow the steps below.

      1. From the Constraint on dropdown list, select what the constraint is intended to constrain.

      2. Enter values in either or both of the two tables.

        • In the first table, you can enter values for the minimum and maximum theta cutoffs.

        • In the second table, you can enter custom thetas and their minimum and maximum cutoff values; if the custom table contains the same thetas as the first table, the custom values for those thetas override the ones in the first table. Use the Add Row button to add a new row at the bottom. Use Remove Row to remove the last row in the table.

  1. Optional: Edit the name of the constraint as desired.

  2. Click Save. The Add\Edit Theta Constraint window closes. The theta constraint appears in the table, which indicates whether or not the constraint is met with a green check mark or red X.

To edit or delete the theta constraint, select the row and click Edit or Delete.

You can also use the Add / Edit button below each graph (described in Referring to Technical Constraints) to add or edit whichever constraint corresponds to that graph.

To disable the constraint, clear the checkbox in the Enabled column.

Single-Value Constraints

  1. Click Add. The Add\Edit Technical Constraint window appears.

  2. Enter a meaningful name for the constraint. For example, Average Difficulty, Average P-Value, or Correlation Minimum .23.

  3. From the Attribute dropdown list, select the item statistic for the constraint, such as Difficulty, PValue, or Correlation.

    • Note that difficulty for polytomous items is computed as the average of the IRT difficulty parameters for the item.

  4. From the Base dropdown list, select either Item Based or Point Based. Note that the base option is relevant only for averages, not for sums, minimums, or maximums, which work the same way regardless of base. The base determines whether an average statistic will be multiplied (weighted) by the items’ points. The points are taken from the item parameters or, if statistics are not available, from the IAT Max Score in the rubric.

  5. From the Grouping Operator dropdown list, choose the function that is applied to the item statistics for the set of items selected on the form.

    • Avg: Takes the (weighted) average of the item statistics. For example, you could constrain the average difficulty of items on the form. If the constraint is item-based, the denominator is the number of items. If the constraint is point-based, the denominator is the sum of the score points over items.

    • Sum: Takes the sum of the item statistics. For example, you could constrain the total number of points attainable on the form by selecting the item statistic Points and the function Sum.

    • Min: Takes the minimum of the item statistics, for example, the item with the lowest difficulty.

    • Max: Takes the maximum of the item statistics, for example, the item with the highest difficulty.

  6. Optional: Clear the Is Strict Constraint checkbox to indicate that the constraint need not necessarily be met.

  7. Enter the target value and tolerance. Technical constraints are specified with a target value and a tolerance that specifies the allowed deviation from the target. A tolerance of 0 means the constraint has to be met exactly, which is not realistic for real-valued constraints but may be for integer-valued ones. The target and tolerance values can be translated into a minimum and maximum acceptable value as follows:

    min=target−tolerance

    max=target+tolerance

    Sometimes the user may want to specify a technical constraint directly as a minimum and/or maximum value. Right now, this is not supported, but minimum and maximum values can be translated into target and tolerance values as follows:

    target=(min+max)∕2

    tolerance=max−target

    Or, equivalently:

    tolerance=(max−min)∕2

    target=min+tolerance

    For example, the constraint that the minimum p-value is .1 can be specified as follows:

    target=(.1+1)∕2=.55

    tolerance=1−.55=.45

    Or, equivalently:

    tolerance=(1−.1)∕2=.45

    target=.1+.45=.55

    Note that, because there is no real maximum constraint on the p-value in this case, the highest possible value is used for the maximum (p=1).

    As another example, an average difficulty constraint could have a target of 0 and a tolerance of .1.

  8. Click Save. The Add\Edit TCC Constraint window closes. The new constraint appears in the table, with the actual form value listed in the Current column beside the target and tolerance values.

To edit or delete a constraint, select the radio button in the rightmost column of the table and click Edit or Delete. To disable the constraint, clear the checkbox in the Enabled column.

Proficiency Level Constraints

A proficiency constraint sets the maximum number of points by which proficiency level raw cut scores can differ between the current form and the reference form.

For example, suppose the raw cut score for level 2 (separating it from level 1) is x on the reference form and x+9 on the current form. If a proficiency constraint specifies a maximum difference of 8 points for level 2, the form will not meet the constraint. Likewise, if the current form’s level 2 raw cut score is x−9, it will not meet the constraint.

  1. Click Add Proficiency Constraint. The Add Proficiency Level Constraint window appears.

  2. Enter a meaningful name for the constraint.

  3. In the blank table row provided, select a proficiency level from the dropdown list and enter the maximum difference between the current and reference form raw cut scores for that level.

  4. Optional: To add more levels below, select Add Row and repeat step 3.

  5. Optional: To delete the last row in the table, select Remove Row.

  6. Select Save. The Add Proficiency Level Constraint window closes. The new constraint appears in the table, which indicates whether or not the constraint is met with a green check mark or red X.

To edit or delete a constraint, select the radio button in the rightmost column of the table and click Edit or Delete. To disable the constraint, clear the checkbox in the Enabled column.