Adaptive and Dynamic Assessments

Overview on how adaptive assessments can be set up in the Items API, so the items presented to the students are chosen dynamically based on their performance in the test.

This article outlines the support for delivering dynamic assessments through the Items API.

Three types of dynamic activities are available.

  • Item Branching — items are dynamically selected based on pre-configured paths and branches depending on the student's score.
  • Item Adaptive — items are dynamically selected, one by one, based on psychometrically calibrated difficulty levels.
  • Testlet Adaptive — similar to Item Adaptive, but instead of selecting each individual item in sequence, small groups of items ('testlets') are selected at a time and fetched together as a batch. Each testlet may be a simple activity, with a static set of items and a nominated difficulty rating, or a dynamic Item Adaptive activity.

Each type of dynamic assessment is introduced below. For demos and the full technical details on configuring and initializing dynamic assessments, see the demos and technical reference section.

Note The Item Branching/Adaptive modes are not compatible with Dynamic Content.

In branching activities, items are presented to the sudent according to pre-configured paths. The decision of which item (or branches containing multiple items) can be based on the student's performance, such as correctness of the last answer, or overall score so far. Each branch can rejoin the main path, or not, depending on the assessment requirements.

You can create the path configuration as part of an Items API activity definition. Any item from your Learnosity item bank can be used in an item branching activity.

The configuration is simply a number of steps. Each step either provides the reference of an item to be presented, or a decision to be taken based on the student's performance.

Item steps provide the reference of an item to present to the student. They also indicate which step to visit next, once the student has provided an answer. It is possible to express simple branches by providing two next steps, depending on whether the answer was correct or not.

Global score decision steps can select between two steps to visit next depending on whether the student's score in the assessment so far is greater-or-equal, or lesser, than a percentage of the currently achievable maximum score.

In adaptive activities, items (or groups of items, known as testlets) are selected dynamically such that the difficulty of each item is adapted to the estimated ability of the user. The first requirement for an item level adaptive activity is a pool of items in your Learnosity item bank. The items must be calibrated with a difficulty measure according to the Rasch model.

You can then create an Items API activity definition for an item or testlet adaptive activity. The activity definition allows you to tweak how the item selection algorithm behaves. It also allows you to use tags to restrict the activity to use specific sets of items from the item bank.

The key to selecting items during an item adaptive session is the user's ability estimate. It is calculated after every response and is used to create a target difficulty for the selection of the next item. You may specify an initial ability estimate to be used for the selection of the first item in the session.

The primary method used for generating ability estimates is the Maximum Likelihood Estimation method. A limitation of this method is that it can only provide a finite estimate if the user has given a mixed response pattern i.e. they have given at least one incorrect response, and at least one correct response.

You can configure the Items API to use the Expected A Posteriori method to estimate abilities when a mixed response pattern is not available at the beginning of a user's session. If this information is not provided, the Items API will, in lieu of a finite ability estimate, increase or decrease the target difficulty for each item in fixed increments until a mixed response pattern is available.

  1. Search for all available items within the required tags.
  2. Remove excluded items, and items from excluded tags.
  3. Remove items the user has already seen in the current session, as well as any Related items defined in the item bank.
  4. Calculate an ability estimate for the user, as above, to use as a target difficulty for the next item.
  5. If an offset is defined, apply it to the target difficulty.
  6. From the target difficulty, use the specified difficulty tolerance to determine a range of difficulties from which to select an item.
  7. Select an item at random from within the desired range of difficulties.

The session will end when one of the specified termination criteria is met.

The Items API also allows you to embed uncalibrated seed items during the session. These items do not need to have difficulty measures, and the responses to these items will have no effect on the user's ability estimate in the session.

This feature can be used to gather data on new items so that they can be calibrated and used in future assessments.

Learnosity keeps track of how often items are used in an adaptive session, either as a live item or a seed item.

In a testlet adaptive activity, items are selected in groups, called testlets. This makes it easy to combine multiple, stand-alone activities into a single, seamless session for the user.

The testlet activities must first be created in your Learnosity item bank via either Author Site or Data API. The activities must also be calibrated with a difficulty measure, for example the average difficulty of the items within the testlet.

The configuration and selection algorithm is exactly the same as for an Item Adaptive activity above, except instead of selecting items, testlets are selected based on the respective difficulties and tags of the testlet activities.

Item adaptive activities can also be used as testlets inside a testlet adaptive activity. In this scenario, the ability estimates used for selecting items during the item adaptive testlet will include responses to all items in the branching session, including previous testlets.

The demos take you through the available dynamic and adaptive activities, and allows you to configure some of the parameters discussed above.

For details on configuration options, see the technical reference documentation: