Computerized Adaptive Testing Simulation – `catsim.simulation`

Module containing functions relevant to the process of simulating the application of adaptive tests. Most of this module is based on the work of [Bar10].

class catsim.simulation.Estimator(verbose: bool = False)[source]

Base class for ability estimators

property avg_evaluations

Average number of function evaluations for all tests the estimator has been used

Returns:: average number of function evaluations

property calls

How many times the estimator has been called to maximize/minimize the log-likelihood function

Returns:: number of times the estimator has been called to maximize/minimize the log-likelihood function

abstract estimate(index: int) → float[source]

Returns the theta value that minimizes the negative log-likelihood function, given the current state of the: test for the given examinee.

Parameters:: index – index of the current examinee in the simulator
Returns:: the current \(\hat\theta\)

property evaluations

Total number of times the estimator has evaluated the log-likelihood function during its existence

Returns:: number of function evaluations

class catsim.simulation.FiniteSelector(test_size)[source]: Base class representing a CAT item selector.

class catsim.simulation.Initializer[source]

Base class for CAT initializers

abstract initialize(index: int) → float[source]

Selects an examinee’s initial \(\theta\) value

Parameters:: index – the index of the current examinee
Returns:: examinee’s initial \(\theta\) value

class catsim.simulation.Selector[source]

Base class representing a CAT item selector.

abstract select(index: int | None = None) → int | None[source]

Returns the index of the next item to be administered.

Parameters:: index – the index of the current examinee in the simulator.
Returns:: index of the next item to be applied or None if there are no more items to be presented.

class catsim.simulation.Simulable[source]

Base class representing one of the Simulator components that will receive a reference back to it.

preprocess()[source]: Override this method to initialize any static values the Simulable might use for the duration of the simulation. preprocess is called after a value is set for the simulator property. If a new value if attributed to simulator, this method is called again, guaranteeing that internal properties of the Simulable are re-initialized as necessary.

Class representing the simulator. It gathers several objects that describe the full simulation process and simulates one or more computerized adaptive tests

Parameters:

items – a matrix containing item parameters
examinees – an integer with the number of examinees, whose real \(\theta\) values will be sampled from a normal distribution; or a :py:type:list containing said \(\theta_0\) values

property administered_items: list: List of lists containing the indexes of items administered to each examinee during the simulation.

property bias: float: Bias between the estimated and true abilities. This property is only available after simulate() has been successfully called. For more information on estimation bias, see catsim.cat.bias()

property duration: float: Duration of the simulation, in seconds.

property estimations: list: List of lists containing all estimated \(\hat\theta\) values for all examinees during each step of the test.

property examinees: ndarray: :py:type:numpy.ndarray containing examinees true ability values (\(\theta\)).

property items: ndarray: Item matrix used by the simulator. If the simulation already occurred, a column containing item exposure rates will be added to the matrix.

property latest_estimations: list: Final estimated \(\hat\theta\) values for all examinees.

property mse: float: Mean-squared error between the estimated and true abilities. This property is only available after simulate() has been successfully called. For more information on the mean-squared error of estimation, see catsim.cat.mse()

property overlap_rate: float: Overlap rate of the test, if it is of finite length.

property response_vectors: list: List of boolean lists containing the examinees answers to all items.

property rmse: float: Root mean-squared error between the estimated and true abilities. This property is only available after simulate() has been successfully called. For more information on the root mean-squared error of estimation, see catsim.cat.rmse()

simulate(initializer: Initializer | None = None, selector: Selector | None = None, estimator: Estimator | None = None, stopper: Stopper | None = None, verbose: bool = False)[source]

Simulates a computerized adaptive testing application to one or more examinees

Parameters:

initializer – an initializer that selects examinees \(\theta_0\)
selector – a selector that selects new items to be presented to examinees
estimator – an estimator that reestimates examinees abilities after each item is applied
stopper – an object with a stopping criteria for the test
verbose – whether to periodically print a message regarding the progress of the simulation. Good for longer simulations.

>>> from catsim.initialization import RandomInitializer
>>> from catsim.selection import MaxInfoSelector
>>> from catsim.estimation import NumericalSearchEstimator
>>> from catsim.stopping import MaxItemStopper
>>> from catsim.simulation import Simulator
>>> from catsim.cat import generate_item_bank
>>> initializer = RandomInitializer()
>>> selector = MaxInfoSelector()
>>> estimator = NumericalSearchEstimator()
>>> stopper = MaxItemStopper(20)
>>> Simulator(generate_item_bank(100), 10).simulate(initializer, selector, estimator, stopper)

class catsim.simulation.Stopper[source]

Base class for CAT stop criterion

abstract stop(index: int) → bool[source]

Checks whether the test reached its stopping criterion for the given user

Parameters:: index – the index of the current examinee
Returns:: True if the test met its stopping criterion, else False

Computerized Adaptive Testing Simulation – catsim.simulation

Computerized Adaptive Testing Simulation – `catsim.simulation`