Site menu:

cNORM - Generating Continuous Test Norms

Alexandra Lenhard, Wolfgang Lenhard & Sebastian Gary

cNORM (A. Lenhard, Lenhard & Gary, 2018) is an R package that generates continuous test norms for psychometric and biometric data and analyzes the associated model fit. Originally, cNorm exclusively used an approach that makes no assumptions about the specific distribution of the raw data (A. Lenhard, Lenhard, Suggate & Segerer, 2016). Since version 3.2 (2024), however, the package also offers the option of parametric modeling using the beta-binomial distribution.

cNORM was developed specifically for achievement tests (e.g. vocabulary development: A. Lenhard, Lenhard, Segerer & Suggate, 2015; written language acquisition: W. Lenhard, Lenhard & Schneider, 2017). However, the package can be used wherever mental (e.g. reaction time), physical (e.g. body weight) or other test scores depend on continuous (e.g. age, duration of schooling) or discrete explanatory variables (e.g. sex, test form). In addition, the package can also be used for "conventional" norming based on individual groups, i.e. without including explanatory variables.

The package estimates percentiles as a function of the explanatory variable. This is done either parametrically on the basis of the beta-binomial distribution or distribution-free using Taylor polynomials. Mathematical modeling of the data using continuous variables such as age has the following advantages:

• By optimizing the model based on the overall sample, small deviations from representativeness in individual subsamples caused by random sampling are minimized (A. Lenhard, Lenhard & Gary, 2019).
• Gaps between different discrete levels of the explanatory variable are closed. In intelligence tests, for example, norms can be created not only for relatively wide age intervals (e.g., 6 months), but for any exact age. In achievement tests, the norms are no longer limited to the specific time of the data collection (e.g., middle or end of the school year), but can be extended to any point in time within a school year under certain conditions.
• Norm tables are always determined on the basis of the entire sample instead of a single cohort or grade level. For this reason, only around a quarter of the sample size is required to achieve the same norm score precision with continuous norming compared to conventional norming per subgroub. In general, continuous norming only requires a sample size of around 100 per cohort to achieve sufficient norming quality (W. Lenhard & Lenhard, 2020). Especially in those performance areas that deviate relatively strongly from the population mean, continuous norming usually leads to more precise results than conventional norming. This is specifically important, because it is precisely these areas that are most relevant in diagnostic practice.
• The limits of the model fit can be evaluated graphically and analytically. For example, it is possible to determine at which points the model deviates strongly from the manifest data or where strong floor or ceiling effects occur. This feature allows to specify in which age and performance ranges the test no longer differentiates sufficiently.
• The beta-binomial distribution models the raw data very precisely in many application scenarios, especially for tests that are based on 1-PL IRT models. Fitting with Taylor polynomials, on the other hand, requires very few assumptions about the raw score distributions. This means that a wider range of scales can be standardized, for example, continuous measures such as reaction times, scales that contain negative values or speeded test scales.
• In case of violations of represantativeness, Iterative Proportional Fitting (Raking) can be used to calculate weights and apply these to mitigate biases.

The mathematical principles of fitting with Taylor polynomials are briefly described in the corresponding mathematical derivation of the method. There is also a detailed description of norming based on the beta-binomial distribution, including the mathematical background.

On the following pages, we will demonstrate the necessary steps for the application of the R package with real human performance data, namely, with the normative sample of the sentence comprehension subtest of ELFE 1-6, a reading comprehension test in German language (W. Lenhard & Schneider, 2006) and with the German adaptation of the Peabody Picture Vocabulary Test 4 (A. Lenhard et al., 2015):

1. Installation
2. Data Preparation
3. Weighting
4. Distribution-free Modeling
5. Parametric Modeling
6. Examples
7. Graphical User Interface Web-based or as a Jamovi Module

In A Nutshell

Below you will find a quick guide to distribution-free modeling with the essential cNORM functions and - further below - a quick guide to parametric modeling using the beta-binomial distribution:

Distribution-Free Modeling

## Basic example code for modeling the sample dataset

library(cNORM)

# Start the graphical user interface (needs shiny installed)
# The GUI includes the most important functions. For specific cases,
# please use cNORM on the console.

cNORM.GUI()

# Using the syntax on the console: The function 'cnorm' performs
# all steps automatically. Please specify the raw score and the
# grouping variable. The resulting object contains the ranked data
# via object$data and the model via object$model.

cnorm.elfe <- cnorm(raw = elfe$raw, group = elfe$group)

# Plot different indicators of model fit depending on the number of
# predictors

plot(cnorm.elfe, "subset", type=0) # plot R2
plot(cnorm.elfe, "subset", type=3) # plot MSE

# NOTE! At this point, you usually select a good fitting model and rerun
# the process with a fixed number of terms, e. g. 4. Avoid models
# with a high number of terms:

cnorm.elfe <- cnorm(raw = elfe$raw, group = elfe$group, terms = 4)

# Powers of age can be specified via the parameter 't'.
# Cubic modeling is usually sufficient, i.e., t = 3.
# In contrast, 'k' specifies the power of the person location.
# This parameter should be somewhat higher, e.g., k = 5.

cnorm.elfe <- cnorm(raw = elfe$raw, group = elfe$group, k = 5, t = 3)

# Visual inspection of the percentile curves of the fitted model

plot(cnorm.elfe, "percentiles")

# Visual inspection of the observed and fitted raw and norm scores

plot(cnorm.elfe, "norm")
plot(cnorm.elfe, "raw")

# In order to compare different models, generate a series of percentile
# plots with an ascending number of predictors, in this example between
# 5 and 14 predictors.

plot(cnorm.elfe, "series", start=5, end=14)

# Cross validation in order to choose appropriate number of terms
# with 80% of the data for training and 20% for validation. Due to
# the time consumption, the maximum number of terms is limited to 10
# in this example with 3 repetitions of the cross validation.

cnorm.cv(cnorm.elfe$data, max=10, repetitions=3)

# Cross validation with prespecified terms of an already
# existing model

cnorm.cv(cnorm.elfe, repetitions=3)

# Print norm table (in this case: 0, 3 or 6 months at grade level 3)
# (Note: The data is coded such that 3.0 represents the beginning and
# 3.5 the middle of the third school year)

normTable(c(3, 3.25, 3.5), cnorm.elfe)

Parametric modeling with beta-binomial distribution:

library(cNORM)

# cNORM can also model norm data using the beta-binomial distribution.
# This approach is particularly effective for tests with a fixed
# number of dichotomous items and a continuous distribution of
# item difficulties.

model.betabinomial <- cnorm.betabinomial(ppvt$age, ppvt$raw)

# Again, model fit can be influenced by changing the powers for the
# curve fits of α and β:

model.betabinomial <- cnorm.betabinomial(ppvt$age, ppvt$raw, alpha = 4)

# Plot percentile curves and fit of the manifest and modeled norm scores:

plot(model.betabinomial, ppvt$age, ppvt$raw)
plotNorm(model.betabinomial, ppvt$age, ppvt$raw, width = 1)

# Display fit statistics:

summary(model.betabinomial)

# Prediction of norm scores for new raw scores and preparation of
# norm tables:

predict(model.betabinomial, c(8.9, 10.1), c(123, 98))

tables <- normTable.betabinomial(model.betabinomial, c(2, 3, 4),
          reliability=0.9)

Conventional norming per age group

library(cNORM)

# Application of cNORM for the generation of conventional norms
# for a specific age group (in this case age group 3):

data <- elfe[elfe$group == 3,]
cnorm(raw=data$raw)

Display instructions in cNORM:

library(cNORM)
# Display vignette

vignette("cNORM-Demo", package = "cNORM")
vignette("WeightedRegression", package = "cNORM")
vignette("BetaBinomial", package = "cNORM")

Installation

Terms of Use/Licensing

cNORM is licensed under GNU Affero General Public License v3 (AGPL-3.0). This means that copyrighted parts of cNORM may only be used free of charge in commercial and non-commercial projects that run under this same license, retain the copyright notice, provide their source code and correctly cite cNORM. Copyright protection includes, for example, the reproduction and distribution of source code or parts of the source code of cNORM or of graphics created with cNORM. The integration of the package into a server environment in order to access the functionality of the software (e.g. for online delivery of norm scores) is also subject to this license. However, a regression function determined with cNORM is not subject to copyright protection and may be used freely for commercial or non-commercial projects. If you want to apply cNORM in a way that is not compatible with the terms of the AGPL 3.0 license, please do not hesitate to contact us to negotiate individual conditions.

If you want to use cNORM for scientific publications, we would also ask you to quote the source.

References

In case you need a reference to this page in a scientific paper, please use the following citation:

Lenhard, A., Lenhard, W. & Gary, S. (2024). cNORM - Generating Continuous Test Norms (Revision). Retrieved from: https://www.psychometrica.de/cNorm_en.html. Dettelbach (Germany): Psychometrica. https://doi.org/10.13140/RG.2.2.25821.26082

Copyright © 2024 Drs. Alexandra & Wolfgang Lenhard