Site menu:

cNORM - Generating Continuous Test Norms

Alexandra Lenhard, Wolfgang Lenhard & Sebastian Gary

cNORM (A. Lenhard, Lenhard & Gary, 2018) is an R package that generates continuous test norms for psychometric and biometric data and analyzes the associated model fit. Originally, cNORM exclusively used an approach that makes no assumptions about the specific distribution of the raw data (A. Lenhard, Lenhard, Suggate & Segerer, 2016). However, the package now also offers the option of parametric modeling using the beta-binomial distribution or the sinh-arcsinh distribution.

cNORM was developed specifically for achievement tests (e.g. vocabulary development: A. Lenhard, Lenhard, Segerer & Suggate, 2015; written language acquisition: W. Lenhard, Lenhard & Schneider, 2017). However, the package can be used wherever mental (e.g. reaction time), physical (e.g. body weight) or other test scores depend on continuous (e.g. age, duration of schooling) or discrete explanatory variables (e.g. sex, test form). In addition, the package can also be used for "conventional" norming based on individual groups, i.e. without including explanatory variables.

The package estimates percentiles as a function of the explanatory variable. Mathematical modeling of the data using continuous variables such as age has the following advantages:

• By optimizing the model based on the overall sample, small deviations from representativeness in individual subsamples caused by random sampling are minimized (A. Lenhard, Lenhard & Gary, 2019).
• Gaps between different discrete levels of the explanatory variable are closed. In intelligence tests, for example, norms can be created not only for relatively wide age intervals (e.g., 6 months), but for any exact age. In achievement tests, the norms are no longer limited to the specific time of the data collection (e.g., middle or end of the school year), but can be extended to any point in time within a school year under certain conditions.
• Norm tables are always determined on the basis of the entire sample instead of a single cohort or grade level. For this reason, only around a quarter of the sample size is required to achieve the same norm score precision with continuous norming compared to conventional norming per subgroub. In general, continuous norming only requires a sample size of around 100 per cohort to achieve sufficient norming quality (W. Lenhard & Lenhard, 2020). Especially in those performance areas that deviate relatively strongly from the population mean, continuous norming usually leads to more precise results than conventional norming. This is specifically important, because it is precisely these areas that are most relevant in diagnostic practice.
• The limits of the model fit can be evaluated graphically and analytically. For example, it is possible to determine at which points the model deviates strongly from the manifest data or where strong floor or ceiling effects occur. This feature allows to specify in which age and performance ranges the test no longer differentiates sufficiently.
• In case of violations of represantativeness, Iterative Proportional Fitting (Raking) can be used to calculate weights and apply these to mitigate biases.

cNORM can estimate norms with or without prior assumptions about raw score distributions. The original method implemented in cNORM makes no assumptions whatsoever about raw score distributions. This approach models the distributions at each individual level of the explanatory variable (e.g., at each age level) using Taylor polynomials. This is a versatile, all-purpose method that can be applied to a wide range of scales, including continuous measurements such as reaction times, scales containing negative values, speeded tests, or tests with other discontinuation rules. A brief description of the mathematical foundations of Taylor polynomials can be found in the corresponding mathematical derivation.

However, when a psychometric test instrument meets certain requirements, it may be more appropriate to assume specific parametric raw score distributions. For example, the beta-binomial distribution is typically excellent for calculating norms when test items conform to a 1PL IRT model. A brief description including the mathematical background is also available for fitting based on the beta-binomial distribution.

Fitting using the shash distribution essentially takes a middle position between the two previously described methods. The shash distribution represents a transformation of the normal distribution with four parameters. This allows for more flexible fitting to distributions compared to the beta-binomial distribution, which has only three parameters. The shash distribution typically produces good fits even for speeded tests. The fitting is not quite as flexible as that with Taylor polynomials, since the latter - depending on the chosen parameters - is at least initially performed with more degrees of freedom. However, making specific assumptions about the distribution offers other advantages, such as shape stability extending into extreme performance ranges. Generally, the shash distribution yields very similar results to Taylor polynomial fitting. Which of the two methods is best suited for a particular case cannot be reliably predicted and essentially must be determined through experimentation. The mathematical description of the shash distribution can be found here.

On the following pages, we will demonstrate the necessary steps for the application of the R package with real human performance data, namely, with the normative sample of the sentence comprehension subtest of ELFE 1-6, a reading comprehension test in German language (W. Lenhard & Schneider, 2006) and with the German adaptation of the Peabody Picture Vocabulary Test 4 (A. Lenhard et al., 2015):

1. Installation
2. Data Preparation
3. Weighting
4. Distribution-free Modeling
5. Parametric Modeling
6. Examples
7. Graphical User Interface Web-based or as a Jamovi Module

In A Nutshell

Below you will find a quick guide to distribution-free modeling with the essential cNORM functions and - further below - a quick guide to parametric modeling using the beta-binomial distribution:

Distribution-Free Modeling

## Basic example code for modeling the sample dataset

library(cNORM)

# Start the graphical user interface (needs shiny installed)
# The GUI includes the most important functions. For specific cases,
# please use cNORM on the console.

cNORM.GUI()

# Using the syntax on the console: The function 'cnorm' performs
# all steps automatically. Please specify the raw score and the
# grouping variable. The resulting object contains the ranked data
# via object$data and the model via object$model.

cnorm.elfe <- cnorm(raw = elfe$raw, group = elfe$group)

# Plot different indicators of model fit depending on the number of
# predictors

plot(cnorm.elfe, "subset", type=0) # plot R2
plot(cnorm.elfe, "subset", type=3) # plot MSE

# NOTE! At this point, you usually select a good fitting model and rerun
# the process with a fixed number of terms, e. g. 4. Avoid models
# with a high number of terms:

cnorm.elfe <- cnorm(raw = elfe$raw, group = elfe$group, terms = 4)

# Powers of age can be specified via the parameter 't'.
# Cubic modeling is usually sufficient, i.e., t = 3.
# In contrast, 'k' specifies the power of the person location.
# This parameter should be somewhat higher, e.g., k = 5.

cnorm.elfe <- cnorm(raw = elfe$raw, group = elfe$group, k = 5, t = 3)

# Visual inspection of the percentile curves of the fitted model

plot(cnorm.elfe, "percentiles")

# Visual inspection of the observed and fitted raw and norm scores

plot(cnorm.elfe, "norm")
plot(cnorm.elfe, "raw")

# In order to compare different models, generate a series of percentile
# plots with an ascending number of predictors, in this example between
# 5 and 14 predictors.

plot(cnorm.elfe, "series", start=5, end=14)

# Cross validation in order to choose appropriate number of terms
# with 80% of the data for training and 20% for validation. Due to
# the time consumption, the maximum number of terms is limited to 10
# in this example with 3 repetitions of the cross validation.

cnorm.cv(cnorm.elfe$data, max=10, repetitions=3)

# Cross validation with prespecified terms of an already
# existing model

cnorm.cv(cnorm.elfe, repetitions=3)

# Print norm table (in this case: 0, 3 or 6 months at grade level 3)
# (Note: The data is coded such that 3.0 represents the beginning and
# 3.5 the middle of the third school year)

normTable(c(3, 3.25, 3.5), cnorm.elfe)

Parametric modeling with beta-binomial distribution:

library(cNORM)

# cNORM can also model norm data using the beta-binomial distribution.
# This approach is particularly effective for tests with a fixed
# number of dichotomous items and a continuous distribution of
# item difficulties.

model.betabinomial <- cnorm.betabinomial(ppvt$age, ppvt$raw)

# Again, model fit can be influenced by changing the powers for the
# curve fits of α and β:

model.betabinomial <- cnorm.betabinomial(ppvt$age, ppvt$raw, alpha = 4)

# Plot percentile curves and fit of the manifest and modeled norm scores:

plot(model.betabinomial, ppvt$age, ppvt$raw)
plotNorm(model.betabinomial, ppvt$age, ppvt$raw, width = 1)

# Display fit statistics:

summary(model.betabinomial)

# Prediction of norm scores for new raw scores and preparation of
# norm tables:

predict(model.betabinomial, c(8.9, 10.1), c(123, 98))

tables <- normTable.betabinomial(model.betabinomial, c(2, 3, 4),
          reliability=0.9)

Conventional norming per age group

library(cNORM)

# Application of cNORM for the generation of conventional norms
# for a specific age group (in this case age group 3):

data <- elfe[elfe$group == 3,]
cnorm(raw=data$raw)

Modeling using the shash distribution follows the same procedure as modeling with the beta-binomial distribution. Simply replace 'betabinomial' with 'shash' in the corresponding code.

Display instructions in cNORM:

library(cNORM)
# Display vignette

vignette("cNORM-Demo", package = "cNORM")
vignette("WeightedRegression", package = "cNORM")
vignette("BetaBinomial", package = "cNORM")

Installation

Terms of Use/Licensing

cNORM is licensed under GNU Affero General Public License v3 (AGPL-3.0). This means that copyrighted parts of cNORM may only be used free of charge in commercial and non-commercial projects that run under this same license, retain the copyright notice, provide their source code and correctly cite cNORM. Copyright protection includes the reproduction and distribution of source code or parts of the source code of cNORM. The integration of the package into a server environment in order to access the functionality of the software (e.g. for online delivery of norm scores) is also subject to this license. However, regression functions, norm tables or graphics produced with cNORM are not subject to copyright protection and may be used freely for commercial or non-commercial projects. If you want to apply cNORM in a way that is not compatible with the terms of the AGPL 3.0 license, please do not hesitate to contact us to negotiate individual conditions.

If you want to use cNORM for scientific publications, we would also ask you to quote the source.

References

In case you need a reference to this page in a scientific paper, please use the following citation:

Lenhard, A., Lenhard, W. & Gary, S. (2024). cNORM - Generating Continuous Test Norms (Revision). Retrieved from: https://www.psychometrica.de/cNorm_en.html. Dettelbach (Germany): Psychometrica. https://doi.org/10.13140/RG.2.2.25821.26082

Copyright © 2025 Drs. Alexandra & Wolfgang Lenhard