HomeCalculatorsEngineeringMSA Linearity Calculator
Engineering calculator

MSA Linearity Calculator

Fit ordinary least squares of measurement error ε = ylab − xref versus xref. Reports slope, intercept, and r². Runs locally. Not a single-master bias, not Type-1 Cg, and not a %Linearity pass/fail.

Instant result
Result
—

Enter values to calculate.

Inputs—
Mode—
Formula—
Trust summary Engine tested · Source checked · 9/9 tests · Production surface contract 4/4 · v1.0.0
Input interpretation
Enter values to calculate.
Result
—
Model
OLS of measurement error ε = ylab − xref versus xref: slope, intercept, and r².
Scope
Ordinary least squares of ε = ylab − xref versus xref. slope = dε/d(xref).
Verification
Engine tested · 9/9 tests · Production surface contract 4/4 · Source checked · v1.0.0
Named expert review
Optional · Not performed
Sources
  • AIAG Measurement Systems Analysis (MSA) — Linearity and bias study — error versus reference
  • ISO 11095 — Linear calibration using reference materials — Ordinary least squares of response versus reference
Sources
Evidence
6 boundary · 3 property · Production surface contract 4/4 · Artifact integrity PASS
Production
Embedded snapshot: unpublished · Build schema 1.0.0 ready · Semantic contract ✓ · Attestation report not published on origin · Live production status STALE (1 capability; 156 remain CURRENT) @ 2026-09-14T03:54:53.650Z
Semantic contract
PASS

Formulas

Core equations used by this calculator.

Errorεᵢ = ylabᵢ − xrefᵢ
Slopeb = Σ(xrefᵢ − x̄)(εᵢ − ē) / Σ(xrefᵢ − x̄)²
Intercepta = ē − b·x̄
ir² is the coefficient of determination of ε versus xref. When all errors are equal, slope is 0 and r² is omitted. This page does not take T and does not issue a %Linearity verdict.

How to use

1

Enter xref

Reference values across the range. Comma, space, or semicolon separated. JSON arrays also work. Need at least three points, and they must not all be equal.

2

Enter ylab

Indications paired with xref, same length. Aliases: y, measured.

3

Read slope, then intercept and r²

slope is dε/d(xref). Use Measurement bias for repeats on one master. Use Gauge capability when you have T. This page does not issue pass/fail.

Example calculations

Common configurations with formula and result.

ϟ

Proportional error

xref=0,2,4 · ylab=0,3,6

ε=[0,1,2]; b=0.5; a=0
slope=0.5 · intercept=0 · r²=1
ϟ

Constant offset

xref=0,2,4 · ylab=1,3,5

ε=[1,1,1]; b=0; a=1
slope=0 · intercept=1 · r²=—

MSA Linearity calculator specification

Version 1.0.0 · Engine tested

Calculation status

Review policy · Evidence

Definition
From n≥3 paired reference values xref and indications ylab, ε_i = ylab_i − xref_i. Ordinary least squares of ε versus xref gives slope = dε/d(xref) and intercept at xref = 0.
What it calculates
OLS of measurement error ε = ylab − xref versus xref: slope, intercept, and r².
Inputs
  • xref
  • ylab
Outputs
  • slope
  • intercept
  • r_squared
  • n
  • xbar
  • ebar
  • xref
  • ylab
  • error
Formula
ε=a+b·xref
Assumptions
  • Ordinary least squares of ε = ylab − xref versus xref. slope = dε/d(xref).
  • n≥3 paired points. xref must not all be equal.
  • Not a single-master bias, not Type-1 Cg/Cgk, and not a %Linearity pass/fail.
  • Does not take T.
Units
  • slope is dimensionless when ylab and xref share a unit; intercept same unit as ε
Boundary conditions
  • missing xref or ylab → MISSING_REQUIRED_INPUT
  • n < 3 → VALUE_OUT_OF_RANGE
  • length mismatch → INVALID_INPUT
  • all xref equal → INVALID_INPUT
Example
xref=[0,2,4] ylab=[0,3,6] → slope=0.5 intercept=0 r²=1
Validation cases

3 published on this page · 9/9 tests · Production surface contract 4/4 · View evidence

  • xref=[0,2,4] ylab=[0,3,6] → slope=0.5 intercept=0 r²=1
  • xref=[0,2,4] ylab=[1,3,5] → slope=0 intercept=1 r²=null
  • xref=[0,2] ylab=[0,3] → VALUE_OUT_OF_RANGE
Sources
  • AIAG Measurement Systems Analysis (MSA) — Linearity and bias study — error versus reference
    Supports: Linearity is the change of bias across the operating range. This page reports the OLS slope of ε versus xref and does not issue a %Linearity verdict.
  • ISO 11095 — Linear calibration using reference materials — Ordinary least squares of response versus reference
    Supports: Paired reference and indication values; linear fit. This page fits ε versus xref rather than ylab versus xref.
Calculation version
1.0.0

Background

Interpretation and common distinctions.

Compute MSA linearity as the ordinary-least-squares slope of measurement error versus reference.

Supported and not supported

Supported — slope, intercept, and r² of ε = ylab − xref versus xref · n≥3 paired points · API engineering.msa.linearity

Not supported — %Linearity / process-variation verdict, Type-1 Cg/Cgk, single-master bias, pass/fail

Agent / API notes

Capability id: engineering.msa.linearity · tool id: linearity · pin 1.0.0.

{ "xref": [0, 2, 4], "ylab": [0, 3, 6] }

Aliases: X / ref / reference for xref; y / measured for ylab. Errors: MISSING_REQUIRED_INPUT, INVALID_NUMBER, INVALID_INPUT, VALUE_OUT_OF_RANGE.

Calculator URL stays /calc/engineering/linearity. There is no /calc/metrology.

Other calculators in this family: Measurement bias (trueness), MSA hysteresis, Measurement error, Gauge capability (Cg/Cgk), Gauge R&R, Measurement precision, Maximum permissible error MPE, Tolerance / Uncertainty Workspace .

δ Measurement bias (trueness) › H MSA hysteresis › ε Measurement error › Cg Gauge capability (Cg/Cgk) › GRR Gauge R&R › s Measurement precision › MPE Maximum permissible error MPE › WS Tolerance / Uncertainty Workspace ›
}

Frequently asked questions

Key distinctions behind the calculation.

Is this the same as measurement bias?

No. Accuracy takes repeats on one master and reports signed bias of the mean. This page fits how error changes across a reference range.

Is this Type-1 Cg/Cgk?

No. Gauge capability needs a characteristic tolerance T and a fraction frac. This page does not take T.

Does a slope of 0.1 mean fail?

This page does not issue that verdict. It reports slope, intercept, and r² only. It does not compute %Linearity against process variation.

What if every xref is the same?

INVALID_INPUT. The OLS denominator Σ(xref−x̄)² is zero, so slope is undefined.

Where does this run?

Locally in the browser by default. REST and MCP call the same metrology-engine linearity op.