HomeCalculatorsEngineeringMeasurement Precision Calculator
Engineering calculator

Measurement Precision Calculator

Compute ISO 5725 repeatability from a sample: s, CV = s/|x̄|, and r = 2.8·s. Runs locally. Not Type A u_A, not Gauge R&R EV, and not percent CV.

Instant result
Result
—

Enter values to calculate.

Inputs—
Mode—
Formula—
Trust summary CVP VERIFIED · CVP protocol 1.0.0-proposed · Engineering assurance · Repeatability s, CV = s/|x̄|, and r = 2.8 s from a sample, + O3 mpmath tabulated precision.
Input interpretation
Enter values to calculate.
Result
—
Verified scope
Repeatability s, CV = s/|x̄|, and r = 2.8 s from a sample, + O3 mpmath tabulated precision.
Assurance
Engineering
Declared partition coverage
PASS · 6/6 declared partitions (alias, awkward, ddof0, zero-mean, spread, invalid-domain) · Matrix
Deferred
Not Type A uncertainty and not Gauge R&R.
Numerical scope
O2: repeatability s, CV, and r = 2.8 s vs a separate-module sample identity (≤2 ULP). Not Type A u_A. ≤2 ULP vs O3 applies only to the published tabulated precision vectors. Not Type A uncertainty.
Known limitations
  • Core CVP does not include live graph, viewport, or pointer interaction.
Model
ISO 5725 repeatability s, CV, and r = 2.8·s from a sample.
Scope
ISO 5725 repeatability: s from summarizeSample, CV = s/|x̄| when x̄ ≠ 0, r = 2.8·s.
Verification
Engine tested · Source checked · v1.0.1 · CVP VERIFIED · CVP protocol 1.0.0-proposed · Engineering assurance · Repeatability s, CV = s/|x̄|, and r = 2.8 s from a sample, + O3 mpmath tabulated precision.· View Manifest · CVP overview · Specification
Versions
Calculation 1.0.1 · CVP protocol 1.0.0-proposed · Evidence 2026-09-25.o2-o3
Verification revision
2026-09-25.o2-o3 · 1/1 property · digest 432e5f2065c0
Legacy regression
11/11 tests · Production surface contract 6/6
Trust layers
Verification VERIFIED · Production CURRENT · Overall CVP state: VERIFIED under CVP 1.0.0-proposed
Reference
O1 model · O3 expected_values · O3 numerical_behavior · O2 expected_values · O2 numerical_behavior
Interfaces
PASS · UI (SSR) / REST / MCP — ui-ssr is query-result HTML, not a live browser session. Error-path engine·REST·MCP 1/1 (status, code, calculation_version). SSR compared on URL-canonical requested calculations; empty query is idle (not an error) and JSON-typed object/array inputs are REST/MCP-only.
Supplemental domain review
Not performed
Named expert review
Not performed
CVP suite
4/4 golden · 5/5 CVP boundary · 4/4 invalid · 1/1 property · 1/1 metamorphic · 8/8 O3 · 2/2 cross-interface · 6/6 CVP contract · Manifest
Sources
Sources
Evidence
1 legacy golden · 5 legacy boundary · legacy regression suite · 4/4 oracle-backed golden · 4/4 invalid · Artifact integrity PASS
This calculator CURRENT · Public schema 1.0.1 matches · Semantic contract ✓ · Production attested · Public/cache ✓ · Origin ✓
Semantic contract
PASS
Full verification

Manifest identity, reference classes, interfaces, suite, and production records.

Formulas

Core equations used by this calculator.

Sample ss = √(Σ(xᵢ − x̄)² / (n − ddof))
CVCV = s / |x̄| (x̄ ≠ 0)
Repeatability limitr = 2.8 · s
i2.8 is the ISO 5725 conventional factor, not coverage factor k. CV is not multiplied by 100. This is not Type A u_A = s/√n.

How to use

1

Enter repeated observations

Comma, space, or semicolon separated. JSON arrays also work. Sample s needs at least two values when ddof=1.

2

Read r, then s and CV

r is the repeatability limit. Use Type A when you need u_A of the mean. Use Accuracy when you have a master xm.

Example calculations

Common configurations with formula and result.

ϟ

Unit s

values=7,8,9

s=1; r=2.8; CV=1/8
r=2.8 · CV=0.125
ϟ

Zero mean

values=1,-1,0

s=1; r=2.8; CV undefined
r=2.8 · CV=—

Measurement Precision calculator specification

Version 1.0.1 · Engine tested

Calculation status

Review policy · Evidence

Definition
From n repeated observations, sample s (ddof=1 default), coefficient of variation CV = s/|x̄| when x̄ ≠ 0, and ISO 5725 repeatability limit r = 2.8·s.
What it calculates
ISO 5725 repeatability s, CV, and r = 2.8·s from a sample.
Inputs
  • values
  • ddof
Outputs
  • s
  • cv
  • r
  • xbar
  • n
Formula
r=2.8·s; CV=s/|x̄| (x̄≠0)
Assumptions
  • ISO 5725 repeatability: s from summarizeSample, CV = s/|x̄| when x̄ ≠ 0, r = 2.8·s.
  • Not Type A u_A = s/√n, not Gauge R&R EV, and not percent CV (×100).
Units
  • same as values; CV is dimensionless
Boundary conditions
  • missing values → MISSING_REQUIRED_INPUT
  • n=1 with ddof=1 → VALUE_OUT_OF_RANGE
  • cv op with x̄ = 0 → INVALID_INPUT
  • s=0 is valid
Example
values=[7,8,9] → s=1 r=2.8 CV=0.125
Validation cases

3 published on this page · 11/11 tests · Production surface contract 6/6 · View evidence

  • values=[7,8,9] → s=1 r=2.8 CV=0.125
  • values=[1,-1,0] cv op → INVALID_INPUT
  • values=[1] → VALUE_OUT_OF_RANGE
Sources
  • ISO 5725-1 — Accuracy (trueness and precision) of measurement methods and results — Precision as repeatability; repeatability limit r = 2.8 σ_r
    Supports: This page reports sample s and the conventional r = 2.8·s; it does not report a between-laboratory R
  • JCGM 200:2012 — International vocabulary of metrology (VIM) — 2.15 measurement precision
    Supports: Precision is closeness of agreement under specified conditions, quantified here by s and CV
Calculation version
1.0.1

Background

Interpretation and common distinctions.

Compute ISO 5725 repeatability: sample s, CV, and r = 2.8·s.

Supported and not supported

Supported — s · CV = s/|x̄| when x̄ ≠ 0 · r = 2.8·s · API engineering.measurement.precision

Not supported — Type A u_A, Gauge R&R EV, percent CV (×100), reproducibility R, pass/fail

Agent / API notes

Capability id: engineering.measurement.precision · tool id: precision · pin 1.0.1.

{ "values": [7, 8, 9] }

Alias: sample for values. Errors: MISSING_REQUIRED_INPUT, INVALID_NUMBER, INVALID_INPUT, VALUE_OUT_OF_RANGE.

Calculator URL stays /calc/engineering/precision. There is no /calc/metrology.

Other calculators in this family: Type A uncertainty, Measurement bias (trueness), Gauge R&R, ISO 5725 reproducibility, Uncertainty of a mean from precision, Pooled standard deviation, Repeatability from duplicates, Cochran's C, Gauge capability (Cg/Cgk), Process performance (Pp/Ppk), Tolerance / Uncertainty Workspace .

uA Type A uncertainty › δ Measurement bias (trueness) › GRR Gauge R&R › R ISO 5725 reproducibility › u Uncertainty of a mean from precision › s_p Pooled standard deviation › s_r Repeatability from duplicates › C Cochran's C › Cg Gauge capability (Cg/Cgk) › Pp Process performance (Pp/Ppk) › WS Tolerance / Uncertainty Workspace ›

Frequently asked questions

Key distinctions behind the calculation.

Is this Type A uncertainty?

No. Type A reports u_A = s/√n of the mean. This page reports s, CV, and the ISO 5725 repeatability limit r = 2.8·s.

Is this ISO 21748 u of a mean?

No. Precision takes a sample. Uncertainty of a mean from precision takes already-stated s_r, s_L, and n≥2.

Is this repeatability from duplicates?

No. Precision takes a sample of observations and reports s and r = 2.8·s. Repeatability from duplicates takes pair differences d_i and reports s_r = √(Σd²/(2m)).

Is this Cochran's C?

No. Cochran's C takes k≥2 already-stated laboratory s_i. This page takes one sample of observations.

Is r the same as Gauge R&R repeatability?

No. Gauge R&R repeatability is AIAG EV (equipment variation). This page is a one-factor sample: r = 2.8·s.

Is 2.8 the coverage factor k?

No. Coverage factor k is a different page. 2.8 is the ISO 5725 conventional factor for the repeatability limit.

Is CV a percentage?

No. CV = s/|x̄| is dimensionless and is not multiplied by 100. When x̄ = 0, CV is omitted (null) and the cv op returns INVALID_INPUT.

Where does this run?

Locally in the browser by default. REST and MCP call the same metrology-engine precision ops.