Skip to content

Select your region & language

Global

Region

Error and precision

A value measured without error is called the "true value." Values obtained through measurement, such as the "display value" of a measuring instrument or the "indicated value" of a meter, have an "error," which is the difference from the true value.
To describe the state of error, we use "bias," which is the difference between the average value (population mean) of the measured values and the true value; "variability," which is the degree to which the measured values are inconsistent; and the standard deviation, which represents the magnitude of the variation.
Errors are expressed using "uncertainty," which represents the degree of uncertainty in the measurement results using the "error limit," which is the limit of the estimated overall error; "accuracy," which uses the "accuracy" limit of the estimated bias; and "precision," which uses the "precision" value, which is the standard deviation of the variation or an integer multiple of that value.
In practical terms, the overall quality of a measuring instrument, including the accuracy and precision of its measurements, is expressed as "precision." However, the accuracy is also expressed as "percentage error," which is the ratio of the instrument's error to the upper limit of its effective measurement range, which is the "baseline value." For example, analog output is expressed as ±0.5%/FS (FS = full scale = baseline value).
When the same object is measured repeatedly in a short period of time using the same measurement method and conditions, the degree to which the measured values match is called "repeatability," and is sometimes expressed as half the difference between the maximum and minimum measured values. Conversely, when the same object is measured using the same method but at different times and locations, the degree to which the measured values match is called "reproducibility."
Error factors include "influence errors" due to differences in measurement environment such as temperature, "random errors" of unknown cause, and "mistakes," "personal errors," and "parallax" caused by the measurer.
It is common practice to "correct" measured values by adding a certain value, such as correcting for temperature, to make the measured values closer to the true value. When the correction amount is expressed as a ratio, it is called the "influence coefficient."
The difference between a standard value and the permissible limit value is called "tolerance," and the difference between a specified maximum value and a specified minimum value is called "tolerance," and these are used in management objectives such as machining drawings and quality control.

(Excerpt from the email newsletter issued on February 22, 2002)