Digital Gauge Frequently Asked Questions
Question number FAQ-0112
Indication accuracy of linear gauges.
What is instruction accuracy?
The measurement error of the gauge sensor.
The error is measured for each specified measurement value, and when viewed over the entire length, the sum of the maximum error in the positive direction (maximum value) and the minimum error in the negative direction (absolute value) constitutes the indication accuracy of that gauge sensor (see table below).
- Despite a minimum unit of 10 μm, the indication accuracy is 3 μm
At our company, we measure the error using a reference instrument with a minimum reading value of 0.1 μm.
We start the inspection by aligning the zero of the gauge sensor with the zero of the reference instrument,
for example, in the case of GS−102, we read the values of the reference instrument when the gauge sensor values are 2.01, 4.02, 6.03, 8.04, and 10.05 mm. Those values become the error.
Even with a gauge sensor that has a resolution of 10 μm, the indication accuracy is 3 μm (catalog value) because the minimum reading value of the reference instrument is smaller than 10 μm.
Additionally, the values being inspected are not integers to confirm the multiplication accuracy.
Measurement Example
| Measurement point (mm) | Reading of the gauge (mm) | Error (μm) | |
|---|---|---|---|
| 0 | 0 | 0 | |
| 2.01 | 2.0095 | 0.5 | Max |
| 4.02 | 4.021 | -1 | |
| 6.03 | 6.0322 | -2.2 | Min |
| 8.04 | 8.0413 | -1.3 | |
| 10.05 | 10.052 | -2 |
Precision = Max - Min = 0.5 - (-2.2) = 2.7 μm
What is resolution?
It refers to the minimum reading value that can be read with the combination of gauge sensors and displays.
For example, in the case of GS-102, the minimum reading value is 10 μm (0.01 mm).
What is incremental precision?
If the resolution and the scale increments of the sensor are the same, the accuracy of the scale directly corresponds to the accuracy of the resolution. However, it is generally common to electrically divide the scale increments to improve resolution. The errors generated from this division are referred to as 'multiplication accuracy' by our company.
Supplementary explanation
For the GS-551 (minimum unit: 1 μm), electrical processing of slits with an 8 μm pitch produces a pulse signal of one pulse per 1 μm.
The accuracy at each 8 μm slit point is shown in Table 1. Between the slits, errors occur during electrical processing; these values are shown in Table 2. To check for electrical processing errors, the fractional portions are rounded off for inspection.
Because this is a pulse signal, an error of one count (1 pulse = μm) is added on the display side.
Note that the slit spacing varies by model.
< Table 1>Example of data measured at two points of 5 mm stroke
| Measurement point mm | Error 1 (between 1 and 3 mm) μm | Error 2 (between 1.7 and 3.7 mm) μm |
|---|---|---|
| 0.2 | 0.0 | 0.1 |
| 0.4 | 0.0 | 0.0 |
| 0.6 | 0.0 | 0.0 |
| 0.8 | 0.0 | −0.1 |
| 1.0 | 0.1 | 0.0 |
| 1.2 | 0.1 | 0.1 |
| 1.4 | 0.2 | 0.0 |
| 1.6 | 0.1 | 0.1 |
| 1.8 | 0.2 | 0.0 |
| 2.0 | 0.2 | 0.0 |
< Table 2>Example of measurement data for two units
| Measurement point mm | No.1 Error μm | No.2 Error μm |
|---|---|---|
| 1.002 | −0.8 | 1.2 |
| 2.004 | −0.6 | 1.4 |
| 3.006 | −0.6 | 0.8 |
| 4.008 | −0.7 | 0.2 |
| 5.010 | −1.1 | 1.2 |
| Accuracy (Maximum Value) | 1.1 | 1.4 |
Sensor error is
Mechanical errors (slitting adhesion errors, spindle bending, play, etc.)
Electrical errors (amplification errors) are a combination.
The GS-551 has a slit pitch of 8 μm (0.008 mm), so as long as accuracy measurements are taken at intervals that are multiples of this, cumulative errors do not appear. Therefore, the errors are small.
Last updated: March 15, 2002