Digital Gauges: Frequently Asked Questions
Question number FAQ-0112
Linear gauge indication accuracy
■ What is instruction accuracy?
This is the measurement error inherent in the gauge sensor.
The error is measured for each predetermined measurement, and the sum of the maximum positive error (maximum value) and the minimum negative error (absolute value) over the entire length represents the indication accuracy of the gauge sensor (see table below).
- How can it have an indication accuracy of 3 μm when the smallest unit is 10 μm?
Our company has a minimum reading of 0.1 μm. reference device The error is being measured using this method.
The zero and reference device Start the test by setting the zeros together.
For example, in the case of GS-102, when the gauge sensor values are 2.01, 4.02, 6.03, 8.04, and 10.05 mm... reference device Read the value. That value will be the error.
Even with a gauge sensor with a resolution of 10 μm, the indication accuracy is 3 μm (catalog value). reference device This is because the minimum reading is less than 10 μm.
Furthermore, the reason the values being tested are not integers is to also verify the multiplication precision.
■ Measurement Examples
| Measurement points (mm) | Reading from the reference instrument (mm) | error (μm) | |
|---|---|---|---|
| 0 | 0 | 0 | |
| 2.01 | 2.0095 | 0.5 | Max |
| 4.02 | 4.021 | -1 | |
| 6.03 | 6.0322 | -2.2 | Min |
| 8.04 | 8.0413 | -1.3 | |
| 10.05 | 10.052 | -2 |
Indication accuracy = Max- Min = 0.5-(-2.2) = 2.7 μm
■ What is resolution?
This refers to the minimum reading that can be obtained using a combination of a gauge sensor and a display unit.
For example, with the GS-102, the minimum reading is 10 μm (0.01 mm).
■ What is multiplication precision?
If the resolution and the size of the scale increments within the sensor are the same, the accuracy of the scale directly corresponds to the accuracy of the resolution. However, generally, the resolution is increased by electrically dividing (multiplying) the scale increments. We refer to the error caused by this division as the multiplication accuracy.
(2) Supplementary explanation
In the case of the GS-551 (for minimum units of 1 μm), a pulse signal of 1 pulse per 1 μm is generated by electrical processing from a slit with an 8 μm pitch.
The accuracy at 8 μm intervals corresponding to the slit points is shown in Table 1. Between the slits, electrical processing errors occur, and these values are shown in Table 2. Fractional values are rounded off during inspection to check for electrical processing errors.
Because it is a pulse signal, an error of 1 count (1 pulse = μm) is added to the display.
Please note that the spacing of the slits varies depending on the model.
< Table 1>Example of data measured at two points of 5 mm stroke
| Measurement points mm | error 1 (Between 1 and 3 mm) μm | error 2 (Between 1.7 and 3.7 mm) μm |
|---|---|---|
| 0.2 | 0.0 | 0.1 |
| 0.4 | 0.0 | 0.0 |
| 0.6 | 0.0 | 0.0 |
| 0.8 | 0.0 | −0.1 |
| 1.0 | 0.1 | 0.0 |
| 1.2 | 0.1 | 0.1 |
| 1.4 | 0.2 | 0.0 |
| 1.6 | 0.1 | 0.1 |
| 1.8 | 0.2 | 0.0 |
| 2.0 | 0.2 | 0.0 |
< Table 2>Example of measurement data for two units
| Measurement point mm | No.1 error μm | No.2 error μm |
|---|---|---|
| 1.002 | −0.8 | 1.2 |
| 2.004 | −0.6 | 1.4 |
| 3.006 | −0.6 | 0.8 |
| 4.008 | −0.7 | 0.2 |
| 5.010 | −1.1 | 1.2 |
| Precision (Max value) | 1.1 | 1.4 |
Sensor error is
Mechanical errors (slit attachment errors, spindle bending, play, etc.)
This is a combination of electrical errors (multiplier errors).
* The GS-551 has a slit pitch of 8 μm (= 0.008 mm), so as long as accuracy measurements are taken at intervals that are multiples of this, multiplicative errors will not appear. Therefore, the error is small.
Last updated: 2002-03-15