The sensor behavior is measured using the laser interferometer discussed below in a temperature controlled environment, comparable to that in a high accuracy CMM.
The repeatability of the Gannen probe is measured using a custom build calibration setup. Over a 10 micrometer measurement range, the standard deviation of the measurement signal is typically 2 nm or less for a single measurement direction. The combined 3D standard deviation of the probe is typically 4 nm over this range.
The graph below shows the result of a repeatability test with the Gannen XP over a 5 micrometer displacement. Over a period of 6 hours, 2000 measurement points are taken at each position. It can be seen that the contribution of drift and creep during this 6 hour measurement are neglectable.
The graph above shows the residual between a displacement as measured by the laser interferometer setup and the same displacement as measured by the Gannen XP.
The maximum top-top deviation during the measurement was 11 nanometers with a standard deviation of about 2 nanometers at each measurement position and for all three measurement directions, i.e. x- y- and z-direction.
This is shown in the graph below, where the average value at each measurement position is given, as well as the 95% confidence interval (plusminus two times the standard deviation).
The contribution of hysteresis during this measurement can be seen to be about 1 nanometer for a 5 micrometer displacement, i.e. << 0.1%.