We often get asked about accuracy. How accurate is your scanner? How can you prove accuracy? What kind of testing was done and how was is done? In this article we explain in detail.
The Calibry Mini’s stated accuracy is up to 0.07mm (0.0027 inches). What does that mean? It means that under normal circumstances, you can expect to get a deviation of any single point of data to be plus or minus of roughly 70 microns in any direction. This does not mean that the deviation couldn’t be smaller or larger. The number is an approximation and can vary greatly depending on several factors which we will mention later in this article.
How did we arrive at this number? What kind of testing was done?
First, we ordered the manufacturing of a ball-bar (two aluminum oxide balls attached to a bar spaced approximately 70mm apart). Then, this ball-bar was measured by a certified metrological measuring machine which guarantees accurate measurements within 6 microns.
The certificate states that the centers of the two spheres are exactly 2.75512 inches apart (69.98mm). The temperature in the room during certification was 68F (20C).
Then what?
We scanned the ball-bar with Calibry Mini. The temperature in the room was 20C degrees and the device was calibrated right before the procedure. The scanner was kept at a 90 degree angle to the object during scanning and the shades to the windows were shut to avoid excessive light in the room. We used “Geometry” tracking mode in version 3.3.41 and version 3.4.141 (soon to be released) of Calibry Nest to finalize the models into the standard STL format. The scans were assembled with three different resolution settings: 0.15mm, 0.2mm, and 0.3mm. Then, each data set was loaded into Geomagic Design X (v.2020.0.4). The software identified two spheres and their respective mid points. Then it measured the exact distance between these two points.
STL file imported into Design X
Identifying a sphere
Spheres, Sketch and Measurement
Sketch of spheres and measurement between midpoints
These are the results:
|
Calibry Nest Version |
Scan # |
Resolution 0.15mm |
Deviation @ 0.15mm |
Resolution 0.2mm |
Deviation @ 0.2mm |
Resolution 0.3mm |
Deviation @ 0.3mm |
|
3.3.41 |
1 |
70.0080 |
0.0280 |
70.0036 |
0.0236 |
69.9892 |
0.0092 |
|
2 |
69.9431 |
0.0369 |
69.9531 |
0.0269 |
69.9411 |
0.0389 |
|
|
3 |
70.0181 |
0.0381 |
70.0167 |
0.0367 |
70.0212 |
0.0416 |
|
|
3.4.141 |
1 |
69.9949 |
0.0149 |
69.9992 |
0.0192 |
69.9983 |
0.0183 |
|
2 |
69.9676 |
0.0124 |
69.9442 |
0.0358 |
69.9750 |
0.0050 |
|
|
3 |
69.9899 |
0.0099 |
69.9714 |
0.0086 |
69.9691 |
0.0109 |
The average deviation in these 18 scans was 0.02305mm (23 microns or 0.0009 inches). To recreate our experiments download original data here (150MB).
At the same time, we tried various other tests to see how one or several factors can affect accuracy. For example, we tried to use a non-calibrated scanner to complete the same task. The deviation reached as high as 192 microns (0.1926mm or 0.0075 inches). We also tried to cool down the room and scanned in direct sunlight. The results were all negatively affected by these factors.
So, what do we recommend to consistently achieve high-accuracy data?
1. Calibrate scanner before scanning using the board provided with your device.
2. Adjust room temperature to 20C (68F) degrees and close window shades to avoid excessive light.
3. Scan at a 90 degree angle to the object.
4. Keep a proper distance to the object. The built-in screen on the device will indicate, in real-time, when you are in the “green zone”.
5. Gather enough data. For the ball-bar, we gathered about 800-1000 frames for each test scan.
6. If scanning in Geometry tracking mode, use crumpled pieces of paper spread around the object to create additional geometry which will assist during post-processing.
Why 0.07mm stated accuracy?
Yes, the scanner can achieve higher accuracy consistently. However, sometimes, a user will be unable to keep to all the recommendations. Before announcing the Mini in 2020, we conducted dozens of tests: some were done under perfect conditions and, deliberately, others were not. We stated the average number because we knew that at times our users won’t be scanning in a perfectly lit room, won’t have the time to calibrate the device before each scan or be in a room with the recommended temperature.
Final thoughts
A certified, metrological device guarantees certain accuracy each time you use it, as long as you follow manufacturer’s precise instructions for operation. These scanners are typically very expensive and difficult to operate. The Calibry Mini scanner is not a certified metrological device, but if you follow our recommendations above, you will receive extremely accurate results each time and for a fraction of the cost.
PS If you have questions about accuracy or anything else, please email us at support@thor3Dscanner.com. We love talking to Calibry owners!