Precision Measurement Complete Guide: Ask What You Are Measuring First, Then Pick the Instrument
A reading that is off can fail in more than one way: the wrong instrument, a setup that violates the underlying principle, or a specification figure you read as something it does not mean. This guide strings the 30 measurement articles on this site into one route - start from what you are measuring, then pick the instrument, read the specification, and trace the error. Each section points to the article that covers it; this page does not repeat their content.
1. Which of the Four Problems Do You Have?
Nine times out of ten a measurement problem falls into one of four. Find yours, then read only that section.
| Your situation | Section |
|---|---|
| You are not sure how to hold or set up the instrument | 2 |
| Two instruments give different readings on the same feature | 3 |
| The specification sheet is unclear - can you reject a part on it? | 4 |
| The drawing calls out something and you do not know what to measure it with | 5 |
2. How to Use the Instrument in Your Hand
Hand-held contact instruments cover most shop-floor work. Their accuracy usually fails on setup, not on the instrument itself.
| Instrument | What that article answers |
|---|---|
| Vernier Caliper Complete Guide | Reading, the four measuring modes, and when to switch to a micrometer |
| Outside Micrometer Complete Guide | Reading, zeroing before every measurement, and periodic calibration |
| Height Gauge Guide | Scribing and comparison; how far the scriber sticks out multiplies the error |
| Dial and Lever Indicators | Mounting, measuring attitude and four error types |
| Why a Lever Indicator Needs cosθ | A tilted stylus always reads high; correction table and worked example |
| Inside Measuring Instruments | Contact point types; controlling eccentricity and tilt error |
| Where to Support a Long Bar Gauge | Airy and Bessel points - a long gauge bends under its own weight |
| Micrometer Head Selection | Choosing the fine-adjust element for fixtures and automation |
3. Why the Reading Is Off: Structure First, Then Error
When two instruments disagree, the one that is wrong is usually the one that is structurally at a disadvantage - not the one held by the less careful operator. This section is the foundation for everything else here.
| Article | Why read it |
|---|---|
| The Abbe Principle | The structural reason a caliper is less accurate than a micrometer, and why a height gauge has an even larger offset |
| Three Classes of Measurement Error | Systematic error can be subtracted, random error can only be reduced, a gross error must be discarded |
| Gauge Blocks and Calibration | The length standard behind every comparison; wringing, interference fringes and temperature |
| Why Calibration Certificates Matter | Accuracy only means something if it traces back to a national standard; how that chain works and where it breaks |
4. Reading the Specification: Two Articles for Buyers and QC
This section is for the buyer and the incoming-inspection desk: for the same caliper, the figure on the specification sheet decides whether you can reject it.
| Article | What it answers |
|---|---|
| What MPE, EMPE and SMPE Mean | When JIS replaced "instrumental error" with "maximum permissible error", what changed was not the term but the definition of conforming |
| How to Read an Indicator Specification | The MPE tables of JIS B 7503 (mechanical) and B 7563 (digital), listed cell by cell |
5. What Are You Measuring: Look It Up by Feature
Different callouts need different methods. Size, form, surface, thread and hardness are five separate routes.
| What you measure | Which article |
|---|---|
| Geometric tolerance (flatness, parallelism, squareness) | How to Measure Geometric Tolerance - the five inspection principles |
| Roundness, cylindricity, runout | Roundness and Cylindricity - four evaluation methods and UPR analysis |
| Surface roughness Ra | How to Measure Surface Roughness - choosing among seven methods |
| Thread pitch diameter | Three-Wire Method - a gauge only says go or no-go; for a number you need this |
| Buttress thread | Measuring and Designating a Buttress Thread - 7° on one flank and 45° on the other, so the ordinary three-wire formula does not apply |
| Whether the thread gauge itself is still right | Calibrating Thread Gauges - a worn GO gauge passes bad threads; temperature soak and two-point pitch diameter |
| Which dimension failed on a rejected thread | Thread Rejected? Find the Dimension - a GO gauge that will not enter is not always a small pitch diameter; how to measure internal pitch diameter, pitch and flank angle |
| Hardness | Hardness Testing Complete Guide - Vickers vs Rockwell, and the specimen thickness limit |
6. Instrument-Grade Equipment: When Hand Tools Cannot Reach
Complex parts, 100% inspection, or non-contact requirements move you up to instruments. These five cover selection and accuracy verification.
| Equipment | What that article covers |
|---|---|
| CMM Accuracy Verification | How ISO 10360 is applied; length measurement error and uncertainty |
| Vision Measuring Systems | Edge detection, sub-pixel accuracy and auto focus |
| Laser Micrometers | Non-contact outside diameter; optical axis alignment is the accuracy driver |
| Microscope Measurement Optics | NA, resolution, depth of field and magnification |
| Linear Encoders | Position measurement on machine tools; interpolation and error sources |
7. On the Machine and in QC: Measurement Feeding Back Into Production
Measurement does not only happen in the inspection room. Edge Finders covers on-machine measurement - bringing the spindle centre onto the workpiece edge to set the work coordinate origin. Quality Control and SPC turns single readings into a process judgement: Cp, Cpk and control charts answer "is this process stable", not "does this part pass". Whether the machine itself runs true is a separate question: Measuring Spindle Dynamic Accuracy uses Lissajous curves to see how the spindle behaves at different speeds — a static check, however precise, cannot see what happens once it spins.
8. Frequently Asked Questions (FAQ)
Q: If I can only buy one, a caliper or a micrometer?
It depends on the accuracy you need. A caliper covers more (outside, inside, depth, step) but is structurally in violation of the Abbe principle; a micrometer is more accurate but each one covers only one range. Read the Abbe article first, then the last section of the caliper guide on when to switch.
Q: Five measurements of one part give five numbers. Is the instrument faulty?
Not necessarily. First establish whether it is systematic, random or gross error - the diagnosis and the remedy differ for each. See Three Classes of Measurement Error in section 3.
Q: The sheet says EMPE +/-0.03. Does a reading within 0.03 conform?
No. Once JIS replaced instrumental error with maximum permissible error, the specification range and the conformance range stopped being the same thing. Go straight to section 4; that article works through a 200 mm vernier caliper as a full example.
Q: Does this page list accuracy figures for each instrument?
No. A guide only routes you; the numbers live in the individual articles - MPE permissible values in the two articles of section 4, JIS tolerance tables in the gauge block article. That way each figure has one home and nothing is missed when it changes.
Published: 2026-09-08 | Last updated: 2026-10-21
