Accuracy vs. Precision

What accuracy and precision are

Two words that get used interchangeably in casual speech, but mean different things:

• Accuracy is how close a measurement is to the true value.
• Precision is the resolution or repeatability of the measurement — how finely the result is reported, and how reproducibly the same measurement device produces the same result.

The two are independent. A coordinate written to one-meter precision can be 200 meters from the actual location (precise but inaccurate — common with the wrong map datum). A coordinate written to 100-meter precision can be exactly correct within that 100-meter square (accurate, less precise). And the worst combination — imprecise and inaccurate — looks no different on the page from the best one until you compare it against truth.

The distinction is worth treating on its own because it's the conceptual frame for several rules in navigation practice — coordinate truncation, tool-precision matching, declination rounding, GPS EPE discipline. Those rules are easier to follow once you see that they're all variations of the same idea: match what you write to what you actually measured.

The classic illustration

A dartboard. Four hypothetical shooters, each throwing five darts:

The high-precision-low-accuracy case is the dangerous one in navigation. The tight cluster looks confident. It's wrong by a known offset that no amount of additional measurement with the same equipment will reveal. The classic example: a GPS reading positions to 1-meter precision while configured for the wrong map datum — the answers are tightly grouped and consistently wrong by about 200 meters. That's not a rounding problem; that's a systematic accuracy problem masquerading as confidence.

Precision is set by the measurement floor — not by the printed marks

Every measurement has a precision floor: the finest distinction you can honestly make with this tool, on this medium, under these conditions. Reading a result past that floor doesn't add information — it adds the appearance of information.

The common shortcut is to equate the floor with the tool's finest graduation. That's wrong in both directions, and it's worth getting right — it's the difference between under-using a good tool and over-trusting a number.

You can read between the marks. Interpolating a reading to a fraction of the smallest graduation is normal, expected practice — not a cheat. The working convention is that a careful eye resolves about one-tenth of a division under good conditions, or conservatively one-half when the read is coarse or the instrument is jittery. This is exactly why MapTools grid tools print a dot in the center of each 100-meter cell: the dot is an interpolation aid — a built-in halfway reference so you can estimate position within the cell instead of only naming which cell you're in. A tool's graduations are a floor on what it labels, not a ceiling on what you can read.

But the real floor is set by whatever gives out first — and it usually isn't the graduation:

• The medium. On a 1:24,000 map, 100 meters is 4.2 mm of paper and 10 meters is 0.42 mm. Interpolating a 100-meter grid down to ~10 meters means splitting that 4.2 mm by eye, with the center dot halving the span you're judging — squarely doable. Going finer is not: 1 meter is 0.042 mm, below the width of the printed line, and the paper itself expands and contracts with humidity. About 10 meters is the practical floor on a 1:24,000 paper map, regardless of what any tool's marks claim. See Map Scale for the ratio that drives this. The fix for needing finer precision is a finer-scale map — not a finer tool or a steadier hand.
• Instrument repeatability. A compass dial graduated every 2° can honestly be interpolated to 1°, and a sighting or mirror compass marked in 1° to about ½° — because the instrument is steady enough to support it. What stops you going finer on a baseplate compass usually isn't the dial spacing; it's the needle's settling oscillation (a degree or two) and how steadily you hold the capsule. The better-mounted instrument earns the finer read. The marks aren't the limit; the repeatability is.
• Systematic accuracy. Even a legitimate interpolated read hits a wall where extra precision stops buying accuracy. The National Map Accuracy Standard puts a 1:24,000 map's own horizontal error at about ±12 meters. So a clean 10-meter interpolated position is already brushing the map's own registration error — read "to the meter" and you'd be reporting a precision the map itself can't back. This is the accuracy-vs-precision distinction biting in the field: you can refine the reading past the point where the map is true.

So the honest grid-tool statement has two cases, not one:

• Read to the nearest cell, a 100-meter grid tool gives 100-meter precision — the abbreviated coordinate names the cell, and the 10- and 1-meter digits are genuinely absent.
• Read with interpolation (using the center dot), the same tool gives roughly 10-meter precision — a real, honestly-estimated 10-meter digit. It does not give you a 1-meter digit; the paper gave out first.

And the GPS, as always, is its own case:

• A GPS reports coordinates to 1-meter precision by convention. The actual accuracy is usually 5–20 meters depending on conditions — see the GPS EPE section below. The 1-meter digits on the display are precision; the 5–20-meter ground reality is accuracy.

Interpolating is not inventing. The two look alike on the page and are opposites in practice. Interpolating to a 10-meter read and writing that 10-meter digit records something you actually estimated, within the tool's and the medium's floor. Inventing the 1-meter digit — writing 559734 when you read 559.7 and the paper can't resolve finer — fabricates a digit you never saw. The truncate-don't-round discipline below polices the second; it has no quarrel with the first.

How that precision gets reported in a coordinate depends on the format — and the formats fall into two camps.

How coordinate formats express precision

Coordinate formats fall into two camps in how they handle precision:

• Lat/lon coordinates name points. A latitude/longitude pair identifies a specific point on the Earth; precision is set by how many digits you write past the smallest unit.
• UTM, MGRS, and USNG coordinates name squares. A truncated coordinate identifies a square on the ground rather than a point. The square's size is set by how many digits you write, and the coordinate refers to the SW corner of that square.

The two camps handle the "report only what you measured" rule differently.

UTM, MGRS, and USNG: drop or pad the unmeasured digits

• Abbreviated MGRS / USNG lets you literally drop the unmeasured digits. A 100-meter grid-tool reading becomes a six-digit abbreviated coordinate (597 372); a 10-meter reading becomes eight digits (5974 3722). The number of digits is the precision claim.
• Full UTM, written in meters, has no such freedom — the easting and northing are always integers to the meter. A 100-meter measurement gets written as 0559700 4137200, with the trailing zeros as a positional placeholder. This is a convention, not a lie: an experienced reader treats 0559700 as either "exactly on the 100-meter line" or "measured to 100 meters and zero-padded," and uses context to tell which. But the ambiguity is real, and it's one of the places where the UTM convention is awkward.
  • What not to do: invent the 10-meter and 1-meter digits to "fill in" the coordinate. Writing 0559734 4137268 when you actually measured 559.7 and 4137.2 overstates precision by a factor of ten and silently misleads anyone who acts on the number.

Lat/lon: precision is set by digit count

Lat/lon coordinates name points, not squares. Precision is set by how many digits you write past the smallest unit, and the format choice — DMS, DDM, DD — is a notation choice, not a precision choice. Any of the three lat/lon formats can express any precision; you just have to use enough digits.

DD reads as more precise at a glance because the digit-count arithmetic is more visible — decimal degrees are base-10, while DMS hides precision behind a mixed-base notation (60 minutes per degree, 60 seconds per minute). The appearance is misleading: the same point can be expressed at the same precision in any of the three notations.

DMS (hddd° mm′ ss″) — latitude:

DDM (hddd° mm.mmm′) — latitude:

DD (hddd.ddddd°) — latitude:

37° 25′ 30.0″ N, 37° 25.500′ N, and 37.42500° N all describe the same point to roughly 1-meter precision. The notation differs; the precision doesn't.

Latitude vs. longitude. The ground distances above are for latitude — 1° of latitude is about 111 km / 69 mi anywhere on Earth.

Longitude distances vary with latitude. A degree of longitude is the same ~111 km / ~69 mi at the equator, shrinks to about 96 km / 60 mi at 30° N (south Texas, north Florida), drops to about 79 km / 49 mi at 45° N (Oregon, Maine, Minnesota), and continues shrinking toward zero at the poles. In CONUS, expect longitude distances at any given precision to be roughly two-thirds to nine-tenths of the latitude values in the tables. UTM doesn't have this asymmetry — a UTM meter is a meter everywhere, which is one of the practical reasons to prefer UTM for land navigation.

The underlying discipline is the same in all formats: write what you measured, in the format's idiom, and don't make up the digits you didn't.

Truncate, don't round

When you abbreviate a coordinate to a coarser precision, truncate the extra digits — do not round them.

Worked example (from the MGRS / USNG abbreviation rules):

An easting of 0559751 becomes 597 at 100-meter precision, not 598.

The reason: the truncated coordinate names the grid square the point is in. Truncating to 597 says "somewhere in the 597-XXX 100-meter square." That square actually contains the point. Rounding to 598 would name a different square — the one to the east — that doesn't contain the point. Rounding silently moves your position by up to 100 meters in the wrong direction.

The general form of the rule: as precision decreases, the square gets bigger; the reported coordinate names the square the point is in, not the nearest grid intersection.

By convention, the coordinate names the square by its SW corner — the corner with the easting and northing values produced by the truncation.

The FGDC USNG standard formalizes this with a precision ladder (using the same example easting/northing):

Each level adds two digits — one to the easting, one to the northing — and shrinks the square by a factor of ten. Both halves of the coordinate carry the same precision. A coordinate with three easting digits and four northing digits isn't precise; it's malformed.

Match precision to the need, not to what's available

The other direction of the rule: don't over-measure when the application doesn't require it.

From the declination correction worked example: a calculated declination of 15° 27′ is a perfectly precise number, but most field navigation only needs the nearest half-degree or full degree. The honest field answer is "15½°" — or "15°" if the route doesn't require even that. Carrying the 27′ through a chain of mental arithmetic adds work and adds opportunities for error, while delivering a precision the next step in the workflow doesn't use.

A general principle worth stating directly: the right precision is the coarsest one that supports the decision you're using the number for. A backcountry hiker plotting an attack point doesn't need 1-meter accuracy. A search-and-rescue team coordinating a sweep does. A surveyor establishing a property corner needs more than either. Calibrate the precision to the use.

The general field practice for civilian land navigation, per the FGDC USNG standard, is 100-meter or 10-meter precision for "general field applications." Anything finer is "special applications" and should be justified by what the work requires.

GPS EPE — the honest precision number

A modern handheld GPS displays coordinates to 1-meter precision, but its actual accuracy varies with satellite geometry, atmosphere, terrain, and tree cover. Most receivers expose this as an Estimated Position Error (EPE) field — the receiver's own estimate of how far off the displayed position might be.

Get in the habit of checking EPE every time you read a position. Typical values:

• Under good conditions (open sky, multiple satellites high above the horizon): EPE under 10 meters is common.
• Acceptable (light tree cover, moderate terrain): EPE under 20 meters / 60 feet.
• Suspect (canyon, dense canopy, hand inside a pack): EPE may climb to 30 m or more — the reported position is still precise to 1 meter on the display, but inaccurate enough that those last digits are meaningless.

The EPE habit is the operational form of the accuracy-vs-precision discipline: the GPS shows you a precise-looking number, the EPE tells you how much of that precision to trust. See GPS Setup for Map Coordinates for receiver configuration that puts EPE on a screen you actually look at.

Where this concept shows up in practice

A short tour, so the rule is recognizable when you encounter it:

• UTM / MGRS / USNG — the truncate-don't-round rule; the precision-by-digit-count ladder.
• Map Scale — the rule that the tool's marked precision must match the map's scale; the practical 10-meter limit on a 1:24,000 paper map.
• North References (True, Magnetic, Grid) and Declination — the "you probably don't need more than half a degree" guidance for declination correction.
• GPS Setup for Map Coordinates — the EPE field as a routine check; matching coordinate format precision (1 m, 10 m, 100 m) to use case.
• Map Datums — the failure mode where precise coordinates are accurate to the wrong datum and consistently offset by a known amount.

All of these are the same idea wearing different clothes: a number is only as good as what produced it, and writing extra digits doesn't make it better.

When to truncate — and when to write the displayed digits

The truncation discipline above is right for some measurements and wrong for others. Two situations look similar but call for different practices.

Map measurements with a precision tool: truncate to what you measured

When you read a coordinate off a map with a grid tool, slot tool, or ruler, the tool sets your precision. Write the coordinate to match: drop the unmeasured digits in abbreviated MGRS/USNG, zero-pad in full UTM. Don't invent digits the tool didn't give you.

A coordinate written with more precision than the producing tool measured lies about what is known. A reader who sees 10S 0559751 4137247 and knows it came from a map measurement reasonably interprets it as a 1-meter position. If the tool actually measured to 100 meters and the trailing 51 / 47 are invented, the reader has been misled — they may pace the last 50 meters expecting to land on a feature, when the original measurement put the feature anywhere within a 100-meter square.

The honest form of "I measured to 100 meters" depends on the format:

• In abbreviated MGRS/USNG, drop the digits you didn't measure: 597 372. The reader sees a six-digit coordinate and knows it's a 100-meter square.
• In full UTM in meters, write the coordinate with trailing zeros where the measurement ran out: 10S 0559700 4137200. The reader has to read the convention — "ends in 00 00" usually means "measured to 100 m" — but the format doesn't allow anything cleaner.

GPS readings: write all the digits the receiver shows

GPS is the practical exception. A modern handheld displays coordinates to 1-meter precision, but actual accuracy is typically 5–20 meters and depends on satellite geometry, terrain, and tree cover. Everyone who works with GPS data knows this; the EPE field gives the per-reading specifics.

In the field, just write down what the GPS shows. Three reasons:

• The over-precision is well-understood — the next person seeing your 10S 0559751 4137247 will apply GPS-reading judgment (and EPE limits) rather than treating it as a survey-grade position.
• The full digits preserve information. You can always truncate later; you can't recover digits you didn't write down.
• A truncation blunder is harder to recover from than over-precision. Writing the wrong digits because you tried to round or truncate in your head (597 372 when the measurement was 5975 3725) silently moves the point. Writing all the displayed digits preserves the original observation, even when it implies more precision than is physically real.

The principle still holds — don't invent digits a tool didn't give you — but for GPS, the displayed digits are what the tool gave you, even if the underlying accuracy is coarser. The honest record is the receiver's output; the interpretation of that output requires knowing the tool produced it.

Common misconceptions