TL;DR
Blanket DEM quality statements are deceptive.
That said, Copernicus DEM is usually better than SRTM.
For terrain applications, FABDEM might be even better.
Intro
We use Digital Elevation (DEM) and Land Cover Models to identify obstructions to wireless communications over long distances.
DEM quality is important to our business and the success of our customers.
SRTM has been the go-to DEM for most industry applications.
SRTM has served us well.
But, it now has competition.
Should we replace SRTM with a competitor?
How can we know if a competitor is better than SRTM?
Here are actual quality claims from DEM marketing and scientific material:
- < 4m (90% linear error),
- Mean Absolute Error is 0.49m less (urban) and 2.27m (forests),
- RMSE is 2.44m for NED and 3.53m for SRTM, and
- RMSE of SRTM is 6.61m in floodplains.
These claims are like the claim that a basketball team with taller players is better than a team with shorter players.
Yes, height is important, but secondary to the players' many quirks & features.
DEM quality is similar, and its many quirks & features elude the statistics above.
DEM quality begins by considering how the DEM will be used — if you don't know how the DEM will be used, you can make no claims as to its quality / suitability.
This article introduces DEM Explorer, a graphing tool that lets you explore DEM quality and see for yourself that DEM quality cannot be captured by a single number.
DEM Explorer graphs the behavior of the 14 DEMs below, across 3 land covers and 4 slopes.
Each curve plots the distribution of error [ elevDEM - elevLiDAR ] between a DEM and highly accurate LiDAR ground truths.
Pan, zoom, pinch & swipe the graph to discover more quirks & features.
DEM Contenders
| DEM |
Version |
Released |
Notes |
| ASTER |
v003 |
2019-06 |
Put it out to pasture. |
| AW3D30 |
v3.2 (Feb 2022) |
2022-02 |
|
| COPernicus DEM 30 |
DGED 2022_1 |
2023-01 |
Pixels 32 bit float |
| COPernicus DEM 30 0.5m |
DGED 2022_1 |
2023-01 |
Pixels rounded to 0.5m |
| COPernicus DEM 30 1m |
DGED 2022_1 |
2023-01 |
Pixels rounded to 1m |
| COPernicus DEM 90 |
DGED 2022_1 |
2023-01 |
Pixels 32 bit float |
| COPernicus DEM 90 1m |
DGED 2022_1 |
2023-01 |
Pixels rounded to 1m |
| FABDEM |
V1-2 |
2023-01 |
COP30 with reduced forest & building bias |
| ICESat-2_2 |
v005 |
2023-02 |
h_te_uncertainty < 2 |
| ICESat-2_10 |
v005 |
2023-02 |
h_te_uncertainty < 10 |
| MERIT |
v1.0.3 |
2018-10 |
SRTM, with less forest bias |
| NASADEM |
HGT v001 |
2020 |
Reprocessed SRTM |
| SRTM |
SRTMGL1 v003 |
2016 |
Very popular |
| TDX90 |
v3 |
2016 |
Foundation of COP30 and COP90 |
Try for Yourself
Click a graph on the right to launch DEM Explorer and see
- why DEM quality cannot be captured by one statistic,
- where SRTM is superior to Copernicus DEM 30 (COP30),
- why NASADEM is a good replacement for SRTM,
- where COP30 is an asset and where it is a liability,
- where FABDEM amazingly reduces COP30 surface bias, but at the cost of exaggerating flooding risks,
- how rounding COP30 heights to integers (useful on resource constrained devices) affects quality (or not),
- where ICESat-2 could make an excellent global DEM, if more ATL08 segments can be captured,
- how much MERIT reduces SRTM forest bias,
- that SRTM has less forest bias than COP30,
- when COP90 is a good substitute for COP30, and
- why ASTER should be put out to pasture.
Methodology
We compare each DEM listed above to billions of LiDAR ground truths with a point density > 6 / m2.
Each colored curve on the graph is a distribution of error, created by comparing one DEM to all LiDAR ground truths with the same land cover and slope; eg. forest with moderate slope.
Each curve stresses the DEM in a different way, teasing out biases.
A smooth curve requires at least one million LiDAR ground truths; most curves use many more (billions in some cases) producing the smooth curves you see on the right.
These LiDAR ground truths have a vertical accuracy better than the height of a
chipmunk
(5 to 10 cm).
We quote RMSE, MAE and other statistics to 0.1m precision; a higher precision captures only terrain noise, like chipmunks, acorns and other ephemeral clutter.
ESA WorldCover 10m 2021 V200 identifies a land cover for each LiDAR elevation:
Grass / Crop, Forest or Developed.
(We combine Grassland and Cropland, as they present remote sensing with a similar, short and easily permeable surface.)
Slope is calculated
from a high-resolution 0.5m elevation grid created from the LiDAR ground truths, providing the most accurate ground slope possible:
| Label | Slope (%) |
|---|
| Level | < 1 |
| Gentle | 1 to 4 |
| Moderate | 4 to 12 |
| Steep | 12 to 100 |
Elevation Normalization (Geoid to Ellipsoid)
LiDAR and DEM surveys capture ellipsoidal elevation which are later converted to geoidal (eg. EGM96, EGM2008, NAVD88, CGG2013) for public use.
Our analysis require all LiDAR and DEM elevations normalized to the WGS84 ellipsoid.
Normalization applies an interpolation method (eg. bilinear, bicubic) to a geoid grid; each interpolation & grid size combination produces slightly different results.
Normalization error occurs if the combination we use does not match what was used when the elevation data was packaged for public use.
This error can vary from centimeters to meters.
A geoid's continuous surface is defined by spherical harmonic coefficients.
These coefficients are too computationally expensive to work with directly, so they are digitized once into a grid of pixels, which approximate the geoid's surface, and interpolated, on demand, to obtain geoid offsets.
Our normalization (from geoid to WGS84) should use the same grid size and interpolation as when the DEM was created.
However, only NASADEM publishes these details (ie. 15 arcsecond grid with linear interpolation).
We applied various geoid grid sizes and interpolation methods to COP90 to discover how it was derived from TanDEM-X 90m (ie. 60 arcsecond grid with linear interpolation); we assume the same for COP30 but cannot confirm because TanDEM-X does not publish a 1 arcsecond spacing DEM.
We used a 60 arcsecond grid and spline interpolation for other DEMs, as that is the interpolation method used by the US National Geospatial Intelligence Agency (masters of the geoid) in their calculations.
These details are important wherever the geoid undulates strongly, such as Hawaii, a place we are currently studying.
Conclusion
DEM quality is not a constant and depends on use-case, nature-of-bias, budget, license terms, file size and coverage area.
DEM Explorer can help you understand a DEM's nature-of-bias from comparisions with 627 billion LiDAR ground truths in
Southern Ontario
and 12.3 billion more in
Newfoundland,
Canada.
Marketing and scientific literature often use RMSE (root mean square error) as a proxy for DEM quality.
RMSE must be used with caution, because a few bad apples can spoil the results.
To that end, DEM Explorer provides >10m and >20m threshold statistics, measuring the percentage of bad apples (ie. percentage of error above 10m and 20m) which sends RMSE soaring.
DEM Explorer also provides MAE (mean absolute error) which is less sensitive to extreme outliers.
But, RMSE or MAE — alone — are as much a sign of DEM quality as player height is to basketball team quality.
What's the answer to replacing SRTM?
Switching DEMs is not a simple exercise.
A new DEM brings its own quirks & features that will improve some things and worsen others.
Will the mix of quirks & features net a positive outcome?
Copernicus DEM provides much more accurate surface elevations, useful for our work in wireless propagation analysis.
FABDEM is a derivative of Copernicus DEM that reduces this surface bias, which you need for flood analysis.
FABDEM performs this task well, at a cost of some negative bias.
As well, FABDEM has restrictive
terms of license
which a commercial application must consider.
We use DEMs for wireless propagation analysis, which favors a DEM that captures all surface clutter (forests, shrubs, but not chipmunks or acorns).
Other use-cases, like floodplain analysis, need a no-clutter DEM.
These and other quirks & features are what DEM Explore can help you discover, on your search for a better DEM.
![For Southern Ontario, Canada, distribution graph of error of SRTM DEM in [ forest moderate slope, rmse 5.4 mae 4.2 ] [ developed moderate rmse 3.0, mae 2.1 ] [ grass/crop level rmse 1.7, mae 1.3 ]](/img/news/2023/2023-03-02-CA_ON-SRTM.png)
![For Southern Ontario, Canada, distribution graph of error of Copernicus DEM 30 vs NASADEM grass/crop level slope [ COP30 rmse 0.9, mae 0.4 ] [ NASADEM rmse 1.7, mae 1.3 ]](/img/news/2023/2023-03-02-CA_ON-COP30-NASADEM.png)
![For Southern Ontario, Canada, distribution graph of error of Copernicus DEM 30 vs FABDEM forest moderate slope [ COP30 rmse 8.8, mae 7.3 ] [ FABDEM rmse 3.8, mae 2.8 ]](/img/news/2023/2023-03-02-CA_ON-COP30-FABDEM-forest.png)
![For Avalon Peninsula, Newfoundland, Canada, distribution graph of error of Copernicus DEM 30 vs FABDEM forest moderate slope [ COP30 rmse 8.8, mae 7.3 ] [ FABDEM rmse 3.8, mae 2.8 ]](/img/news/2023/2023-03-02-CA_NL-COP30-FABDEM-forest.png)
![For Southern Ontario, Canada, distribution graph of error of Copernicus DEM 30 with float and integer pixels in level forest surfaces [ float and integer rmse 7.3, mae 5.7 ]](/img/news/2023/2023-03-02-CA_ON-COP30-float-1m-forest-level.png)
![For Southern Ontario, Canada, distribution graph of error of Copernicus DEM 30 with float and integer pixels in level grass/crop terrain [ float rmse 0.9, mae 0.4 ] [ integer rmse 1.0 mae 0.5 ]](/img/news/2023/2023-03-02-CA_ON-COP30-float-1m-grass-level.png)
![For Southern Ontario, Canada, distribution graph of error of ICESat-2 with h_te_uncertainty less than 2 and 10 [ lt 2 rmse 1.17, mae 0.65 ] [ lt 10 rmse 1.73, mae 0.76 ]](/img/news/2023/2023-03-02-CA_ON-ICESAT2-forest-moderate.png)
![For Avalon Peninsula, Newfoundland, Canada, distribution graph of error of Copernicus DEM 90 and TanDEM-X 90 for level developed terrain [ COP90 rmse 1.2, mae 0.9 ] [ TDX90 rmse 1.6, mae 0.9 ]](/img/news/2023/2023-03-02-CA_NL-COP90-TDX90-developed-level.png)
