# Validation Report — Petrel Fuel Layer v1.0 Global FM40 Crosswalk

## Validation status: plausibility + US family agreement

**This is a deterministic crosswalk, so it is validated differently from a
trained model.** There is no held-out test set and no statistical accuracy
score, because the product is a fixed rule table rather than a fitted predictor.
What follows is an honest record of what *was* checked — biome plausibility
across the world's major fuel regimes, and family-level agreement against US
LANDFIRE ground truth — and what a categorical crosswalk deliberately does not
establish. Users who need a per-pixel accuracy figure should validate v1.0 in
their own region of interest first.

## Checks performed

### 1. Biome plausibility spot-checks

The crosswalk was run over sample windows in the world's major fuel biomes and
the assigned FM40 **family** compared against the physically expected family
(`petrel-global-fuel/scripts/crosswalk_prototype_validate.py`):

| Biome | Expected family | Crosswalk result |
|---|---|---|
| Borneo tropics | TL / TU timber | TL timber ✓ |
| Siberia taiga | TU / TL timber | TU timber ✓ |
| Mediterranean Spain | **SH shrub** (not grass) | SH shrub ✓ — the flagship fix |
| Kansas / Great Plains | GR grass | GR grass ✓ |
| Sahel | GR sparse grass | GR grass ✓ |
| Australian savanna | GR / GS grass-savanna | GR / GS ✓ |
| Brazil cerrado | GR / GS savanna (not timber) | GR / GS ✓ |
| Great Lakes / open water | NB non-burnable | NB ✓ |

The families land where fuel physics says they should — critically, the
Mediterranean now reads as **shrub (SH)**, correcting the ML line's
shrub-as-grass error, and open water reads as **non-burnable (NB)**, correcting
its water-as-grass error.

### 2. US in-domain family agreement vs LANDFIRE

Over CONUS test regions (Great Plains, Sierra Nevada, Southeast pine) the
crosswalk's FM40 assignment is compared against **LANDFIRE 2022 FBFM40 ground
truth** at the **family** level (NB / GR / GS / SH / TU / TL). LANDFIRE is
reprojected from EPSG:5070 to the working grid and compared cell-for-cell; the
family-agreement fraction is reported per region at build time. This is an
in-domain sanity check where an authoritative fuels map exists — not a global
accuracy claim.

### 3. Automated product integrity (promote audit)

The Petrel promote audit machine-checks the shipped raster: encoding tags
(`PETREL_ENCODING`, coast-clip provenance), overview pyramids, required
documentation, and land test points — Madrid, the Amazon, Siberia taiga, the
Kansas plains, and the Australian interior must all return a valid FM40 code
(not NoData), and coastal reference points (Carolina Beach NC, Malibu CA) must
resolve to land, not ocean. These are integrity checks on the shipped raster,
not accuracy checks on the crosswalk rules.

## What v1.0 does NOT establish

- **No per-pixel accuracy percentage or FM40-code confusion matrix.** Validation
  is at the family level and by plausibility; the specific in-family code (GR2
  vs GR3, TL2 vs TL9) is a climate-rule assignment, not an independently
  verified per-pixel label.
- **No global ground-truth comparison.** LANDFIRE is CONUS-only; outside the US
  there is no comparable authoritative FM40 map, so global agreement is argued
  from biome plausibility, not measured.
- **No canopy validation.** v1.0 ships no canopy attributes, so none are
  validated. (This is the main content difference from the ML line.)
- **No temporal validation.** WorldCover 2021 is a single epoch; post-2021
  disturbance is not reflected and not assessed.

## Known limitations

1. **Humid-C savanna over-read.** Without a canopy-height input, open savanna
   tree cover in humid temperate climates (Köppen Cf/Cw/Cs) is assigned
   closed-canopy timber-litter (TL). This is the crosswalk's main systematic
   soft spot; the v1.1 canopy layer is the fix.
2. **Coarse alpine / tundra transitions.** Köppen is ~1 km, so fuel transitions
   across elevation and at the tundra edge are blocky. v1.1 adds elevation.
3. **Family, not fuel load.** The reliable unit is the family; the in-family
   code is representative, not a measured fuel load.
4. **Class-only.** No canopy cover, height, base height, or bulk density ships
   in v1.0.
5. **~270 m native resolution, ~1 km climate driver.** Sub-cell fuel
   heterogeneity is unresolved. Not for parcel-level fuel typing.
6. **Coverage:** global land 60° S – 75° N. Ocean is NoData (GSHHG-f coast clip,
   `all_touched`); Antarctica is excluded by Petrel standard.

## v1.1 refinement plan

1. Add **real per-pixel canopy** (cover, stand height, canopy base height, bulk
   density) from the CC-BY **ETH global canopy-height** layer — replacing the
   climate proxy for open-vs-closed forest and resolving the humid-C savanna
   over-read.
2. Add **elevation** to sharpen alpine / tundra fuel transitions.
3. Extend the LANDFIRE family-agreement check to more CONUS ecoregions and,
   where authoritative national fuels maps exist, spot-check outside the US.

## How to validate locally

For your region of interest:

1. Overlay the FM40 family against any local fuels map or land-cover-derived
   fuel classification and check the families agree.
2. Confirm shrub-dominated and savanna-woodland landscapes read as SH / TU
   rather than grass or closed timber.
3. Reproject with **nearest-neighbour** — never bilinear — when comparing.
4. Report systematic biases to layers@petreldata.io — findings feed the v1.1
   plan.

## References

- Scott, J. H., & Burgan, R. E. (2005). Standard fire behavior fuel models.
  *USDA Forest Service, RMRS-GTR-153.* https://doi.org/10.2737/RMRS-GTR-153
- Zanaga, D., et al. (2022). ESA WorldCover 10 m 2021 v200.
  https://doi.org/10.5281/zenodo.7254221
- Beck, H. E., et al. (2018). Present and future Köppen-Geiger climate
  classification maps at 1-km resolution. *Scientific Data*, 5, 180214.
- LANDFIRE (2022). Scott and Burgan Fire Behavior Fuel Model 40 (FBFM40),
  CONUS. USGS / USFS.