Why Your Satellite Imagery Model Works in the Notebook and Fails Over a New Region
A road- or infrastructure-detection model that hits strong validation numbers on one region and then degrades noticeably the moment it's pointed at imagery from a different city isn't a bad model — it's what happens by default when a geospatial pipeline is built and validated on too narrow a slice of the earth's actual variety.
The environment is the adversary, not the architecture
Atmospheric conditions, seasonal foliage changes, cloud cover, and sensor calibration differences between satellite passes create real variability in image quality and spectral characteristics — and a model trained on a clean, cloud-free summer dataset from one region will quietly degrade when deployed against winter imagery, a different sensor, or a region with a different building density and material palette. This is domain shift, and in geospatial work it's closer to the default condition than the exception, because unlike a factory floor, you can't control the lighting on a satellite pass.
The bar is higher than standard computer vision benchmarks
Production geospatial systems — infrastructure monitoring, land-use classification, damage assessment — typically need intersection-over-union scores well above what's considered a strong result in general object detection, because the downstream use case (a road-network record, an insurance claim, a permitting decision) doesn't tolerate the same margin of error a product recommendation does. That bar interacts badly with the annotation problem: labeling satellite imagery accurately requires specialized expertise, it's slow, and it's expensive, which means most teams have less high-quality labeled data than the accuracy target actually requires.
- Train and validate resolution should match deployment resolution — ultra-fine training data can introduce noise that doesn't match what the model sees in production.
- Hold out validation imagery from a genuinely different region and season, not a random split of the same collection pass.
- Budget for spatio-temporal drift monitoring after deployment — a model's performance on new geography isn't static, it degrades as conditions diverge from training.
- Treat annotation quality as a first-class cost, not a preprocessing line item — inter-annotator disagreement on complex geospatial labels caps model performance before training even starts.
None of this means geospatial computer vision doesn't work in production — it clearly does, at real scale, for infrastructure and land-monitoring use cases. It means the validation set has to be built to lie to you as little as possible, by deliberately including the regional, seasonal, and sensor variation the model will actually face — because a single clean benchmark number from one region tells you almost nothing about how the model behaves on the next one.