Filling gaps in satellite imagery
Research on how geospatial foundation models can help reconstruct satellite images that were never taken sharply enough, including when parts of the data are missing.
Work in progress in Dr. Di Yang's GeoDI Lab (Google Scholar). This page describes the problem and my role; results stay with the lab until they are published.
01The problem
Earth-observation satellites such as MODIS and Landsat trade off how often they see a place against how sharply. Some sensors revisit daily at coarse resolution; others capture fine detail but only every couple of weeks, and clouds take out more of those passes. Spatiotemporal fusion predicts a fine-resolution image for a date where only coarse imagery exists, using a fine image from a nearby date as reference.
Real archives are also incomplete across the spectrum: a band (one of the colour or infrared channels a sensor records) can be missing or unusable. The project looks at how fusion holds up when that happens, and whether foundation models trained on many kinds of satellite data can help.
Diagram: a fine-resolution image from an earlier date and a coarse image from the target date are fused to predict a fine image at the target date, with one spectral band withheld.
02The model
We are studying a specific geospatial foundation model, a large model pretrained on many kinds of satellite data, which adjusts to the wavelengths of whichever bands it is given. One model can take imagery from different sensors and band combinations, which makes it a natural fit when bands are missing, a practical problem with real satellite archives. The project looks at how to apply it to fusion with incomplete input, and how well it performs there.
03The lab
The GeoDI Lab also works with NASA-supported citizen-science programmes, including EMERGE and GLOBE Observer, where volunteers submit environmental observations that feed research and teaching. More on the lab's citizen-science work.
Alongside the research I wrote ask-globe, a skill that lets an AI agent answer plain-language questions from live GLOBE Observer data, so people who don't write code can use the dataset too.
04Further reading
The satellites
- MODIS · NASA Earthdata
- MODIS spectral bands · NASA Earthdata
- Landsat · NASA Science
Spatiotemporal fusion
- On the blending of the Landsat and MODIS surface reflectance (STARFM) · Gao et al., Remote Sensing of Environment, 2006
- An enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM) · Zhu et al., Remote Sensing of Environment, 2010
- A flexible spatiotemporal method for fusing satellite images (FSDAF) · Zhu et al., Remote Sensing of Environment, 2016
Geospatial foundation models
- Neural plasticity-inspired multimodal foundation model for Earth observation (DOFA) · Xiong et al., arXiv, 2024
Missing and degraded bands
- Missing information reconstruction of remote sensing data: a technical review · Shen et al., IEEE Geoscience and Remote Sensing Magazine, 2015
- Restoration of Aqua MODIS band 6 using histogram matching and local least squares fitting · Rakwatin et al., IEEE Transactions on Geoscience and Remote Sensing, 2009
- Scientific impact of MODIS C5 calibration degradation and C6+ improvements · Lyapustin et al., Atmospheric Measurement Techniques, 2014