import { pointMap, legend, blues, loadJSON, loadText, loadSeries, loadTrainAnim, observeTheme, seriesChart, harmonicChart, segmented, buttonRow, reconChart, embeddingStrip, trainFrame, scrubber, lossChart, ethiopiaChart, ethiopiaColors, rmseTable } from "./web/js/ui.js"
import { BANDS, windowIdx, fromDate } from "./web/js/core.js"
import { getModel } from "./web/js/model.js"
theme = Generators.observe(observeTheme)
points = loadJSON("points.json")
disforLabels = loadJSON("labels.json")
// years with enough clear acquisitions for a window, and the default (most acquisitions)
yearsOf = (series) => {
const n = d3.rollup(Array.from(series.day, (d) => new Date(d * 864e5).getUTCFullYear()), (v) => v.length, (y) => y);
const ys = [...n].filter(([, c]) => c >= 8).sort((a, b) => a[0] - b[0]);
return { years: ys.map(([y]) => y), best: d3.greatest(ys, ([, c]) => c)?.[0] };
}
yearStart = (y) => fromDate(new Date(Date.UTC(y, 0, 1)))
// the year an example button asked for, used while its point is selected
mutable preset = nullBudgetGFM
remote sensing
deep learning
Sentinel-2
Training a tiny geospatial foundation model on a single GPU and the CDSE free tier.
Everyone is doing Geospatial Foundation Models (GFMs) lately. See: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231]
The motivation for training these Foundation Models is pretty obvious: If you have a GFM which understands remote sensing data well, you need a lot less tedious manual feature engineering and less reference samples to create models which work well enough for a variety of tasks. Also in my opinion no model has really hit a home run with usability yet. This still leaves lots of room to innovate and come up with unique approaches.
Because of this and since all the cool kids are doing it, I want to join in on the fun.
There’s only one small issue: All the cool kids have large amounts of compute and training data available. I don’t. So let’s see what we can find laying around in the junk drawer to still do something.

Full to the brim! Let’s start with the training data, because this will guide which model architecture we go for.
Data
I’ve previously assembled a dataset for forest disturbances[232] it provides Sentinel-2 time-series and labels for 3772 samples in Europe. These are time-series for single pixels from 2015-2025 but only for forests. But let’s stick with single pixel time-series and just extend this dataset with other land use classes.
We use the Copernicus Dataspace Ecosystem (CDSE) free tier to download time-series for other land use classes1, based on the LUCAS dataset. CDSE is limited to 30000 Processing Units per month which allows us to extend the dataset with ~700 more time-series.
We will use the train/val split from the DISFOR dataset and just randomly split the extended LUCAS dataset into train/val.
Here’s the distribution, you can click on any point to see its time-series:
Model
I’ve previously worked a lot with time-series modelling of forests for anomaly detection. The bread-and-butter approach to do this is with harmonic regression models [233, 234, 235, 236]. Optical satellite time-series of basically everything on earth exhibit “phenology”. Meaning that the spectral response of a single point on earth changes through the year based on the season.
Click on basically any sample in the overview map to see this in action. Especially for forests this seasonal pattern is very predictable.
Harmonic regression models capture this phenology by fitting sines and cosines of different frequencies to the data using linear regression. The model outputs the intercept and two coefficients for each harmonic.
See the example below for how it works. You can click on any point on the map to visualize it here, or look through the examples.
This works and it actually already outputs something which can be thought of as a very basic form of an embedding. The model coefficients basically capture brightness of the pixel (intercept) and seasonality (harmonic coefficients). These coefficients were already benchmarked by AlphaEarth alongside other GFMs and performed surprisingly well for many tasks.
There are some downsides though:
- This model only works univariate. So coefficients would have to be created for every band individually
- Large number of harmonics can make the model unstable
- It does not capture correlation between spectral bands
So here’s the thinking: Do the same modelling of the phenology like a linear harmonic model but do it using a non-linear deep learning model which can learn general patterns and correlations between different spectral bands.
This is the model architecture:
- Encoder: A simple TransformerEncoder (405K Params) and TimeEncoder, which adds harmonics from 8 frequencies to each Sentinel-2 acquisition.
- Bottleneck: a single 32 dimension embedding for all acquisitions and bands
- Decoder: MLP head, gets the embedding and TimeEncoder to reconstruct the time-series.
The hope is that this 32 dimension embedding can more robustly capture time-series dynamics.
Training
Look at it learn!
The training approach is really simple2, for each epoch we sample a random year slice for each point, discard all cloudy or snowy acquisitions and sample 90% of the remaining acquisitions.
We back off on the learning rate from 1e-3 to 1e-6 during training.
Even after 1000 epochs, the model does not overfit to the train data, which is a good sign with that little training data.
Results
As seen above, the results for the validation set actually look quite good! But how does the model perform outside of Europe?
To answer that question, we used a colleagues CDSE account to sample 700 more points randomly over the entire land mass of Earth (-Antarctica). Below is a map of RMSE for the in-domain validation set, as well as the extra points.
This still works surprisingly well for being completely outside the training domain. There are only a few points in very arid regions like the Sahara or snowy regions with sparse time-series where you can see that the model really breaks down.
Fun fact: The model is so small that when you click around the validation points, only the Sentinel data is loaded. Encoding and inference is running on the fly, locally on your computer3.
If we compare the RMSE for the validation set vs the set of the entire earth, we can see that there isn’t a huge difference.
Finally we would like to run a benchmark and compare to other results. For that we picked ethiopia_crops as available in the eval suite for AlphaEarth.
In the AlphaEarth eval, probes on XY coordinates for the sample points performed unreasonably well. That makes the quality of that dataset a bit dubious. However in my opinion the evaluation approach for this dataset, carried out by AlphaEarth is no less dubious4.
However all other evals needed way more points than we had free CDSE credits for. So against better judgement here’s the results:
These results are… interesting? As said before, the classes are well stratified geographically. Because of this, classifying just based on distance to the training classes works well (see results for knn-1 and knn-3 coordinates).
One thing that stands out is that compared to the plot of the same benchmark published by AlphaEarth, the pre-computed AlphaEarth embeddings I am using here consistently show some of the worst results. In their paper, AlphaEarth reached above 0.4 balanced accuracy with each classifier, while here it struggles to get above 0.35.
The very basic harmonic regression coefficients based on NDMI worked really well with the k-NN heads. However far and away the best general results are by Tessera v1.1, which isn’t that surprising since they are specifically designed to handle crop classification well.
And how do we do? Not too bad actually for being out-of-domain and being way out-of-depth. We just barely get the second highest accuracy overall in the linear head with 49 training samples per class. We also do not degrade too badly in any of the other evaluations.
So what does this all mean?
Not much really I think. The siren song of foundation models is that you get a single vector which can be applied to many different tasks. Right now we only evaluated our model on a single task which fit its time-series based architecture well. I doubt that our model would generalize paricularly well to other tasks. This has one particular reason:
What I implemented here is a basic autoencoder structure. The whole field of image foundation models moved away from this model architecture a while ago because while it has an ability to capture structure, it tends to capture low level information (how bright, which color, which frequencies) instead of capturing semantics (dog, cat, banana). The solution was masked autoencoders. By masking out large parts of images, the model now really had to learn what was represented instead of just being good at regurgitating frequencies at different scales.
The fact that our model works reasonably well outside of its domain and does not overfit on the little amount of training data is a strong indication that we also “just” learn frequencies and physical dependencies with no higher level semantic understanding (agriculture, maize, wheat, forest, grassland, etc.).
Masked autoencoders have now also been extensively applied to remote sensing imagery. Both to single-pixel time-series[18], single timestamp imagery[47, 129], single-modal spatio-temporal time-series[70] and even more recently to multi-modal spatio-temporal time-series[115, 122, 201]. All of them approach masking slightly differently.
References
NoteAll the foundation models (and the other references)
[1]
[2]
[3]
“GASSL: Geography-Aware Self-Supervised Learning,” ICCV, 2021.
[4]
[5]
“RSP: An Empirical Study of Remote Sensing Pretraining,” TGRS, 2022.
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
“AST: Adaptive Self-supervised Transformer for Optical Remote Sensing Representation,” ISPRS JPRS, 2023.
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
“EarthPT: a time series foundation model for Earth Observation,” NeurIPS CCAI workshop, 2023.
[30]
[31]
“Cross-Scale MAE: A Tale of Multiscale Exploitation in Remote Sensing,” NeurIPS, 2023.
[32]
[33]
[34]
[35]
“Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps,” NeurIPSW, 2023.
[36]
[37]
[38]
[39]
[40]
[41]
[42]
“SpectralGPT: Spectral Remote Sensing Foundation Model,” TPAMI, 2024.
[43]
[44]
[45]
[46]
“BFM: A Billion-scale Foundation Model for Remote Sensing Images,” IEEE JSTARS, 2024.
[47]
“Clay Foundation Model,” Arxiv, 2024.
[48]
“Hydro–A Foundation Model for Water in Satellite Imagery,” Arxiv, 2024.
[49]
[50]
[51]
[52]
[53]
“MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining,” IEEE JSTARS, 2024.
[54]
[55]
[56]
[57]
[58]
[59]
[60]
[61]
[62]
[63]
[64]
[65]
[66]
“FG-MAE: Feature Guided Masked Autoencoder for Self-Supervised Learning in Remote Sensing,” IEEE JSTARS, 2024.
[67]
[68]
[69]
[70]
[71]
[72]
[73]
[74]
“RemoteCLIP: A Vision Language Foundation Model for Remote Sensing,” IEEE TGRS, 2024.
[75]
[76]
[77]
[78]
[79]
[80]
[81]
[82]
[83]
[84]
[85]
[86]
[87]
[88]
[89]
[90]
[91]
“HSIGene: A Foundation Model For Hyperspectral Image Generation,” Arxiv, 2024.
[92]
“GeoBind: Binding Text, Image, and Audio through Satellite Images,” IGARSS, 2024.
[93]
[94]
“GeoLLM-QA: Evaluating Tool-Augmented Agents in Remote Sensing Platforms,” ICLR ML4RS Workshop, 2024.
[95]
[96]
[97]
[98]
[99]
[100]
[101]
“SARATR-X: Toward Building a Foundation Model for SAR Target Recognition,” IEEE TIP, 2025.
[102]
[103]
[104]
[105]
[106]
“HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model,” IEEE TPAMI, 2025.
[107]
“HyperSL: A Spectral Foundation Model for Hyperspectral Image Interpretation,” IEEE TGRS, 2025.
[108]
[109]
“Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation,” CVPRW (EarthVision Best Paper), 2025.
[110]
[111]
“SpectralEarth: Training Hyperspectral Foundation Models at Scale,” IEEE JSTARS, 2025.
[112]
[113]
[114]
[115]
“Galileo: Learning Global & Local Features of Many Remote Sensing Modalities,” ICML TerraBytes Workshop, 2025.
[116]
[117]
“RingMamba: Remote Sensing Multisensor Pretraining With Visual State Space Model,” IEEE TGRS, 2025.
[118]
[119]
“SeaMo: A Multi-Seasonal and Multimodal Remote Sensing Foundation Model,” Information Fusion, 2025.
[120]
“MoSAiC: Multi-Modal Multi-Label Supervision-Aware Contrastive Learning for Remote Sensing,” IEEE Sensors Journal, 2025.
[121]
[122]
[123]
“SkySense++: A semantic-enhanced multi-modal remote sensing foundation model for Earth observation,” Nature Machine Intelligence, 2025.
[124]
[125]
[126]
[127]
[128]
[129]
[130]
[131]
[132]
[133]
[134]
“RoMA: Scaling up Mamba-based Foundation Models for Remote Sensing,” NeurIPS, 2025.
[135]
[136]
[137]
[138]
[139]
[140]
[141]
“Falcon: A Remote Sensing Vision-Language Foundation Model,” Arxiv, 2025.
[142]
[143]
[144]
[145]
[146]
[147]
“SegEarth-R1: Geospatial Pixel Reasoning via Large Language Model,” Arxiv, 2025.
[148]
[149]
[150]
[151]
[152]
“RSGPT: A Remote Sensing Vision Language Model and Benchmark,” ISPRS JPRS, 2025.
[153]
[154]
[155]
[156]
[157]
[158]
[159]
[160]
[161]
[162]
[163]
[164]
[165]
[166]
[167]
[168]
“RemoteSAM: Towards Segment Anything for Earth Observation,” ACMMM, 2025.
[169]
[170]
[171]
[172]
[173]
[174]
[175]
[176]
[177]
[178]
[179]
[180]
“Sat2Sound: A Unified Framework for Zero-Shot Soundscape Mapping,” Arxiv, 2025.
[181]
[182]
“Towards LLM Agents for Earth Observation: The UnivEARTH Dataset,” Arxiv, 2025.
[183]
[184]
[185]
[186]
[187]
[188]
“FlexiMo: A Flexible Remote Sensing Foundation Model,” IEEE TGRS, 2026.
[189]
“MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Models,” IEEE TGRS, 2026.
[190]
[191]
[192]
[193]
“Alliance: All-in-One Spectral-Spatial-Frequency Awareness Foundation Model,” IEEE TPAMI, 2026.
[194]
[195]
[196]
[197]
[198]
[199]
[200]
[201]
[202]
[203]
“SARMAE: Masked Autoencoder for SAR Representation Learning,” CVPR, 2026.
[204]
“Aquila: A Hierarchically Aligned Vision-Language Model for Enhanced Remote Sensing Image Comprehension,” Journal of Remote Sensing, 2026.
[205]
[206]
[207]
[208]
[209]
[210]
[211]
[212]
[213]
[214]
[215]
[216]
[217]
[218]
[219]
[220]
[221]
[222]
[223]
[224]
[225]
[226]
[227]
[228]
[229]
[230]
[231]
[232]
J. Viehweger et al., “Dense labelled time-series for mapping european forest disturbance agents,” Earth System Science Data Discussions, 2026.
[233]
E. B. Brooks et al., “On-the-Fly Massively Multitemporal Change Detection Using Statistical Quality Control Charts and Landsat Data,” IEEE Transactions on Geoscience and Remote Sensing, 2014.
[234]
Z. Zhu and C. E. Woodcock, “Continuous change detection and classification of land cover using all available Landsat data,” Remote Sensing of Environment, 2014.
[235]
J. Verbesselt, A. Zeileis, and M. Herold, “Near real-time disturbance detection using satellite image time series,” Remote Sensing of Environment, 2012.
[236]
L. Dutrieux and J. Viehweger, “Nrt: Operational monitoring of satellite image time-series in Python,” Journal of Open Source Software, 2024.
Footnotes
I’m using CDSE because it is the only (?) provider with all Sentinel-2 scenes reprocessed to > baseline 4. For anyone interested, I’ve written previously about how to extract time-series from CDSE↩︎
We did do some ablations to arrive at our set of model size and embedding dimensions. I haven’t prepared the data properly, but if there’s any interest I can cobble together a table.↩︎
Phone? Haven’t tested it.↩︎
I decided that this will be its own blog post. But the gist is, that the eval as proposed by Google AlphaEarth evaluates on the wrong growing season for around half the sampled points↩︎