EFAST fusion with phenocam validation.

End-to-end pipeline from selecting sites from the global PhenoCam Network to run EFAST spatio-temporal fusion with Sentinel-2 / Sentinel-3 and validate GCCs accross sensors. The numbered steps cover site selection, Sentinel data acquisition, different fusion orders, accuracy metrics, and sample-level statistics, all feeding a static web QA viewer.

---

Pipeline overview

Step	Script	What it does
1	1-phenocam.py	Download PhenoCam metadata and one_day_summary GCC CSVs
2	2-phenocam-screening.py	Apply PhenoCam count, SNR, and proximity gates to select feasible sites
3	3-sentinel-data.py	Acquire S2 (Earth Search COG) and S3 OLCI SYN L2 (CDSE OpenEO); prepare REFL, DIST_CLOUD, and composite GeoTIFFs
4	4-fusion.py	Run EFAST BtI (fuse reflectance → GCC) and ItB (fuse GCC directly) for each screened site
5	5-metrics.py	Extract PhenoCam-matched timeseries, compute NSE/RMSE/r baselines and fusion metrics, emit per-site JSON and webapp manifest
6	6-statistics-fusion-order.py	Paired ItB-vs-BtI significance test (Wilcoxon + t-test) across all sites

---

Quick start

Run pipeline wrapper (recommended)

uv sync
uv run python run-pipeline.py --evaluation-year 2025

Runs all five steps in order. Steps 1 and 2 are skipped when their output already exists. Each site in steps 3–5 is skipped when data/metrics/{year}/{site}/metrics.json is present. Any failure stops the run immediately, so one can fix the issue and re-run; completed work is never repeated.

# single site (steps 1 and 2 still skip if already done)
uv run python run-pipeline.py --evaluation-year 2025 --site ICOSFR-Fon1

Step by step

uv sync
uv run python 1-phenocam.py --evaluation-year 2025
uv run python 2-phenocam-screening.py --evaluation-year 2025
uv run python 3-sentinel-data.py --evaluation-year 2025
uv run python 4-fusion.py --evaluation-year 2025
uv run python 5-metrics.py --evaluation-year 2025
uv run python 6-statistics-fusion-order.py --evaluation-year 2025

Steps 1–5 accept --evaluation-year (default 2025) and --site (optional, for single-site runs). Step 6 is a full-sample aggregate and only accepts --evaluation-year and --alpha (default 0.05). Steps 3–5 are resumable — existing output files are skipped.

# single site
uv run python 3-sentinel-data.py --evaluation-year 2025 --site ICOSFR-Fon1
uv run python 4-fusion.py --evaluation-year 2025 --site ICOSFR-Fon1
uv run python 5-metrics.py --evaluation-year 2025 --site ICOSFR-Fon1

Credentials

Step 3 S3 download uses CDSE OpenEO (SENTINEL3_SYN_L2_SYN). Set CDSE_USER and CDSE_PASSWORD in ../.env at the workspace root. S2 uses AWS Earth Search COG range reads (no auth required).

---

Outputs (under data/)

Artifact	Step	Role
phenocam/{year}.json	1	Site list + sites_dir pointer
phenocam/{year}/{site}.json, {site}_1day.csv	1	Raw API payload and GCC CSV
phenocam_screening/{year}.json / .csv	2	Gate results (pass/fail per site)
sentinel_data/{year}/{site}/prepared/s2/	3	S2 REFL + DIST_CLOUD GeoTIFFs
sentinel_data/{year}/{site}/prepared/s3/	3	S3 composite GeoTIFFs
fusion/{year}/{site}/bti/, .../itb/	4	BtI fused reflectance + GCC; ItB fused GCC
metrics/{year}/{site}/	5	Per-site timeseries, metrics, covariates JSON
metrics/manifest.json	5	Webapp manifest (years + site metadata)
statistics_fusion_order/{year}.json	6	Paired ItB-vs-BtI test summary (NSE, RMSE, nRMSE, r)

---

Web viewer

python3 -m http.server 8080 runs the webapp on http://localhost:8000/index.html. Requires step 5 output (data/metrics/manifest.json).

---

License

MIT