NASA/IBM paper

[brunosan]

FYI, the NASA/IBM/Clark geospatial model paper just came out.

It would be interesting to see what we can learn from them for our model.

https://arxiv.org/pdf/2310.18660.pdf

Cc @danhammer

[brunosan]

Great read. It has many relevant bits of experience and information.
Bottom line to me is that they validated the approach, and points towards much better utility as we increase the training data, which our effort is.

One of the interesting bits is that the foundational model proved also useful when fine tuned on higher resolution data.

Technical Note: Update on Foundation Models for Geospatial AI Paper

Pretraining Details: (edited GPT-4 summary)

• Optimizer & Learning Rate: AdamW with β1 = 0.9, β2 = 0.999; one-cycle cosine learning rate scheduler, max learning rate 5e-4.
• Model Architecture: Experiments with ViT-base and ViT-large backbones.
• Training Epochs: 1000 epochs.
• Image & Patch Size: Input image size 224 × 224; patch size 1 × 16 × 16 (time × x × y).
• Data Bands: Six HLSL30 bands used (B02, B03, B04, B05, B06, B07).
• GPUs Used: Training on IBM watsonx platform with up to 64 NVIDIA A100 GPUs.
• Data Loading Efficiency: Utilized Zarr files for efficient data handling.
• Minimal Training Loss Achieved: MSE loss of 0.0283.

Dataset Specifics for Multi-Temporal Cloud Gap Imputation: (edited GPT-4 summary)

• Data Collection Area: Contiguous United States, 2022.
• Data Specification: TIFF files, 224 x 224 pixels, 30 meters resolution, 6 spectral bands, 3 temporal snapshots (18 total channels).
• Data Selection Criteria: Less than 5% cloud coverage, zero missing values.
• Final Dataset: 7852 image chips.

Fine-Tuning Module:(edited GPT-4 summary)

• Module Basis: Pytorch and mmsegmentation library, customized for spatiotemporal data.
• Decoder Architecture: Lightweight, 4 ConvTranspose2D layers, single 2D convolutional layer.
• Fine-Tuning Hyperparameters:
  • Batch size: 4 (flood mapping, wildfire scars), 8 (multi-temporal crop segmentation).
  • Epochs: 60 (flood mapping), 50 (wildfire scars), 80 (multi-temporal crop segmentation).
  • Learning rate: 6e-5 (flood mapping), 1.3e-05 (wildfire scars), 1.5e-05 (multi-temporal crop segmentation).
  • Loss metrics: Weighted cross entropy (flood mapping, crop segmentation), unweighted dice loss (wildfire scars).
  • Evaluation Metrics: IoU, F1-score, Accuracy.
• Cloud Gap Imputation:
  • Batch size: 16.
  • Learning rate: 1e-4.
  • Epochs: Max 200.
  • Loss metric: RMSE.
  • Evaluation metrics: RMSE, MAE, SSIM.

Gap Imputation Task Specifics(edited GPT-4 summary):

• Model Modification: Prithvi 100M model encoder adapted for input masks.
• Baseline Comparison: Modified CGAN architecture, derived from Pix2Pix.
• Learning Rates: 1.0e-4 (discriminator), 5.0e-4 (generator), with an alpha of 5 (weighting of MSE loss).