Update README.md

README.md

@@ -2,17 +2,19 @@
 license: mit
 ---
 
-# OmniSat: Self-Supervised Modality Fusion for Earth Observation (ECCV 2024)
+# AnySat: An Earth Observation Model for Any Resolutions, Scales, and Modalities (ArXiv 2024)
 
 [Guillaume Astruc](https://gastruc.github.io/), [Nicolas Gonthier](https://ngonthier.github.io/), [Clement Mallet](https://www.umr-lastig.fr/clement-mallet/), [Loic Landrieu](https://loiclandrieu.com/)  
 
 
-Official models for [_OmniSat: Self-Supervised Modality Fusion for Earth Observation_](https://arxiv.org/pdf/2404.08351.pdf)
+Official models for [_AnySat: An Earth Observation Model for Any Resolutions, Scales, and Modalities_](https://arxiv.org/pdf/2404.08351.pdf)
 
 ## Abstract
 
-We introduce OmniSat, a novel architecture that exploits the spatial alignment between multiple EO modalities to learn expressive multimodal representations without labels. We demonstrate the advantages of combining modalities of different natures across three downstream tasks (forestry, land cover classification, and crop mapping), and  propose two augmented datasets with new modalities: PASTIS-HD and TreeSatAI-TS.
-For more details and results, please check out our [github](https://github.com/gastruc/OmniSat) and [project page](https://gastruc.github.io/projects/omnisat.html).  
+We introduce AnySat: a JEPA-based multimodal Earth Observation model that train simultaneously on diverse datasets with different scales, resolutions (spatial, spectral, temporal), and modality combinations.
+
+
+For more details and results, please check out our [github](https://github.com/gastruc/AnySat) and [project page](https://gastruc.github.io/projects/omnisat.html).  
 
 <p align="center">
   <img src="https://github.com/gastruc/OmniSat/assets/1902679/9fc20951-1cac-4891-b67f-53ed5e0675ad" width="800" height="400">
@@ -48,23 +50,32 @@ To get features from an observation of a batch of observations, you need to prov
   - "aerial-flair": Single date tensor (Bx5xHxW) with 5 channels (RGB NiR Elevation), 0.2m resolution
   - "spot": Single date tensor (Bx3xHxW) with 3 channels (RGB), 1m resolution
   - "naip": Single date tensor (Bx4xHxW) with 3 channels (RGB), 1.25m resolution
-  - "s2": Time series tensor (BxTx10xHxW) with 10 channels (B0,B1???), 10m resolution
+  - "s2": Time series tensor (BxTx10xHxW) with 10 channels (B2 B3 B4 B5 B6 B7 B8 B8a B11 B12), 10m resolution
   - "s1-asc": Time series tensor (BxTx2xHxW) with 2 channels (VV VH), 10m resolution
-  - "s1": Time series tensor (BxTx3xHxW) with 3 channels, 10m resolution
-  - "alos": Time series tensor (BxTx3xHxW) with 3 channels, 30m resolution
-  - "l7": Time series tensor (BxTx6xHxW) with 6 channels, 30m resolution
-  - "l8": Time series tensor (BxTx11xHxW) with 11 channels, rescaled to 10m resolution
-  - "modis": Time series tensor (BxTx7xHxW) with 7 channels, 250m resolution
-
-
+  - "s1": Time series tensor (BxTx3xHxW) with 3 channels (VV VH Ratio), 10m resolution
+  - "alos": Time series tensor (BxTx3xHxW) with 3 channels (HH HV Ratio), 30m resolution
+  - "l7": Time series tensor (BxTx6xHxW) with 6 channels (B1 B2 B3 B4 B5 B7), 30m resolution
+  - "l8": Time series tensor (BxTx11xHxW) with 11 channels (B8 B1 B2 B3 B4 B5 B6 B7 B9 B10 B11), rescaled to 10m resolution
+  - "modis": Time series tensor (BxTx7xHxW) with 7 channels (B1 B2 B3 B4 B5 B6 B7), 250m resolution
 
 Time series keys require a "{key}_dates" (for example "s2_dates") tensor of size BxT that value an integer that represent the day of the year.
+Then you have to choose at which scale you want te produce features. Scale argument is in meters and represent the size of the desired patch size. 
+Outputs will be composed of the concatenation of a class token and a flattened feature map where each feature encodes a scale x scale zone 
 Then, you can run:
 
 ```python
-features = AnySat(data)
+features = AnySat(data, scale=scale) #
+```
+
+And then you can apply those features to the desired downstream task!
+
+If you want to get a feature map at the density of a specific modality you can specify:
+
+```python
+features = AnySat(data, scale=scale, keep_subpatch=True, modality_keep=modality) #where modality is the name of the desired modality
 ```
-And then apply those features to the desired downstream task
+
+Note that the features will be of size 2*D. If you have several modalities of the same desired resolution, you should pick the most informative one (or modify the code to concatenante also the other modalities)
 
 To reproduce results, add new modalities, or do more experiments see the full code on [github]('https://github.com/gastruc/AnySat').