|
--- |
|
thumbnail: "https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/RSiM_Logo_1.png" |
|
tags: |
|
- mixer_b16_224 |
|
- BigEarthNet v2.0 |
|
- Remote Sensing |
|
- Classification |
|
- image-classification |
|
- Multispectral |
|
library_name: configilm |
|
license: mit |
|
widget: |
|
- src: example.png |
|
example_title: Example |
|
output: |
|
- label: Agro-forestry areas |
|
score: 0.000028 |
|
- label: Arable land |
|
score: 0.000100 |
|
- label: Beaches, dunes, sands |
|
score: 0.001113 |
|
- label: Broad-leaved forest |
|
score: 0.000188 |
|
- label: Coastal wetlands |
|
score: 0.000413 |
|
--- |
|
|
|
[TU Berlin](https://www.tu.berlin/) | [RSiM](https://rsim.berlin/) | [DIMA](https://www.dima.tu-berlin.de/menue/database_systems_and_information_management_group/) | [BigEarth](http://www.bigearth.eu/) | [BIFOLD](https://bifold.berlin/) |
|
:---:|:---:|:---:|:---:|:---: |
|
<a href="https://www.tu.berlin/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/tu-berlin-logo-long-red.svg" style="font-size: 1rem; height: 2em; width: auto" alt="TU Berlin Logo"/> | <a href="https://rsim.berlin/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/RSiM_Logo_1.png" style="font-size: 1rem; height: 2em; width: auto" alt="RSiM Logo"> | <a href="https://www.dima.tu-berlin.de/menue/database_systems_and_information_management_group/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/DIMA.png" style="font-size: 1rem; height: 2em; width: auto" alt="DIMA Logo"> | <a href="http://www.bigearth.eu/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/BigEarth.png" style="font-size: 1rem; height: 2em; width: auto" alt="BigEarth Logo"> | <a href="https://bifold.berlin/"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/BIFOLD_Logo_farbig.png" style="font-size: 1rem; height: 2em; width: auto; margin-right: 1em" alt="BIFOLD Logo"> |
|
|
|
# Mixer_b16_224 pretrained on BigEarthNet v2.0 using Sentinel-1 bands |
|
|
|
<!-- Optional images --> |
|
<!-- |
|
[Sentinel-1](https://sentinel.esa.int/web/sentinel/missions/sentinel-1) | [Sentinel-2](https://sentinel.esa.int/web/sentinel/missions/sentinel-2) |
|
:---:|:---: |
|
<a href="https://sentinel.esa.int/web/sentinel/missions/sentinel-1"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/sentinel_2.jpg" style="font-size: 1rem; height: 10em; width: auto; margin-right: 1em" alt="Sentinel-2 Satellite"/> | <a href="https://sentinel.esa.int/web/sentinel/missions/sentinel-2"><img src="https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/sentinel_1.jpg" style="font-size: 1rem; height: 10em; width: auto; margin-right: 1em" alt="Sentinel-1 Satellite"/> |
|
--> |
|
|
|
|
|
> **_NOTE:_** This version of the model has been trained with a different band order that is not compatible with the newer versions and does not match the order proposed in the technical documentation of Sentinel-2. |
|
> |
|
> The following bands (in the specified order) were used to train the models with version 0.1.1: |
|
> - For models using Sentinel-1 only: Sentinel-1 bands `["VH", "VV"]` |
|
> - For models using Sentinel-2 only: Sentinel-2 10m bands and 20m bands `["B02", "B03", "B04", "B08", "B05", "B06", "B07", "B11", "B12", "B8A"]` |
|
> - For models using Sentinel-1 and Sentinel-2: Sentinel-2 10m bands and 20m bands and Sentinel-1 bands = `["B02", "B03", "B04", "B08", "B05", "B06", "B07", "B11", "B12", "B8A", "VH", "VV"]` |
|
> |
|
> Newer models are compatible with the order in the technical documentation of Sentinel-2 and were trained with the following band order: |
|
> - For models using Sentinel-1 only: Sentinel-1 bands `["VV", "VH"]` |
|
> - For models using Sentinel-2 only: Sentinel-2 10m bands and 20m bands `["B02", "B03", "B04", "B05", "B06", "B07", "B08", "B8A", "B11", "B12"]` |
|
> - For models using Sentinel-1 and Sentinel-2: Sentinel-1 bands and Sentinel-2 10m bands and 20m bands `["VV", "VH", "B02", "B03", "B04", "B05", "B06", "B07", "B08", "B8A", "B11", "B12"]` |
|
|
|
This model was trained on the BigEarthNet v2.0 (also known as reBEN) dataset using |
|
the Sentinel-1 bands. |
|
It was trained using the following parameters: |
|
- Number of epochs: up to 100 (with early stopping after 5 epochs of no improvement based on validation average |
|
precision macro) |
|
- Batch size: 512 |
|
- Learning rate: 0.001 |
|
- Dropout rate: 0.15 |
|
- Drop Path rate: 0.15 |
|
- Learning rate scheduler: LinearWarmupCosineAnnealing for 1000 warmup steps |
|
- Optimizer: AdamW |
|
- Seed: 0 |
|
|
|
The weights published in this model card were obtained after 15 training epochs. |
|
For more information, please visit the [official BigEarthNet v2.0 (reBEN) repository](https://git.tu-berlin.de/rsim/reben-training-scripts), where you can find the training scripts. |
|
|
|
](https://raw.githubusercontent.com/wiki/lhackel-tub/ConfigILM/static/imgs/combined_2000_600_2020_0_wide.jpg) |
|
|
|
The model was evaluated on the test set of the BigEarthNet v2.0 dataset with the following results: |
|
|
|
| Metric | Macro | Micro | |
|
|:------------------|------------------:|------------------:| |
|
| Average Precision | 0.560443 | 0.740646 | |
|
| F1 Score | 0.507159 | 0.648535 | |
|
| Precision | 0.560443 | 0.740646 | |
|
|
|
# Example |
|
| A Sentinel-1 image (VV, VH and VV/VH bands are used for visualization) | |
|
|:---------------------------------------------------:| |
|
| ](example.png) | |
|
|
|
| Class labels | Predicted scores | |
|
|:--------------------------------------------------------------------------|--------------------------------------------------------------------------:| |
|
| <p> Agro-forestry areas <br> Arable land <br> Beaches, dunes, sands <br> ... <br> Urban fabric </p> | <p> 0.000028 <br> 0.000100 <br> 0.001113 <br> ... <br> 0.000067 </p> | |
|
|
|
|
|
To use the model, download the codes that define the model architecture from the |
|
[official BigEarthNet v2.0 (reBEN) repository](https://git.tu-berlin.de/rsim/reben-training-scripts) and load the model using the |
|
code below. Note that you have to install [`configilm`](https://pypi.org/project/configilm/) to use the provided code. |
|
|
|
```python |
|
from reben_publication.BigEarthNetv2_0_ImageClassifier import BigEarthNetv2_0_ImageClassifier |
|
|
|
model = BigEarthNetv2_0_ImageClassifier.from_pretrained("path_to/huggingface_model_folder") |
|
``` |
|
|
|
e.g. |
|
|
|
```python |
|
from reben_publication.BigEarthNetv2_0_ImageClassifier import BigEarthNetv2_0_ImageClassifier |
|
|
|
model = BigEarthNetv2_0_ImageClassifier.from_pretrained( |
|
"BIFOLD-BigEarthNetv2-0/mixer_b16_224-s1-v0.1.1") |
|
``` |
|
|
|
If you use this model in your research or the provided code, please cite the following papers: |
|
```bibtex |
|
@article{clasen2024refinedbigearthnet, |
|
title={reBEN: Refined BigEarthNet Dataset for Remote Sensing Image Analysis}, |
|
author={Clasen, Kai Norman and Hackel, Leonard and Burgert, Tom and Sumbul, Gencer and Demir, Beg{\"u}m and Markl, Volker}, |
|
year={2024}, |
|
eprint={2407.03653}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CV}, |
|
url={https://arxiv.org/abs/2407.03653}, |
|
} |
|
``` |
|
```bibtex |
|
@article{hackel2024configilm, |
|
title={ConfigILM: A general purpose configurable library for combining image and language models for visual question answering}, |
|
author={Hackel, Leonard and Clasen, Kai Norman and Demir, Beg{\"u}m}, |
|
journal={SoftwareX}, |
|
volume={26}, |
|
pages={101731}, |
|
year={2024}, |
|
publisher={Elsevier} |
|
} |
|
``` |
|
|
