HYPERVIEW - Vision Transformer Model (https://ai4eo.eu/challenge/hyperview-challenge/):
This repository is based on the original code from:
https://github.com/ridvansalihkuzu/hyperview_eagleeyes/tree/master/experimental_1
Below are the instructions to set up the environment and run the code:
Table of Contents
Setup and Usage
Loading the Pre-Trained Model
To load a pre-trained model ("VisionTransformer.pt"), use the following code snippet:
import clip
import torch # Make sure to import torch
from clip.downstream_task import TaskType
device = "cpu" # Change to 'cuda' if you have a GPU
num_classes = 4 # Number of classes in the original HYPERVIEW dataset
# Load the CLIP model with the downstream task configuration
model, _ = clip.load(
"ViT-L/14", device, downstream_task=TaskType.HYPERVIEW,
class_num=num_classes
)
# Load the pre-trained weights
model.load_state_dict(torch.load("VisionTransformer.pt"))
model.eval() # Set the model to evaluation mode
Loading training data
import numpy as np
from clip.hyperview_data_loader import HyperDataloader, DataReader
im_size = 224 # Image size
num_classes = 4 # Number of classes in the original HYPERVIEW dataset
# Paths to training data and ground truth
train_path = "<TRAIN_PATH>"
train_gt_path = "<TRAIN_PATH>/train_gt.csv"
# Initialize the dataset reader and transformations
target_index = list(np.arange(num_classes))
trans_tr, _ = HyperDataloader._init_transform(im_size)
train_dataset = DataReader(
database_dir=train_path, label_paths=train_gt_path,
transform=trans_tr, target_index=target_index
)
Acknowledgment
This model is based on the Vision Transformer architecture developed by Google Research, as detailed in their repository Vision Transformer. The original models were trained on the ImageNet and ImageNet-21k datasets and are licensed under the Apache License, Version 2.0.
We would like to express our gratitude to the authors and contributors of the Vision Transformer project for their valuable work, which has significantly influenced my model's development. For more information on the license and usage, please refer to the Apache License 2.0.
Citations:
User Request
If you feel that this model is useful in your research, we would appreciate if you could refer to the following paper while using the model:
[1] J. Nalepa et al., "Estimating Soil Parameters From Hyperspectral Images: A benchmark dataset and the outcome of the HYPERVIEW challenge," in IEEE Geoscience and Remote Sensing Magazine, DOI: 10.1109/MGRS.2024.3394040.