---
license: apache-2.0
---

# HYPERVIEW - Vision Transformer Model (https://ai4eo.eu/challenge/hyperview-challenge/):

This repository is based on the original code from:

https://github.com/ridvansalihkuzu/hyperview_eagleeyes/tree/master/experimental_1

Below are the instructions to set up the environment and run the code:

## Table of Contents

- [Setup and Usage](#setup-and-usage)
  - [Loading the Pre-Trained Model](#loading-the-pre-trained-model)
  - [Loading the Training Data](#loading-the-training-data)
  - [Code Modifications](#code-modifications)
- [Environment Setup](#environment-setup)

## Setup and Usage

### Loading the Pre-Trained Model

To load a pre-trained model ("VisionTransformer.pt"), use the following code snippet:

```python
import clip
import torch  # Make sure to import torch

from clip.downstream_task import TaskType


device = "cpu"  # Change to 'cuda' if you have a GPU
num_classes = 4  # Number of classes in the original HYPERVIEW dataset

# Load the CLIP model with the downstream task configuration
model, _ = clip.load(
    "ViT-L/14", device, downstream_task=TaskType.HYPERVIEW,
    class_num=num_classes
)

# Load the pre-trained weights
model.load_state_dict(torch.load("VisionTransformer.pt"))
model.eval()  # Set the model to evaluation mode
```

### Loading training data

```python
import numpy as np

from clip.hyperview_data_loader import HyperDataloader, DataReader


im_size = 224  # Image size
num_classes = 4  # Number of classes in the original HYPERVIEW dataset

# Paths to training data and ground truth
train_path = "<TRAIN_PATH>"
train_gt_path = "<TRAIN_PATH>/train_gt.csv"

# Initialize the dataset reader and transformations
target_index = list(np.arange(num_classes))
trans_tr, _ = HyperDataloader._init_transform(im_size)
train_dataset = DataReader(
    database_dir=train_path, label_paths=train_gt_path,
    transform=trans_tr, target_index=target_index
)
````

## Acknowledgment

This model is based on the Vision Transformer architecture developed by Google Research, as detailed in their repository [Vision Transformer](https://github.com/google-research/vision_transformer). The original models were trained on the ImageNet and ImageNet-21k datasets and are licensed under the Apache License, Version 2.0.

We would like to express our gratitude to the authors and contributors of the Vision Transformer project for their valuable work, which has significantly influenced my model's development. For more information on the license and usage, please refer to the [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0).

**Citations:**
1. [Acknowledgment Note](https://github.com/google-research/vision_transformer?tab=Apache-2.0-1-ov-file#readme)
2. [Vision Transformer GitHub Repository](https://github.com/google-research/vision_transformer)

## User Request

If you feel that this model is useful in your research, we would appreciate if you could refer to the following paper while using the model:

[1] J. Nalepa et al., "Estimating Soil Parameters From Hyperspectral Images: A benchmark dataset and the outcome of the HYPERVIEW challenge," in IEEE Geoscience and Remote Sensing Magazine, DOI: 10.1109/MGRS.2024.3394040.

(https://ieeexplore.ieee.org/document/10526314)