README.md

CVRP: A Rice Image Dataset with High-Quality Annotations for Image Segmentation and Plant Phenomics Research

Multi-cultivar and multi-view rice plant image dataset (CVRP) consists of 2,303 field images with their annotated masks and 123 indoor images of individual panicles.

Annotation Workflow

To optimize the process of annotation, we combine deep learning methods with manual curation. The workflow comprises two stages: manual annotation and model-based prediction followed by manual curation.

Getting Started

Recommend python 3.7, CUDA v11.3 and Pytorch 1.10.0.

Clone MMSegmentation

git clone https://github.com/open-mmlab/mmsegmentation.git -b v1.1.2 

pip install -U openmim
mim install mmengine
mim install mmcv==2.0.0

pip install -r run/requirements.txt
cd mmsegmentation
pip install -v -e .

Creating a Dataset

1. Directory structure of the dataset:

   📁 CVRPDataset/
   ├─📁 images/
   └─📁 labelme_jsons/
   

2. Convert labelme files to mask:

   python run/labelme2mask.py
   

   now, the structure looks like:

   📁 CVRPDataset/
   ├─📁 img_dir/
   └─📁 ann_dir/
   

3. Split the training set and test set.

   python run/split_dataset.py
   

   now, the structure looks like:

   📁 CVRPDataset/
   ├─📁 img_dir/
   │  ├─📁 train/
   │  └─📁 val/
   └─📁 ann_dir/
      ├─📁 train/
      └─📁 val/
   

   You can download our training set and test set here.

Dataset Configs

cd mmseg/datasets
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/CVRP.py
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/__init__.py
cd ../../configs/_base_/datasets
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/CVRP_pipeline.py

If you want to register your own dataset,

1. Import and register datasets in mmseg/datasets/__init__.py

   from .CVRP import CVRPDataset
   

   Add,

   # other datasets
   __all__ = ['CVRPDataset']
   

2. Register dataset class in `mmseg/datasets/CVRP.py'

   class CVRPDataset(BaseSegDataset):
    METAINFO = {
        'classes':['background','panicle'],
        'palette':[[127,127,127],[200,0,0]]
    }
   

3. Modify pipeline of data process in config/_base_/CVRP_pipeline.py

   dataset_type = 'CVRPDataset' 
    data_root = 'CVRPDataset/'
   

   you'll need to specify the paths for the train and evalution data directories.

   # train_dataloader: 
   data_prefix=dict(img_path='img_dir/train', seg_map_path='ann_dir/train')
   # val_dataloader:
   data_prefix=dict(img_path='img_dir/val', seg_map_path='ann_dir/val')
   

Model Configs You can generate model config files using run_configs.py

mkdir 'work_dirs' 'CVRP_configs' 'outputs'
python ../run/run_configs.py --model_name deeplabv3plus -m configs/deeplabv3plus/deeplabv3plus_r101-d8_4xb4-160k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name knet -m configs/knet/knet-s3_swin-l_upernet_8xb2-adamw-80k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name mask2former -m configs/mask2former/mask2former_swin-l-in22k-384x384-pre_8xb2-160k_ade20k-640x640.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name segformer -m configs/segformer/segformer_mit-b5_8xb2-160k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

Also, you can download model config files here.

cd CVRP_configs
wget https://huggingface.co/CVRPDataset/Model/resolve/main/model_configs/CVRP_mask2former.py

Train

python tools/train.py CVRP_configs/CVRP_mask2former.py

Also, you can download checkpoint here.

Test

python ../run/test.py -d CVRPDataset/val -m CVRP_configs/CVRP_mask2former.py -pth work_dirs/CVRP_mask2former/Mask2Former.pth -o outputs/CVRP_mask2former

UI

We create a web user interface for annotation based on gradio:

python run/app.py

The UI :

1.Users can upload an image or use a sample image at ①.Then, they can select one of four models at ②. We recommend Mask2Former. After that, click Run. 2.We provide two forms of segmentation results for download at ③.

Labelme

If you need to manually adjust the annotation, you can use labelme.

python run/mask2json.py
pip install labelme==3.16.7
labelme

python run/json2png.py

Citation

Please considering cite our paper if you find this work useful!

Acknowledgements

We thank Mr.Zhitao Zhu, Dr. Weijie Tang, and Dr. Yunhui Zhang for their technical support.