CVRPDataset/Model

CVRP: A Rice Image Dataset with High-Quality Annotations for Image Segmentation and Plant Phenomics Research

Multi-cultivar and multi-view rice plant image dataset (CVRP) consists of 2,303 field images with their annotated masks and 123 indoor images of individual panicles.

Annotation Workflow

To optimize the process of annotation, we combine deep learning methods with manual curation. The workflow comprises two stages: manual annotation and model-based prediction followed by manual curation.

Getting Started

Recommend python 3.7, CUDA v11.3 and Pytorch 1.10.0.

Clone

git clone https://huggingface.co/CVRPDataset/Model
cd Model
git clone https://github.com/open-mmlab/mmsegmentation.git -b v1.1.2 mmsegmentation

pip install -U openmim
mim install mmengine
mim install mmcv==2.0.0

pip install -r run/requirements.txt
cd mmsegmentation
pip install -v -e .

Creating a Dataset Here is an example if you want to make own dataset.

1. Directory structure of the dataset:

   📁 CVRPDataset/
   ├─📁 images/
   └─📁 labelme_jsons/
   

2. Convert labelme files to mask:

   python run/labelme2mask.py
   

   now, the structure looks like:

   📁 CVRPDataset/
   ├─📁 img_dir/
   └─📁 ann_dir/
   

3. Split the training set and test set.

   python run/split_dataset.py
   

   now, the structure looks like:

   📁 CVRPDataset/
   ├─📁 img_dir/
   │  ├─📁 train/
   │  └─📁 val/
   └─📁 ann_dir/
      ├─📁 train/
      └─📁 val/
   

   You can download our training set and validation set here.

Dataset Configs

cd mmsegmentation/mmseg/datasets
rm -rf __init__.py                  # delete original file
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/CVRP.py
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/__init__.py  
cd ../../configs/_base_/datasets
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/CVRP_pipeline.py

If you want to register your own dataset,

1. Import and register datasets in mmseg/datasets/__init__.py

   from .CVRP import CVRPDataset
   

   Add,

   # other datasets
   __all__ = ['CVRPDataset']
   

2. Register dataset class in `mmseg/datasets/CVRP.py'

   class CVRPDataset(BaseSegDataset):
    METAINFO = {
        'classes':['background','panicle'],
        'palette':[[127,127,127],[200,0,0]]
    }
   

3. Modify pipeline of data process in config/_base_/CVRP_pipeline.py

   dataset_type = 'CVRPDataset' 
    data_root = 'CVRPDataset/'
   

   you'll need to specify the paths for the train and evalution data directories.

   # train_dataloader: 
   data_prefix=dict(img_path='img_dir/train', seg_map_path='ann_dir/train')
   # val_dataloader:
   data_prefix=dict(img_path='img_dir/val', seg_map_path='ann_dir/val')
   

Model Configs

You can generate model config files using run_configs.py

cd mmsegmentation
mkdir 'work_dirs' 'CVRP_configs' 'outputs' 'CVRPDataset'
python ../run/run_configs.py --model_name deeplabv3plus -m configs/deeplabv3plus/deeplabv3plus_r101-d8_4xb4-160k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name knet -m configs/knet/knet-s3_swin-l_upernet_8xb2-adamw-80k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name mask2former -m configs/mask2former/mask2former_swin-l-in22k-384x384-pre_8xb2-160k_ade20k-640x640.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name segformer -m configs/segformer/segformer_mit-b5_8xb2-160k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

Also, you can download model config files here.

cd mmsegmentation/CVRP_configs
wget https://huggingface.co/CVRPDataset/Model/resolve/main/model_configs/CVRP_mask2former.py

Train

cd mmsegmentation 
python tools/train.py CVRP_configs/CVRP_mask2former.py

Also, you can download checkpoint here.

Test

python ../run/test.py -d mmsegmentation/CVRPDataset/val -m mmsegmentation/CVRP_configs/CVRP_mask2former.py -pth mmsegmentation/work_dirs/CVRP_mask2former/Mask2Former.pth -o mmsegmentation/outputs/CVRP_mask2former

UI

We create a web user interface for annotation based on gradio:

python app.py

The UI :

1.Users can upload an image or use a sample image at ①.Then, they can select one of four models at ②. We recommend Mask2Former. After that, click Run. 2.We provide two forms of segmentation results for download at ③.

LabelMe

If you need to manually adjust the annotation, you can use LabelMe.

python run/mask2json.py
pip install labelme==3.16.7
labelme

python run/json2png.py

Data and Code Availability

The CVRP dataset and accompanying codes are publicly available from Hugging Face at https://huggingface.co/datasets/CVRPDataset/CVRP & https://huggingface.co/CVRPDataset/Model, and Bioinformatics service in Nanjing Agricultural University at http://bic.njau.edu.cn/CVRP.html.

Acknowledgements

We thank Mr.Zhitao Zhu, Dr. Weijie Tang, and Dr. Yunhui Zhang for their technical support.