PointSO

Paper_link: https://arxiv.org/abs/2502.13143
Project Page: https://qizekun.github.io/sofar/
Github Code: https://github.com/qizekun/SoFar

We introduce the concept of semantic orientation, representing the object orientation condition on open vocabulary language. The model is part of SoFar, a 6-DoF system for spatial reasoning and robotic manipulation. The released weights is on Huggingface PointSO, and the code is in the orientation folder.

Pretrain

Download the PointMAE as initialization.

wget https://github.com/Pang-Yatian/Point-MAE/releases/download/main/pretrain.pth -P orientation/

Perpare the OrienText300K dataset following DATASET.md.

cd orientation
sh train_ddp.sh

Finetune

Perpare the Open6DOR finetuning dataset following DATASET.md. The dataset is generated from isaac sim with different assets from Open6DOR. Finetune PointSO will significantly improve the performance on Open6DOR rotation track & 6-DoF track. We recommend using this version of the model for the Open6DOR V2 evaluation.

cd orientation
sh train_ddp_ft.sh