Follows https://github.com/huggingface/alignment-handbook/issues/45#issuecomment-1845598205

From HuggingFaceH4/mistral-7b-sft-beta