These models underwent supervised fine-tuning and direct preference optimization for instruction following on top of Microsoft's Phi 2 base LLM