apple/ml-reversal-blessing

Reversal Blessing: Thinking Backward May Outpace Thinking Forward in Multi-choice Questions

📃 Paper • 📝 Blog

This model card accompanies the research paper, Reversal Blessing: Thinking Backward May Outpace Thinking Forward in Multi-choice Questions.

Here we release 4 models' checkpoints trained with simulation data described in out paper Section 4. Please follow our github README to download and evaluate these models.

Results of the Controlled Simulation Study of 4-Digits Multiplication

		Forward X			Reverse X
	L2R	R2L(m,n)	R2L(m)	R2L	L2R(m,n)	L2R(n)
Test Accuracy (%)	99.81±0.15	59.71±1.99	60.93±0.88	100±0	97.82±0.35	99.85±0.10
Train Accuracy (%)	99.76±0.15	59.03±1.66	61.22±1.12	100±0	97.90±0.42	99.98±0.04
Test Cond. Ent. (nats)	0.06	1.18	0.08	0	0.84	0.01
Train Cond. Ent. (nats)	0.06	1.17	0.08	0	0.83	0.01
Theo. Cond. Ent. (nats)	0	1.49	0	0	1.49	0
Training loss	0.86	0.94	0.94	0.86	0.94	0.94