RLHF, DPO, and the Evolution of Alignment Training

Pretraining produces capable models, but raw pretrained models are not useful assistants. Alignment training is what shapes them into the helpful, honest, and harmless systems users actually interact with. The techniques have evolved rapidly from RLHF to DPO to constitutional AI, each addressing limitations of the previous approach.