RLHF Explained: How AI Learns Preferences Nobody Can Write Down
Summary
You cannot write a loss function for 'helpful and polite.' RLHF sidesteps that: humans rank pairs of model outputs, a reward model learns to predict those preferences, and the language model is optimized against the reward model with reinforcement learning. The same feedback-loop shape powers modern search ranking, where clicks and saves act as the preference signal that tunes retrieval.
About this video
You cannot write a loss function for 'helpful and polite.' RLHF sidesteps that: humans rank pairs of model outputs, a reward model learns to predict those preferences, and the language model is optimized against the reward model with reinforcement learning. The same feedback-loop shape powers modern search ranking, where clicks and saves act as the preference signal that tunes retrieval.