Reinforcement Learning with Human Feedback (RLHF)

• Use human feedback to train a reward model for predicting rewards when there isn’t a well-defined reward function
• Two models are trained: policy network and reward model
• Can be used for training deep RL agents, alignment of LLMs, and fine-tuning language models