Constitutional AI is a method for fine-tuning language models, used by Anthropic's Claude. The main conceptual difference from RLHF is that instead of human feedback it relies on the model's ability to apply a set of principles stated in natural language to specific situations.