Why Jailbreaks Keep Working on Aligned Language Models
Why jailbreaks keep working on aligned language models comes down to one fact: alignment training changes what a model tends to do, and it does not remove what the model can do. A jailbreak is a prompt that finds a path…