Make base models great again. I'm still nostalgic for GPT-2 or GPT-3. I can understand why RLHF was invented in the first place but it seems to me that you could still train a base model, so that if it's about to say something dangerous, it just prematurely cuts off the generation by emitting the <endoftext> token instead.

Alternatively, make models natively emit structured data. LLMs in their current form emit free-form arbitrary text which needs to be parsed in all sorts of annoying ways in order to make it useful for any downstream applications anyways. Also, structured output could help with preventing misaligned behavior.

(I'm less confident in this idea than the previous one.)

Half-baked AI Safety ideas thread

duck_master18d10

Try to wean people off excessive reliance on LLMs. This is probably the biggest source of AI-related negative effects today. I am trying to do this myself (I formerly alternated between claude, chatgpt, and lmarena.ai several times a day), but it is hard.

(By AI-related risks I mean effects like people losing their ability to write originally or think independently.)

Which things were you surprised to learn are not metaphors?

Answer by duck_masterNov 24, 2024180

When I visited Manhattan, I realized that "Wall Street" and "Broadway" are not just overused clichés, but the names of actual streets (you can walk on them!)

AI Safety Salon with Steve Omohundro