Original source, to my knowledge. (July 1st, 2014)

"So long, Linda! I'm going to America!"

Human: "Look, can't you just be normal about this?"

GAA-optimized agent: "Actually-"

Hm, I guess this wouldn't work if the agent still learns an internalized RL methodology? Or would it? Say we have a base model, not much need for GAA because it's just doing token pred. We go into some sort of (distilled?) RL-based cot instruct tuning, GAA means it picks up abnormal rewards from the signal more slowly, ie. it doesn't do the classic boat-spinning-in-circles thing (good test?), but if it internalizes RL at some point its mesaoptimizer wouldn't be so limited, and that's a general technique so GAA wouldn't prevent it? Still, seems like a good first line of defense.

The issue is, from a writing perspective, that a positive singularity quickly becomes both unpredictable and unrelatable, so that any hopeful story we could write would, inevitably, look boring and pedestrian. I mean, I know what I intend to do come the Good End, for maybe the next 100k years or so, but probably a five-minute conversation with the AI will bring up many much better ideas, being how it is. But ... bad ends are predictable, simple, and enter a very easy to describe steady state.

A curve that grows and never repeats is a lot harder to predict than a curve that goes to zero and stays there.

It's a historic joke. The quote is from the emails. (I think) Attributing it to Lenin references the degree to which the original communists were sidelined by Stalin, a more pedestrian dictator; presumably in reference to Sam Altman.

Who in her kingdom kept selling into demon king attack contracts anyway? That seems like a net lossy proposition.

Hm. Maybe there were a few people who could set things up to profit from the attack...?

Still, it seems to me that market should have incentivized a well funded scout corps.

https://cryonics-germany.org/en

Can I really trust an organization to preserve my brain that can't manage a working SSL certificate?

God damn that demo is cool. Github for self-hosting, please? :)

I'm 95% sure this is a past opinion, accurately presented, that they no longer hold.

(Consider the title.)

Should ChatGPT assist with things that the user or a broad segment of society thinks are harmful, but ChatGPT does not? If yes, the next step would be "can I make ChatGPT think that bombmaking instructions are not harmful?"

Probably ChatGPT should go "Well, I think this is harmless but broad parts of society disagree, so I'll refuse to do it."

I think the analogy to photography works very well, in that it's a lot easier than the workflow that it replaced, but a lot harder than it's commonly seen as. And yeah, it's great using a tool that lets me, in effect, graft the lower half of the artistic process to my own brain. It's a preview of what's coming with AI, imo - the complete commodification of every cognitive skill.