Sorry if this is obvious, but where does the “irreducible” loss come from? Wouldn’t that also be a function of the data, or I guess the data’s predictability?
Constant, predictable gains vs. Black Swans
Did the survey! ...And now to upvote everything.
It reminds me of Justice Potter Stewart: "I know it when I see it!"
I knew we shouldn't have spent all that funding on awakening the Elder God Cthulhu!
Oh god. That... makes a scary amount of sense. If an AI told me that I would probably believe it. I'd also start training myself to be more of a "night-time person".
Hi, my name is Joe. I live in North Jersey. I was born into a very religious Orthodox Jewish family. I only recently realized I how badly I was doublethinking.
I started with HPMOR (as, it seems, do most people) and found my way into the Sequences. I read them all on OB, and was amazed at how eloquently someone else could voice what seems to be my thoughts. It laid out bare the things I had been struggling with.
Then I found LW and was mostly just lurking for a while. I only made an account when I saw this post and realized how badly I wanted to upvote some of the comments :).
I think this site and the Sequences on it have changed my life, and I'm glad to finally be part of it.
I think that solving the alignment for EV maximizers is a much stronger version of alignment than eg prosaic alignment of LLM-type models. Agents seem like they’ll be more powerful than Tool AIs. We don’t know how to make them, but if someone does, and capabilities timelines shorten drastically, it would be awesome to even have a theory of EV maximizer alignment before then