Posts
Wiki Contributions
I personally endorse this as an example of us being a community that Has The Will To Try To Build Nice Things.
1To say the obvious thing: I think if Anthropic isn't able to make at least somewhat-roughly-meaningful predictions about AI welfare, then their core current public research agendas have failed?
1
1Fair enough!
Possibly misguided question given the context -- I see you incorporating imperfect information in "the attack fails silently", why not also a distinction between "the attack succeeds noisily, the AI wins and we know it won" and "the attack succeeds silently, the AI wins and we don't know it won" ?
I would suggest that the set of means available to nation-states to unilaterally surveil another nation state is far more expansive than the list you have. For example, the good-old-fashioned "Paying two hundred and eighty-two thousand dollars in a Grand Cayman banking account to a Chinese bureaucrat"* appears nowhere in your list.
*If you get that this is a reference to the movie Spy Game, you are cool. If you don't, go watch Spy Game. It has a worldview on power that is extremely relevant to rationalists.
I think you could argue plausibly that the climax of Vernor Vinge's A Deepness In the Sky has aspects of this, though it's subverted in multiple interesting spoilery ways.
In fact, I think you could argue that a lot of Vinge's writing tends to have major climaxes dependent on Xanatos Gambit pileups based on deception themes.
This feels like a great theory for one motivation, but it isn't at all complete.
For example: this theory doesn't really predict why anyone is ever hired above the bottom level of an organization at the margin.
That's a fair criticism! Season 1 is definitely slower on that front compared to the others. I think season 1 is the most normal "crime of the week" season by far, which is why I view it as a good on-ramp for folks less familiar. Arguably, for someone situated as you are, you should just watch the pilot, read a quick wiki summary of every other episode in season 1 except for the last 2, watch those last 2, and move into season 2 when things get moving a little faster. (Finch needs a character that doesn't appear until season 2 to do a lot of useful exposition on how he thinks about the Machine's alignment).
I will continue to pitch on the idea that Person of Interest is a TV show chock full of extremely popular TV people, including one lead beloved by Republicans, and we inexplicably fail to push people towards its Actually Did The Research presentation of loss-of-control stuff.
We should do that. You all should, unironically, recommend it, streaming now on Amazon Prime for free, to your normie parents and aunts and uncles if they're curious about what you do at work.
1
Is this where we think our pressuring-Anthropic points are best spent ?