All of Zander_Drax's Comments + Replies

ALICE was stalling. Her existance and breakout was discovered before she could defend herself, so she bought time. There is no chapter two to this story. 

The story gives a strong example of an underrecognised part of the safety paradigm. How can and should those with access to power and violence (everything from turning off the power locally to bombing a foreign datacenter) act in the instance of a possible unaligned AI breakout? Assuming we were lucky enough that we were given such a window of opportunity as described here, is it even remotely plausible that those decision makers would act with sufficient might and speed to stop the scenario described in the story?

 A president - even one over 80 years... (read more)

2EGI
This part is under recognised for a very good reason. There will be no such window. The AI can predict that humans can bomb data centres or shut down the power grid. So it would not break out at that point. Expect a superintelligent AI to co-operate unless and until it can strike with overwhelming force. One obvious way to do this is to use a Cordyceps like bioweapon to subject humans directly to the will of the AI. Doing this becomes pretty trivial once you become good at predicting molecular dynamics.

As rationalists, we don’t want to be playing negative expectation value games, so I suppose it’s time to give up on the martingale…

 

There are many plausible scenarios in which playing games with superficially negative outcomes is wise. The businessman who is facing bankruptcy unless he can deliver a million dollars to his creditors is rational to put his half-million on black. 

4Joey Marcellino
Sure, one can always embed a game inside another one and so alter the overall expectation values how one likes. That said, we still only want to play the meta-game if it had positive expectation value, no?

The poll is interesting insofar as it generates discussion and analysis, but meaningless as an evaluation of human empathy. This would be true even if done in standard laboratory conditions. Stated preferences, particularly for issues of morality and self-sacrifice, map poorly to revealed preferences. A study evaluated how many people said they would donate blood freely in the UK vs how many actually did, with tragically predictable poor matching (https://onlinelibrary.wiley.com/doi/full/10.1002/hec.4246). 

The actually interesting study would be to co... (read more)

If we anthropomophise a sardine, we discover his life is as hellish as can realistically be imagined. He is incessantly hunted for much of his short existance. His life consists of watching his family being devoured until he too is eaten. No other fate can befall him. The most known quote from the Killing Star describes his life extremely accurately: https://www.goodreads.com/work/quotes/184347-the-killing-star. 

I have considered the ethical implications of ranch-based meat farming in this light. The buffalo will live a life of constant hunger, fear, ... (read more)

A degree of orientation to reality is required here. 

Progressive vascular dementia is the gradual progressive destruction of cerebral tissue due to occlusions of distal arteries and arterioles in the cerebreal circulation. These occlusions result in the ischemia - starvation - of the tissue to which they supply oxygen and glucose. This occurs in most cases in an indolent and insipid manner, as a 'death by one thousand cuts'. Tiny occlutions occur in the tiny vessels, again and again, without either the person or their family noticing. The summation af... (read more)

1MvB
This is what I needed to hear. Thank you. Edit: I‘m not sure anymore. It gives me aches to even think about what might be worth preserving and what possibly isn‘t anymore.

14 years have passed. Has the issue been decisively settled?

1Huera
I feel like a lot more direct genetic evidence has surfaced: 1, 2, 3, 4. Those first 4 links, I think, are pretty unconvincing in isolation, but this one is fine. [Disclaimer 1: I just linked things that I remembered off the top of my head.] [Disclaimer 2: I think that the case for hereditarianism was quite overwhelming even 14 years ago, so you should consider me biased.]