Rationality Quotes August 2012

Alejandro1

Here's the new thread for posting quotes, with the usual rules:

Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
Do not quote yourself
Do not quote comments/posts on LW/OB
No more than 5 quotes per person per monthly thread, please.

Here's the new thread for posting quotes, with the usual rules:

Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
Do not quote yourself
Do not quote comments/posts on LW/OB
No more than 5 quotes per person per monthly thread, please.

I of course agree that the thought of being able to read the AI's mind is ridiculous.

It's not transparently obvious to me why this would be "ridiculous", care to enlighten me? Building an AI at all seems ridiculous to many people, but that's because they don't actually think about the issue because they've never encountered it before. It really seems far more ridiculous to me that we shouldn't even try to read the AIs mind, when there's so much at stake.

AIs aren't Gods, with time and care and lots of preparation reading their thoughts should be doable. If you disagree with that statement, please explain why? Rushing things here seems like the most awful idea possible, I really think it would be worth the resource investment.

AIs aren't Gods, with time and care and lots of preparation reading their thoughts should be doable.

Humans reading computer code aren't gods either. How long until an uFAI would get caught if it did stuff like this?

0DanielLC14y

If it's a self-modifying AI, the main problem is that it keeps changing. You might find the memory position that corresponds to, say, expected number of paperclips. When you look at it next week wondering how many paperclips there are, it's changed to staples, and you have no good way of knowing. If it's not a self-modifying AI, then I suspect it would be pretty easy. If it used Solomonoff induction, it would be trivial. If not, you are likely to run into problems with stuff that only approximates Bayesian stuff. For example, if you let it develop its own hanging nodes, you'd have a hard time figuring out what they correspond to. They might not even correspond to something you could feasibly understand. If there's a big enough structure of them, it might even change.

4wedrifid14y

Sure, possible. Just a lot harder than creating an FAI to do it for you---especially when the AI has an incentive to obfuscate.

9

Rationality Quotes August 2012

9

9

9

Rationality Quotes August 2012

9

9