# gjm comments on AI is not enough - Less Wrong Discussion

-22 07 February 2012 03:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

Comment author: 07 February 2012 11:02:47PM 5 points [-]

No, I didn't say "it's all algorithmic, basta"; I said "so far as we know, it's all algorithmic". Of course it's possible that we'll somehow discover that actually our minds run on magic fairies and unicorns or something, but so far as I can tell all the available evidence is consistent with everything being basically algorithmic. You're the one claiming to know that that isn't so; I invite you to explain how you know.

I haven't claimed that the axioms of arithmetic are derived from something simpler. I have suggested that for all we know, the process by which we found those axioms was basically algorithmic, though doubtless very complicated. (I'm not claiming that that algorithmic process is why the axioms are right. If you're really arguing not about the processes by which discoveries are made but about why arithmetic is the way it is, then we need to have a different discussion.)

it is easily possible to invent new axioms. This is essential for intelligence, yet an AI can't do it, since it only works by its axioms.

I'm afraid this is very, very wrong. Perhaps the following analogy will help: suppose I said "It is easily possible to contemplate arbitrarily large numbers, even ones bigger than 2^32 or 2^64. This is essential for intelligence, yet an AI can't do it, since it only works with 32-bit or 64-bit arithmetic." That would be crazy, right?, because an AI (or anything else) implemented on a hardware substrate that can only do a very limited set of operations can still do higher-level things if it's programmed to. A computer can do arbitrary-precision arithmetic by doing lots of 32-bit arithmetic, if the latter is organized in the right way. Similarly, it can cook up new axioms and rules by following fixed rules satisfying fixed axioms, if the latter are organized in the right way.

And how are these rules determined?

Depends how far back the chain of causation you want to go. There'll be some rules programmed into the computer by human beings. Those were determined by whatever complicated algorithms human brains execute. Those were determined by whatever complicated algorithms human cultures and biological evolution execute. Those were determined by ... etc. As you go further back, you get algorithms with less direct connection to intelligence (ours, or a computer's, or whatever). Ultimately, you end up with whatever the basic laws of nature are, and no one knows those for sure. (But, again, so far as anyone knows they're algorithmic in nature.)

So: no infinite chain, probably (though it's not clear to me that there's anything actually impossible about that); you start with whatever the laws of nature are, and so far as anyone knows they just are what they are. (I suppose you could try to work that up into some kind of first-cause argument for the existence of God, but I should warn you that it isn't likely to work well.)

Really? I think it [emergence] is [magic] ... It seems to me nature is inherently magical.

Oh. Either you're using the word "magical" in a nonstandard way that I don't currently understand, or at least one of us is so terribly wrong about the nature of the universe that further discussion seems unlikely to be helpful.