Will_Newsome comments on Journal of Consciousness Studies issue on the Singularity - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (85)
Tipler paper
Wow, that's all kinds of crazy. I'm not sure how much as I'm not a mathematical physicist - MWI and quantum mechanics implied by Newton? Really? - but one big flag for me is pg187-188 where he doggedly insists that the universe is closed, although as far as I know the current cosmological consensus is the opposite, and I trust them a heck of a lot more than a fellow who tries to prove his Christianity with his physics.
(This is actually convenient for me: a few weeks ago I was wondering on IRC what the current status of Tipler's theories were, given that he had clearly stated they were valid only if the universe were closed and if the Higgs boson was within certain values, IIRC, but I was feeling too lazy to look it all up.)
And the extraction of a transcendent system of ethics from a Feynman quote...
This is just too wrong for words. This is like saying that looking both ways before crossing the street is obviously a part of rational street-crossing - a moment's thought will convince the reader (Dark Arts) - and so we can collapse Hume's fork and promote looking both ways to a universal meta-ethical principal that future AIs will obey!
Show me this morality in the AIXI equation or GTFO!
A map from range to domain, a proof in propositional logic, or a series of lambda equations and reductions all come to mind...
One man's modus ponens is another man's modus tollens. That the 'honestly' requires other entities is proof that this cannot be an ethical system which encompasses all rational beings.
Any argument that rests on a series of rhetorical questions is untrustworthy. Specifically, sure, I can in 5 seconds come up with a reason they would not preserve us: there are X mind-states we can be in while still maintaining identity or continuity; there are Y (Y < X) that we would like or would value; with infinite computing power, we will exhaust all Y. At that point, by definition, we could choose to not be preserved. Hence, I have proven we will inevitably choose to die even if uploaded to Tipler's Singularity.
(Correct and true? Dunno. But let's say this shows Tipler is massively overreaching...)
What a terrible paper altogether. This was a peer-reviewed journal, right?
The "AIXI equation" is not an AI in the relevant sense.
Fine, 'show me this morality in a computable implementation of AIXI using the speed prior or GTFO' (what was it called, AIXI-tl?).
That also isn't an AI in the relevant sense, as it doesn't actually exist. Tipler would simply deny that such an AI would be able to anything for Searlian reasons. You can't prove that an AIXI-style AI will ever work, and it's presumably part of Tipler's argument that it won't work, so simply asserting that it will work is sort of pointless. I'm just saying that if you want to engage with his argument you'll have to get closer to it 'cuz you're not yet in bowshot range. If your intention was to repeat the standard counterargument rather than show why it's correct then I misinterpreted your intention; apologies if so.
The AIXI proofs seem pretty adequate to me. They may not be useful, but that's different from not working.
More to the point, nothing in Tipler's paper gave me the impression he had so much as heard of AIXI, and it's not clear to me that he does accept Searlian reasons - what is that, by the way? It can't be Chinese room stuff since Tipler has been gung ho on uploading for decades now.
It's really not obvious that if you run an AIXI-like AI it will actually do anything other than self-destruct, no matter how much juice you give it. There have been various papers on this theme recently and it's a common LW meme ("AIXI drops an anvil on its head").
By "Searlian reasons" I mean something like emphasizing the difference between syntax and semantics and the difficulty of the grounding problem as representative of this important dichotomy between narrow and general intelligence that philosophers of mind get angry with non-philosophers of mind for ignoring.
I don't think Tipler's not having heard of AIXI is particularly damning, even if true.
I don't think it's obvious it would self-destruct - any more than it's obvious humans will not self-destruct. (And that anvil phrase is common to Eliezer.) The papers you allude to apply just as well to humans.
I believe you are the one who is claiming AIXI will never work, and suggesting Tipler might think like you.