= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on [Link] A superintelligent solution to the Fermi paradox - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (75)
Former, not the latter. And yes, the anti-Penrose point applies, but we can skirt it by postulating that the superintelligence is limited in its decision theory—it can recognize good results when it seems them, much as TDT can recognize that UDT beats it at counterfactual mugging, but it's architecturally constrained not to self-modify into the winning thing. So humans might run native hypercomputation or native super-awesome decision theory that an AI could exploit but that the AI would know it couldn't emulate given its knowledge of its own limited architecture.
I guess you're distantly alluding to the old discussion of 'what would AIXI do if it ran into a hypercomputing oracle?' in modern guise. I'm afraid I know too little about TDT or UDT to appreciate the point. It just seems a little far-fetched - so not only are we thinking about hypercomputation, which I believe is generally regarded as being orders of magnitude less likely than say P=NP, we're also thinking about a superintelligent and superpowerful agent with a decision theory that just happens to be broken in the right way?
If we were being mined for our computational potential, I can't help but feel human lives ought to be less repetitive than they are.
Haven't seen any surveys, but I don't think so. I think hypercomputation is considered by some important people to be more likely than P=NP. I believe very few people have really considered it, so you shouldn't take anyone's off-the-cuff impressions as meaning very much unless you know they've thought a lot about the limitations of theoretical computer science. I don't really have any ax to grind on the matter, but I think hypercomputation is neglected.
I think my points were supposed to be disjunctive, not conjunctive. A broken decision theory or a limited theory of computation can both result in humans outcompeting superintelligences on certain very specific decision problems or (pseudo-)computations. Wei Dai's "Metaphilosophical Mysteries" is relevant.
Given some models, yes. Given other models, the AI might not be able to locate what parts of the system have the special sauce and what parts don't, so it's more likely to let humans be.
Your link isn't a stupid person, but to some extent, the lack of interest in hypercomputation says what the field thinks of it. Compare it to quantum computation, where people were avidly researching it and coming up with algorithms decades before even toy quantum computers showed up in cutting-edge labs.
Wei Dai's link is pretty controversial.
But only after it was discovered that the theory of quantum mechanics implied it was theoretically possible.
My understanding of the history is that everyone believed the extended Church-Turing thesis until someone noticed that the (already established) theory of quantum mechanics contradicted it.
I don't think I've ever seen anyone invoke the extended Church-Turing thesis by either name or substance before quantum computing came around.
People were talking about P-time before quantum computing and implicitly assuming that it applied to any computer they could build.
I don't see how one would apply "P-time" to "any computer they could build".
I meant "apply" in the sense that one applies a mathematical model to a phenomenon. Specifically, it was implicitly assumed the the notion of polynomial time captured what was actually possible to compute in polynomial time.
Not sure, but it seems that whenever I get into discussions with you it's usually about some potentially-important edge case or something. Strange.
But anyway, yeah. I just want to flag hypercomputation as a speculative thing that it might be worth taking an interest in, much like mirror matter. One or two of my default models are probably very similar to yours when it comes down to betting odds.
Um, you do realize you're comparing apples and oranges there, since one is a statement about physics and the other a statement about mathematics.
In this area, I do not think there is such a hard and fast distinction.
So, how would you phrase the existence of hypercomputation as a mathematical statement?
Presumably something involving recursively enumerable functions...
As someone who understands computational theory, I strongly suspect you're seriously confused about how computational complexity theory works. As I don't have the time or interest to give a course in computational complexity, might I recommend asking the original question on mathoverflow if you are interested.
Apologies if that came off as rude.