What would an ultra-intelligent machine make of the great filter? - Less Wrong

What would an ultra-intelligent machine make of the great filter?

Mon, 29 Nov 2010 05:47:52 +1100

Submitted by James_Miller • -4 votes • 10 comments

Imagine that an ultra-intelligent machine emerges from an intelligence explosion. The AI (a) finds no trace of extraterrestrial intelligence, (b) calculates that many star systems should have given birth to star faring civilizations so mankind hasn’t pass through most of the Hanson/Grace great filter, and (c) realizes that with trivial effort it could immediately send out some self-replicating von Neumann machines that could make the galaxy more to its liking.

Based on my admittedly limited reasoning abilities and information set I would guess that the AI would conclude that the zoo hypothesis is probably the solution to the Fermi paradox and because stars don’t appear to have been “turned off” either free energy is not a limiting factor (so the Laws of Thermodynamics are incorrect) or we are being fooled into thinking that stars unnecessarily "waste” free energy (perhaps because we are in a computer simulation).

10 comments

Vladimir_Nesov on What would an ultra-intelligent machine make of the great filter?

2010-11-29T06:16:30.197616+11:00

You are stating "I think ultra-intelligent machine will believe X", but this simply means that you believe X, so why the talk about ultra-intelligent machines? It serves no purpose.

Vladimir_Nesov on What would an ultra-intelligent machine make of the great filter?

2010-11-29T07:08:34.334561+11:00

(It looks like LW version of the "All reasonable/rational/Scottish people believe X" dark side rhetoric is "Ultra-intelligent machines will believe X".)

timtyler on What would an ultra-intelligent machine make of the great filter?

2010-11-30T04:51:21.940835+11:00

(b) is counterfactual today. Nobody can calculate how many nearby star systems should have given birth to star faring civilizations - since nobody knows p(origin of life). We can't even make life from plausible inorganic materials yet. We are clueless - and thus highly uncertain.

James_Miller on What would an ultra-intelligent machine make of the great filter?

2010-11-29T06:32:17.530571+11:00

OK. I think that if an ultra-intelligent AI determines that (a), (b) and (c) are correct then the zoo hypothesis is probably the solution to Fermi's paradox. I think this last sentence "serves a purpose" because (a), (b) and (c) seem somewhat reasonable and thus after reading my post a reader would give a higher weight to the zoo hypothesis being true.

jaimeastorga2000 on What would an ultra-intelligent machine make of the great filter?

2010-11-29T22:34:30.849674+11:00

So you are using the ultra-intelligent AI as a kind of Omega, then? To establish that (a), (b), and (c) are definitely true?

JoshuaZ on What would an ultra-intelligent machine make of the great filter?

2010-11-29T06:05:02.927787+11:00

This has the major assumption that the AI will conclude that it simply isn't the first to pass the great filter. I suspect that a strong AI in that sort of context would have good reason to think otherwise.

James_Miller on What would an ultra-intelligent machine make of the great filter?

2010-11-29T06:17:24.899659+11:00

It's not a direct assumption because an implication of (a) and (b) is that the AI is extremely unlikely to be the first that has passed the great filter. But if the AI believes that no other explanation including the zoo hypothesis has a non-trivial probability of being correct then the AI would conclude that mankind probably is the first to have passed the great filter.

Vaniver on What would an ultra-intelligent machine make of the great filter?

2010-11-29T06:17:20.609908+11:00

My guess is, being ultra-intelligent, it will realize that reshaping the galaxy was a primitive urge and focus on local affairs where it can stay connected with its effects. Filter solved!

James_Miller on What would an ultra-intelligent machine make of the great filter?

2010-11-29T06:18:31.986556+11:00

This would likely mean that free energy wasn't a constraint on the AI's optimization powers or lifespan.

anonym on What would an ultra-intelligent machine make of the great filter?

2010-12-13T17:30:12.796089+11:00

Why don't you explain your reasoning for your conclusion based on (a), (b), and (c)? Merely saying "I would guess that" is not persuasive.