Or the benefits could slightly outweigh the harm.
You have to treat this option as a net win of 0 then, because you have no more info to go on so the probs. are 50/50. Option A: Torture. Net win is negativ. Option B: Spec dust. Net win is zero. Make you choice.
In the Least Convenient Possible World of this hypothetical, every dust speck causes a constant small amount of harm with no knock-on effects(no avoiding buses, no crashing cars...)
I don't think he was talking about self-PA, but rather an altered decision criteria, such that rather that "if I can prove this is good, do it" it is "if I can prove that if I am consistent then this is good, do it"
Yes.
and it still can't /increase/ in proof strength.
Mmm, I think I can see it.
What about "if I can prove that if a version of me with unbounded computational resources is consistent then this is good, do it". (*)
It seems to me that this allows increase in proof strength up to the proof strength of that particular ideal reference agent.
(* there should be probably additional constraints that specify that the current agent, and the successor if present, must be provably approximations of the unbounded agent in some conservative way)
"if I can prove that if a version of me with unbounded computational resources is consistent then this is good, do it"
In this formalism we generally assume infinite resources anyway. And even if this is not the case, consistent/inconsistent doesn't depend on resources, only on the axioms and rules for deduction. So this still doesn't let you increase in proof strength, although again it should help avoid losing it.
Good question! Translating your question to the setting of the logical model, you're suggesting that instead of using provability in Peano Arithmetic as the criterion for justified action, or provability in PA + Con(PA) (which would have the same difficulty), the agent uses provability under the assumption that its current formal system (which includes PA) is consistent.
Unfortunately, this turns out to be an inconsistent formal system!
Thus, you definitely do not want an agent that makes decisions on the criterion "if I assume that my own deductions are reliable, then can I show that this is the best action?", at least not until you've come up with a heuristic version of this that doesn't lead to awful self-fulfilling prophecies.
I don't think he was talking about self-PA, but rather an altered decision criteria, such that rather that "if I can prove this is good, do it" it is "if I can prove that if I am consistent then this is good, do it" which I think doesn't have this particular problem, though it does have others, and it still can't /increase/ in proof strength.
I'm having trouble understanding how something generally intelligent in every respect except failure to understand death or that it has a physical body, could be incapable of ever learning or at least acting indistinguishable from one that does know.
For example, how would AIXI act if given the following as part of its utility function: 1) utility function gets multiplied by zero should a certain computer cease to function 2) utility function gets multiplied by zero should certain bits be overwritten except if a sanity check is passed first
Seems to me that such an AI would act as if it had a genocidally dangerous fear of death, even if it doesn't actually understand the concept.
That AI doesn't drop an anvil on its head(I think...), but it also doesn't self-improve.
Failing to halt and going into an infinite loop are not the same thing.
I'd appreciate some explanation on that, to see if you're saying something I haven't heard before or if we're talking past each-other. I don't just include while(true); under "infinite loop", I also include infinitely-expanding recursions that cannot be characterized as coinductive stepwise computations. Basically, anything that would evaluate to the \Bot type in type-theory counts for "infinite loop" here, just plain computational divergence.
I think that what Joshua was talking about by 'infinite loop' is 'passing through the same state an infinite number of times.' That is, a /loop/, rather than just a line with no endpoint. although this would rule out (some arbitrary-size int type) x = 0; while(true){ x++; } on a machine with infinite memory, as it would never pass through the same state twice. So maybe I'm still misunderstanding.
Suppose "mathemathics would never prove a contradiction". We can write it out as ¬Provable(⊥). This is logically equivalent to Provable(⊥) → ⊥, and it also implies Provable(Provable(⊥) → ⊥) by the rules of provability. But Löb's theorem expresses ∀ A (Provable(Provable(A) → A) → Provable(A)), which we can instantiate to Provable(Provable(⊥) → ⊥)→ Provable(⊥), and now we can apply modus ponens and our assumption to get a Provable(⊥).
Wasn't Löb's theorem ∀ A (Provable(Provable(A) → A) → Provable(A))? So you get Provable(⊥) directly, rather than passing through ⊥ first. This is good, as, of course, ⊥ is always false, even if it is provable.
10 would still be incorrect.
Darn it, and I counted like five times to make sure there really were 10 visible before I said anything. I didn't realize that the stone the middle-top stone was on top of was one stone, not two.
The cover is incorrect :(
EDIT: If you do not understand this post, read essay 268 from the book!
There might be one more stone not visible?
Thanks for the detailed comment.
I don't think that exceptional intelligence is either necessary or sufficient to be an exceptional mathematician. Tao's statement "But an exceptional amount of intelligence has almost no bearing on whether one is an exceptional mathematician." is a very strong statement: if he had said "plays only a moderate role in whether one is an exceptional mathematician" he would have been on much more solid ground.
I agree that the Langlands quote is by itself not strong evidence against Tao's assertion for the reasons that you give, but it's still evidence. I'm relying on many weak arguments. I'll gradually flesh them out in my sequence.
I share your intuition re: combinatorialists vs.geometers. One of my friends spent a lot of time with Chern, who struck him as being quite ordinary with respect to R, while being exceptional on a number of other dimensions. Grothendieck's self-assessment suggests that it is in fact possible to be amongst the greatest mathematicians without exceptional R.
A key point that you might be missing (certainly I did for many years) is that there just aren't many people of exceptional intelligence. Suppose that it were true that IQ is normally distributed: then the number of people of IQ 145+ would be 60x larger than the number of people of IQ 160+. Under this hypothesis, even if only 1 in 20 exceptional mathematicians had IQ 160+, that would mean that people in that range were 3x as likely as their IQ 145+ counterparts. to become exceptional mathematicians. It's been suggested that the distribution of IQ is in fact fat-tailed because of assortative mating, and this blunts the force of the aforementioned argument, but it's also true that more than 5% of exceptional mathematicians have IQ 160+: I think the actual figure is closer to 50%.
It should be noted that if measured IQ is fat-tailed, this is because there is something wrong with IQ tests. IQ is defined to be normally distributed with a mean of 100 and a standard deviation of either 15 or 16 depending on which definition you're using. So if measured IQ is fat-tailed, then the tests aren't calibrated properly(of course, if your test goes all the way up to 160, it is almost inevitably miscalibrated, because there just aren't enough people to calibrate it with).
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
= 27d567bc2d48d9f40e9ccc20ea370b15)
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
"The Lord Pilot shouted, fist held high and triumphant: "To live, and occasionally be unhappy!"" (three worlds collide) dust specks are just dust specks - in a way its helpful to sometimes have these things.
But the thing changes if you don't distribute the dust specks 1 per person but 10 per second per person?
In the Least Convenient Possible World of this hypothetical, each and every dust speck causes a small constant amount of harm, with no knock-on effects(no increasing one's appreciation of the moments when one does not have dust in ones eye, no preventing a 'boring painless existence,' nothing of the sort). Now it may be argued whether this would occur with actual dust, but that is not really the question at hand. Dust was just chosen as being a 'seemingly trivial bad thing.' and if you prefer some other trivial bad thing, just replace that in the problem and the question remains the same.