Comment author: [deleted] 19 March 2015 08:13:05AM *  0 points [-]

Technically speaking, you can observe the loop encoded in the Turing machine's code somewhere -- every nonhalting Turing machine has some kind of loop. The Halting theorems say that you cannot write down any finite program which will recognize and identify any infinite loop, or deductively prove the absence thereof, in bounded time. However, human beings don't have finite programs, and don't work by deduction, so I suspect, with a sketch of mathematical grounding, that these problems simply don't apply to us in the same way they apply to regular Turing machines.

EDIT: To clarify, human minds aren't "magic" or anything: the analogy between us and regular Turing machines with finite input and program tape just isn't accurate. We're a lot closer to inductive Turing machines or generalized Turing machines. We exhibit nonhalting behavior by design and have more-or-less infinite input tapes.

In response to comment by [deleted] on Second-Order Logic: The Controversy
Comment author: orthonormal 20 March 2015 05:54:57PM 1 point [-]

every nonhalting Turing machine has some kind of loop

Formal proof needed; I in fact expect there to be something analogous to a Penrose tiling.

Moreover, to adapt Keynes' apocryphal quote, a Turing machine can defer its loop for longer than you can ponder it.

And finally, as a general note, if you find that your proof that human beings can solve the halting problem can't be made formal and concise, you might consider the possibility that your intuition is just plain wrong. It is in fact relevant that theoretical computer scientists seem to agree that the halting problem is not solvable by physical processes in the universe, including human beings.

New forum for MIRI research: Intelligent Agent Foundations Forum

36 orthonormal 20 March 2015 12:35AM

Today, the Machine Intelligence Research Institute is launching a new forum for research discussion: the Intelligent Agent Foundations Forum! It's already been seeded with a bunch of new work on MIRI topics from the last few months.

We've covered most of the (what, why, how) subjects on the forum's new welcome post and the How to Contribute page, but this post is an easy place to comment if you have further questions (or if, maths forbid, there are technical issues with the forum instead of on it).

But before that, go ahead and check it out!

(Major thanks to Benja Fallenstein, Alice Monday, and Elliott Jin for their work on the forum code, and to all the contributors so far!)

EDIT 3/22: Jessica Taylor, Benja Fallenstein, and I wrote forum digest posts summarizing and linking to recent work (on the IAFF and elsewhere) on reflective oracle machines, on corrigibility, utility indifference, and related control ideas, and on updateless decision theory and the logic of provability, respectively! These are pretty excellent resources for reading up on those topics, in my biased opinion.

Comment author: orthonormal 19 March 2015 07:24:10PM 3 points [-]

By the way, there are other reasons that we use quining to study decision theories within (virtual) mathematical universes. Most importantly, it lets us play with the logic of provability in a straightforward way, which gives us some really nice polynomial-time tools for analyzing the outcomes. See Benja's modal UDT implementation in Haskell and my intro to why this works (especially Sections 6 and 7).

Of course, there are things outside that scope we want to study, but for the moment provability logic is a good lamppost under which we can search for keys.

Comment author: Squark 19 March 2015 06:49:36PM 11 points [-]

Great idea, well done!

However: Is it really the case that it's impossible to login without Facebook? Why?

Comment author: orthonormal 19 March 2015 07:11:46PM 2 points [-]

Yes, for now. Authentication with Facebook (or Google) is one of the simpler methods for avoiding spambots.

Comment author: Evan_Gaensbauer 19 March 2015 03:56:24PM *  5 points [-]

As a supporter of effective altruism who's interested in risks from superintelligence, where should I post now? What I mean by "interested" is someone who is sympathetic to the reasoning but skeptical about the tractability of the work MIRI and their allies do. For myself as well as on behalf of friends who share my concern, I would ask of MIRI what they think of the cause as a whole, how they think it will change in light of e.g., Elon Musk's donation to FLI and growing publicity, and the Open Philanthropy Project's investigation of the cause. I have one friend earning to give, a few acquaintances doing the same, and at least a couple friends my age in university still who may do so. As potential future donors and vocal supporters, provision of information would help each of us as individuals reach better conclusions. I wouldn't demand Eliezer Yudkowsky or Luke Muehlhauser respond, really just someone affiliated with the organization. Could I expect something like that on LessWrong still, or has everyone working for MIRI disappeared from LessWrong forever to go to this new forum? My questions wouldn't be on technical research, but I'd like to know where and how to address questions and concerns to the organization.

Comment author: orthonormal 19 March 2015 05:38:18PM 7 points [-]

The new forum will not cause people to disappear from Less Wrong, I expect. In fact, note the narrow focus of the IAFF:

Examples of subjects that are off-topic (and which moderators might close down) include recent advances in artificial intelligence and machine learning, existential risks, effective altruism, human rationality, general mathematical logic, general philosophy of mind, and anything that cannot (yet) be usefully formalized or modeled mathematically.

Less Wrong continues to be a great place to discuss those topics (which includes the topics you're interested in). And as for the technical topics, I hope to make some posts on here for the more digestible forum work.

Comment author: Vladimir_Nesov 19 March 2015 02:10:32PM *  14 points [-]

From the welcome post:

Examples of subjects that are off-topic (and which moderators might close down) include [...] anything that cannot (yet) be usefully formalized or modeled mathematically.

This seems open to an undesirable interpretation (discussion of how to formulate things that haven't yet been formulated and exercises in working through pre-formal examples, in an attempt to isolate principles that can then be explored more systematically). It might also encourage cargo cult formalization.

Comment author: orthonormal 19 March 2015 05:33:41PM 6 points [-]

Right. I wanted to encourage semi-formalized topics, but not completely non-technical philosophizing. Can someone suggest a better wording?

Comment author: orthonormal 19 March 2015 04:58:54AM 38 points [-]

Why a separate forum, you ask? For one thing, a narrower focus is better for this purpose: sending academic researchers to Less Wrong led to them wondering why people were talking about polyamory, cryonics, and Harry Potter on the same site as the decision theory posts...

Comment author: skeptical_lurker 21 February 2015 01:35:32AM -2 points [-]

I have a cunning plan:

What constraints are Harry working under?

The deadman switch hostage killer cannot have go off because they are timeturned, and the deaths of hundreds of students would surely cause a paradox by disrupting the match. Its possible that it sets a timer to go off after Harry jumps back in time, but then presuming that Harry knows exactly when he jumped back and Quirrel does not, Harry can then warn the teachers before the bomb goes off. At the end of the day, even if the hostages die, hundreds of lives are a small price to pay for stopping Voldemort.

If Quirrel dies, he will come back and torture Harry's parents. But presumably it will take Quirrel a little time to possess his new host. If the accessible Horocruxes are scattered throughout the world, then he will probably have to apperate back to the UK, which could take several jumps. Meanwhile, all Harry has to do is send his patronus to Dumbledoor, who can then Phoenix travel to Oxford, and should get there first, bringing Harry's parents back to the comparative safety of Hogwarts.

Quirrel cannot directly hurt Harry using magic, but he has a gun and the reflexes of a martial artist, plus he has the strength of an adult, so despite being ill he can probably overpower Harry if Harry managed to grab the gun.

Harry could transfigure a knife, but this wouldn't work, mostly because it would take some time, Voldemort would notice, and shoot him. So, a possible solution: transfigure a knife, but with the electrons swapped out for muons. Muons have higher mass, so the muon orbitals are far smaller than electron orbitals, so the material will be denser, and I think also stronger. Because of this, the blade can be made much thinner, and so easily concealed, perhaps to the extent of being invisible. The blade could be attached to Harry's wand, which could serve as a handle. It can penetrate Quirrel's shields, because his magic cannot interact with an object Harry has transfigured.

This still leaves the problem of fighting an immortal dark wizard with hundreds of Horocruxes. The next time Voldemort attacks, he will have got Bellatrix to cast shielding spells over him, which can stop any transfigured objects Harry throws at him. But by that time, Harry can have the order of the phoenix fighting at his side.

I imagine Volde attacking Hogwarts again and again, and everytime he is killed, possessing a new body and returning to the fight minutes later, while dead teachers and aurors stay dead (or do they, given the stone?) meanwhile, Harry desperately tries to think of a plan before the defenders are slowly worn down.

Comment author: orthonormal 21 February 2015 04:04:20AM 3 points [-]

Voldemort took Harry's wand away already.

Comment author: Douglas_Knight 12 February 2015 02:04:12AM 0 points [-]

Right, Dunning-Kruger is just regression to the mean.

Comment author: orthonormal 17 February 2015 01:42:13AM 0 points [-]

No, that's false. It's possible (and common) for a person to be wildly overconfident on a pretty broad domain of questions.

In response to 2014 Survey Results
Comment author: orthonormal 09 January 2015 02:00:09AM 6 points [-]

It looks to me like everyone was horrendously underconfident on all the easy questions, and horrendously overconfident on all the hard questions.

I think that this is what correct calibration overall looks like, since you don't know in advance which questions are easy and which ones are tricky. I would be quite impressed if a group of super-calibrators had correct calibration curves on every question, rather than on average over a set of questions.

View more: Prev | Next