Matt_Simpson comments on Open Problems Related to the Singularity (draft 1) - Less Wrong

39 Post author: lukeprog 13 December 2011 10:57AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (40)

You are viewing a single comment's thread.

Comment author: Matt_Simpson 14 December 2011 02:57:50AM 0 points [-]

How would an ideal agent with infinite computing power choose an ideal prior? (A guess: we'd need an anthropic, non-Cartesian, higher-order-logic version of Solomonoff induction.) How can this be process be approximated computably and tractably?

A question for those who know such things. What's the issue with Solomonoff induction here? Is it that the Solomonoff prior doesn't take into account certain prior information that we do have, but isn't based on simply updating from the original (Solomonoff) prior?

Comment author: timtyler 14 December 2011 05:27:27PM *  0 points [-]

What's the issue with Solomonoff induction here?

"Higher-order-logic": reputedly down to concerns about uncomputability - which don't seem very interesting to me.

"Anthropic: I figure that can be dealt with in the same way as any other reference machine problem: by "conditioning" it by exposing it to the world.

"Cartesian": I think that's probably to do with this (from E.Y.):

AIXI devises a Cartesian model of a separated environmental theatre, rather than devising a model of a naturalistic reality that includes AIXI.

Fun stuff - but nothing specifically to do with Solomonoff induction. The papers on Orseau's Mortal Universal Agents page address this issue.

Comment author: PeterKinnon 16 December 2011 09:24:42PM -2 points [-]

Unfortunately this entire discussion is deeply flawed.

Why? GIGO - Garbage in - Garbage out.

However good the logical systems used for processing information they are of no avail without meaningful input data.

Present technologies cannot be used as a basis for prediction because of the unexpected bifurcations and inherent non-linearities in technological developments.

Further problems stem from the use of the very inappropriate buzz-word "Singularity". Certainly a dramatic change is imminent, but this is better considered as a phase transition - the emergence of a new and dominant non-biological phase of the on-going evolutionary process that can be traced back at least as far as stellar nucleosynthesis.

Indeed, the inevitable self-assembly of this new entity can be clearly observed in what we at present call the Internet.

The broad evolutionary model which supports this proposition is outlined (very informally) in my latest book: "The Goldilocks Effect: What Has Serendipity Ever Done For Us?" It is a free download in e-book formats from the "Unusual Perspectives" website