Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: alex_zag_al 22 April 2014 10:58:06PM *  1 point [-]

I'm pretty sure that the Cauchy likelihood, like the other members of the t family, is a weighted mixture of normal distributions. (Gamma distribution over the inverse of the variance)

EDIT: There's a paper on this, "Scale mixtures of normal distributions" by Andrews and Mallows, if you want the details

Comment author: Cyan 23 April 2014 03:23:42AM *  1 point [-]

Oh, for sure it is. But that only gives it a conditionally conjugate prior, not a fully (i.e., marginally) conjugate prior. That's great for Gibbs sampling, but not for pen-and-paper computations.

In the three years since I wrote the grandparent, I've found a nice mixture representation for any unimodal symmetric distribution:

Suppose f(x), the pdf for a real-valued X, is unimodal and symmetric around 0. If W is positive-valued with pdf g(w) = -w f'(w) and U ~ Unif(-W, W), then U's marginal distribution is the same as X. Proof is by integration-by-parts.

I don't think it would be too hard to convert this width-weighted-mixture-of-uniforms representation to a precision-weighted-mixture-of-normals representation.

Comment author: Vulture 19 April 2014 04:49:35PM *  0 points [-]

Gratuitous image + obscure reference + anti-deathism not firewalled from rationalism = downvote, sorry.

Comment author: Cyan 19 April 2014 07:36:39PM -1 points [-]

The term "rationalism" has a previously-established meaning quite different from LW-style rationality.

Comment author: So8res 16 April 2014 04:57:13PM *  5 points [-]

Incorrect -- your implementation itself also affects the environment via more than your chosen output channels. (Your brain can be scanned, etc.) If you define waste heat, neural patterns, and so on as "output channels" then sure, we can say you only interact via I/O (although the line between I and O is fuzzy enough and your control over the O is small enough that I'd personally object to the distinction).

However, AIXI is not an agent that communicates with the environment only via I/O in this way: if you insist on using the I/O model then I point out that AIXI neglects crucial I/O channels (such as its source code).

until I see the actual math

In fact, Botworld is a tool that directly lets us see where AIXI falls short. (To see the 'actual math', simply construct the game described below with an AIXItl running in the left robot.)

Consider a two-cell Botworld game containing two robots, each in a different cell. The left robot is running an AIXI, and the left square is your home square. There are three timesteps. The right square contains a robot which acts as follows:

1. If there are no other robots in the square, Pass.
2. If an other robot just entered the square, Pass.
3. If an other robot has been in the square for a single turn, Pass.
4. If an other robot has been in the square for two turns, inspect its code.
.. If it is exactly the smallest Turing machine which never takes any action,
.. move Left.
5. In all other cases, Pass.

Imagine, further, that your robot (on the left) holds no items, and that the robot on the right holds a very valuable item. (Therefore, you want the right robot to be in your home square at the end of the game.) The only way to get that large reward is to move right and then rewrite yourself into the smallest Turing machine which never takes any action.

Now, consider the AIXI running on the left robot. It quickly discovers that the Turing machine which receives the highest reward acts as follows:

1. Move right
2. Rewrite self into smallest Turing machine which does nothing ever.

The AIXI then, according to the AIXI specification, does the output of the Turing machine it's found. But the AIXI's code is as follows:

1. Look for good Turing machines.
2. When you've found one, do it's output.

Thus, what the AIXI will do is this: it will move right, then it will do nothing for the rest of time. But while the AIXI is simulating the Turing machine that rewrites itself into a stupid machine, the AIXI itself has not eliminated the AIXI code. The AIXI's code is simulating the Turing machine and doing what it would have done, but the code itself is not the "do nothing ever" code that the second robot was looking for -- so the AIXI fails to get the reward.

The AIXI's problem is that it assumes that if it acts like the best Turing machine it found then it will do as well as that Turing machine. This assumption is true when the AIXI only interacts with the environment over I/O channels, but is not true in the real world (where eg. we can inspect the AIXI's code).

Comment author: Cyan 17 April 2014 01:40:31AM *  0 points [-]

Actually, an AI that believes it only communicates with the environment via input/output channels cannot represent the hypothesis that it will stop receiving input bits.

But I am an intelligence that can only communicate with the environment via input/output channels!

Incorrect -- your implementation itself also affects the environment via more than your chosen output channels.

Okay, fair enough. But until you pointed that out, I was an intelligence that believed it only communicated with the environment via input/output channels (that was your original phrasing, which I should have copied in the first place), and yet I did (and do) believe that it is possible for me to die.

Thus, what the AIXI will do is this: it will move right, then it will do nothing for the rest of time.

Incorrect. I'll assume for the sake of argument that you're right about what AIXI will do at first. But AIXI learns by Solomonoff induction, which is infallible at "noticing that it is confused" -- all Turing machines that fail to predict what actually happens get dropped from the hypothesis space. AIXI does nothing just until that fails to cause the right-room robot to move, whereupon any program that predicted that merely outputting "Pass" forever would do the trick gets zeroed out.

The AIXI's problem is that it assumes that if it acts like the best Turing machine it found then it will do as well as that Turing machine.

If there are programs in the hypothesis space that do not make this assumption (and as far as I know, you and I agree that naturalized induction would be such a program), then these are the only programs that will survive the failure of AIXI's first plan.

Has Paul Christiano looked at this stuff?

ETA: I don't usually mind downvotes, but I find these ones (currently -2) are niggling at me. I don't think I'm being conspicuously stupid, and I do think that discussing AIXI in a relatively concrete scenario could be valuable, so I'm a bit at a loss for an explanation. ...Perhaps it's because I appealed to Paul Christiano's authority?

Comment author: So8res 15 April 2014 07:51:47PM 2 points [-]

Actually, an AI that believes it only communicates with the environment via input/output channels cannot represent the hypothesis that it will stop receiving input bits. See this post for a discussion of the finer details.

Comment author: Cyan 16 April 2014 03:37:59PM *  1 point [-]

But I am an intelligence that can only communicate with the environment via input/output channels! And so are you!

How is it that we are able to represent the hypothesis that one can die? I refuse to accept that humans do something that AIXI can't until I see the actual math. (I don't affirm the opposite claim, mind you.)

Comment author: Cyan 15 April 2014 12:53:20PM *  2 points [-]

Such formalisms are perhaps ill-suited for real self-modifying agents, which are embedded within their environments. Indeed, the agent/environment separation is somewhat reminiscent of Cartesian dualism: any agent using this framework to reason about the world does not model itself as part of its environment. For example, such an agent would be unable to understand the concept of the environment interfering with its internal computations, e.g. by inducing errors in the agent’s RAM through heat.

I'm not sure that "unable" is the best way of describing this state of affairs. It's fairer to say that such agents start with no model of itself as part of the environment and would have to build such a model from scratch -- rather like how humans tend to believe that we have souls that will persist beyond death, and have to get educated about how the world works before we understand the implausibility of that belief.

Comment author: EHeller 08 April 2014 03:51:17AM 4 points [-]

So in my limited experience with physics and statistics phds I've sent here, its north of 80%.

Comment author: Cyan 08 April 2014 04:12:36AM *  2 points [-]

Ouch. (The statistics PhD thing is saddening to me but not surprising, alas.)

Comment author: Eliezer_Yudkowsky 07 April 2014 11:59:42PM 4 points [-]

The primary failure mode of writing is that nobody reads it. I don't know how to write like Bostrom in a way that people will read. I'm already worried about things like the tiling agents paper dropping off the radar.

Comment author: Cyan 08 April 2014 03:36:36AM 8 points [-]

The steps you could take to avoid the nobody-reads-it failure mode seem to me to be orthogonal to the steps you could take to avoid the author-is-a-colossal-prick failure mode. Given that you started this whole damn web site and community as insurance against the possibility that there might be someone out there with more innate talent for FAI, lukeprog's suggestion that you take steps to mitigate the author-is-a-colossal-prick failure mode in furtherance of that mission seems like a pretty small ask to me. And I say this as one who has always enjoyed your writing.

Comment author: eli_sennesh 07 April 2014 11:10:26AM 0 points [-]

Basically, I accept that critique, but only at an engineering level. Ditto on the "how much" issue: it's engineering. Neither of these issues actually makes me believe that a welfare state strapped awkwardly on top of a fundamentally industrial-capitalist, resource-capitalist, or financial-capitalist system - and constantly under attack by anyone perceiving themselves as a put-upon well-heeled taxpayer to boot - is actually a better solution to poverty and inequality than a more thoroughly socialist system in which such inequalities and such poverty just don't happen in the first place (because they're not part of the system's utility function).

I certainly believe that we have not yet designed or located a perfect socialist system to implement. What I do note, as addendum to that, is that nobody who supports capitalism believes the status quo is a perfect capitalism, and most people who aren't fanatical ideologues don't even believe we've found a perfect capitalism yet. The lack of a preexisting design X and a proof that X Is Perfect do not preclude the existence of a better system, whether redesigned from scratch or found by hill-climbing on piecemeal reforms.

All that lack means is that we have to actually think and actually try -- which we should have been doing anyway, if we wish to act according to our profession to be rational.

Comment author: Cyan 07 April 2014 01:15:08PM 1 point [-]

Good answer. (Before this comment thread I was, and I continue to be, fairly sympathetic to these efforts.)

Comment author: eli_sennesh 07 April 2014 08:10:09AM 2 points [-]

For what it's worth, it seems to me that you've used the term "socialism" to refer to two different, conflated, specific policies. In the OP you seem to be talking about direct redistribution of money, which isn't necessarily equivalent to the notion of worker control of the means of production that you introduce in the parent; and the term "socialism" doesn't pick out either specific policy in my mind.

Ah, here's the confusion. No, in the OP I was talking about worker control of the means of production, and criticizing Effective Altruism for attempting to fix poverty and sickness through what I consider an insufficiently effective intervention, that being direct redistribution of money.

Comment author: Cyan 07 April 2014 10:53:18AM *  1 point [-]

Oh, I see. Excellent clarification.

How would you respond to (what I claim to be) Krugman's account, i.e., in current conditions poor households are budget-constrained and would, if free to do so, liquidate their ownership of the means of production for money to buy the things they need immediately? Just how much redistribution of ownership are you imagining here?

Comment author: eli_sennesh 06 April 2014 04:10:12PM *  0 points [-]

The whole point of the second quotation and the paragraph after that was to... Oh never mind, should I just assume henceforth that contrary to its usage in socialist discourse, to outsiders "socialism" always means state-owned monopoly? In that case, what sort of terminology should I use for actual worker control of the means of production, and such things?

Comment author: Cyan 07 April 2014 01:48:06AM *  1 point [-]

"Anarcho-syndicalism" maybe? All's I know is that my socialized health insurance is a state-run oligopsony/monopoly (and so is my province's liquor control board). In any event, if direct redistribution of wealth is the key identifier of socialism, then Milton Friedman was a socialist, given his support for negative income taxes.

Prolly the best thing would be to avoid jargon as much as possible when talking to outsiders and just state what concrete policy you're talking about. For what it's worth, it seems to me that you've used the term "socialism" to refer to two different, conflated, specific policies. In the OP you seem to be talking about direct redistribution of money, which isn't necessarily equivalent to the notion of worker control of the means of production that you introduce in the parent; and the term "socialism" doesn't pick out either specific policy in my mind. (An example of how redistribution and worker ownership are not equivalent: on Paul Krugman's account, if you did direct redistribution right now, you'd increase aggregate demand but not even out ownership of capital. This is because current household consumption seems to be budget-constrained in the face of the ongoing "secular stagnation" -- if you gave poor people a whack of cash or assets right now, they'd (liquidate and) spend it on things they need rather than investing/holding it. )

View more: Next