Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: KnaveOfAllTrades 17 September 2014 10:37:12PM *  0 points [-]

This post is brilliant.

(Sensations of potential are fascinating to me. I noticed a few weeks ago that after memorizing a list of names and faces, I could predict in the first half second of seeing the face whether or not I'd be able to retrieve the name in the next five seconds. Before I actually retrieved the name. What??? I don't know either.)

Right! When telling people about Anki, I often mention the importance of not self-deluding about whether one knows the answer. But sometimes I also mention how I mark a card as 'Easy' before I've retrieved or subvocalized the answer. It definitely felt like the latter was not self-delusion (especially when Anki was asking me what the capital of the UK was, say). But I felt unable to communicate why it was not self-delusion, and worried that without the other person understanding that mental phenomenon, they would think I was self-deluding and conclude that self-delusion is actually okay after all.

I vaguely noticed that awkwardness to some degree, but I still need to work on the skill of noticing such impasses and verbalizing them. And I certainly wasn't conscious enough of it, or didn't dwell enough on it, to think more about noticing.

Comment author: skeptical_lurker 17 September 2014 09:10:59PM *  0 points [-]

Are you implying that rational agents can be successfully blackmailed by other agents that simulate them inaccurately? (This does seem plausible to me, and is an interesting rare example of accurate knowlage posing a hazard.)

Comment author: therufs 17 September 2014 09:05:45PM 0 points [-]

why anybody would either want to call it marriage

I don't think anyone suggested that?

or frame it as being an alternative to marriage.

Some marriages are of convenience, and the honorific sense doesn't apply as well to people who don't fit the romantic ideal of marriage.

Comment author: simplicio 17 September 2014 08:50:24PM 1 point [-]

If traditional marriage is a sparrow, then marriage with no-fault divorce is a penguin, and 5 college kids sharing a house is a centipede. Type specimen, non-type specimen, wrong category.

Social expectations are mutable, yes - what of it? Do you think it's desirable or inevitable that marriage just become a fancy historical legal term for income splitting on one's tax return? Do you think sharing a house in college is going to be, or ought to be, hallowed and encouraged?

Comment author: Lumifer 17 September 2014 08:29:27PM *  1 point [-]

which makes me wonder why anybody would either want to call it marriage

I could make exactly the same argument about divorce-able marriage and wonder why would anyone call this get-out-whenever-you-want-to arrangement "marriage" :-D

The point is, the "thick layer of social expectations" is not immutable.

Comment author: Diadem 17 September 2014 08:02:24PM 0 points [-]

I wonder if this can not be partially explained by people wanting to answer quickly. The teacher says you can make as many guesses as you like, but we still instinctively feel like we do better if we do it faster.

Imagine the same test, but now with the last line reading: "You can make as many guesses as you like, but you get graded on how fast you get the right result". With the rule it is a lot more rational to not spend too much time on verification of your hypothesized rule. I have no idea what the best strategy is, I guess it depends on your priors about the rule-space, but it probably does not involve spending a lot of questions on falsification.

My guess is that many people approach the problem as if it is of the above variety, even though it isn't. So while positive bias no doubt plays a part, I think a desire to answer quickly also factors hugely.

This is testable. Give people a 10 dollar reward for giving the correct answer, and explicitly tell them that the number of guesses does not affect this reward. I hypothesize that the fraction of people getting the correct answer will go up significantly.

(I know this is a very old thread, but this sequence still features prominently on the site, so I have some hopes that people still read this occasionally :P)

Comment author: simplicio 17 September 2014 07:46:47PM 0 points [-]

This framing is marginally saner, but the weird panicky eschatology of pop-environmentalism is still present. Apparently the author thinks that using up too many resources, or perhaps global warming, currently represent human extinction level threats?

Comment author: simplicio 17 September 2014 07:38:59PM *  1 point [-]

To my mind, the giving of tax breaks etc. to married folks occurs because (rightly or wrongly) politicians have wanted to encourage marriage.

I agree that in principle there is nothing wrong with 3 single moms or 5 college students forming some sort of domestic partnership contract, but why give them the tax breaks? Do college kids living with each other instead of separately create some sort of social benefit that "we" the people might want to encourage? Why not just treat this like any other contract?

Apart from this, I think the social aspect of marriage is being neglected. Marriage for most people is not primarily about joint tax filing, but rather about publicly making a commitment to each other, and to their community, to follow certain norms in their relationship (e.g., monogamy; the specific norms vary by community). This is necessary because the community "thinks" pair bonding and childrearing are important/sacred/weighty things. In other words, "married" is a sort of honorific.

Needless to say, society does not think 5 college students sharing a house is an important/sacred/weighty thing that needs to be honoured.

This thick layer of social expectations is totally absent for the kind of arm's-length domestic partnership contract you propose, which makes me wonder why anybody would either want to call it marriage or frame it as being an alternative to marriage.

Comment author: dankane 17 September 2014 07:14:43PM 1 point [-]

I think you mean that rational agents cannot be successfully blackmailed by others agents that for which it is common knowledge that the other agents can simulate them accurately and will only use blackmail if they predict it to be successful. All of this of course in the absence of mitigating circumstances (including for example the theoretical likelihood of other agents that reward you for counterfactualy giving into blackmail under these circumstances).

Comment author: Eliezer_Yudkowsky 17 September 2014 06:11:39PM 2 points [-]

Rational agents cannot be successfully blackmailed by other agents that simulate them accurately, and especially not by figments of their own imagination.

Comment author: dankane 17 September 2014 05:15:03PM *  1 point [-]

I suppose. On the other hand, is that because other people can read your mind or because you have emotional responses that you cannot suppress and are correlated to what you are thinking? This is actually critical to what counterfactuals you want to construct.

Consider for example the terrorist who would try to bring down an airplane that he is on given the opportunity. Unfortunately, he's an open book and airport security would figure out that he's up to something and prevent him from flying. This is actually inconvenient since it also means he can't use air travel. He would like to be able to precommit to not trying to take down particular flights so that he would be allowed on. On the other hand, whether or not this would work depends on what exactly airport security is picking up on. Are they actually able to discern his intent to cause harm, or are they merely picking up on his nervousness at being questioned by airport security. If it's the latter, would an internal precommitment to not bring down a particular flight actually solve his problem?

Put another way, is the TSA detecting the fact that the terrorist would down the plane if given the opportunity, or simply that he would like to do so (in the sense of getting extra utils from doing so).

Comment author: Manfred 17 September 2014 05:00:48PM 2 points [-]

Your blog posts are great!

Comment author: dankane 17 September 2014 04:18:33PM 1 point [-]

I'm sure we could think of some

OK. Name one.

Comment author: army1987 17 September 2014 04:04:08PM 2 points [-]

Humans are not perfect deceivers.

Comment author: Jiro 17 September 2014 04:01:46PM *  1 point [-]

The problem requires that Omega be always able to figure out what you do. If Omega can only figure out what you can do under a limited set of circumstances, you've changed one of the fundamental constraints of the problem.

You seem to be thinking of this as "the only time someone won't come to a decision fast enough is if they deliberately stall", which is sort of the reverse of fighting the hypothetical--you're deciding that an objection can't apply because the objection applies to an unlikely situation.

Suppose that in order to decide what to do, I simulate Omega in my head as one of the steps of the process? That is not intentionally delaying, but it still could result in halting problem considerations. Or do you just say that Omega doesn't give me the money if I try to simulate him?

Comment author: dankane 17 September 2014 03:46:45PM 1 point [-]

Fine. Your opponent actually simulates what UDT would do if Omega had told it that and returns the appropriate response (i.e. it is CooperateBot, although perhaps your finite prover is unable to verify that).

Comment author: AABoyles 17 September 2014 03:25:57PM 3 points [-]

I experienced a far less conscious and intentional version of noticing reflexively throughout my childhood. Specifically, I became very highly attuned to the act of stepping on cracks in pavings in response to the schoolyard rhyme "Don't step on the crack or you'll break your Momma's back." I never labored under the delusion that there was some mystical force which would cause gross harm to my mother if I did (or didn't) step on a crack--it was more of a game. A game which lasted from early in elementary school through puberty. I have other gamified (if immature) examples of passive noticing--the game) comes to mind. (Apologies if anyone still cares about the game, by the way). Now, these parallels are shallow in that I wasn't meta-noticing as Brienne was. But it does lend a concept I'll find useful in applying the principles of noticing and meta-noticing: namely, the act of gamification. I question, however, whether gamification lends itself to moving the intention from conscious searching to subconscious noticing.

Comment author: nshepperd 17 September 2014 03:18:47PM 2 points [-]

If Omega only puts the million in if it finds a proof fast enough, it is then possible that you will one-box and not get the million.

Yes, it's possible, and serves you right for trying to be clever. Solving the halting problem isn't actually hard for a large class of programs, including the usual case for an agent in a typical decision problem (ie. those that in fact do halt quickly enough to make an actual decision about the boxes in less than a day). If you try to deliberately write a very hard to predict program, then of course omega takes away the money in retaliation, just like the other attempts to "trick" omega by acting randomly or looking inside the boxes with xrays.

In response to comment by Decius on Fake Explanations
Comment author: Jiro 17 September 2014 02:47:01PM *  0 points [-]

and going from "almost certain" to "certain" would add a small value to a correct answer but a large penalty to a wrong answer.

It's hard to come up with a system where the student doesn't benefit from lying about his certainty. What you describe would fix the case from 4 (almost certain) to 5 (certain), but you need to get all the cases to work and it's plausible that fixing the 4 to 5 case (and, in general, increasing the incentive to pick 4) breaks the 3 to 4 case.

After all, you can't have all the transitions between certainty values add a small value to a correct answer. You must have a transition where a large value is added for a correct answer and your system may break down around such transitions.

Comment author: Lumifer 17 September 2014 02:39:06PM 0 points [-]

I don't particularly like your "ellipse" generalization, either, because it's just wrong. ... Bringing ellipses into the issue is just an intuitive, illustrative fiction, which I really don't appreciate very much because it's not particularly informative and it isn't scientifically sound at all.

I think you're mistaken about that. An ellipse is the shape of a multivariate normal distribution, for example. In fact, there is the entire family of elliptical distributions which are, to quote Wikipedia, "a broad family of probability distributions that generalize the multivariate normal distribution. Intuitively, in the simplified two and three dimensional case, the joint distribution forms an ellipse and an ellipsoid, respectively, in iso-density plots."

a perfect correlation would be linear

That's a meaningless phrase, correlation is linear by definition. Moreover, it's a particular measure of dependency which can be misleading.

View more: Next