Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: AshwinV 24 November 2014 11:59:09AM *  0 points [-]

What do I mean by ‘ambition’?

I know this is completely outta sync with what you were going for, but I couldnt resist quoting good ol' rational Quirell:

There was a half-smile on Professor Quirrell's face as he replied, "Not really, Miss Davis. In truth I do not care about that sort of thing in the slightest. But it is futile to count the witches among Ministers of Magic and other such ordinary folk leading ordinary existences, when Grindelwald and Dumbledore and He-Who-Must-Not-Be-Named were all men." The Defense Professor's fingers idly spun the button, turning it over and over. "Then again, only a very few folk ever do anything interesting with their lives. What does it matter to you if they are mostly witches or mostly wizards, so long as you are not among them? And I suspect you will not be among them, Miss Davis; for although you are ambitious, you have no ambition."

"That's not true! " said Tracey indignantly. "And what's it mean?"

Professor Quirrell straightened from where he had been leaning against the wall. "You were Sorted into Slytherin, Miss Davis, and I expect that you will grasp at any opportunity for advancement which falls into your hands. But there is no great ambition that you are driven to accomplish, and you will not make your opportunities. At best you will grasp your way upward into Minister of Magic, or some other high position of unimportance, never breaking the bounds of your existence."

Comment author: AshwinV 24 November 2014 11:17:04AM 0 points [-]

Update: I'm over it now. :D

Comment author: TheAncientGeek 24 November 2014 11:09:50AM 0 points [-]

Asking "is this reductive" and nothing else is not a good way to do philosophy.

Comment author: CCC 24 November 2014 09:13:43AM 0 points [-]

It would seem to me that Omega's actions would be as follows:

  • IF (Two box when empty And Two box when full) THEN Empty
  • IF (One box when empty And One box when full) THEN Full
  • IF (Two box when empty And One box when full) THEN Empty or Full
  • IF (One box when empty And Two box when full) THEN Refuse to present boxes

Cases 1 and 2 are straightforward. Case 3 works for the problem, no matter which set of boxes Omega chooses to leave.

In order for Omega to maintain its high prediction accuracy, though, it is necessary - if Omega predicts that a given player will choose option 4 - that Omega simply refuse to present the transparent boxes to this player. Or, at least, that the number of players who follow the other three options should vastly outnumber the fourth-option players.

Comment author: wedrifid 24 November 2014 07:30:07AM *  0 points [-]

When discussing transparent Newcomb, though, it's hard to see how this point maps to the latter two situations in a useful and/or interesting way.

Option 3 is of the most interest to me when discussing the Transparent variant. Many otherwise adamant One Boxers will advocate (what is in effect) 3 when first encountering the question. Since I advocate strategy 2 there is a more interesting theoretical disagreement. ie. From my perspective I get to argue with (literally) less-wrong wrong people, with a correspondingly higher chance that I'm the one who is confused.

The difference between 2 and 3 becomes more obviously relevant when noise is introduced (eg. 99% accuracy Omega). I choose to take literally nothing in some situations. Some think that is crazy...

In the simplest formulation the payoff for three is undetermined. But not undetermined in the sense that Omega's proposal is made incoherent. Arbitrary as in Omega can do whatever the heck it wants and still construct a coherent narrative. I'd personally call that an obviously worse decision but for simplicity prefer to define 3 as a defect (Big Box Empty outcome).

As for 4... A payoff of both boxes empty (or both boxes full but contaminated with anthrax spores) seems fitting. But simply leaving the large box empty is sufficient for decision theoretic purposes.

Out of interest, and because your other comments on the subject seem well informed, what do you choose when you encounter Transparent Newcomb and find the big box empty?

In response to comment by lalaithion on On Caring
Comment author: AmagicalFishy 24 November 2014 05:30:36AM *  0 points [-]

For the most part, I follow—but there's something I'm missing. I think it lies somewhere in: "It would be trivial for me to increase how much I care about one fo them, and therefore I should care about them as if I had already completed that process, even if I hadn't."

Is the underlying "axiom" here that you wish to maximize the number of effects that come from the caring you give to people, because that's what an altruist does? Or that you wish to maximize your caring for people?

To contextualize the above question, here's a (nonsensical, but illustrative) parallel: I get cuts and scrapes when running through the woods. They make me feel alive; I like this momentary pain stimuli. It would be trivial for me to woods-run more and get more cuts and scrapes. Therefore I should just get cuts and scrapes.

I know it's silly, but let me explain: A person usually doesn't want to maximize their cuts and scrapes, even though cuts and scrapes might be appreciated at some point. Thus, the above scenario's conclusion seems silly. Similarly, I don't feel a necessity to maximize my caring—even though caring might be nice at some point. Caring about someone is a product of my knowing them, and I care about a person because I know them in a particular way (if I knew a person and thought they were scum, I would not care about them). The fact that I could know someone else, and thus hypothetically care about them, doesn't make me feel as if I should.

If, on the other hand, the axiom is true—then why bother considering your intuitive "care-o-meter" in the first place?

I think there's something fundamental I'm missing.

(Upon further thought, is there an agreed-upon intrinsic value to caring that my ignorance of some LW culture has lead me to miss? This would also explain wanting to maximize caring.)

(Upon further-further thought, is it something like the following internal dialogue? "I care about people close to me. I also care about the fate of mankind. I know that the fate of mankind as a whole is far more important than the fate of the people close to me. Since I value internal consistency, in order for my caring-mechanism to be consistent, my care for the fate of mankind must be proportional to my care for the people close to me. Since my caring mechanism is incapable of actually computing such a proportionality, the next best thing is to be consciously aware of how much it should care if it were able, and act accordingly.")

Comment author: dxu 24 November 2014 05:13:32AM *  1 point [-]

Any difficulty here is in choosing the set of rewards that most usefully illustrate the interesting aspects of the problem.

I'd say that about hits the nail on the head. The permutations certainly are exhaustively specifiable. The problem is that I'm not sure how to specify some of the branches. Here's all four possibilities (written in pseudo-code following your example):

  1. IF (Two box when empty And Two box when full) THEN X
  2. IF (One box when empty And One box when full) THEN X
  3. IF (Two box when empty And One box when full) THEN X
  4. IF (One box when empty And Two box when full) THEN X

The rewards for 1 and 2 seem obvious; I'm having trouble, however, imagining what the rewards for 3 and 4 should be. The original Newcomb's Problem had a simple point to demonstrate, namely that logical connections should be respected along with causal connections. This point was made simple by the fact that there's two choices, but only one situation. When discussing transparent Newcomb, though, it's hard to see how this point maps to the latter two situations in a useful and/or interesting way.

Comment author: nshepperd 24 November 2014 03:37:38AM *  2 points [-]

Yes, I am assuming that I am capable of executing arbitrarily many instructions when computing my strategy.

But apparently you want to ignore the part when I said Omega has to have his own computing power increased to match. The fact that Omega is vastly more intelligent and computationally powerful than you is a fundamental premise of the problem. This is what stops you from magically "predicting him".

Look, in Newcomb's problem you are not supposed to be a "perfect reasoner" with infinite computing time or whatever. You are just a human. Omega is the superintelligence. So, any argument you make that is premised on being a perfect reasoner is automatically irrelevant and inapplicable. Do you have a point that is not based on this misunderstanding of the thought experiment? What is your point, even?

Comment author: nshepperd 24 November 2014 03:10:56AM 1 point [-]

This contradicts the accuracy stated at the beginning. Omega can't leave both boxes empty for people who try to adopt a mixed strategy AND also maintain his 99.whatever accuracy on one-boxers.

He can maintain his 99% accuracy on deterministic one-boxers, which is all that matters for the hypothetical.

Alternatively, if we want to explicitly include mixed strategies as an available option, the general answer is that Omega fills the box with probability = the probability that your mixed strategy one-boxes.

Comment author: SystemsGuy 24 November 2014 02:52:42AM *  0 points [-]

Some individuals (and I presume more here than most venues) struggle with any internal inconsistency, while others readily compartmentalize and move on. I am an engineer by training and of course most of my workmates are engineers, yet they represent a variety of religions as well. Most have some questions and doubts about their own, and plenty more about others, and yet that doesn't make a huge difference for day-to-day life.

Some would quickly conclude that such an engineer's judgement is questionable, and discount their work, but most seem to be adequately logical in other spheres.

Perhaps the better questions is one of utility -- what value does the individual get for their beliefs? I graduated with many Elect Engrs; let's presume one went to work on microprocessor design (driven by quantum theory) and another does correction math for GPS satellites (driven by relativity). It is well understood that the two theories have been objectively demonstrated to work well in their respective domains, and yet are mathematically incompatible (at best each may a simplification of a more universal rule). Both cannot be 'true', and while both could be false and likely are to some degree, they are both incredibly useful.

From a systems perspective I tend to fall back on the Systems rules-of-thumb, like "all models are wrong; some are useful", and "draw a box around what is working together to do what you're interested in, and analyze within". Compartmentalization allows one to get down to the work at hand, in support of a utilitarian view.

I am here to learn, though. Must inconsistency be driven out, or simply embraced as part of the imperfect human machine?

Comment author: wedrifid 24 November 2014 02:09:27AM *  1 point [-]

If I consider my predictions of Omega's predictions, that cuts off more branches, in a way which prevents the choices from even having a ranking.

It sounds like your decision making strategy fails to produce a useful result. That is unfortunate for anyone who happens to attempt to employ it. You might consider changing it to something that works.

"Ha! What if I don't choose One box OR Two boxes! I can choose No Boxes out of indecision instead!" isn't a particularly useful objection.

Comment author: ike 24 November 2014 02:09:16AM 0 points [-]

I haven't seen all the comments on this post nor kept up with the different versions, but: have you considered doing an analysis with added information as how long into the flu season the person is? That is, the likelihood of my getting the flu this year goes down as time goes on, while the costs of getting the shot remain constant: at which point would the costs outweigh the gains?

Comment author: wedrifid 24 November 2014 02:04:41AM *  1 point [-]

It's me who has to run on a timer.

No, Nshepperd is right. Omega imposing computation limits on itself solves the problem (such as it is). You can waste as much time as you like. Omega is gone and so doesn't care whether you pick any boxes before the end of time. This is a standard solution for considering cooperation between bounded rational agents with shared source code.

When attempting to achieve mutual cooperation (essentially what Newcomblike problems are all about) making yourself difficult to analyse only helps against terribly naive intelligences. ie. It's a solved problem and essentially useless for all serious decision theory discussion about cooperation problems.

Comment author: Jiro 24 November 2014 01:54:24AM 1 point [-]

I don't see how omega running his simulation on a timer makes any difference for this,

It's me who has to run on a timer. If I am only permitted to execute 1000 instructions to decide what my answer is, I may not be able to simulate Omega.

Though it may be convenient to postulate arbitrarily large computing power

Yes, I am assuming that I am capable of executing arbitrarily many instructions when computing my strategy.

the intended options for your strategy are clearly supposed to be "unconditionally one-box" and "unconditionally two-box", with potentially a mixed strategy allowed. Which is why you are provided wth no information whatsoever that would allow you to predict omega

I know what problem Omega is trying to solve. If I am a perfect reasoner, and I know that Omega is, I should be able to predict Omega without actually having knowledge of Omega's internals.

Actually, if you look at the decision tree for Newcomb's, the intended options for your strategy are clearly supposed to be "unconditionally one-box" and "unconditionally two-box",

Deciding which branch of the decision tree to pick is something I do using a process that has, as a step, simulating Omega. It is tempting to say "it doesn't matter what process you use to choose a branch of the decision tree, each branch has a value that can be compared independently of why you chose the branch", but that's not correct. In the original problem, if I just compare the branches without considering Omega's predictions, I should always two-box. If I consider Omega's predictions, that cuts off some branches in a way which changes the relative ranking of the choices. If I consider my predictions of Omega's predictions, that cuts off more branches, in a way which prevents the choices from even having a ranking.

Comment author: wedrifid 24 November 2014 01:54:18AM *  1 point [-]

As I argued in this comment, however, the scenario as it currently is is not well-specified; we need some idea of what sort of rule Omega is using to fill the boxes based on his prediction.

Previous discussions of Transparent Newcomb's problem have been well specified. I seem to recall doing so in footnotes so as to avoid distraction.

I have not yet come up with a rule that would allow Omega to be consistent in such a scenario, though, and I'm not sure if consistency in this situation would even be possible for Omega. Any comments?

The problem (such as it is) is that there is ambiguity between the possible coherent specifications, not a complete lack. As your comment points out there are (merely) two possible situations for the player to be in and Omega is able to counter-factually predict the response to either of them, with said responses limited to a boolean. That's not a lot of permutations. You could specify all 4 exhaustively if you are lazy.

IF (Two box when empty AND One box when full) THEN X
IF ...

Any difficulty here is in choosing the set of rewards that most usefully illustrate the interesting aspects of the problem.

Comment author: EHeller 24 November 2014 01:43:35AM *  0 points [-]

If your goal is to show that Omega is "impossible" or "inconsistent", then having Omega adopt the strategy "leave both boxes empty for people who try to predict me / do any other funny stuff" is a perfectly legitimate counterargument. It shows that Omega is in fact consistent if he adopts such strategy. You have no right to just ignore that counterargument.

This contradicts the accuracy stated at the beginning. Omega can't leave both boxes empty for people who try to adopt a mixed strategy AND also maintain his 99.whatever accuracy on one-boxers.

And even if Omega has way more computational than I do, I can still generate a random number. I can flip a coin thats 60/40 one-box, two-box. The most accurate Omega can be, then, is to assume I one box.

Comment author: soreff 24 November 2014 01:23:39AM 0 points [-]

The pain from the needle during the injection lasts just a few seconds, but the muscle pain at the injection site is noticeable for hours. That said, I'd rate it as much lower than ericyu3 rated it. For me, this is one of those situations where having the explanation for a sensation in hand, and knowing that it is self-limiting and harmless, makes a large difference. I'd be quite concerned if I had a pain of identical magnitude but with no explanation for what caused it.

Comment author: wedrifid 24 November 2014 01:03:37AM *  0 points [-]

I am too; I'm providing a hypothetical where the player's strategy makes this the least convenient possible world for people who claim that having such an Omega is a self-consistent concept.

It may be the least convenient possible world. More specifically it is the minor inconvenience of being careful to specify the problem correctly so as not to be distracted. Nshepperd gives some of the reason typically used in such cases.

Moreover, the strategy "pick the opposite of what I predict Omega does" is a member of a class of strategies that have the same problem

What happens when you try to pick the the opposite of what you predict Omega does is something like what happens when you try to beat Deep Fritz 14 at chess while outrunning a sports car. You just fail. Your brain is a few of pounds of fat approximately optimised for out-competing other primates for mating opportunities. Omega is a super-intelligence. The assumption that Omega is smarter than the player isn't an unreasonable one and is fundamental to the problem. Defying it is a particularly futile attempt to fight the hypothetical by basically ignoring it.

Generalising your proposed class to executing maximally inconvenient behaviours in response to, for example, the transparent Newcomb's problem is where it gets actually gets (tangentially) interesting. In that case you can be inconvenient without out-predicting the superintelligence and so the transparent Newcomb's problem requires more care with the if clause.

In response to comment by AmagicalFishy on On Caring
Comment author: lalaithion 23 November 2014 11:32:52PM 1 point [-]

For me, personally, I know that you could choose a person at random in the world, write a paragraph about them, and give it to me, and by doing that, I would care about them a lot more than before I had read that piece of paper, even though reading that paper hadn't changed anything about them. Similarly, becoming friends with someone doesn't usually change the person that much, but increases how much I care about them an awful lot.

Therefore, I look at all 7 billion people in the world, and even though I barely care about them, I know that it would be trivial for me to increase how much I care about one of them, and therefore I should care about them as if I had already completed that process, even if I hadn't

Maybe a better way of putting this is that I know that all of the people in the world are potential carees of mine, so I should act as though I aready care about these people in deference to possible future-me.

In response to comment by Decius on On Caring
Comment author: AmagicalFishy 23 November 2014 10:39:05PM *  0 points [-]

I second this question: Maybe I'm misunderstanding something, but part of me craves a set of axioms to justify the initial assumptions. That is: Person A cares about a small number of people who are close to them. Why does this equate to Person A having to care about everyone who isn't?

View more: Next