Comment author: wedrifid 25 August 2010 07:07:02AM 6 points [-]

Hereinafter, "to Know x" means "to be objectively right about x, and to be subjectively 100 percent certain of x,

I know nothing! Nothing!

Comment author: GrateGoo 01 September 2010 09:13:52PM *  1 point [-]

My previous post resulted in 0 points, despite being very thoroughly thought-through. A comment on it, consisting of the four words "I know nothing! Nothing!" resulted in 4 points. If someone could please explain this, I'd be a grateful Goo.

Comment author: [deleted] 11 August 2010 06:38:26AM *  3 points [-]

Where should the line be drawn regarding the status of animals as moral objects/entities? E.G Do you think it is ethical to boil lobsters alive? It seems to me there is a full spectrum of possible answers: at one extreme only humans are valued, or only primates, only mammals, only veterbrates, or at the other extreme, any organism with even a rudimentary nervous system (or any computational, digital isomorphism thereof), could be seen as a moral object/entity.

Now this is not necessarily a binary distinction, if shrimp have intrinsic moral value it does not follow that they must have a equal value to humans or other 'higher' animals. As I see it, there are two possibilities; either we come to a point where the moral value drops to zero, or else we decide that entities approach zero to some arbitrary limit: e.g. a c. elegans roundworm with its 300 neurons might have a 'hedonic coefficient' of 3x10^-9. I personally favor the former, the latter just seems absurd to me, but I am open to arguments or any comments/criticisms.

In response to comment by [deleted] on Open Thread, August 2010
Comment author: GrateGoo 25 August 2010 05:59:56AM *  1 point [-]

Suppose sentient beings have intrinsic value in proportion to how intensely they can experience happiness and suffering. Then the value of invertebrates and many non-mammal vertebrates is hard to tell, while any mammal is likely to have almost as much intrinsic value as a human being, some possibly even more. But that's just the intrinsic value. Humans have a tremendously greater instrumental value than any non-human animal, since humans can create superintelligence that can, with time, save tremendous amounts of civilisations in other parts of the universe from suffering (yes, they are sparse, but with time our superintelligence will find more and more or them, in theory ultimately infinitely many).

The instrumental value of most humans is enormously higher than the intrinsic value of the same persons - given that they do sufficiently good things.

Comment author: PaulAlmond 22 August 2010 11:56:59PM 1 point [-]

I disagree with that. The being in Newcomb's problem wouldn't have to be all-knowing. He would just have to know what everyone else is going to do conditional on his own actions. This would mean that any act of prediction would also cause the being to be faced with a choice about the outcome.

For example:

Suppose I am all-knowing, with the exception that I do not have full knowledge about myself. I am about to make a prediction, and then have a conversation with you, and then I am going to sit in a locked metal box for an hour. (Theoretically, you could argue that even then I would affect the outside world, but it will take time for chaos to become an issue, and I can factor that in.) You are about to go driving.

I predict that if I tell you that you will have a car accident in half an hour, you will drive carefully and will not have a car accident.

I also predict that if I do not tell you that you will have a car accident in half an hour, you will drive as usual and you will have a car accident.

I lack full self-knowledge. I cannot predict whether I will tell you until I actually decide to tell you.

I decide not to tell you. I get in my metal box and wait. I know that you will have a car accident in half an hour.

My lack of complete self-knowledge merely means that I do not do pure prediction: Instead any prediction I make is conditional on my own actions and therefore I get to choose which of a number of predictions comes true. (In reality, of course, the idea that I really had a "choice" in any free will sense is debatable, but my experience will be like that.)

It would be the same for Newcomb's boxes. Now, you could argue that a paradox could be caused if the link between predictions and required actions would force Omega to break the rules of the game. For example, if Omega predicts that if he puts the money in both boxes, you will open both boxes, then clearly Omega can't follow the rules. However, this would require some kind of causal link between Omega's actions and the other players. There could be such a causal link. For example, while Omega is putting the money in the boxes, he may disturb weather patterns with his hands, and due to chaos theory make it rain on the other player on his way to play game, causing him to open both boxes. However, it should seem reasonable that Omega could manage his actions accordingly to control this: He may have to move his hands a particular way, or he may need to ensure that the game is played very soon after the boxes are loaded.

Comment author: GrateGoo 25 August 2010 03:36:22AM *  0 points [-]

Hereinafter, "to Know x" means "to be objectively right about x, and to be subjectively 100 percent certain of x, and to have let the former 'completely scientifically cause' the latter (i.e. to have used the former to create the latter in a completely scientific manner), such that it cannot, even theoretically, be the case that something other than the former coincidentally and crucially misleadingly caused the latter - and to Know that all these criteria are met".

Anything that I merely know ("know" being defined as people usually seem to implicitly define it in their use of it), as opposed to Know, may turn out to be wrong (for all that I know). It seems that the more our scientists know, the more they realize that they don't know. Perhaps this "rule" holds forever, for every advancing civilisation (with negligible exceptions)? I think there could not even theoretically be any Knowing in the (or any) world. I conjecture that, much like it's universally theoretically impossible to find a unique integer for every unique real, it's universally theoretically impossible for any being to Know anything at all, such as for example what box(es) a human being will take.

Nick Bostrom's Simulation Argument seems to show that any conceivable being that could theoretically exist might very well (for all he (that being) knows) be living in a computer simulation controlled by a mightier being than himself. This universal uncertainty means that no being could Know that he has perfect powers of prediction over anything whatsoever. Making a "correct prediction" partly due to luck isn't having perfect powers of prediction, and a being who doesn't Know what he is doing cannot predict anything correctly without at least some luck (because without luck, Murphy's law holds). This means that no being could have perfect powers of prediction.

Now let "Omeg" be defined as the closest (in terms of knowledge of the world) to an all Knowing being (Omega) that could theoretically exist. Let A be defined as the part(s) of an Omeg that are fully known by the Omeg itself, and let B be defined as whatever else there may be in an Omeg. I suggest that in no Omeg of at least the size of the Milky Way can the B part be too small to secretly contain mechanisms that could be stealthily keeping the Omeg arbitrarily ignorant by having it falsely perceive arbitrarily much of its own wildest thought experiments (or whatever other unready thoughts it sometimes produces) to be knowledge (or even Knowledge). I therefore suggest that B, in any Omeg, could be keeping its Omeg under the impression that the A part is sufficient for correct prediction of, say, my choice of boxes, while in reality it isn't. Conclusion: no theoretically possible being could perfectly predict any other being's choice of boxes.

You may doubt it, but you can't exclude the possibility. This means you also can't exclude the possibility that whatever implications Newcomb's problem seems to produce that wouldn't occur to people if Omega were replaced by, say, a human psychologist, are implications that occur to people only because the assumption, that there could be such a thing as a perfect predictor of something/anything, is an assumption too unreasonable to be worthy of acceptance, as its crucial underpinnings don't make sense (like it doesn't make sense to assume that there is an integer for every real) - and as it can, because of this, be expected to produce arbitrarily misleading conclusions (about decision theory in this case) - much like many seemingly reasonable but heavily biased extreme thought experiments designed to smear utilitarianism scare even very skilled thinkers into drawing false conclusions about utilitarianism.

Or suppose someone goes to space, experiences weightlessness, thinks: "hey, why doesn't my spaceship seem to exert any gravity on me?" and draws the conclusion: "it's not gravity that keeps people down on Earth; it's just that the Earth sucks". Like that conclusion would be flawed, the conclusion that Newcomb's problem shows that we should replace Causal Decision Theory with Evidential Decision Theory is flawed.

So, to be as faithful to the original Newcomb thought-experiment as is possible within reason, I'd interpret it in the way that just barely rids its premises of theoretical impossibility: I'd take Omega to mean Omeg, as defined above. An Omeg is fallible, but probably most of the time better than me at predicting my behavior, so I should definitely one-box, for the same reason that I should one-box if the predictor were a mere human being who just knew me very well. To risk a million dollar just to possibly get another 1000 dollar just isn't worth it. Causal Decision Theory leads me to this conclusion just fine.

*) You might think B would be "the real" (or "another, smarter") Omeg, by controlling A. But neither B nor A can rationally completely exclude the possibility that the other one of them is in secret control of both of them. So no one of them can have "perfect powers of prediction" over any being whatsoever.

Comment author: Perplexed 22 August 2010 05:49:55PM 7 points [-]

Reminds me of a story, set in a lazy Mark Twain river town. Two friends walking down the street. First says to second, "See that kid? He is really stupid." Second asks, "Why do you say that?" First answers, "Watch". Approaches kid. Holds out nickel in one hand and dime in the other. Asks kid which he prefers. "I'll take the nickel. It's bigger". Man hands nickel to kid with smirk, and the two friends continue on.

Later the second man comes back and attempts to instruct the kid. "A dime is worth twice the value, that is it buys more candy", says he, "even though the nickel looks bigger." The kid gives the man a pitying look. "Ok, if you say so. But I've made seven nickels so far this month. How many dimes have you made?"

Which brings me to my real point - empirical research, I'm sure you have seen it, in which player 1 is asked to specify a split of $10 between himself and player 2. Player 2 then chooses to accept or reject. If he rejects, neither player gets anything. As I recall, when greedy player 1 specifies more than about 70% for himself, player 2 frequently rejects even though he is costing himself money. This can only be understood in classical "rational agent" game theory by postulating that player 2 does not believe researcher claims that the game is a one-shot.

What is the point? Well, perhaps people who have read about Newcomb problems are assuming (like most people in the research) that, somehow or other, greed will be punished.

Comment author: GrateGoo 23 August 2010 01:04:45AM *  -1 points [-]

Is it plausible that evolution would gradually push those 70% down to 30% or even lower, given enough time? There may not yet have been enough time for a strong enough group selection in evolution to create such an effect, but sooner or later it should happen, shouldn't it? I'm thinking a species with such a great degree of selflessness would be more likely to survive than the present humanity is, because a larger percentage of them would cooperate about existential risk reduction than is the case in present humanity. Yet, 10-30% is still not 0%, so even with 10% there would still be enough of selfishness to make sure they wouldn't end up refusing each other's gifts until they all starve to death or something.

Can group selection of genes for different psychological constitution in humans already explain why player 1 takes only 70% and not, say, at least 90%, on average, in the game you describe?

What do chimps do? Does a chimp player 1 take more or less than 70%?

Comment author: dv82matt 16 August 2010 11:56:42PM -6 points [-]

Newcomb’s problem is a poor vehicle for illustrating points about rationality. It is a minefield of misconceptions and unstated assumptions. In general the one boxers are as wrong as the two boxers. When Omega is not infallible the winning strategy depends on how Omega arrives at the prediction. If that information is not assumed or somehow deducible then the winning strategy is impossible to determine.

Your point about casual decision theory being flawed in some circumstances may be correct but using Newcomb’s problem to illustrate it detracts from the argument.

Consider a condensed analogy. Someone will roll a standard six sided die. You can bet on six or not-six to come up. Both bets double your money if you win. Assume betting on six wins. Since six wins any decision theory that has you betting not-six is flawed.

Comment author: GrateGoo 22 August 2010 11:16:03PM -1 points [-]

Any conclusions, about how things work in the real world, drawn from Newcomb's problem, crucially rest on the assumption that an all-knowing being might, at least theoretically, as a logically consistent concept, exist. If this crucial assumption is flawed, then any conclusions drawn from Newcomb's problem are likely flawed too.

To be all-knowing, you'd have to know everything about everything, including everything about yourself. To contain all that knowledge, you'd have to be larger than it - otherwise there would be no matter or energy left to perform the activity of knowing it all. So, in order to be all-knowing, you'd have to be larger than yourself. Which is theoretically impossible. So, the Newcomb problem crucially rests on a faulty assumption: that something that is theoretically impossible might be theoretically possible.

So, conclusions drawn from Newcomb's problem are no more valid than conclusions drawn from any other fairy tale. They are no more valid than, for example, the reasoning: "if an omnipotent and omniscient God would exist who would eventually reward all good humans with eternal bliss, all good humans would eventually be rewarded with eternal bliss -> all good humans will eventually be rewarded with eternal bliss whether the existence of an omnipotent and omniscient God is even theoretically possible or not".

One might think that Newcomb's problem could be altered; one might think that instead of an "all-knowing being" it could assume the existence a non-all-knowing being that however knows what you will choose. But if the MWI is correct, or if the universe is otherwise infinitely large, not all of the infinitely many identical copies of you would be controlled by any such being. If they would, that would mean that that being would have to be all-knowing. Which, as shown, is not possible.