Comment author: cousin_it 11 December 2013 02:51:14PM *  9 points [-]

A sorcerer has two ways to manipulate people:

1) Move things around in the world.

2) Directly influence people's minds.

I'm not going to talk about option 2 because it stops people from being perfect reasoners. (If there's a subset of option 2 that still lets people be perfect reasoners, I'd love to hear it - that might be the most interesting part of the puzzle). That leaves option 1.

Here's a simple model of option 1. Nature shuffles a deck of cards randomly, then a sorcerer (if one exists) has a chance to rearrange the cards somehow, then the deck is shown to an observer, who uses it as Bayesian evidence for or against the sorcerer's existence. We will adopt the usual "Nash equilibrium" assumption that the observer knows the sorcerer's strategy in advance. This seems like a fair idealization of "moving things around in the world". What would the different types of sorcerers do?

Note that if both Bright and Dark might exist, the game becomes unpleasant to analyze, because Dark can try to convince the observer that Bright exists, which would mean Dark doesn't exist. To simplify the game, we will let the observer know which type of sorcerer they might be playing against, so they only need to determine if the sorcerer exists.

A (non-unique) best strategy for Bright is to rearrange the cards in perfect order, so the observer can confidently say "either Bright exists or I just saw a very improbable coincidence". A (non-unique) best strategy for Dark is to leave the deck alone, regardless of the observer's prior. Invisible has the same set of best strategies as Dark. I won't spell out the proofs here, anyone sufficiently interested should be able to work them out.

To summarize: if sorcerers can only move things around in the world and cannot influence people's minds directly, then Bright does as much as possible, Invisible and Dark do as little as possible, and the observer only looks at things in the world and doesn't do anything like "updating on the strength of their own beliefs". The latter is only possible if sorcerers can directly influence minds, which stops people from being perfect reasoners and is probably harder to model and analyze.

Overall it seems like your post can generate several interesting math problems, depending on how you look at it. Good work!

Comment author: BT_Uytya 13 December 2013 05:59:47PM 0 points [-]

A (non-unique) best strategy for Dark is to leave the deck alone, regardless of the observer's prior

If I were a Dark, I would try to rearrange the cards so they look random to an unsophisticated observer. No long runs of same color, no obvious patterns in numbers (people are bad random number generators, they think that random string is string without any patterns, not string without big patterns, 17 is the most random number, blah blah blah).

(It's possible that the variation of it can be a good strategy even against more sophisticated agents, because if by a pure chance string of cards has low Kolmogorov complexity, agent is going to take this as evidence for Bright, and I don't want him to believe in Bright)

Comment author: BT_Uytya 07 December 2013 07:05:04PM 9 points [-]

No one makes the wrong decisions for reasons they think are wrong. The more clever the man, as the Nroni were fond of saying, the more apt he was to make a fool of himself. We all argue ourselves into our mistakes.

Scott R. Bakker, The White-Luck Warrior

Comment author: BT_Uytya 07 December 2013 06:58:09PM 5 points [-]

If you find yourself taken unawares by someone you thought you knew, recall that the character revealed is as much your own as otherwise. When it comes to Men and their myriad, mercenary natures, revelation always comes in twos.

– Managoras, Ode to the Long-Lived Fool

Scott R. Bakker, The White-Luck Warrior

Comment author: BT_Uytya 07 December 2013 07:04:49PM *  7 points [-]

Another good fictional epigraph from the same book:

Any fool can see the limits of seeing, but not even the wisest know the limits of knowing. Thus is ignorance rendered invisible, and are all Men made fools.

– Ajencis, The Third Analytic of Men

Comment author: BT_Uytya 07 December 2013 07:01:55PM 14 points [-]

This was what made the fall of Iothiah so disastrous. <...> Strategically, the loss of Iothiah was little more than a nuisance.

Symbolically, however…

The crisis she faced was a crisis in confidence, nothing more, nothing less. The less her subjects believed in the Empire, the less some would sacrifice, the more others would resist. It was almost arithmetic. The balance was wobbling, and all the world watched to see which way the sand would spill. She had made a resolution to act as if she believed to spite all those who doubted her as much as anything else, and paradoxically, they had all started believing with her. It was a lesson Kellhus had drummed into her countless times and one she resolved never to forget again.

To know is to have power over the world; to believe is to have power over men.

Scott R. Bakker, The White-Luck Warrior

Comment author: BT_Uytya 07 December 2013 06:58:09PM 5 points [-]

If you find yourself taken unawares by someone you thought you knew, recall that the character revealed is as much your own as otherwise. When it comes to Men and their myriad, mercenary natures, revelation always comes in twos.

– Managoras, Ode to the Long-Lived Fool

Scott R. Bakker, The White-Luck Warrior

Comment author: BT_Uytya 07 December 2013 06:57:11PM *  1 point [-]

(potential spoilers removed, so if this dialogue doesn't make sense, be assured that it makes sense in context)

"Just wait," Zsoronga said. "Something auspicious will happen. Some twist will keep you here, where you can discharge your fate! Wait and see."

"And what if they know?" Sorweel finally asked, voicing the one alternative they had passed over in silence.

"They don't know."

"But wh-"

"They don't know."

Zsoronga, Sorweel was beginning to realize, possessed the enviable ability to yoke his conviction to his need - to believe, absolutely, whatever his heart required. For Sorweel, belief and want always seemed like ropes too short to bind together, forcing him to play the knot as a result.

Scott R. Bakker, The White-Luck Warrior

Comment author: gjm 16 November 2013 10:18:45AM 17 points [-]

Nice example of how using a probability of exactly zero can screw you over. Two observations.

  1. Could have done with a link to Eliezer's 0 and 1 are not probabilities from back in 2008.

  2. I say you can get to 99.99% confidence that 1159 is prime (if it actually is; I haven't checked); probably 99.9999%. Suppose you (a) write a program to check all possible divisors, test that it gives the right answers for everything up to 100, and run it multiple times in case of cosmic rays; (b) look it up in a table of prime numbers; (c) apply, by hand, one of the fancy number-theoretical primality tests (most of these are probabilistic -- but again you can find statements of the form "If a number less than 10^12 passes these specific tests and isn't one of the following short list of exceptions, it is prime"). Then I reckon that apart from theories where what's wrong is your brain a,b,c are clearly extremely close to independent; (a) has well less than 0.001 chance of failure, (b) well less than 0.01, and (c) well less than 0.1; so the only hypothesis you need to consider that might take the probability above 10^-6 is that you're delusional in some way that specifically messes up your ability to tell whether 1159 is prime. Now (d) this is surely extraordinarily rare -- delusions in general aren't so rare, but this is something very specific and weird; and (e) if your mind is that badly screwed up then attempting to work with probabilities is probably kinda meaningless anyway. (Theorems like Cox's say that you should use probabilities as measures of confidence, and compute with them in the standard ways, if you are a rational agent. If you are known to be a catastrophically irrational agent, then what probabilities you assign is probably not the greatest of your worries.)

Comment author: BT_Uytya 22 November 2013 03:28:53PM *  0 points [-]

apart from theories where what's wrong is your brain

(just amused by the possibility)

Also, it is possible that Peano arithmetic isn't consistent; if so, either the very concept of 'primality' doesn't make any sense, or it can just mess up the primality tests which were used in creation of (b) and (c), and the connection between "1159 if prime" and "this program outputs True and halts" as well.

Of course, it screws up any application of Cox's theorem here, even worse than in delusion case.

Comment author: BT_Uytya 28 August 2013 09:32:18PM 3 points [-]

In the new version of Newcomb's problem, you have to choose between a box containing the map and a box containing warm fuzzies.

Self-delusion or accurate beliefs? We all can empathize with this choice.

Comment author: jasonsaied 08 August 2013 05:07:40AM 2 points [-]

I'd say it's highlighting the human fallacy to try to ignore and escape from bad news. Instead of facing this prophecy, they just destroyed the ship that delivered it to them and told themselves they were safe.

Comment author: BT_Uytya 09 August 2013 11:26:41PM 1 point [-]

Actually, prophesy was about the ship; the spaceship crashed into Aragena, their planet, and then curious inhabitants looked inside (and found nothing dangerous). After that came the messenger of their King and told them that they all are doomed.

And they indeed were.

Comment author: RolfAndreassen 03 August 2013 08:54:44PM 5 points [-]

Not quite seeing the applicability as a rationality quote; but in "it's bed" you should drop the apostrophe.

Comment author: BT_Uytya 09 August 2013 11:09:13PM 0 points [-]

Probably I'm incredible late with that, but:

a) thank you, embarrassing mistake fixed

b) I was fascinated with the "volatile atoms" bit. It feels like a line taken from a poem on reductionism. I'm not sure that I managed to convey it because I'm not so much versed in English fiction and poetry.

Also, I liked their safety measures, it's a pity they hadn't worked in the end.

View more: Prev | Next