Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: private_messaging 21 May 2013 04:54:16AM *  -1 points [-]

The point with akrasia was to illustrate that more than 1 volition inside 1 head isn't even rare here to begin with. The actual issue is that, of course, you aren't creating some demon out of nothing. You are re-purposing existing part of your brain, involved in the internal monologue or even mental visualization as well, making this part not integrate properly with the rest under one volition. There's literally less of your brain under your volition.

This topic is extremely retarded. This tulpa stuff resembles mental illness. Now, you wanna show off your "rationality" according to local rules of showing off your rationality, by rejecting the simple looking argument that it should be avoided like mental illness is. "Of course" it's pattern matching, "non central fallacy" and other labels that you were taught here to give to equally Bayesian reasoning when it arrives at conclusions you don't like. Here's the thing: Yeah, it is in some technical sense not mental illness. It most closely resembles one. And it is as likely to be worse as it is likely to be better*, and it's expected badness is equal to that of mental illness, and the standard line of reasoning is going to approximate utility maximization much better than this highly biased reasoning where if it is not like mental illness it must be better than mental illness, or worse, depending to which arguments pop into your head easier. In good ol caveman days, people with this reasoning fallacy, they would eat a mushroom, get awfully sick, and then eat another mushroom that looks quite similar to the first, but is a different mushroom of course, in the sense that it's not the exact same physical mushroom body, and get awfully sick, and then do it again, and die.

Let's suppose it was self inflicted involuntary convulsion fits, just to make an example where you'd not feel so much like demonstrating some sort of open mindness. Now the closest thing would have been real convulsion fits, and absent other reliable evidence either way expected badness of self inflicted convulsion fits would clearly be equal.

Also, by the way, what ever mental state you arrive at by creating a tulpa, is unlikely to be a mental state not achievable by one or the other illness.

  • if its self inflicted, for example standard treatments might not work.
Comment author: private_messaging 19 May 2013 07:29:33PM *  1 point [-]

Yes. The issue is that the argument "look at periodic table, it's so big, there would be at least one" requires that the fact of fission releasing neutrons would be assumed independent across nuclei.

Comment author: private_messaging 18 May 2013 06:31:31PM 5 points [-]

Humans seem to have something similar, e.g. humans who are unable to form mental imagery can talk normally but perform badly on any task which is difficult to do without mental imagery. And in programming there are people who seem intelligent on basis of how they talk about programming but are surprisingly bad at constructing anything that works for a given, fixed task.

Comment author: private_messaging 17 May 2013 04:54:15PM *  1 point [-]

The point is that when someone "hears voices" - which do not respond to the will in the same way in which internal monologues do, there's no demons, there's no new brain added. It is existing brain regions involved in the internal monologue failing to integrate properly with the rest. Less dramatically, when people claim they e.g. want to get on a diet but are mysteriously unable to - their actions do not respond to what they think is their will but instead respond to what they think is not their will - it's the regions which make decisions about food intake not integrating with the regions that do the talking (Proper integration either results in the diet or absence of the belief that one wants to be on a diet). The bottom line is, brain is not a single CPU of some kind. It is a distributed system parts of which are capable of being in conflict, to the detriment of the well being of the whole.

Comment author: private_messaging 13 May 2013 10:16:17AM 0 points [-]

There's only that much brain to go around with, the brain, being for the most part a larger version of australopithecus brain, as it is can have trouble seeing itself as a whole (just look at that "akrasia" posts where you can see people's talkative parts of the brain disown the decisions made by the decision-making parts). Why do you expect anything but detrimental effects from deepening the failure of the brain to work as a whole?

Comment author: private_messaging 12 May 2013 04:32:38PM *  0 points [-]

It would be pretty weird to argue that human lives decay in utility based on how many there are.

Well, suppose there's mind uploads, and one mind upload is very worried about himself so he runs himself multiply redundant with 5 exact copies. Should this upload be a minor utility monster?

3^^^3 is far more than there are possible people.

If you found out that the universe was bigger than you thought, that there really were far more humans in the universe somehow, would you just stop caring about human life?

Bounded doesn't mean it just hits a cap and stays there. Also, if you scale all utilities that you can effect down it changes nothing about actions (another confusion - mapping the utility to how much one cares).

And yes there are definitely cases where money are worth small probability of saving lives, and everyone agrees on such - e.g. if we find out that an asteroid has certain chance to hit Earth, we'd give money to space agencies, even when chance is rather minute (we'd not give money to cold fusion crackpots though). There's nothing fundamentally wrong with spending a bit to avert a small probability of something terrible happening. The problem arises when the probability is overestimated, when the consequences are poorly evaluated, etc. It is actively harmful for example to encourage boys to cry wolf needlessly. I'm thinking people sort of feel innately that if they are giving money away - losing - some giant fairness fairy is going to make the result more likely good than bad for everyone. World doesn't work like this; all those naive folks who jump on opportunity to give money to someone promising to save the world, no matter how ignorant, uneducated, or crackpotty that person is in the fields where correctness can be checked at all, they are increasing risk, not decreasing.

Comment author: private_messaging 12 May 2013 06:51:26AM *  0 points [-]

I abhor using unnecessary novel jargon.

I'm curious what you think the problem with Pascal's Mugging is though.

Bad math being internally bad, that's the problem. Nothing to do with any worlds, real or imaginary, just a case of internally bad math - utilities are undefined, it is undefined if you pay up or not, the actions chosen are undefined. Akin to maximizing blerg without any definition of what blerg even is - maximizing "expected utility" without having defined it.

Speed prior works, for example (it breaks some assumptions of Blanc. Namely, the probability is not bounded from below by any computable function of length of the hypothesis).

Comment author: private_messaging 12 May 2013 06:11:18AM *  1 point [-]

The best "priors" about the universe are 1 for what that universe right around you is, and 0 for everything else. Other priors are a compromise, an engineering decision.

What I am thinking is that

  • there is a considerably better way to assign priors which we do not know of yet - the way which will assign equal probabilities to each side of a die if it has no reason to prefer one over the other - the way that does correspond to symmetries in the evidence.

  • We don't know that there will still be same problem when we have a non-stupid way to assign priors (especially as the non-stupid way ought to be considerably more symmetric). And it may be that some value systems are intrinsically incoherent. Suppose you wanted to maximize blerg without knowing what blerg even really is. That wouldn't be possible, you can't maximize something without having a measure of it. But I still can tell you i'd give you 3^^^^3 blergs for a dollar, without either of us knowing what blerg is supposed to be or whenever 3^^^^3 blergs even make sense (if blerg is an unique good book of up to 1000 page length, it doesn't because duplicates aren't blerg).

Comment author: private_messaging 11 May 2013 05:57:07PM *  2 points [-]

The actual probability is either 0 or 1 (either happens or doesn't happen). Values in-between quantify ignorance and partial knowledge (e.g. when you have no reason to prefer one side of the die to the other), or, at times, are chosen very arbitrarily (what is the probability that a physics theory is "correct").

I don't know if those things have such extremes in low probability vs high utility to be called pascal's mugging.

New names for same things are kind of annoying, to be honest, especially ill chosen... if it happens by your own contemplation, I'd call it Pascal's Wager. Mugging implies someone making threats, scam is more general and can involve promises of reward. Either way the key is the high payoff proposition wrecking some havoc, either through it's prior probability being too high, other propositions having been omitted, or the like.

But even so, the human brain doesn't operate on anything like Solomonoff induction, Bayesian probability theory, or expected utility maximization.

People are still agents, though.

Comment author: private_messaging 11 May 2013 10:29:47AM *  2 points [-]

There is no real way of doing that without changing your probability function or your utility function. However you can't change those.

Last time I checked, priors were fairly subjective even here. We don't know what is the best way to assign priors. Things like "Solomonoff induction" depend to arbitrary choice of machine.

Any agent subject to Pascal's Mugging would fall pray to this problem first, and it would be far worse.

Nope, people who end up 419-scammed or waste a lot of money investing into someone like Randel L Mills or Andrea Rossi live through their life ok until they read a harmful string in a harmful set of circumstances (bunch of other believers around for example).

View more: Next