Peter,
As a general strategy for considering a black box, great. As a vehicle for defining a mysterious 'something' you want to understand, potentially useful but dangerous. Labelling can make a job harder in cases where the 'thing' isn't a thing at all but a result of your confusion. 'Free will' is a good example. It's like naming an animal you plan to eat: makes it harder to kill.
Ben
Ciphergoth - just coming back to this post to say a repeated, enormous, heartfelt thank you for taking what must have been a lot of time on this. Well laid out, wouldn't have done anything differently, and as good a read as when I was swept up in it on OB back in the day.
Cheers
Hey, I think this link's dead. Converting from ePub to Mobi isn't difficult but if someone's already taken the time to get the formatting right, add chapters, ToC etc....
Apologies for coming to this party a bit late. Particularly as I find my own answer really, really frustrating. While I wouldn't say it was an origin per se, getting into reading Overcoming Bias daily a few years back was what crystallised it for me. I'd find myself constantly somewhere between "well, yeah, of course" and "ohhhhhhhhhhhhhhh!" Guess the human brain doesn't tend to do Damascene revelations. We need overwhelming evidence, over a long period of time, to even begin chipping away at our craziest beliefs, and even then it's a s...
Very true, and well put. A combination of quantum events could probably produce anything you wanted, at whatever vanishingly tiny probability. Bear in mind that it's the configuration that evolves every which way, not 'this particle can go here, or here, or here....' But we're into Greg Egan territory here.
Suffice it to say that anyone who says they subscribe to quantum suicide but isn't either dead or richer than god is talking out of their bottom.
Voted up for sheer balls. You have my backing sir.
(B) if he thanks you for the free $100, does he ask for another one of those nice free hundred dollar note dispensers? (This is the "quantum suicide" option
I laugh in the face of anyone who attests to this and doesn't commit armed robbery on a regular basis. If 'at least one of my branches will survive' is your argument, why not go skydiving without a parachute? You'll survive - by definition!
So many of these comments betray people still unable to think of subjective experience as anything other than a ghostly presence sitting outside the quan...
The phrase "for me to be an animal" may sound nonsensical, but "why am I me, rather than an animal?" is not obviously sillier than "why am I me, rather than a person from the far future?".
Agreed - they are both equally silly. The only answer I can think of is 'How do you know you are not?" If you had, in fact, been turned into an animal, and an animal into you, what differences would you expect to see in the world?
What if I hack & remove $100 from your bank account. Are you just as wealthy as you were before, because you haven't looked?
Standard Dispute. If wealthy = same amount of money in the account, no. If wealthy = how rich you judge yourself to be. The fact that 'futures diverge' is irrelevant up until the moment those two different pieces of information have causal contact with the brain. Until that point, yes, they are 'the same
I'm not as versed in this trilemma as I'd like to be, so I'm not sure whether that final question is rhetorical or not, though I suspect that it is. So mostly for my own benefit:
While there's no denying that subjective experience is 'a thing', I see no reason to make that abstraction obey rules like multiplication. The aeroplane exists at a number of levels of abstraction above the atoms it's composed of, but we still find it a useful abstraction. The 'subjective experiencer' is many, many levels higher again, which is why we find it so difficult to talk a...
This is a pretty good summary of my standpoint. While I agree with the overarching view that rationality isn't a value in its own right, it seems like a pretty good thing to practise for general use.
+1 rationality point for reading comments without checking the author. -1 social point for the faux pas.
Sounds like a good one, count me in. I work at King's Cross to UCL is ideal. I'd have been at the FAI thing this weekend but for other arrangements.
Sorry, should have given more context.
Given the sky-high utility I'd place on living, I wouldn't expect to see the numbers crunch down to a place where a non-huge sum of money is the difference between signing up and not.
So when someone says 'if it were half the price maybe I'd sign up' I'm always interested to know exactly what calculations they're performing, and exactly what it is that reduces the billions of utilons of living down to a marginal cash sum. The (tiny?) chance of cryonics working? Serious coincidence if those factors cancel comfortably. ...
Being dead != Not doing anything
Not doing something because you're lazy != Not existing
I don't believe that you put low utility on life. You're just putting low utility on doing stuff you don't like.
I often have this thought, and then get a nasty sick feeling along the lines of 'what the hell kind of expected utility calculation am I doing that weighs a second shot at life against some amount of cash?' Argument rejected!
"If you could reason with religious people, there would be no religious people."
Robin, I'm a little surprised to read you saying that topics on which it's difficult to stay on track should be skirted. As far as I'm concerned, 'What are your religious views?' is the first question on the Basic Rationality test. I know that encouraging compartmentalisation isn't your goal by any means, but it sounds to me as though it would be the primary effect.
I can also see a need for a place for people to gather who want to be rational about all topics.
Now you're talking. No topics should be off-limits!
physical materialism feels bereft of meaning compared to the theistic worldview.
On what are you basing your assumption that the world should have whatever you mean by 'meaning'?
Just by the by, it might be a good party piece for you, but it would be a truly horrible party piece for half the people you performed it to.
I, Eliezer Yudkowsky, do now publicly announce that I am not planning to commit suicide, at any time ever, but particularly not in the next couple of weeks
ROFLcopters.
18 months too late, but http://xkcd.com/505/
By Eliezer's line of reasoning above - that the subjective experience is in the causal change between one state and the 'next' then yes, symbols are as good a substrate as any. FWIW, this is how I see things too.
4 years too late but... this is missing the point of both Eliezer and IL. Eliezer/Barbour's timeless physics has no changing state over time, because there is no time. Both states exist in a timeless configuration space, and the causal connection between them is only inferred. IL is trying to illustrate this leads to some pretty rediculous conclusions - such as that all you have to do is write down the states on a piece of paper, and then viola - you have created conscious beings even though no computation is actually going on.
EDIT: For what it's worth I think the Barbour's physics is a mysterious answer that doesn't actually dissolve any of the questions it purports to solve..
Ha, never noticed this. What I meant was 'Stupid me forgetting to log in.' So yes, we're worried! ;)
Ben
I think you are right that paperclip maximizers would not care at all about ethics.
Correct. But neither would they 'care' about paperclips, under the way Eliezer's pushing this idea. They would flarb about paperclips, and caring would be as alien to them as flarbing is to you.
Seconded. One of the many modern connotations of 'Singularity' is 'Geek Apocalypse'.
Which is happening, like, a good couple of years afterwards.
Intelligence explosion does away with that, and seems to nail the concept much better anyway.
How about the middle ground - "If constant PR consideration stops you from expressing yourself all the time, maybe it's time to reconsider your priorities"?
Posting stuff on Facebook that might get you in trouble is the archetype these day I suppose, but I really can't bring myself to care about things like that.
Maybe I just don't have a strong enough terminal value to protect right now, but I find it easier to imagine myself thinking, 50 years hence, "I wish I'd just decided 'to hell with it' and said what I thought" than "I wish I'd shut up, gone with the flow and eased my path."
I'll hit you up in late 2059 and let you know how that went.
There's a thesis in there somewhere.
We all know what's really going down. The Dark Lords of the Matrix are currently cacking themselves and coming up with semi-plausible reasons to break the thing until they can decide on a long-term strategy.
Favourite album post-1960?
Who actually gets off on earning loads of karma across multiple accounts with no-one knowing?
Please stop allowing your practical considerations get in the way of the pure, beautiful counterfactual!
Seriously though, either you allow yourself to suspend practicalities and consider pure decision theory, or you don't. This is a pure maths problem, you can't equate it to 'John has 4 apples.' John has 3^^^3 apples here, causing your mind to break. Forget the apples and years, consider utility!
My commiserations, to the extent that you seem to need them.
I'd like to imagine I'd have a similar reaction, this is an inspiring post. All the best.
Cracking idea, like it a lot. Hofstadter would jump for joy, and in his honour:
Beware of generalising across people you haven't spent much time around, however tempting the hypothesis. Drawing a map of the city from your living room etc.
My first 18 years were spent attending a Catholic church once a week. To the extent that we can ever know what other people actually believe (whatever that means), most of them have genuinely internalised the bits they understand. Like, really.
We can call into question what we mean by 'believe', but I can't agree that a majority of the world population is just cynically going with the flow. Finally, m...
Also upvoted, and very succintly put.
Rationality is a tool we use get to our terminal value. And what do we do when that tool tells us our terminal value is irrational?
Never ask that question.
I wonder whether you can hold to any meaningful 'individual', whether the difference be bit-wise or no.
Indeed, that's what I'm driving at.
Harking back to my earlier comment, changing a single bit and suddenly having a whole new person is where my problem arises. If you change that bit back, are you back to one person? I might not be thinking hard enough, but my intuition doesn't accept that. With that in mind, I prefer to bite that bullet than talk about degrees of person-hood.
At some point you will surely admit that we now have 2 people and not just 1
Actually I won't. While I grok your approach completely, I'd rather say my concept of 'an individual' breaks down once I have two minds with one bit's difference, or two identical minds, or any of these borderline cases we're so fond of.
Say I have two optimisers with one bit's difference. If that bit means one copy converts to Sufism and the other to Mennonism, then sure, two different people. If that one bit is swallowed up in later neural computations due to the coarse-graine...
Voted this down, then changed my mind and undid it. This is a genuine question, the answer to which was graciously accepted. Downvoting people who need guidance to understand a concept and are ready to learn is exactly what we don't want to do.
Thanks for the link ;).
OK, on the one hand we have many-worlds. As you say, no direct subjective corroborating evidence (it’s what we’d see either way). What’s more, it’s the simplest explanation of what we see around us.
On the other hand, we have one-world. Again, ‘it’s what we’d see either way’. However, we now have to postulate an extra mechanism that causes the ‘collapse’.
I know which of these feels more like a privileged complex hypothesis pulled out of thin air, like a dragon.
Could whomever downvoted me above let me know where I’m going wrong here?
Yeah I get into trouble there. It feels as though two identical copies of a person = 1 pattern = no more people than before copying. But flip one bit and do you suddenly have two people? Can't be right.
That said, the reason we value each person is because of their individuality. The more different two minds, the closer they are to two separate people? Erk.
Silas, looking forward to that post.
Appears 3 times in my top 10.
That aside, though, I'm now so much better at stopping myself and saying 'hang on, is this really going to work/is this really true/is this really right?' Very, very generic, but certainly something I've noticed in myself.
Surely spontaneous collapse is the garage dragon here. Zero evidence, highly unlikely.
I find myself simultaneously convinced and unconvinced by this! Anticipation (dependent, of course, on your definition) is surely a vital tool in any agent that wants to steer the future? Or do you mean 'human anticipation' as differentiated from other kinds? In which case, what demarcates that from whatever an AI would do in thinking about the future?
However, Dai, your top level comment sums up my eventual thoughts on this problem very well. I've been trying for a long time to resign myself to the idea that a notion of discrete personal experience is inco...
Gwern, I refer you to http://xkcd.com/137/
At the risk of violent downvoting, one of the many reference points that jumped into my mind while reading was 'the closest thing I've experienced to jumping between nested levels of reality is on drugs'.
"Don't let yourself unreflectively fall into a routine" and "don't be emotionally uncomfortable with nonconformity" are of course good advice; "be indifferent to PR when you're trying to do something for which PR actually matters" is bad advice.
...said Achilles to his friend Mr Tortoise.
Three identical comments, all beginning 'Two comments'?
Head a splode?
It is a cracking read, though the quality does dip and dive. No doubt that's just the nature of the beast.
I've read much better treatises on the Chinese Room that have been written since though - H & D seem to attack it in strange and abstract ways in The Mind's I.
And the GEB sections just made me want to pick that up again....
For the record, I didn't get a huge amount out of I Am A Strange Loop, one of Hofstadter's more recent efforts. A bit too travelogue, a bit too 'voyage of personal discovery', though his style of writing is still striking in its own very particular way. Anyone else have a different experience here?
Feeling this one. So odd how having a really solid grounding in a subject allows you to work out what later seems to be basic truths.
Funny how those highly unlikely borderline cases whisk away a lot of the confusion, huh? No committed physicalist can postulate a serious difference between seeing something red and having a virtual red-thing pumped into your optic nerve, I would hope. I think that’s a far more useful scenario than thinking about someone suddenly be able to see colour. In fact, you could probably keep moving a step towards your magical ‘inner perceiver’ and asking whether ‘it’s a real experience of redness’. That’s not to say that qualia are fundamentally dualistic, simply...
If you're contemplating picking the book up, do, it's really excellent. Conceptually very dense but worth taking it nice and slowly.