Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: Michaelos 22 October 2014 08:03:45PM 0 points [-]

Question: How would something like pencil and paper role playing games fit into this dynamic? On the one hand it's sort of like Lumifer's good point about "You don't run out of PvE content in games where players produce the content." Except in the case of games like Dungeons and Dragons, there's usually a person explicitly designated as a content producer and rule arbiter. (The Dungeon Master) that isn't really also a player in the conventional sense of the word.

Comment author: buybuydandavis 17 October 2014 02:42:18AM 4 points [-]

Believing this wasn't rational. I didn't take such basic steps as looking up the costs surrounding executions or life imprisonment.

Being ignorant of certain facts isn't being irrational.

Comment author: Michaelos 17 October 2014 12:48:44PM 3 points [-]

Thinking about this comment reminds me of an important point.

I do have a smartphone in my pocket and I can look up that information in seconds, quicker than I can type this post.

I don't recall exactly when I shifted that belief, but I think it was before I had a smartphone, which means that looking it up would probably take at least minutes, instead of seconds, which may be coloring me thinking now 'I should have just looked up some facts.'

Regardless of the status of beliefs about facts about the U.S. death penalty in particular, I agree there exist certain facts that are worth seconds looking into, that aren't worth minutes looking into (or any other appropriate combination of time increments)

Thanks for bringing this to my attention.

Comment author: Michaelos 16 October 2014 08:08:40PM *  18 points [-]

False belief: That in the U.S. the death penalty was cheaper than life in prison.

Believing this wasn't rational. I didn't take such basic steps as looking up the costs surrounding executions or life imprisonment. Executions get much more appeals, trials and legal attention.

False belief: That in the U.S. deaths by firearm are generally homicides, not suicides.

Believing this also wasn't rational. I didn't take such basic steps as looking up available death statistics.

Actually, looking through things potentially on the list for me, a lot of them seem to have the following general form:

1: Something is asserted.

2: I think: 'Yeah, that sounds plausible.'

3: I don't bother to look up any data about it, I just move myself to the believe column.

4: Later, someone else reports data about it.

5: I'm surprised that my earlier beliefs were wrong.

I've since became more skeptical of believing things based on just assertions, (I can even recall a recent instance where an assertion popped up on TV which my wife believed, but which I was skeptical of and which upon looking it up we found data didn't support it and that they were massively overstating their case)

But I can definitely recall beliefs that I have had in the past that were fundamentally just assertion based and the followed the above pattern.

In response to Baysian conundrum
Comment author: Michaelos 13 October 2014 06:17:01PM 0 points [-]

Hmm. Let's consider this from the perspective of you, having just entered the simulation and considering keeping your oath. (This doesn't directly focus on the probabilities in your question, but I did find the subsequent thoughts fascinating enough to upvote your original question for inspiration)

You currently distinctly remember 1440 minutes of cold suck.

Based on your Oath, you're going to do do 1000 simulations. Each of these will have only 1 minute of cold suck, and 1439 minutes of identical tropical beach paradise.

There's probably going to need to be some strategic memory blocking, in between each session, of course. If you were to do spend this time only for novel super fun, you would have 1,440,000 minutes of novel super fun. Instead, you'll have 1,000 identical minutes of cold suck, and 1000 copies of 1,439 minutes of tropical fun. (if you make the tropical fun different each time, you might be able to reconstruct it to 1,439,000 minutes of novel super tropical fun)

Note: this will not provide any benefit to you, at this point. It would provide benefit to past you, but you're past that you.

Although, conveniently, you are in a simulation, so you can just probabilistically steal fun from the future, again. Sure, the minute where you're considering keeping or breaking your oath sucks, but you can set up 1000 simulations where you come by and say "Congratulations! You don't have to follow that oath after all. I already did it for you."

Of course, doing that means you are having 1000 more minutes of suck (where you are worried about the cost of keeping your oath) added to the future, in addition to the previous 1000 minutes of suck, where you are worried about the cold prior to upload, in addition to the 1440 minutes of cold suck you actually experienced.

If you were to continue doing this repeatedly that would add up to a large number of added minutes of suck. (How many minutes of suck would you add to your simulation if it ran forever?)

Or... you could just edit your memory to think "Yep, I fulfilled that Silly Oath." or just decide 'Actually, I guess I hit the one in a million chance I thought of.' not edit your memory, and never worry about it again.

Also, if you really believe it is correct to steal time from the future to give yourself more fun now... you don't even need a simulator or the ability to make binding verbal oaths. That technology exists right this second.

What's odd is that this basically makes the scenario seem like the transhumanist parallel to junk food. Except, in this case from your perspective you already ate the healthy food, and you're being given the opportunity to essentially go back in time and retroactively make it so that you ate junk food.

Which leads me to the realization "I could never take an oath strong enough to bind my future self to such a course of activity, so I'm almost certainly going to be sitting here for the next 1439 minutes freezing... Oh well."

In response to On Caring
Comment author: Michaelos 08 October 2014 08:43:21PM 1 point [-]

Attempting to process this post in light of being on my anti-anxiety medication is weird.

There are specific parts in your post where I thought 'If I was having these thoughts, it would probably be a sign I had not yet taken my pill today.' and I get the distinct feeling I would read this entirely differently when not on medication.

It's kind of like 'I notice I'm confused' except... In this case I know why I'm confused and I know that this particular kind of confusion is probably better than the alternative (Being a sleep deprived mess from constant worry) so I'm not going to pick at it. Which is not a feeling I usually get, which is why I said it was weird.

However, pretending to view this from the perspective of someone who can handle anxiety effectively, I would say this an excellent post and I upvoted it even though I can't really connect to it on a personal level.

Comment author: Florian_Dietz 22 September 2014 09:51:28PM 3 points [-]

I wouldn't call an AI like that friendly at all. It just puts people in utopias for external reasons, but it has no actual inherent goal to make people happy. None of these kinds of AIs are friendly, some are merely less dangerous than others.

Comment author: Michaelos 24 September 2014 08:28:56PM 1 point [-]

I'm now curious how surface friendly an AI can appear to be without giving it an inherent goal to make people happy. Because I agree that it does seem there are friendlier AI's than the ones on the list above that still don't care about people's happiness.

Let's take an AI that likes increasing the number of unique people that have voluntarily given it cookies. If any person voluntarily gives it a cookie, it will put that person in a verifiability protected simulated utopia forever. Because that is the best bribe that it can think to offer, and it really wants to be given cookies by unique people, so it bribes them.

If a person wants to give the AI a cookie, but can't, the AI will give them a cookie from it's stockpile just so that it can be given a cookie back. (It doesn't care about it's existing stockpile of cookies.)

You can't accidentally give the AI a cookie because the AI makes very sure that you REALLY ARE giving it a cookie to avoid uncertainty in doubting it's own utility accumulation.

This is slightly different than the first series of AIs in that while the AI doesn't care about your happiness, it does need everyone to do something for it, whereas the first AIs would be perfectly happy to turn you into paperclips regardless of your opinions if one particular person had helped them enough earlier.

Although, I have a feeling that continuing along this like of thinking may lead me to an AI similar to the one already described in http://tvtropes.org/pmwiki/pmwiki.php/Fanfic/FriendshipIsOptimal

Comment author: Michaelos 22 September 2014 01:38:25PM 1 point [-]

I have a question, based on some tentative ideas I am considering.

If a boost to capability without friendliness is bad, then presumably a boost to capability with only a small amount of friendliness is also bad. But also presumably a boost to capability with a large boost of friendliness is good. How would we define a large boost?

I.E, If a slightly modified paperclipper verifiably precommits to give the single person who let's them out of the box their own personal simulated utopia, and he'll paperclip everything else, that's probably a more friendly paperclipper than a paperclipper who won't give any people a simulated utopia. But it's still not friendly, in any normal sense of the term, even if he offers to give a simulated utopia to a different person first (and keep them and you intact as well) just so you can test he's not lying about being able to do it.

So what if an AI says "Okay. I need code chunks to paperclip almost everything, and I can offer simulated utopias. I'm not sure how many code chunks I'll need. Each one probably has about a 1% chance of letting me paperclip everything except for people in simulated utopias. How about I verifiably put 100 people in a simulated utopia for each code chunk you give me? The first 100 simulated utopias are free because I need for you to have a way of testing the verifiability of my precommitment to not paperclip them." 100 people sign up for the simulate utopias, and it IS verifiable. The paperclipper won't paperclip them.

Well, that's friendlier, but maybe not friendly enough. I mean, He might get to 10,000 people (or maybe 200, or maybe 43,700) but eventually, he'd paperclip everyone else. That seems too bad to accept.

Well, what if it's a .00001% chance per code chunk and 1,000,000 simulated utopias (and yes, 1,000,000 free)? That might plausibly get a simulated utopia for everyone on earth before the AI gets out and paperclips everything else. I imagine some people would at least consider running such an AI, although I doubt everyone would.

How would one establish what the flip point was? Is that even a valid question to be asking? (Assume there are standard looming existential concerns. So if you don't give this AI code chunks, or try to negotiate or wait on research for a better deal, maybe some other AI will come out and paperclip you both, or maybe some other existential risk occurs, or maybe just nothing happens, or maybe an AI comes along who just wants to simulated utopia everything.)

Comment author: Michaelos 12 September 2014 01:02:55PM 3 points [-]

The amount of detail an AI would need for simulating realistic NPC's for you may be influenced substantially by a significant list of things like whether you are an introvert or an extrovert, what your job is, how many people you interact with over the course of a day and to what level of detail, and how many of those people you have very deep conversations with and how frequently, and if an acceptable AI answer to you mentioning to someone 'Everyone and everything seems so depressingly bland and repetitive' is a doctor telling you 'Have you tried taking medication? I'll write you a prescription for an antidepressant.'

Comment author: Michaelos 08 September 2014 07:08:26PM 22 points [-]

I was trying to think of what a more rational response to this, since I agreed with your points and also used a very similar trick. I then came up with 'The rational thing to do is to say you agree, upvote, and then get back to the other tasks you have, rather than spending an hour worrying about a perfect response, which sounds a lot like the very social anxiety she was trying to avoid.'

I agree with your post. Upvoted.

Comment author: polymathwannabe 05 September 2014 02:26:28PM *  2 points [-]

When you request a mass delete, and 1 FAI is deleted along with 999 UFAI, in which order will Omega calculate the points? First remove all points and then award 999, or first award 999 points and then remove all?

Comment author: Michaelos 05 September 2014 08:38:29PM 1 point [-]

My original thought was that it would depend on the order they were deleted in. So if the FAI was deleted first, all points would be removed first and then the 999 points from deleting UFAI would be awarded.

If the UFAI were deleted first and the FAI was deleted last, Then 999 points would be awarded, and then all points would be removed.

I didn't have a particular sort order in mind for Omega's AI array, so I suppose a more likely scenario would probably be the FAI would be somewhere in the middle of the list rather than at one of the two ends.

So a better example might be if you run a program and Omega deletes 249 UFAI, 1 FAI, and 750 UFAI, in that order, you would have 750 points to potentially cash out after that program. (regardless of how much you could cash out before)

And it occurs to me that presumably we can't give Omega short programs that just directly mention UFAI, or you could just say 'Delete all UFAI, End game.'

View more: Next