Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: -necate- 26 March 2017 08:35:07AM 0 points [-]

Of course I can no longer change the framing while the survey is running, but for future surveys how would you frame it? Do you think it is worth specifying time-frame, other wealth an so on? I am not sure if most participants of an online survey would carefully read the instructions and actually try to visualise the specified situation.

What I have seen often is the framing "on top of what you already own you have been given..." which implies that you should make your decision based on your current situation. Do you think this would be superior?

Comment author: Dagon 26 March 2017 04:26:24PM 0 points [-]

it depends on what you're trying to discern with the question. If you want to know what people would do with a windfall in addition to their current allocation, ask that. but then you probably need to ask some intrusive quesitons about what they previously had.

You could state "X is all you have in the world, and you make enough to live on but not to save additional. which of the following would you choose to save for a vacation in 6 months" or "for retirement in 20 years" or "for emergencies".

Comment author: Dagon 26 March 2017 12:04:45AM 3 points [-]

I always hate these kinds of things which say "you're given X, what do you want to do" and don't specify what timeframe, goals, or other resources/investments I already have.

My answers are very different if this is a marginal investment on top of my current allocations vs a sole investment and I have nothing else. Likewise very different if I expect to need the results in 6 months or 20 years, and whether I can change/rebalance in the meantime.

Comment author: Dagon 25 March 2017 11:57:43PM 1 point [-]

Umm, stop waving your hands and start putting some estimates down. Especially when you say things like

Over a long enough timeline, the probability of a copy of any given uploaded mind falling into the power of a sadistic jerk approaches unity.

You show an inability to actually figure out the relative frequencies that would make this true or false. There's lots of ways this could be false - most notably there may be dozens of orders of magnitude more uploaded minds than sadistic jerks, and any nonzero cost of running a mind means the SJs simply can't afford to torture most of them.

Once an uploaded mind has fallen under the power of a sadistic jerk, there is no guarantee that it will ever be 'free', and the quantity of experienced sufferring could be arbitrarily large, due in part to the embarrassingly parallel nature of torture enabled by running multiple copies of a captive mind.

More unstated assumptions (with which I think I disagree). How are you aggregating suffering (or value generally) for minds? Do you think that identical tortures for two copies of a mind is different than torture of one? Why? Do you think that any amount of future potential torture can remove the value of current pleasure? Why?

Even if you try to just quantify "value * experienced-seconds" and simply multiply, it's going to be hard to think anyone is better off NOT being uploaded.

Feel free to make choices for yourself, and even to advocate others to securely erase their information-patterns before it's too late. But without a lot more clear probability estimates and aggregation methodology, I think I'll take my chances and seek to continue living.

Comment author: Dagon 21 March 2017 01:36:53PM 1 point [-]

This is obvious, because you and the copy will reach the same decision.

Only if you reject the idea of free will and there are no environmental microvariations.

That justification implicitly assumes that you and your copy as (somewhat) antagonistic

No, it doesn't need to be antagonistic, just independent. Also, it's part of the PD setup, not assumed by the twins justification.

Your main point is fair, though - this version of copy-of-you PD and Newcomb's problem if it's predictor is equivalent to copying you are similar.

Comment author: Dagon 20 March 2017 09:15:49PM *  0 points [-]

Had to search to find the rest of the problem (like what happens if he predicted you to be in Aleppo and you're there - you die). This was helpful, and I came across a 2008 paper which argues that CDT works here.

I'm still not sure how this is any different from Newcomb's problem: if Death predicts you perfectly, your best plan is to just accept it and leave your heirs the maximum amount (one-box). And CDT works just fine if you phrase it as "what is the probability that Death/Omega has correctly predicted your action" (but it does somewhat bend the "causal" part. I prefer the C stand for Classical, though).

Comment author: Dagon 20 March 2017 04:42:00PM 1 point [-]

downvoting still disabled, but I didn't find this link worth following.

Weird defensive tone. Political references to things I don't follow closely (and that are actively mind-killing). Generalizations that are simply wrong.

Comment author: Lumifer 16 March 2017 08:16:09PM *  1 point [-]

but to the extent that any agent makes coherent goal-driven decisions, it has a utility function

That is not obvious to me. Why is it so? (defining "utility function" might be helpful)

Comment author: Dagon 16 March 2017 10:54:50PM *  0 points [-]

The most defensible use of the term is described as Ordinal Utility, but this is a little weaker than I commonly see it used around here. I'd summarize as "a predictive model for how much goodness an agent will experience conditioned on some decision". Vincent Yu has a more formal description in (this comment)[http://lesswrong.com/lw/dhd/stupid_questions_open_thread_round_3/72z3].

There's a lot of discussion about whether humans have a utility function or not, with the underlying connotation being that a utility function implies consistency in decisionmaking, so inconsistency proves lack of utility function. One example: Do Humans Want Things? I prefer to think of humans as having a utility function at any given point in time, but not one that's consistent over time.

A semi-joking synonym for "I care about X" for some of us is "I have a term for X in my utility function". Note that this (for me) implies a LOT of terms in my function, with very different coefficients that may not be constant over time.

Comment author: Dagon 16 March 2017 07:57:02PM *  2 points [-]

Humans don't have a utility function and make very incoherent decisions

Wait. Most of what I've read about utility functions applied to humans before anyone seriously talked about AGI. It doesn't have to be a simple function, and nobody may be able to algebraically express their utility function, but to the extent that any agent makes coherent goal-driven decisions, it has a utility function.

Humans that make incoherent decisions are either following a complex utility function that's hard to reverse-engineer, or have an inconstant utility function that changes across time.

In response to Excuses and validity
Comment author: Dagon 13 March 2017 08:45:41PM 0 points [-]

I'm confused about the meta - is it intended to be an algebra problem? Was the halfway point in words or time? I guess only words makes sense - if it took 2 hours total, and you slowed down considerably 1 hour in, how could the second hour take longer than an hour? You must mean you wrote quickly for some time X, during which you got half the text completed, and the second half of the text took Y time, Y > X and X + Y = 2h.

Or you mean it should have taken two hours to write, but actually took longer because after an hour you slowed down.

Comment author: Lumifer 09 March 2017 04:00:23PM *  1 point [-]

The closest thing we have to magic in real world is advanced technology.

Move your mind a hundred years back and look at your phone.

Comment author: Dagon 10 March 2017 11:46:29PM *  1 point [-]

"Any technology that's distinguishable from magic is insufficiently advanced"

-- Contrapositive Arthur Clarke

View more: Next