MugaSofer comments on The $125,000 Summer Singularity Challenge - Less Wrong

20 Post author: Kaj_Sotala 29 July 2011 09:02PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (259)

You are viewing a single comment's thread. Show more comments above.

Comment author: MugaSofer 26 April 2013 11:28:42AM *  -1 points [-]

I mean, if I try to imagine living in a world where only 10,000 people exist, I conclude that I would be significantly more motivated to extend the lives of an arbitrary person (e.g., by preventing them from starving) than I am now. (Leaving aside any trauma related to the dieback itself.)

Well, if the population is that low saving people is guarding against an existential risk, so I would feel the same. Does your introspection yield anything on why smaller numbers matter more?

ETA: your brain can't grasp numbers anywhere near as high as a billion. How sure are you murder matters now?

Comment author: TheOtherDave 26 April 2013 01:12:36PM 3 points [-]

How sure are you murder matters now?

It's pretty clear that individual murder doesn't matter to me.

I mean, someone was murdered just now, as I write this sentence, and I care about that significantly less than I care about the quality of my coffee. I mean, I just spent five seconds adjusting the quality of my coffee, which is at least a noticeable quantity of effort if not a significant one. I can't say the same about that anonymous murder.

Oh look, there goes another one. (Yawn.)

The metric I was using was not "caring whether someone is murdered", which it's clear I really don't, but rather "being willing to murder someone," which it's relatively clear that I do, but not nearly as much as I could. (Insert typical spiel here about near/far mode, etc.)

Comment author: nshepperd 26 April 2013 02:11:49PM 0 points [-]

I think the resolution to that is that you don't have to have an immediate emotional reaction to care about it. There are lots of good and bad things happening in the world right now, but trying to feel all of them would be pointless, and a bad fit for our mental architecture. But we can still care, I think.

Comment author: TheOtherDave 26 April 2013 03:38:08PM 0 points [-]

Well, I certainly agree that I don't have to have an emotional reaction to each event, or indeed a reaction to the event at all, in order to be motivated to build systems that handle events in that class in different ways. I'm content to use the word "care" to refer to such motivation, either as well as or instead of referring to such emotional reactions. Ditto for "matters" in questions like "does murder matter", in which case my answer to the above would change, but that certainly isn't how I udnerstood MugaSofer's question.

Comment author: [deleted] 26 April 2013 05:03:50PM *  1 point [-]

So the question now is: if you could prevent someone you would most likely never otherwise interact with from being murdered, but that would make your coffee taste worse, what would you do?

Comment author: shminux 26 April 2013 06:00:41PM *  7 points [-]

Don't we make this choice daily by choosing our preferred brand over Ethical Bean at Starbucks?

Comment author: Eliezer_Yudkowsky 26 April 2013 11:26:06PM 7 points [-]

I hear the ethics at Starbucks are rather low-quality and in any case, surely Starbucks isn't the cheapest place to purchase ethics.

Comment author: gwern 27 April 2013 12:23:31AM *  18 points [-]

Bah! Listen, Eliezer, I'm tired of all your meta-hipsterism!

"Hey, let's get some ethics at Starbucks" "Nah, it's low-quality; I only buy a really obscure brand of ethics you've probably never heard of called MIRI". "Hey man, you don't look in good health, maybe you should see a doctor" "Nah, I like a really obscure form of healthcare, I bet you're not signed up for it, it's called 'cryonics'; it's the cool thing to do". "I think I like you, let's date" "Oh, I'm afraid I only date polyamorists; you're just too square". "Oh man, I just realized I committed hindsight bias the other day!" "I disagree, it's really the more obscure backfire effect which just got published a year or two ago." "Yo, check out this thing I did with statistics" "That's cool. Did you use Bayesian techniques?"

Man, forget you!

/angrily sips his obscure mail-order loose tea, a kind of oolong you've never heard of (Formosa vintage tie-guan-yin)

Comment author: Eliezer_Yudkowsky 27 April 2013 08:24:01AM 6 points [-]

If you can't pick something non-average to meet your optimization criteria, you can't optimize above the average.

This comment has been brought to you by my Dvorak keyboard layout.

Comment author: TsviBT 27 April 2013 10:11:04AM 6 points [-]

If you keep looking down the utility gradient, it's harder to escape local maxima because you're facing backwards.

This comment has been brought to you by me switching from Dvorak to Colemak.

Comment author: gwern 27 April 2013 07:22:26PM *  3 points [-]

If you can't pick something non-average to meet your optimization criteria, you can't optimize above the average.

But at the same time, there's only so many possible low-hanging fruits etc, and at some level of finding more fruits, that indicates you aren't optimizing at all...

Comment author: [deleted] 27 April 2013 12:42:20AM 2 points [-]

Ouch, that cuts a bit close to home...

Comment author: [deleted] 27 April 2013 10:14:48AM *  1 point [-]

(Had to google “backfire effect” to find out whether you had made it up on the spot.)

EDIT: Looks like I had already heard of that effect, and I even seem to recall E.T. Jaynes giving a theoretical explanation of it, but I didn't remember whether it had a name.

Comment author: gwern 27 April 2013 07:20:47PM 0 points [-]

Had to google “backfire effect” to find out whether you had made it up on the spot.

"Like I said, it's a really obscure bias, you've probably never heard of it."

I even seem to recall E.T. Jaynes giving a theoretical explanation of it

Really? I don't remember ever seeing anything like that (although I haven't read all of PT:TLoS yet). Maybe you're conflating it with the thesis using Bayesian methods I link in http://www.gwern.net/backfire-effect ?

Comment author: Vaniver 27 April 2013 01:14:53AM 1 point [-]

"Yo, check out this thing I did with statistics" "That's cool. Did you use Bayesian techniques?"

I can't tell if I should feel good or bad that this was the only one where I said "well, actually..."

Comment author: [deleted] 27 April 2013 10:12:18AM *  1 point [-]

BTW, for some reason, certain “fair trade” products at my supermarket are astoundingly cheap (as in, I've bought very similar but non-“fair trade” stuff for more); I notice that I'm confused.

Comment author: TheOtherDave 26 April 2013 05:27:29PM 3 points [-]

Judging from experience, the answer is that it depends on how the choice is framed.

That said, I'd feel worse afterwards about choosing the tastier coffee.