waitingforgodel comments on Best career models for doing research? - Less Wrong

27 Post author: Kaj_Sotala 07 December 2010 04:25PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (999)

You are viewing a single comment's thread. Show more comments above.

Comment author: waitingforgodel 09 December 2010 04:21:46PM 4 points [-]

accusations stick in the mind even when one is explicitly told they are false

Actually that citation is about both positive and negative things -- so unless you're also asking pro-SIAI people to hush up, you're (perhaps unknowingly) seeking to cause a pro-SIAI bias.

Another thing that citation seems to imply is that reflecting on, rather than simply diverting our attention away from scary thoughts is essential to coming to a correct opinion on them.

One of the interesting morals from Roko's contest is that if you care deeply about getting the most benefit per donated dollar you have to look very closely at who you're giving it to.

Market forces work really well for lightbulb-sales businesses, but not so well for mom-and-pop shops, let alone charities. The motivations, preferences, and likely future actions of the people you're giving money to become very important. Knowing if you can believe the person, in these contexts, becomes even more important.

As you note, I've studied marketing, sales, propaganda, cults, and charities. I know that there are some people who have no problem lying for their cause (especially if it's for their god or to save the world).

I also know that there are some people who absolutely suck at lying. They try to lie, but the truth just seeps out of them.

That's why I give Roko's blurted comments more weight than whatever I'd hear from SIAI people who were chosen by you -- no offence. I'll still talk with you guys, but I don't think a reasonably sane person can trust the sales guy beyond a point.

As far as your question goes, my primary desire is a public, consistent moderation policy for LessWrong. If you're going to call this a community blog devoted to rationality, then please behave in sane ways. (If no one owns the blog -- if it belongs to the community -- then why is there dictatorial post deletion?)

I'd also like an apology from EY with regard to the chilling effects his actions have caused.

But back to what you replied to:

What would SIAI be willing to lie to donors about?

Do you have any answers to this?

Comment author: AnnaSalamon 09 December 2010 05:51:40PM 13 points [-]

To answer your question, despite David Gerard's advice:

I would not lie to donors about the likely impact of their donations, the evidence concerning SIAI's ability or inability to pull off projects, how we compare to other organizations aimed at existential risk reduction, etc. (I don't have all the answers, but I aim for accuracy and revise my beliefs and my statements as evidence comes in; I've actively tried to gather info on whether we or FHI reduce risk more per dollar, and I often recommend to donors that they do their own legwork with that charity comparison to improve knowledge and incentives). If a maniacal donor with a gun came searching for a Jew I had hidden in my house, or if I somehow had a "how to destroy the world" recipe and someone asked me how to use it, I suppose lying would be more tempting.

While I cannot speak for others, I suspect that Michael Vassar, Eliezer, Jasen, and others feel similarly, especially about the "not lying to one's cooperative partners" point.

Comment author: David_Gerard 09 December 2010 05:58:06PM *  3 points [-]

I suppose I should add "unless the actual answer is not a trolley problem" to my advice on not answering this sort of hypothetical ;-)

(my usual answer to hypotheticals is "we have no plans along those lines", because usually we really don't. We're also really good at not having opinions on other organisations, e.g. Wikileaks, which we're getting asked about A LOT because their name starts with "wiki". A blog post on the subject is imminent. Edit: up now.)

Comment author: David_Gerard 09 December 2010 04:46:58PM *  8 points [-]

Another thing that citation seems to imply is that reflecting on, rather than simply diverting our attention away from scary thoughts is essential to coming to a correct opinion on them.

Well, uh, yeah. The horse has bolted. It's entirely unclear what choosing to keep one's head in the sand gains anyone.

What would SIAI be willing to lie to donors about?

Although this is a reasonable question to want the answer to, it's obvious even to me that answering at all would be silly and no sensible person who had the answer would.

Investigating the logic or lack thereof behind the (apparently ongoing) memory-holing is, however, incredibly on-topic and relevant for LW.

Comment author: wedrifid 09 December 2010 04:55:40PM *  3 points [-]

Although this is a reasonable question to want the answer to, it's obvious even to me that answering at all would be silly and no sensible person who had the answer would.

Total agreement here. In Eliezer's words:

Ambiguity is their ally. Both answers elicit negative responses, and they can avoid that from most people by not saying anything, so why shouldn't they shut up?

Comment author: David_Gerard 09 December 2010 04:57:37PM *  4 points [-]

Did you make up the 'memory-holing' term?

A fellow called George Orwell.

Comment author: wedrifid 09 December 2010 05:02:38PM 2 points [-]

Ahh, thankyou.

Comment author: David_Gerard 09 December 2010 05:05:34PM *  4 points [-]

I presume you're not a native English speaker then - pretty much any moderately intelligent native English speaker has been forced to familiarity with 1984 at school. (When governments in the UK are being particularly authoritarian, there is often a call to send MPs copies of 1984 with a note "This is not a manual.") Where are you from? Also, you really should read the book, then lots of the commentary on it :-) It's one of the greatest works of science fiction and political fiction in English.

Comment author: wedrifid 09 December 2010 05:31:37PM 4 points [-]

I can tell you all about equal pigs and newspeak but 'memory-holing' has not seemed to make as much of a cultural footprint - probably because as a phrase it is rather awkward fit. I wholeheartedly approve of Orwell in principle but actually reading either of his famous books sounds too much like highschool homework. :)

Comment author: Jack 09 December 2010 06:12:21PM 4 points [-]

Animal Farm is probably passable (though it's so short). 1984 on the other hand is maybe my favorite book of all time. I don't think I've had a stronger emotional reaction to another book. It makes Shakespeare's tragedies look like comedies. I'd imagine you'd have similar feelings about it based on what I've read of your comments here.

Comment author: wedrifid 09 December 2010 07:02:49PM 1 point [-]

That's some high praise there.

It makes Shakespeare's tragedies look like comedies.

So I take it there isn't a romantic 'happily ever after' ending? :P

Comment author: [deleted] 10 December 2010 10:49:14PM 0 points [-]

Actually, there is... ;)

Comment author: Vaniver 09 December 2010 05:36:54PM 1 point [-]

Both are short and enjoyable- I strongly recommend checking them out from a library or picking up a copy.

Comment author: David_Gerard 09 December 2010 05:34:10PM *  1 point [-]

Read them. They're actually really good books. His less-famous ones are not as brilliant, but are good too.

(We were taught 1984 in school, I promptly read to the end with eyes wide. I promptly borrowed Animal Farm of my own accord.)

Comment author: [deleted] 10 December 2010 10:50:52PM 1 point [-]

His less-famous novels aren't as good. On the other hand, some of his essays are among the clearest, most intelligent thinking I've ever come across, and would probably be of a lot of interest to LessWrong readers...

Comment author: David_Gerard 10 December 2010 11:12:44PM *  0 points [-]

Oh yeah. Politics and the English Language is a classic on a par with the great two novels. I first read that in 1992 and wanted to print copies to distribute everywhere (we didn't have internet then).

Comment author: waitingforgodel 09 December 2010 05:10:31PM 1 point [-]

why shouldn't they shut up?

Because this is LessWrong -- you can give a sane response and not only does it clear the air, people understand and appreciate it.

Cable news debating isn't needed here.

Sure we might still wonder if they're being perfectly honest, but saying something more sane on the topic than silence seems like a net-positive from their perspective.

Comment author: wedrifid 09 December 2010 05:36:20PM 2 points [-]

By way of a reminder, the question under discussion was:

What would SIAI be willing to lie to donors about?

Comment author: wnoise 09 December 2010 05:17:02PM *  1 point [-]

LessWrongers are not magically free of bias. Nor are they inherently moral people that wouldn't stoop to using misleading rhetorical techniques, though here they are more likely to be called on it.

In any case, an answer here is available to the public internet for all to see.

Comment author: waitingforgodel 09 December 2010 05:06:13PM 2 points [-]

no sensible person who had the answer would

I respectfully disagree, and have my hopes set on Carl (or some other level-headed person in a position to know) giving a satisfying answer.

This is LessWrong after all -- we can follow complicated arguments, and at least hearing how SIAI is actually thinking about such things would (probably) reduce my paranoia.

Comment author: David_Gerard 09 December 2010 05:11:39PM 1 point [-]

Yeah, but this is on the Internet for everyone to see. The potential for political abuse is ridiculous and can infect even LessWrong readers. Politics is the mind-killer, but pretending it doesn't affect almost everyone else strikes me as not smart.