That's a surprising conclusion to me which I hadn't seen before, but also doesn't seem too hard to come up with, so I'm curious where I've gone off the rails. This argument has a very Will_Newsomey flavor to it to me.

Perhaps it is not wise to speculate out loud in this area until you've worked through three rounds of "ok, so what are the implications of that idea" and decided that it would help people to hear about the conclusions you've developed three steps back. You can frequently find interesting things when you wander around, but there are certain neighborhoods you should not explore with children along for the ride until you've been there before and made sure its reasonably safe.

Perhaps you could send a PM to Will?

Establish a scholarship to collect information on young talent

Related: Reaching young math/compsci talent

Create a merit scholarship for the type of young talent that SI wants to attract – this can reveal valuable information about this group of people, and can potentially be used a targeted publicity tool if handled well.

Information that could be collected from applications

• Basic personal details (age, location, contact methods, etc.)
• Education (past and future)
• Academic interests
• Career goals
• Awards and competition results
• Third-party reviews (i.e., letters of recommendation)
• Basic personality assessment (see previous LW discussion on correlates with Big Five personality traits: [1], [2], [3])
• Ideas about and attitudes toward x-risks/FAI/SI/FHI (these could be responses to prompts – as a bonus, applicants are introduced to the content in the prompts)
• ... Pretty much anything else (personal anecdote: I've revealed things about myself in college and scholarship applications that I have never expressed to anyone else)

Uses of this information

• Check whether SI is effectively reaching the right people with its current plans.
• The compiled list of young talent could be directly used to advertise things like SPARC to the right people.
• General survey tool.

Potential problems and difficulties

• Its use as an information gathering tool could be seen negatively.
• Legal issues?
• Publicity. The scholarship has to be made known to the relevant people, and this has to be done in such a way that SI is seen as a reputable institute. However, a scholarship does open up new avenues for publicity.
• Cost and manpower.

Is anyone else doing this?

As with many ideas, we ought to be cautious if we see no one else doing something similar. Indeed, I cannot think of any high school scholarship that is used primarily to collect information for the sponsoring organization (is this really the case?). However, there is good reason for this – no one else is interested in reaching the same group of high school students. SI is the only organization I know of who wants to reach high school students for their research group.

FHI had a competition that could be an attempt to collect information, but I'm not sure.

High school scholarships

It would be wise to consult current high school scholarships, and AoPS has a good list.

The reason I thought you didn't understand what I was talking about was that I was calling on examples from day to day life, this is what I took "instrumentalist" to mean, and you starting talking about killing people, which is not an event from day to day life.

If you are interested in continuing this discussion (which if not I won't object) let's take this one step at a time; does that difference seem reasonable to you?

Certainly there's a difference between what I said and the traditional phrasing of the dilemma; certainly the idea of sacrificing oneself versus another is a big one.

But the OP was asking for an instrumentalist reason to choose torture over dust specks. It is pretty far-fetched to imagine that literally torturing someone will actually accomplish... well, almost anything, unless they're a supervillain creating a contrived scenario in which you have to torture them.

When you will actually be trading quality of life for barely-tangible benefit on a large scale is torturing yourself working at a startup. This is an actual decision that people make to make lives miserable in exchange for minor but widespread public goods. And I fully support the actual trades of this sort that people actually make.

That's my instrumentalist argument for, as a human being, accepting the metaphor of dust specks versus torture, not my philosophical argument for a decision theory that selects it.

If I recall correctly Alicorn made a reference to reversing the utilities in this argument... would you think it better for someone to give up a life of the purest and truest happiness, if in exchange they created all of the 10 second or less cat videos that will ever be on youtube throughout all of history and the future?

My intuitions here say yes; it can be worth sacrificing your life (i.e. torturing yourself working at a startup) to create a public good which will do a small amount for a lot of people (i.e. making standard immunization injections also give people immunity to dust specks in their eyes)

I expected that my intuitive preference for any number of dust specks over torture would be easy to formalize without stretching it too far. Does not seem like it.

On the other hand, given the preference for realism over instrumentalism on this forum, I'm still waiting for a convincing (for an instrumentalist) argument for this preference.

I don't think it was strictly speaking "persuasive" to anyone

Just as a data point, I was rather greatly persuaded by Karnofsky's argument here. As someone who reads LW more often for the cognitive science/philosophy stuff and not so much for the FAI/Singularity stuff, I did not have a very coherent opinion of the SI, particularly one that incorporated objective critiques (such as Karnofsky's).

Furthermore, I certainly did not, as you assert, know that it is a bad idea to donate to the Singularity Institute. In fact, I had often heard the opposite here.

Apart from the value of having a smart, sincere person who likes and has seriously tried to appreciate you give you their opinion of you ... Holden's post directly addresses "why the hell should people give money to you?" Particularly as his answer - as a staff member of a charity directory - is "to support your goals, they should not give money to you." That's about as harsh an answer as anyone could give a charity: "you are a net negative."

My small experience is on the fringes of Wikimedia. We get money mostly in the form of lots of small donations from readers. We have a few large donations (and we are very grateful indeed for them!) but we actively look for more small donations (a) to make ourselves less susceptible to the influence of large donors (b) to recruit co-conspirators: if people donate even a small amount, they feel like part of the team, and that's worth quite a lot to us.

The thing is that Wikimedia has never been very good at playing the game. We run this website and we run programmes associated with it. Getting money out of people has been a matter of shoving a banner up. We do A/B testing on the banners! But if we wanted to get rabid about money, there's a lot more we could be doing. (At possible expense of the actual mission.)

SIAI doesn't have the same wide reader base to get donations from. But the goal of a charity that cares about its objectives should be independence. I wonder how far they can go in this direction: to be able to say "well, we don't care what you say about us, us and our readers are enough." I wonder how far the CMR will go.