One problem is that if we, say, start admiring people for acting in "more utilitarian" ways, what we may actually be selecting for is psychopathy.

Utilitarianism is not in our nature. Show me a man who would hold a child’s face in the fire to end malaria, and I will show you man who would hold a child’s face in the fire and entirely forget he was originally planning to end malaria.

James A. Donald

A good effort doesn't result in valuable software, but it could result in you learning to program better, increasing your human capital.

To encourage yourself to do some massive, granular task:

• Upon completion of each granule, give yourself a reward with some probability.

• A reward is a small piece of food or a sip of a drink, etc.

• Never eat or drink anything except as a reward for working on the task.

This really works extremely well for me; I have been doing this for about 2 months, at first only with anki reviews and more recently for several other things. The feeling is very similar to addictions like video games or entertaining websites; I often think "I should probably go do X, but let me instead do just one more anki card" and a half-hour later I realize I still haven't done X.

More things:

• Make the rewards unlikely and small so that you stay constantly hungry. Bonus: caloric restriction.

• Create a timed reminder, say half-hourly, to do just a few granules of the task. This encourages episodes of the "just one more" effect.

• Put reinforcers within arm's reach, both temporally (make granules easy and quick, so that hunger feels like an urge to do the task rather than an urge to cheat the system) and spacially (so that you are constantly reminded of your hunger and tempted to do the task).

I repeat: this works extremely well for me and I strongly encourage other people to try it. More details here.

Here is a graph showing the number of Anki reviews I've done every month for the past year, as an example of the results this method can produce.

If knowledge can create problems, it is not through ignorance we can solve them.

-- Isaac Asimov

Is that true or is Gombrich just handling a needle convincingly?

As I understand it from reading the sequences, Eliezer's position roughly boils down to "most AI researchers are dilettantes and no danger to anyone at the moment. Anyone capable of solving the problems in AI at the moment will have to be bright enough, and gain enough insights from their work, that they'll probably have to solve Friendliness as part of it - or at least be competent enough that if SIAI shout loud enough about Friendliness they'll listen. The problem comes if Friendliness isn't solved before the point where it becomes possible to build an AI without any special insight, just by throwing computing power at it along with a load of out-of-the-box software and getting 'lucky'."

In other words, if you're convinced by the argument that Friendly AI is the most important problem facing us, the thing to do is work on Friendly AI rather than prevent other people working on unFriendly AI. Find an area of the problem no-one else is working on, and do that. That might sound hard, but it's infinitely more productive than finding the baddies and shooting at them.

Given that (redacted) It is a very, very, VERY bad idea to start talking about (redacted), and I would suggest you should probably delete this post to avoid encouraging such behaviour.

EDIT: Original post has now been edited, and so I've done likewise here. I ask anyone coming along now to accept that neither the original post nor the original version of this comment contained anything helpful to anyone, and that I was not suggesting censorship of ideas, but caution about talking about hypotheticals that others might not see as such.

I think that would just yield your revealed preference function. As I said, trying to optimize that is like a falling apple trying to optimize "falling". It doesn't describe what you want to do; it describes what you're going to do next no matter what.

I'm thinking either "lazy" or "irresponsible".