Meta: I'd love to know whether the downvotes are because people don't like the presentation of undeveloped ideas like this, or because they don't think the actual idea is a good one.

(The first would put me off posting similar things in the future, the second would encourage me as a feedback mechanism.)

I would suggest that software is not a good domain to start this project in because it is by nature able to be replicated in various forms. (Ctrl-c, CTRL-V for one).

If you consider other areas; for example - refining a piece of wood. In fact in any other domain where working with a rare natural formation that has value due to its form; (Rare minerals or crystals; Ceramics where once fired you must start again if it doesn't work, pure substances where once contaminated will need to be purified again, )

even simpler models of a perfection process are possible - peeling an egg perfectly, shelling a nut.

Its a matter of putting extra value on a perfect finish product. Where a premium diamond is worth a lot more than a 99% pure one.

Side note: the advantage of software is that it is easily scaleable to teach many people at once.

I think you may indeed be missing the point, which is not

• that formal bug-minimizing software development techniques will suffice to produce safe AI

but

• that producing safe AI requires (what is currently) extraordinary success at "getting things right first time", so that
  • it would be beneficial to get better at doing that, and to foster a culture of trying to do it,
  • and practice at making bug-free software might be an effective way to do so
  • not least because value-specification and programming are sufficiently parallel that some of the same techniques or patterns of thought might be useful in both.

However, what I take to be your main point -- that value-specification and software development are in fact not terribly similar, so that practice at one may not help much with the other -- is as applicable either way.

Having been in the industry for longer than most people around here, I am fairly confident that what you are suggesting is, for all practical purposes, impossible. I mean, it's possible to train and set up prizes, it's impossible to write bug-free code of any complexity.

If you look through the history of such attempts (e.g. Dijkstra's formal verification, various procedural and functional languages specifically developed for this purpose), even the successful ones failed at the stated goals of producing bug-free code.

As Gram_Stone mentioned earlier, the reason is nearly always the same: humans are incapable of producing bug-free non-trivial formal requirements. And that you cannot teach without being a fraud, because human mind itself is too weak and buggy to analyze a complex enough problem.

By using a slightly different offset you get a slightly different nonlinear transformation, and one that may work even better.

That seems pretty unlikely. There's always some subjectivity to the details of coding and transformations, but what constant you add to make logs behave is not one I have ever seen materially change anyone's analysis; I don't think this bikeshedding makes a lick of difference. Again, if you think it does make a difference, I have provided all the code and data.

For example say instead of denominating everything in dollars you'd denominated in cents (and added 1 cent before logging). Then everyone would move up the graph by pretty much log(100), except the people who gave nothing, who would be pulled further from the main part of the graph. I think this would make your fit worse.

Maybe. But would it change any of the conclusions?

In this case we might think that people tend to give something back to society even when they don't do this explicitly as charity donations, so add on a figure to account for this.

...why? One 'gives back to society' just by buying stuff in free markets and by not going out and axe-murdering people, does that mean we should credit everyone as secretly being generous?

My feeling is that $1 is probably smaller than optimal under either interpretation. This would fit with the intuition that going from donating $1 to $9 is likely a smaller deal at a personal level than going from $199 to $999 (counted the same in the current system).

Disagree here as well. As you already pointed out, a more interesting property is the apparent split between people who give nothing and people who give something; someone who gives $199 is already in the habit and practice of donations just like someone who is giving $999, while going from $1 to $9 might represent a real change in personal propensity. ($1 might be tossing a beggar a dollar bill and that person really is not a giver, while $9 might be an explicit donation through Paypal for a fundraiser.)

It's a nonlinear transformation to turn nonlinear totals back into something which is linear, and it does so very well, as you can see by comparing the log graph with an unlogged graph. Again, I'm not seeing what the problem here is. What do you think this changes? Ordering is preserved, zeros are preserved, and dollar amounts become linear which avoids a lot of potential problems with the usual statistical machinery.

I don't see why adding +1 to all responses would make any difference to any of the comparisons; it shifts all datapoints equally. (And anyway, log1p(0) ~> 0. The point of using log1p is simply to avoid log(0) ~> -Inf`.)

R(0) should be total historical spend right? Rather than annual spend?

I put down the first of my thoughts here: http://tinyurl.com/ok3loj7. If I get things where I want them I may post it to LW or turn it into a sequence.

Fair enough. I've edited my original comment.

(For posterity: the text for my original comment's first hyperlink originally read "0 and 1 are not probabilities".)