I want to propose a variant of the Counterfactual Mugging problem discussed here. BE CAREFUL how you answer, as it has important implications, which I will not reveal until the known dumb humans are on record.
Here is the problem:
Clipmega is considering whether to reveal to humans information that will amplify their paperclip production efficiency. It will only do so if it expects that, as a result of revealing to humans this information, it will receive at least 1,000,000 paperclips within one year.
Clipmega is highly accurate in predicting how humans will respond to receiving this information.
The smart humans' indifference curve covers both their current condition and the one in which Clipmega reveals the idea and steals 1e24 paperclips. (In other words, smart humans would be willing to pay a lot to learn this if they had to, and there is an enormous "consumer surplus".)
Without Clipmega's information, some human will independently discover this information in ten years, and the above magnitude of the preference for learning now vs later exists with this expectation in mind. (That is, humans place a high premium on learning it how, even though they will eventually learn it either way.)
The human Alphas (i.e., dominant members of the human social hierarchy), in recognition of how Clipmega acts, and wanting to properly align incentives, are considering a policy: anyone who implements this idea in making paperclips must give Clipmega 100 paperclips within a year, and anyone found using the idea but not having donated to Clipmega is fined 10,000 paperclips, most of which are given to Clipmega. It is expected that this will result in more than 1,000,000 paperclips being given to Clipmega.
Do you support the Alphas' policy?
Problem variant: All of the above remains true, but there also exist numerous "clipmicros" that unconditionally (i.e. irrespective of their anticipation of behavior on the part of other agents) reveal other, orthogonal paperclip production ideas. Does your answer change?
Optional variant: Replace "paperclip production" with something that current humans more typically want (as a result of being too stupid to correctly value paperclips.)