Qiaochu_Yuan comments on Rationality Quotes February 2013 - Less Wrong

2 Post author: arundelo 05 February 2013 10:20PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (563)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 06 February 2013 11:33:28PM 22 points [-]

I've just come across a fascinatingly compact observation by I. J. Good:

Public and private utilities do not always coincide. This leads to ethical problems. Example - an invention is submitted to a scientific adviser of a firm...

The probability that the invention will work is p. The value to the firm if the invention is adopted and works is V, and the loss if the invention is adopted and fails is L. The value to the adviser personally if he advises the adoption of the invention and it works is v, and the loss if it fails to work is l. The losses to the firm and the adviser if he recommends the rejection of the invention are both negligible...

Then the firm's expected gain if the invention is adopted is pV - (1-p)L and the adviser's expected gain in the same circumstances is pv - (1-p)l. The firm has positive expected gain if p/(1-p) > L/V, and the adviser has positive expected gain if p/(1-p) > l/v.

If l/v > p/(1-p) > L/V, the adviser will be faced with an ethical problem, i.e. he will be tempted to act against the interests of the firm.

This is a beautifully simple recipe for a conflict of interest:

Considering absolute losses assuming failure and absolute gains conditioned on success, an adviser is incentivized to give the wrong advice, precisely when:

  • The ratio of agent loss to agent gain,
  • exceeds the odds of success versus failure
  • which in turn exceeds the ratio of principal loss to principal gain.

You can see this reflected in a lot of cases because the gains to an advisor often don't scale anywhere near as fast as the gains to society or a firm. It's the Fearful Committee Formula.

Comment author: Qiaochu_Yuan 06 February 2013 11:36:16PM 5 points [-]

You can see this reflected in a lot of cases because the gains to an advisor often don't scale anywhere near as fast as the gains to society or a firm.

Name three?

Comment author: Vaniver 06 February 2013 11:44:29PM 7 points [-]

The success of Market-Based Management / Koch Industries appears to be due at least in part to their focus on NPV at the managerial level. You get stories like (from memory, and thus subject to fuzz) the manager of a refining plant selling the land the plant was on to a casino which was moving to the area, which he was rewarded for doing because the land the plant was on was more valuable to the casino than the company, even after factoring in the time lost because the plant was shut down and relocated. The corporate culture (and pay incentive structure) rewarded that sort of lateral use of resources, whereas a culture which compartmentalized people and departments would have balked at the lost time and disruption.