= 27d567bc2d48d9f40e9ccc20ea370b15)
Vaniver comments on Histocracy: Open, Effective Group Decision-Making With Weighted Voting - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (62)
I like the idea of measurement. The problem, though, is that you get what you measure, not what you wanted to measure.
Suppose Charlie is risk-averse, and only approves projects with a 95% chance of meeting expectations. David is risk-neutral, and will approve projects that have a positive EV that are significantly higher than other available projects. Oftentimes, they're speculative and will only exceed expectations about 10% of the time, since they only have about a 10% chance of succeeding.
Charlie will get about nine times as many votes as David, eventually. If David votes against Charlie's projects as too bland and too low EV, this will go even worse for David, as eventually only Charlie's projects will be approved and David will be recorded as pessimistic on all of them.
Decision-making is not a logistic regression problem, and so I am pessimistic about logistic regression approaches applied to it. I agree that measuring decision-making ability is a very important task, but approaches like Market-Based Management seem far more promising.
Also known as Goodhart's law.
If the organization is risk-averse, it doesn't want risk-neutral voters to gain influence. If it's risk-neutral, then it should incorporate opportunity costs when judging projects in hindsight. Furthermore, if in hindsight a rejected project still appears to have had a high positive EV, the org should register the rejection of the project as a mistake.
Suppose the organisation is risk-neutral, and Charlie abstains from the sub-95% chance projects rather than rejecting them (in a large organisation that makes many decisions you can't expect everyone to vote on everything). He also rejects the sub-5% projects.
By selectively only telling you what you already knew, Charlie builds up a reputation of being a good predictor, as opposed to David, who is far more often wrong but who is giving actual useful input.
This misses the heart of that criticism: mistakes have different magnitudes.