DSimon comments on Human errors, human values - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (135)
Saying it's encoding human irrationality is taking the viewpoint that the human reaction to the fat-man trolley problem is an error of reasoning, where the particular machinery humans use to decide what to do gives an answer that does not maximize human values.
It makes some sense to say that a human is a holistic entity that can't be divided into "values" and "algorithms". I argued that point in "Only humans can have human values". But taking that view, together with the view that you should cling to human values, means you can't be a transhumanist. You can't talk about improving humans, because implementing human values comes down to being human. Any "improvement" to human reasoning means giving different answers, which means getting "wrong" answers. And you can't have a site like LessWrong, that talks about how to avoid errors that humans systematically make - because, like in the trolley problem case, you must claim they aren't errors, they're value judgements.
Even when holding a view that human values can't be improved, rationality techniques are still useful, because human values conflict with each other and have to be prioritized or weighted.
If value knowing the truth, and I also in the holistic sense "value" making the conjunction fallacy, then LessWrong is still helpful to me provided I value the first more than the second, or if the weighting is such that the net value score is increased even though the individual conjunction fallacy value is decreased.