Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Tom_McCabe2 comments on To Lead, You Must Stand Up - Less Wrong

28 Post author: Eliezer_Yudkowsky 29 December 2007 06:38AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (38)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Tom_McCabe2 29 December 2007 11:48:12PM 1 point [-]

"Only people with mistaken ideas about "rationality" will find themselves believing that "irrationality" will help."

Conjecture: Take two arbitrary, identical optimization processes. Pick some verifiable piece of information A. Give the first process the belief "A", and the second process the belief "~A". The average end utility of the first process will always be equal to or greater than the average end utility of the second process.

Attempted disproof: Suppose that both processes were programmed by naive human rationalists, who give the processes the supergoal of correcting mistaken beliefs. Also suppose that the universe is finite, and has a finite complexity. Both processes will improve their models of the universe, in accordance with the laws of rationality, until they eventually reach an asymptote due to the finite amount of modelable information. Because the utility is determined by the amount of new information that had to be added to the model, the second process will have a higher end utility, as it had to add the information "A" to correct the false information "~A".