khafra comments on Intelligence explosion in organizations, or why I'm not worried about the singularity - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (187)
I think the simplifying features on the other side outweigh those--ie., it's built from atomic units that do exactly what you tell them to, and there are probably fewer abstraction layers between those atomic units and the goal system. But I do think Mechanism Design is an important field, and will probably form an important part of any friendly optimizing process.
Organisations are likely to build machine intelligence and imbue it with their values. That is reason enough to be concerned with organisation values. One of my proposals to help with this is better corporate repuatation systems.