= 27d567bc2d48d9f40e9ccc20ea370b15)
hairyfigment comments on Intelligence explosion in organizations, or why I'm not worried about the singularity - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (187)
At a glance this seems pretty silly, because the first premise fails. Organizations don't have goals. That's the main problem. Leaders have goals, which frequently conflict with the goals of their followers and sometimes with the existence of the organization.
Do humans have goals in this sense? Our subsystems seem to conflict often enough.
We have goals, but they are not consistent over time. The worries about artificial agents(with more power) is that, these values if bad implemented, would create losses we could not accept, like extinction.
In this case it doesn't seem like much of a conflict. I think that barring more-or-less obvious signs of disarray we can count on organizations trying to serve their leaders' self-perceived interests - which, while evil, entail not killing humanity - unless and until the singularity changes the game.
James Q. Wilson wrote a book explaining why this often isn't so.
You might also consider looking a Essence of Decision, which analyzes problems JFK had trying to control various government organizations during the Cuban Missile Crisis. If you want to say that the relevant leaders were the heads of those organizations (eg. the Secretaries of State and Defense), you need to articulate a non-circular theory to identify who the leader of an organization is.
The frak? If an organization like America contains multiple parties explicitly and publicly promising to defeat each other - eg, because people in the other one secretly serve a hostile organization - that falls under "more-or-less obvious signs of disarray".
Can you play that out a little? I think what I'm trying to assert and what you are interpreting aren't the same thing.
My intended assertion was that the sentence:
is false. Further, analyzing that fact in terms of "goals" of the State Department and the Department of Defense leads to insightful and useful conclusions about how organizations work.
As a more concrete addendum to aleksiL, note that McDonalds Corp produces hamburgers for sale. That's how the entity implements the generic policy "maximize shareholder value."
If that is not a "goal" of the entity know as McDonalds, then there is something wrong with our definition of goal.
Sometimes, it is really hard to measure how well an organization achieves its goals - how could we tell if the US DoD is providing the military forces needed to deter war and to protect the security of the United States. But that's different from saying that the DoD does not have any goals.
They have mission statements instead. These serve the same function as most self-proclaimed human goals - public relations.
I think there's a lot to this line of thinking. It's in fact the counterargument I find most threatening to my position.
But I think you are assuming an organization with a particularly autocratic leadership. In some organizations, leadership is broadly distributed.
For example, in many open source software development communities, decisions about how to change the source code are made by a consensus of their developers.
When these developers are using their own software in the process of developing and/or communicating (such as in the case of Git, or Mailman, or Emacs), then I think there's a case for a genuine, distributed sense of organizational intelligence with recursive self-modification.