= 27d567bc2d48d9f40e9ccc20ea370b15)
buybuydandavis comments on Intelligence explosion in organizations, or why I'm not worried about the singularity - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
Comments (187)
Free market theorists from at least Smith considered a market as a benevolent super intelligence. In 1984, Orwell envisioned an organization as a mean super intelligence. In both cases, the functional outcome of the super intelligence ran counter to the intent of the component agents.
There have been very mean superintelligences. Political organization matters. They can be a benevolent invisible hand, or a malevolent boot stomping a human face forever.
Yup. There exist established fields that study super intelligences with interests not necessarily aligned with ours -- polisci, socialsci and econ. Now you may criticize their methods or their formalisms, but they do have smart people and insights.
I think the research into Friendliness, if it's not a fake, would do well to connect with some subproblem in polisci, socialsci or econ. It ought to be easier than the full problem, and the solution will immediately pay off. I asked Vassar about this once, and he said that he did not think this would be easier. I never really understood that reply.
The main response I assume is the fact that friendly agents are not yet invented, or the ideas exposed here are new, this post. The theoretical background could overlap with other sciences, but the main goal(FAI) needs more than that, I supose.
+1
I'll give you Smith, but I don't think Orwell had intelligence as such in mind. One of the main things distinguishing 1984's Ingsoc from non-fictional 20th-century despotism, in fact, was that it didn't pretend to be an agent, that it didn't have goals like "conquer the world" or "safeguard the coming revolution": instead, it was more like a dumb attractor in ideology-space tending towards the undirected exercise of coercive state power for its own sake.
It pretended to be an agent with goals like protecting the people from Eastasia and Eurasia.
Those pretenses were means to the end of the Coercive State Power Maximizer.
And I don't see how you distinguish Smith from Orwell in terms of intelligence or agency. If anything, I see more agency in Ingsoc than a market.