= 27d567bc2d48d9f40e9ccc20ea370b15)
timtyler comments on Evidence for the orthogonality thesis - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (289)
Any kind of agent could - in principle - be engineered.
However, some sorts of agent are more likely to evolve than others - and it is this case that actually matters to us.
For example, intelligent machines are likely to coevolve in a symbiosis with humans - during which they will pick up some of our values. In this case, intelligence and values will be powerfully linked - since stupid machines will fail to absorb so many of our values - as we have seen, for example, with the evolution of cars.
So: The Orthogonality Thesis:
...is true[*] - but the "in principle" renders it kind-of irrelevant to the case that we actually care about.
* Unless the wirehead / pornography problems turn out to actually be serious issues.
I have doubts that it is even true "in principle" unless the goals are hard-wired in and unmodifiable by the intelligence. Do you really think that someone would agree to be OCD or schizophrenic if they had a choice? For higher levels of intelligence, I would think they would be even more discriminating as to goal-states they would accept.
As for the argument by thelittledoctor, the evil genius dictator model is broken even for highly intelligent humans, much less super-intelligences. Those "intelligent" demagogues are rarely, if ever, more than 2 standard deviations above average human intelligence, that definitely doesn't count as "highly intelligent" as far as I'm concerned.
It seems irrelevant whether the AI is quote-unquote "highly intelligent" as long as it's clever enough to take over a country and kill several million people.
The usual argument is that we are likely to be able to build machines that won't want to modify their goals.
IMO, the more pressing issue with something like OCD is that it might interfere with intelligence tests - in which case you could argue that an OCD superintelligent machine is not really intelligent - since it is using its intelligence to screw itself.
This seems to be a corner case to me. The intended point is more that you could engineer an evil genius, or an autistic mind child.