= 27d567bc2d48d9f40e9ccc20ea370b15)
turchin comments on Open thread, Mar. 14 - Mar. 20, 2016 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (212)
Probably everybody had seen it, but EY wrote long post on FB about AlphaGO which get 400 reposts. The post overestimates power of AlphaGO, and in general it seems to me that EY did too much conclusions based on very small available information (3:0 wins at the moment of the post - 10 pages of conclusions). The post's comment section includes contribution from Robin Hanson about usual foom's speed and type topic. EY later updated his predictions based on Segol win on game 4 and stated that even superhuman AI could make dumb mistakes, which may result in new type of AI failures.
https://www.facebook.com/yudkowsky/posts/10154018209759228?pnref=story
So, whats the difference between 'superhuman with dumb mistakes', 'dumb with some superhuman skills', and 'better at some things and worse at others'?
I think the difference here is distribution.
superhuman with dumb mistakes' - 4 brilliant games, one stupid loose.
dumb with some superhuman skills - dumb in one game, unbeatble in another.
better at some things and worse at others - different performance in different domains.
I think that if superhuman AI with bugs will start to self-improve, the bugs will start to accumulate. This will ruin or AIs power, or AIs goal system. The first is good and the second is bad. I also could suggest that first AI which will try to self improve will still have some bugs. The open question is if AI will be able to debug itself. Some bugs may prevent seeing them as bugs, so they are reccurent. The closest thing is human bias of overconfidence. Overconfident human can't understand that there is something wrong with him.