timtyler comments on GiveWell interview with major SIAI donor Jaan Tallinn - Less Wrong

17 Post author: jsalvatier 19 July 2011 03:10PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (8)

You are viewing a single comment's thread.

Comment author: timtyler 20 July 2011 10:14:12AM *  4 points [-]

This bit (from Karnofsky):

I feel like once we basically understand how the human predictive algorithm works, it may not be possible to improve on that algorithm (without massive and time-costly experimentation) no matter what the level of intelligence of the entity trying to improve on it. (The reason I gave: The human one has been developed by trial-and-error over millions of years in the real world, a method that won't be available to the GMAGI. So there's no guarantee that a greater intelligence could find a way to improve this algorithm without such extended trial-and-error)

...is probably not right. Nobody really knows how tough this problem will prove to be once we stop being able to crib from the human solution - and it is possible that progress will get tougher. However, much of the progress on the problem has not been obviously based on reverse-engineering the human prediction algorithm in the first place. Also machine prediction capabilities already far exceed human ones in some domains - e.g. chess, the weather.

Anyway, this problem makes little difference either way. Machines don't have the human pelvis to contend with, and won't be limited to running at 200 Hz.