= 27d567bc2d48d9f40e9ccc20ea370b15)
VAuroch comments on The Magnitude of His Own Folly - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (127)
Eli, sometimes I find it hard to understand what your position actually is. It seems to me that your position is:
1) Work out an extremely robust solution to the Friendly AI problem
Only once this has been done do we move on to:
2) Build a powerful AGI
Practically, I think this strategy is risky. In my opinion, if you try to solve Friendliness without having a concrete AGI design, you will probably miss some important things. Secondly, I think that solving Friendliness will take longer than building the first powerful AGI. Thus, if you do 1 before getting into 2, I think it's unlikely that you'll be first.
Any sufficiently-robust solution to 1 will essentially have to be proof-based programming; if your code isn't mapped firmly to a proof that it won't produce detrimental outcomes, then you can't say in any real sense that it's robust. When an overflow error could result in the 'FAI''s utility value of cheesecake going from 10^-3 to 10^50, you need some damn strong assurance that there won't be an overflow.
Or in other words, one characteristic of a complete solution to 1 is a robust implementation that retains all the security of the theoretical solution, or in short, an AGI. And since this robustness continues to the hardware level, it would be an implemented AGI.
TL;DR: 1 entails 2.