= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on Risks from AI and Charitable Giving - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (126)
No, you are ignoring Xi's context. The claim is not about what a programmer on the team might do, it is about what the AI might write. Notice that the section starts 'The goals of an AI will be under scrutiny at any time...'
Yes. I thought Xi's claim was that if you have an AI and put it to work writing software, the programmers supervising the AI can look at the internal "motivations", "goals", and "planning" data structures and see what the AI is really doing. Obfuscation is beside the point.
I agree with you and XiXiDu that such observation should be possible in principle, but I also sort of agree with the detractors. You say,
Oh, I'm sure they'd try. But have you ever seen a large software project ? There's usually mountains and mountains of code that runs in parallel on multiple nodes all over the place. Pieces of it are usually written with good intentions in mind; other pieces are written in a caffeine-fueled fog two days before the deadline, and peppered with years-old comments to the extent of, "TODO: fix this when I have more time". When the code breaks in some significant way, it's usually easier to write it from scratch than to debug the fault.
And that's just enterprise software, which is orders of magnitude less complex than an AGI would be. So yes, it should be possible to write transparent and easily debuggable code in theory, but in practice, I predict that people would write code the usual way, instead.