= 27d567bc2d48d9f40e9ccc20ea370b15)
satt comments on Superintelligence 15: Oracles, genies and sovereigns - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (30)
I don't think there's one specific common vision about how a scheming AI would emerge.
I'm just extrapolating from my experience as someone who programs computers but can't even pinpoint why my e-book reader freezes up when it does. Twenty-first century computers are already so complex that no person can track what they do in real-time, and as computers and their programs become more complex, tracking their behaviour is going to become an even harder task. (The obvious way out is to implement meta-programs and/or meta-computers which do that job, but then of course we have to ask how we can keep a close eye on those.)
A set of specialized oracles could be used to monitor inputs, internal computations and outputs. One oracle keeps records of every input and output. The question to this oracle is always the same: Is the AI lying? Another oracle is tasked with input steam analysis to filter out any taboo chunks. Other oracles can serve to monitor internal thought processes and self-improvement steps.
If these safeguarding oracles are strictly limited in their capabilities they do not pose a risk to the outside world. The core of such oracles could consist of straight forward defined heuristical rule sets.
Any of above mentioned oracles can be validated and independently certified. This would create a framework for tools, genies and sovereigns with assured friendliness.