In order to set on limits on the kinds of things an AI will do, you need to understand how it works.
How is that impossible with a replicated brain architecture? We can't make one if we don't know how it works.
This is easier when you've got a clearly defined structure to the AI, and know how the parts interact, and why.
Of course. However, how you plan to structure AI what I am asking about. There are many theories about how to structure the AI - so why did the SIAI choose to only focus on a theoretical mathematical logic based approach rather than taking the most advanced, if still flawed, logic device known to man and replicating and improving that?
In order to set on limits on the kinds of things an AI will do, you need to understand how it works.
How is that impossible with a replicated brain architecture? We can't make one if we don't know how it works.
If you have the right tools, you can make a brain without understanding it. Reproductive system can make brains. Whole brain emulation doesn't require understanding of brain architecture, only the dynamics of its lowest-level components.
You "know" how pi works, and how to set up a program that computes it, but you don't know what its q...
I declare this Open Thread open for discussion of Less Wrong topics that have not appeared in recent posts.