= 27d567bc2d48d9f40e9ccc20ea370b15)
dspeyer comments on Thoughts on the Singularity Institute (SI) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (1270)
A tool+human differs from a pure AI agent in two important ways:
The human (probably) already has naturally-evolved morality, sparing us the very hard problem of formalizing that.
We can arrange for (almost) everyone to have access to the tool, allowing tooled humans to counterbalance eachother.
Well, I certainly agree that both of those things are true.
And it might be that human-level evolved moral behavior is the best we can do... I don't know. It would surprise me, but it might be true.
That said... given how unreliable such behavior is, if human-level evolved moral behavior even approximates the best we can do, it seems likely that I would do best to work towards neither T nor A ever achieving the level of optimizing power we're talking about here.
Humanity isn't that bad. Remember that the world we live in is pretty much the way humans made it, mostly deliberately.
But my main point was that existing humanity bypasses the very hard did-you-code-what-you-meant-to problem.
I agree with that point.