J_Thomas2 comments on Invisible Frameworks - Less Wrong

12 Post author: Eliezer_Yudkowsky 22 August 2008 03:36AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (40)

Sort By: Old

You are viewing a single comment's thread.

Comment author: J_Thomas2 22 August 2008 04:08:43PM 0 points [-]

I haven't read Roko's blog, but from the reflection in Eliezer's opposition I find I somewhat agree.

To the extent that morality is about what you *do*, the more you can do the higher the stakes.

If you can drive a car, your driving amplifies your ability to do good. And it amplifies your ability to do bad. If you have a morality that leaves you doing more good than bad, and driving increases the good and the bad you do proportionately, then your driving is a good thing.

True human beings have an insatiable curiousity, and they naturally want to find out about things, and one of the things they like is to find out how to do things. Driving a car is a value in itself until you've done it enough that it gets boring.

But if you have a sense of good and bad apart from curiousity, then it will probably seem like a good thing for good smart people to get the power to do lots of things, while stupid or evil people should get only powers that are reasonably safe for them to have.