= 27d567bc2d48d9f40e9ccc20ea370b15)
lessdazed comments on The 5-Second Level - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (310)
Nitpick: It took me a bit to realize you meant "going back to being among non-rationalists" rather than "going back to the meeting".
Unfortunately I recognize that as the bitter truth, so it's of no use for me for training purposes.
Here's something which might work as an indignation test-- could it be a good move for an FAI to set a limit on human intelligence?
If an AI can be built, it has been shown that humanity is an AI-creating species. As technology and the promulgation of human knowledge improves, it will become easier and easier to make AIs, and the risk of creating a UFAI that the FAI can't defeat goes up.
It will be easier to have people who can't make AIs than to try to control the tech and knowledge comprehensively enough to make sure there are no additional FOOMs.
I considered limiting initiative (imposing akrasia) rather than intelligence, but I think that would impact a wider range of human values.
Apologies for not being at all indignant, but can we generalize to say you have suggested that it could be good to limit something good because that it is a sufficient solution to a specific problem?
I'd appreciate it if someone can show how endorsements of locally bad, merely sufficient solutions are (or aren't) all implicitly arguments from ignorance, confessing to not knowing how to achieve the same results without the negative local consequences..
In other words, sure, limiting locally good thing X could be good on balance if doing so has generally positive consequences (like assassinating Hitler on certain dates), but that really depends on there not being something better on balance in general, of which an interesting case is a solution that has the same positive consequences but fewer negative ones.