how many cool people there are and how willing they would be to participate on a site like LessWrong
A better question is how many of them are willing to create a site like LessWrong.
Also minor nitpick about your use of the word 'cool', since it normally denotes social status rather than rationality.
Have you looked at the rest of the internet recently?
Not really. But are you saying that nowhere else on the internet is close to LessWrong's standards of discourse? I'd figured that but part of me keeps saying "there's no way that can be true" for some reason.
It might be true when it comes to cross-domain rationality (with a few outliers like social abilities). But it certainly isn't true that Less Wrong is anywhere close to the edge in most fields (with a few outliers like decision theory).
One of the most annoying arguments when discussing AI is the perennial "But if the AI is so smart, why won't it figure out the right thing to do anyway?" It's often the ultimate curiosity stopper.
Nick Bostrom has defined the "Orthogonality thesis" as the principle that motivation and intelligence are essentially unrelated: superintelligences can have nearly any type of motivation (at least, nearly any utility function-bases motivation). We're trying to get some rigorous papers out so that when that question comes up, we can point people to standard, and published, arguments. Nick has had a paper accepted that points out the orthogonality thesis is compatible with a lot of philosophical positions that would seem to contradict it.
I'm hoping to complement this with a paper laying out the positive arguments in favour of the thesis. So I'm asking you for your strongest arguments for (or against) the orthogonality thesis. Think of trying to convince a conservative philosopher who's caught a bad case of moral realism - what would you say to them?
Many thanks! Karma and acknowledgements will shower on the best suggestions, and many puppies will be happy.