Comment on Popperian Decision making

Thu, 07 Apr 2011 13:15:52 GMT

Note that Popperians think there is no algorithm that automatically arrives at rational beliefs. There's no privileged road to truth. AIs will not be more rational than people. OK they usually won't have a few uniquely human flaws (like, umm, caring if they are fat). But there is no particular reason to expect this stuff will be replaced with correct ideas. Whatever AIs think of instead will have its own mistakes. It's the same kind of issue as if some children were left on a deserted island to form their own culture. They'll avoid various mistakes from our culture, but they will also make new ones. The rationality of AIs, just like the rationality of the next generation, depends primarily on the rationality of the educational techniques used (education is closely connected to epistemology in my view, because it's about learning, i.e. creating knowledge.

This is mostly irrelevant to your main point, but I'm going to talk about it because it bothered me. I don't think anyone on LessWrong would agree with this paragraph, since it assumes a whole bunch of things about AI that we have good reasons to not assume. The rationality of an AI will depend on its mind design--whether it has biases built into its hardware or not us up to us. In other words, you can't assert that AIs will make their own mistakes because this assumes things about the mind design of the AI, things that we can't assume because we haven't built it yet. Also, even if an AI does have its own cognitive biases, it still might be orders of magnitude more rational than a human being.

Discuss