Harry suggested himself that dementors don't really think. It might not be able to give a meaningful response, and even if it can, there's nothing to suggest that they can learn from the experience of other dementors or instinctively recognize Harry as anathema.
Even if the hypothesis that they don't really think, just behave in a way that people think they will, is true; it's phrased in such a way as to cause doubt in the purple-robes (the use of "just say yes or no" to make them think there's more information then they're getting) and belief in albus. If it just answers "yes" to the first question, it's still plausible within the purple robes belief sets, but hopefully makes harry seem mysterious enough that they believe the next question can also be a yes. Then albus, who's a true believer, pushes the consensus reality over to a yes with his understanding that harry has the power to destroy them.
EDIT: New discussion thread here.
This is a new thread to discuss Eliezer Yudkowsky's Harry Potter and the Methods of Rationality and anything related to it. With two chapters recently the previous thread has very quickly reached 500 comments. The latest chapter as of 17th March 2012 is Ch. 79.
There is now a site dedicated to the story at hpmor.com, which is now the place to go to find the authors notes and all sorts of other goodies. AdeleneDawner has kept an archive of Author's Notes. (This goes up to the notes for chapter 76, and is now not updating. The authors notes from chapter 77 onwards are on hpmor.com.)
The first 5 discussion threads are on the main page under the harry_potter tag. Threads 6 and on (including this one) are in the discussion section using its separate tag system. Also: one, two, three, four, five, six, seven, eight, nine, ten.
As a reminder, it's often useful to start your comment by indicating which chapter you are commenting on.
Spoiler Warning: this thread is full of spoilers. With few exceptions, spoilers for MOR and canon are fair game to post, without warning or rot13. More specifically: