I think it's worth taking a step back from the details of any one of Derren Brown's effects, and looking at the nature of stage magic. A stage magician employs a set of techniques called misdirection to mislead the audience as to how a trick is performed, to direct their attention to irrelevant aspects of the performance, or to encourage them to misinterpret relevant aspects.

An important technique in misdirection is to provide the audience with a false explanation for how the trick is done. A magician who says that a trick is done by magic encourages you watch carefully at the point where he waves his magic wand (knowing that this does the audience no good, because the rabbit was already loaded into the hat). A magician who says that a trick is done by science encourages you to look at the fancy gears of his machine (when actually there's an assistant hidden inside). A magician who says that a trick is done by psychic powers encourages you to watch carefully at the point where he concentrates on reading the subject's mind (when actually the card was marked or forced).

Knowing all this, as I imagine you do, what are we to make of a magician who explains that a trick is done by psychology? I guess this time he might be telling the truth, right?

Now, let's look at your examples with the above in mind, and ask some questions.

In the first example, why does David Frost later agree that he was thinking of a place, after first denying it?

In the second example (guessing the pet name),

1. Is Derren taller or shorter than the woman?
2. Did Derren pick the other volunteer (the man with the shoulder bag) before or after explaining how to read minds?
3. Where was Derren standing while he explained to the woman how to read minds?
4. What was the man with the shoulder bag doing while this was going on?
5. Who exactly was tricked here?

In the third example (the creepy clown),

1. Why is he dressed as a clown?
2. Why a creepy clown in particular?
3. Why does he wave his arms about?
4. What does it look like if you watch it without the sound?

One obvious issue is selection effects: we don't necessarily see the examples where it doesn't work well.

After having read this book, I assign a rather high prior to "he employs some extremely complicated technique, possibly involving some long honed skill ordinary people would never bother with, which no layperson could realistically hope to guess." One of the takeaways I got from that book is that while some magic relies on simple, elegant tricks which seem obvious in hindsight, a lot of the killer tricks are really, really complicated. He may not even be employing the same technique in every permutation of the trick; magicians routinely learn a wide array of methods for accomplishing individual "effects."

I studied a lot about magic routines and can tell you for many big magic tricks on youtube at least one possible way how it would work. But the most important thing about magic is, it is entertainment. And also Derren Brown is an entertainer, that's what he does. I'm a big fan of Derren, also because he exposes much of the supposedly psychic or prayer stuff.

I clearly agree with garethrees' point about misdirection and psychology. Providing a solution that seems plausible to system 1 (sensu Kahneman), makes you stop thinking about it. I think that one of Derren's best tricks is to make the audience feel happy with a fake explanation. In general, I don't like exposing magic tricks. But in some cases when false beliefs spread by magicians clearly threaten to poison people's belief systems, I make sort of an exception:

• The old magician's rule "80% show, 20% magic skills" is still true and also applies to Derren
• Derren is an excellent sleight-of-hand magician. Many tricks that he presents as mental tricks, are card tricks (for example http://www.youtube.com/watch?v=AewhMHhCmNQ)
• Forget suggestion and NLP in the way Derren uses them as explanations. All his tricks are clearly possible without them.
• Often, solutions of "impossible" tricks are very disappointing in the sense that one realises that the actual trick was to make one believe that the magician would not use "cheap" tricks (for example see http://www.youtube.com/watch?v=EUiToBs_YT0), or that the magician would not be able to see through a blindfold, or…
• On TV, you can easily select for what you want (nicely demonstrated by Derren's "The System", http://www.youtube.com/watch?v=9R5OWh7luL4)

I think it's easy to overestimate his skills; I remember a number of occasions where, whilst watching, I assumed psychological brilliance was the only possibility, yet later thought of a much simpler hypothesis. I'll list one example here.

In one trick, which he did on his stage show Enigma, he would have a lot of audience members write a celebritiy's name on a piece of paper and fill a basket; a randomly chosen audience member selected a piece of paper and Derren, from holding the man's hand, painted a picture of the celebrity's face. He makes a good deal of showmanship from the contact with the man's arm, and appeared to be having extreme difficulty - the face looked nothing like a face! Finally he turned it upside down and there was Elton John. For a long time I believed he was a psychological genius, before I realised that somehow forcing the celebrity's name, even in front of the audience looking at the empty bucket being filled and then handed to the audience member, was a far simpler hypothesis.

It seems strange to me that a topic that generated so much discussion and speculation was voted to approximately zero. Perhaps the number of comments on a post is a better indicator of its interest than upvotes - downvotes.

Did I hear correctly in the second video? They both has a goldfish named "George"? No one else finds that worth remarking on? Surely P(both have goldfish named George) << P(She will guess his pet's name|their pets have the same name). So why are we focusing on the latter?

Anyone who came up with a Hypothesis is welcome to explain how he can do it blindfolded to two people sitting more than ten meters away here: http://www.youtube.com/watch?v=ZlAO4U7PG1o

I think that a lot of Derren Brown's stuff, while not exactly fake, is done by hypnotizing them off-camera.

So he either hypnotizes them and finds out some information then wipes the memory of them telling him the information, or he hypnotizes them and gives a post-hypnotic suggestion that they will make a specific choice (but not remember being hypnotized to make the choice).

Edit: This post received 4 downvotes, can someone explain? Is it because of a general skepticism about hypnosis? I gave a source on post-hypnotic amnesia below but I'm still receiving downvotes. Even if you think the supposed effects achieved by stage magicians are illegitimate in some way (e.g. people are just playing along), hypnosis as an explanation for how Derren Brown does these tricks is still valid.