I'm not sure I understand the objection/question, but I'll respond to the objection/question I think it is.

Am I changing the procedure to avoid a counterexample from Wei Dai?

I think the answer is No. If you look at the section titled "An outline of the framework and some guidelines for applying it effectively" you'll see that I say you should try to use a prior that corresponds to an impartial combination of what the people who are most trustworthy in general think. I say a practical approximation of being an "expert" is being someone elite common sense would defer to. If the experts won't tell elite common sense what they think, then what the experts think isn't yet part of elite common sense. I think this is a case where elite common sense just gets it wrong, not that they clearly could have done anything about it. But I do think it's a case where you can apply elite common sense, even if it gives you the wrong answer ex post. (Maybe it doesn't give you the wrong answer though; maybe some better investigation would have been possible and they didn't do it. This is hard to say from our perspective.)

Why go with what generally trustworthy people think as your definition of elite common sense? It's precisely because I think it is easier to get in touch with what generally trustworthy people think, rather than what all subject matter experts in the world think. As I say in the essay:

How should we assign weight to different groups of people? Other things being equal, a larger number of people is better, more trustworthy people are better, people who are trustworthy by clearer indicators that more people would accept are better, and a set of criteria which allows you to have some grip on what the people in question think is better, but you have to make trade-offs....If I went with, say, the 10 most-cited people in 10 of the most intellectually credible academic disciplines, 100 of the most generally respected people in business, and the 100 heads of different states, I would have a pretty large number of people and a broad set of people who were very trustworthy by clear standards that many people would accept, but I would have a hard time knowing what they would think about various issues because I haven’t interacted with them enough. How these factors can be traded-off against each other in a way that is practically most helpful probably varies substantially from person to person.

In principle, if you could get a sense for what all subject matter experts thought about every issue, that would be a great place to start for your prior. But I think that's not possible in practice. So I recommend using a more general group that you can use as your starting point.

Does this answer your question?