If you can identify people who are rational, domain experts, honest, and incentivized to seek the truth, then I think you should trust them over the broader 'expert' community.

If such people don't agree with one another, then they should make clear why they disagree. If they fail to do this, then either they are not very good at practical rationality, or they aren't trying to make themselves clear to external observers.

If they've made it clear why they disagree, and it's a matter of domain expertise you're not qualified to judge yourself, then you're back to the outside view and head-counting. But I think this is a rare case.