It depends on the nature of the analysis I'm doing.

I mean, sure, if the scam artist is smart enough to, for example, completely encapsulate my sensorium and provide me with an entirely simulated world that it updates in real time and perfect detail, then all bets are off... it can make me believe anything by manipulating the evidence I observe. (Similarly, if the scam artist is smart enough to directly manipulate my brain/mind.)

But if my reasoning is reliable and I actually have access to evidence about the real world, then the better I am at evaluating that evidence, the harder I am to scam about things relating to that evidence, even by a scam artist far smarter than me.

The concern here is that if I reject all offers that superficially pattern-match to this sort of scam, I run the risk of turning down valuable offers as well. (I'm reminded of a TV show decades ago where they had some guy dress like a bum and wander down the street offering people $20, and everyone ignored him.)

Of course, if I'm not smart enough to actually evaluate the situation, or don't feel like spending the energy, then superficial pattern-matching and rejection is my safest strategy, as you suggest.

But the question of what analysis a sufficiently smart and attentive agent could do, in principle, to take advantage of rare valuable opportunities without being suckered by scam artists is often worth asking anyway.

Isn't this more of social recognition of a scam?

While there are decision-theoretic issues with the Original Pascal's Wager, one of the main problems is that it is a scam ("You can't afford not to do it! It's an offer you can't refuse!"). It seems to me that you can construct plenty of arguments like you just did, and many people wouldn't take you up on the offer because they'd recognize it as a scam. Once something has a high chance of being a scam (like taking the form of Pascal's Wager), it won't get much more of your attention until you lower the likelihood that it's a scam. Is that a weird form of Confirmation Bias?

But nonetheless, couldn't the AI just function in the same way as that? I would think it would need to learn how to identify what is a trick and what isn't a trick. I would just try to think of it as a Bad Guy AI who is trying to manipulate the decision making algorithms of the Good Guy AI.

Do you consider yourself an example of this? If not (and even if so), what are some other examples?

Note: it might have something to do with the apparently innate human tendency to want to "save face." One way to save face in socially painful situations is to try to appear as though you are doing it all on purpose, to present yourself, for example, as a 'free spirit' rather than a 'loser.' I know for sure that I did this a lot as a child, and even now I have a tendency to emphasize the 'weird' things that I do, to look like I do them confidently and deliberately.

If mathematicians did have to constantly disagree with other people about maths it would be far better to ask an intelligent amateur about maths than a mathemtician.

The best reason I could come up with why someone would think ethicists need to disagree with each other to keep their jobs, is that they need to "publish or perish". But that applies equally well to other academic fields, like mathematics. If it's not true of mathematicians, then I'm left with no reason to think it's true of ethicists.

My guess is you wanted another example?

Ethics is a whole different thing than putative human universals. Very few things that I would assert as ethics would I claim to be human universals. "Normative human essentials" might fit in that context. (By way of illustration, we all likely consider 'Rape Bad' as an essential ethical value but I certainly wouldn't say that's a universal human thing. Just that the ethics of those who don't think Rape Is Bad suck!)

Were those tons of composers like Greenberg doing that sort of work at age 14?

Greenberg on the lack of anything really new in classical music. I think this is publicly available--let me know if it isn't.

Tentative hypothesis: people mostly get hooked by melody and rhythm, but classical has been exploring timbre (to the extent that it's exploring anything) for quite a while.

So in order for Weird Al to be strictly superior to classical music then it must be superior for all times and purposes.

 '

I do, after all, openly rate Weird Al as aesthetically superior to the greatest classical masterpieces.

The second of the above quotes is something I have claimed. The first one is a response to something I have not claimed. There is a straw man at play.

Well if I'm going to contradict you about yourself, I might as well just say it.

On something so straightforward as this doing so makes you look ridiculous and tends to be considered rather rude. Far better to not say it.

I would be trying to claim that you are more sophisticated and intelligent than you yourself will admit.

Valuing Weird Al over classical does not make me less intelligent (albeit certainly less sophisticated). It speaks more about my general aesthetic preference for melding a conceptual meaning in closely with the melody, rhythm, tone, etc. For me the concepts themselves seem to be a part of music to a far greater extent than for most people I have compared myself to. I do not consider this to be a weakness of mine.

So in order for Weird Al to be strictly superior to classical music then it must be superior for all times and purposes.

 '

I do, after all, openly rate Weird Al as aesthetically superior to the greatest classical masterpieces.

The second of the above quotes is something I have claimed. The first one is a response to something I have not claimed. There is a straw man at play.

Well if I'm going to contradict you about yourself, I might as well just say it.

On something so straightforward as this doing so makes you look ridiculous and tends to be considered rather rude. Far better to not say it.

I would be trying to claim that you are more sophisticated and intelligent than you yourself will admit.

Valuing Weird Al over classical does not make me less intelligent (albeit certainly less sophisticated). It speaks more about my general aesthetic preference for melding a conceptual meaning in closely with the melody, rhythm, tone, etc. For me the concepts themselves seem to be a part of music to a far greater extent than for most people I have compared myself to. I do not consider this to be a weakness of mine.