= 27d567bc2d48d9f40e9ccc20ea370b15)
TheOtherDave comments on Pascal's Muggle: Infinitesimal Priors and Strong Evidence - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (404)
Isn't this more of social recognition of a scam?
While there are decision-theoretic issues with the Original Pascal's Wager, one of the main problems is that it is a scam ("You can't afford not to do it! It's an offer you can't refuse!"). It seems to me that you can construct plenty of arguments like you just did, and many people wouldn't take you up on the offer because they'd recognize it as a scam. Once something has a high chance of being a scam (like taking the form of Pascal's Wager), it won't get much more of your attention until you lower the likelihood that it's a scam. Is that a weird form of Confirmation Bias?
But nonetheless, couldn't the AI just function in the same way as that? I would think it would need to learn how to identify what is a trick and what isn't a trick. I would just try to think of it as a Bad Guy AI who is trying to manipulate the decision making algorithms of the Good Guy AI.
The concern here is that if I reject all offers that superficially pattern-match to this sort of scam, I run the risk of turning down valuable offers as well. (I'm reminded of a TV show decades ago where they had some guy dress like a bum and wander down the street offering people $20, and everyone ignored him.)
Of course, if I'm not smart enough to actually evaluate the situation, or don't feel like spending the energy, then superficial pattern-matching and rejection is my safest strategy, as you suggest.
But the question of what analysis a sufficiently smart and attentive agent could do, in principle, to take advantage of rare valuable opportunities without being suckered by scam artists is often worth asking anyway.
But wouldn't you just be suckered by sufficiently smart and attentive scam artists?
It depends on the nature of the analysis I'm doing.
I mean, sure, if the scam artist is smart enough to, for example, completely encapsulate my sensorium and provide me with an entirely simulated world that it updates in real time and perfect detail, then all bets are off... it can make me believe anything by manipulating the evidence I observe. (Similarly, if the scam artist is smart enough to directly manipulate my brain/mind.)
But if my reasoning is reliable and I actually have access to evidence about the real world, then the better I am at evaluating that evidence, the harder I am to scam about things relating to that evidence, even by a scam artist far smarter than me.
I disagree. All the scam artist has to know is your method of coming to your conclusions. Once he knows that then he can probably exploit you depending on his cleverness (and then it becomes an arms race). If anything, trying to defend yourself from being manipulated in that way would probably be extremely difficult in of itself. Either way, my initial guess is that your methodology would still be superficial pattern-matching, but it would just be a deeper, more complex level of it.
This seems to be what Eliezer is doing with all the various scenarios. He's testing his methodology against different attacks and different scenarios. I'm just suggesting is to change your viewpoint to the Bad Guy. Rather than talk about your reliable reasoning, talk about the bad guy and how he can exploit your reasoning.
Fair enough. If I accept that guess as true, I agree with your conclusion.
I also agree that adopting the enemy's perspective is an important -- for humans, indispensible -- part of strategic thinking.