If you start a PR campain about AI risk that results into bringing a lot of luddites into the AGI debate, it might be harder for MIRI to convince AI researchers to treat UFAI as a serious risk not easier because the average AI person might think how the luddites oppose AGI for all the wrong reasons. He's not a luddite so why should he worry about UFAI?

Fair enough. I still believe there could be benefits to gaining wider support but I agree that this is an area that will be mainly determined by the actions of elite specialized thinkers and the very powerful.

I also something that contradicts the goal you layed out above. You said you wanted to spread the meme: "Belief without evidence is bad." If you start pushing memes because you like the effect and not because they are supported by good evidence you don't get "Belief without evidence is bad."

I'm not sure I see a contradiction there. I can see that if I say things that aren't true and people believe them just because I said them, that would be believing without evidence. But "belief without evidence is bad" doesn't have to be true 100% of the time in order for it to be a good, safe meme to spread. If your argument is that the spreading of "Utility > Truth" interferes with "Belief without evidence is bad" so that the two will largely cancel out, then (1) I didn't include "Utility > Truth" on my incomplete list of safe memes precisely because I don't think it's safe and (2) the argument would only be persuasive if the two memes usually interfered with each other, which I don't think is the case. In most situations, people knowing the truth is a really desirable thing. Journalism and marketing are exceptions where it could make sense to oversimplify a message in order for laypeople to understand it, hence making the meme less accurate but more effective at getting people interested (in which case, they'll hopefully continue researching until they have a more accurate understanding). Also, (3) even if two memes contradict each other, using both in tandem could theoretically yield more utilons than using either one alone (or neither), though I'd expect examples to be rare.

By the way, I emailed Adbusters about if/how they measure the effectiveness of their culture jamming campaigns. I'll let you know when I get a response.