I realize this question is contrived, but I figure it might provoke some fun discussion, so here goes:

If you could push a button and have your brain modified to precisely remove a cognitive bias (and have no other unnecessary effects—most convenient possible world), which would you choose? Why?

A putative new idea for AI control; index here.

A lot of the new ideas I've been posting could be parodied as going something like this:

The AI A, which is utility indifferent to the existence of AI B, has utility u (later corriged to v', twice), and it will create a subagent C which believe via false thermodynamic miracles that D does not exist, while D' will hypothetically and counterfactually use two different definitions of counterfactual so that the information content of its own utility cannot be traded with a resource gathering agent E that doesn't exist (assumed separate from its unknown utility function)...

What is happening is that I'm attempting to define algorithms that accomplish a particular goal (such as obeying the spirit of a restriction, or creating a satisficer). Typically this algorithm has various underdefined components - such as inserting an intelligent agent at a particular point, controlling the motivation of an agent at a point, effectively defining a physical event, or having an agent believe (or act as if they believed) something that was incorrect.

The aim is to reduce the problem from stuff like "define human happiness" to stuff like "define counterfactuals" or "pinpoint an AI's motivation". These problems should be fundamentally easier - if not for general agents, then for some of the ones we can define ourselves (this may also allow us to prioritise research directions).

And I also have no doubt that once a design is available, that it will be improved upon and transformed and made easier to implement and generalise. Therefore I'm currently more interested in objections of the form "it won't work" than "it can't be done".

I came across this article this morning via a blog post from

http://solutionfocusedchange.blogspot.com/.

http://sd1.myipcn.org/science/article/pii/S0749597811001099

"Wolves in sheep’s clothing: How and when hypothetical questions influence behavior" by Sarah G. Moore and others. Full article unfortunately unavailable for free.

"We examine how and when hypothetical questions influence judgment and behavior.

Hypotheticals increase the accessibility of the positive or negative information in the question.

Thus, hypotheticals influence behavior according to the valence of the question.

Hypotheticals exert a stronger influence when they are consistent with existing knowledge.

Hypotheticals exert a weaker influence when individuals are aware of their impact."

I think this is a deliberate and obvious application of psychological priming, where we are biased to interpret events, through exposure to positive or negative tone words.

Hypotheticals frame the context of the discussion, and require to you use the hard path of cognition to think in a different way. They are a source of error in social science surveys, and are often used by marketers and political pollsters to lead our response.

I'd like to read the full paper to find out what sort of experimental method they used.