The point is that 'given (output X)=a' may eventually let you prove a contradiction when the output is not, in fact, a, and you have added a false statement as input.

In practice, one does not use a two-way axiom of (output X)=a but an one-way substitution rule 'replace (output X) with a' . The rule may be applied once at start, or through first N steps of deduction process (to catch the cases where deduction manages to deduce X from a slightly modified X that was included inside the 'other algorithm' ; note that one can't do it forever because at some point the proof checker arrives at X from first principles, and the contradiction can be introduced by substitution.

The issue he described is specific to using (outputs X)=a as given, which allows you to e.g. do some algebra, arrive at a number a for any reason, and then replace a with (output X) , which lets you contradict yourself, or make a self fulfilling prophesy. The intent of making it a given, is to make the substitutions one way, but the theorem prover can do substitutions other way around.

Your self-fulfilling prophecy example works for the iteration of the for loop (described in "For each xi, assume the output of X is xi, and try to deduce the expected value of U.") in which the output is assumed to be a, but for the iteration in which the output is assumed to be b, proving that the output is a would be to prove a contradiction. "if (output X)=b then U=0" is one possible outcome, but U could also equal anything else.

I don't see how the NDT algorithm as given allows "(output X)=a" to be proved outside of the for loop at all. I would think it would take (output X)=whatever for each iteration through the for loop as a given before trying to prove anything, in which case in the run of the for loop in which (output X)=b is the given, proving (output X)=a is a clear contradiction, one which I would think our prover could avoid unless our axiomatic system is contradictory in the first place.

Or to rephrase, I don't think "For each xi, assume the output of X is xi, and try to deduce the expected value of U." and "(That is, try and deduce statements of the form "if (output X)=xi then U=ui" for some ui)." are actually equivalent at all, and I think the self-fulfilling prophecy example follows the second and ignores the first.

I am JUST leaving this morning, and I have been staying at the house just a few houses down from you. I will never catch a break!

I think that ''evolved faulty thinking processes'' is the wrong way to look at it and I will argue that some biases are the consequence of structural properties of the brain, which 'cannot' be affected by evolution.

The structure can be affected by evolution, it's just too hard (takes too many coordinated mutations) to get to a structure that actually works better. I think you recognize this by your use of scare quotes, but you would be better off stating it explicitly. This is the flip side of the arguments I think you're alluding to, that the faulty thinking was actually beneficial in the EEA.

There must be an evolutionary explanation for the properties of the brain, but that doesn't mean we need to actually figure out that evolutionary explanation to understand the current behavior. Just like there must be an explanation in terms of physics, but trying to analyze every particle will clearly get us nowhere.

In fact, if you can find an explanation of a phenomenon in terms of current brain structure, I think that screens off evolutionary explanations as mere history (as long as you've really verified that the structure exists and explains the phenomenon).

I do think we're getting sidetracked by your halo effect example, though -- it might be useful to give three or four examples to avoid this (although if each one has a different explanation, that might substantially increase the effort of presenting your idea).

There's a certain issue.

A revision control system I use (GIT) uses 128 bit hashes to identify a much longer piece of program code. There are very many pieces of program code that correspond to same hash.

I don't expect it to ever encounter collision, though. Just because something maps large space to a smaller space, doesn't mean any collisions will actually happen. Even a couple hundred bits is enough to define space so vast, that you can map anything you encounter in your life to it, and never see a collision. Not that our brains necessarily work like this. But they can in principle avoid collisions.

For the halo effect you're speaking of, it is the case that positive qualities weakly correlate - at least the good looks and intelligence do, then the intelligence generally correlates with niceness in so much as intelligence prevents grossly un-nice behaviour that hurts everyone including that person. Still could usually be a fallacy, of course, some sort of signal leakage between 'good looks' and 'good somethingelse'.

I'd say, people just tend to assume by default there's some correlation between two things - either they assume it is positive (so pretty, must be nice), or they assume it is negative (so pretty, must be evil or spoiled or the like), and just a few people assume it is zero by default.

Specifically, it looks like there's a %22 at the end that shouldn't be there. (Here's the corrected version.)