Is the premise that modern sites do not take studies on aesthetics/usability/effectiveness into account even true? I've moved into web development over the past 8 months or so and I regularly search for topics such as "log in vs sign in," "ok cancel button placement" and "optimal web page navigation."

It seems to me that there is no shortage of studies, opinions and hard evidence on display regarding the (in)effectiveness of particular web design choices. Granted not every google hit is going to cite a formal study, but a surprising amount do. Googling the above over the past ten minutes or so has given me references to a study on optimal text layout, a study on label placement/alignment, why dropdown menus apparently suck, and how presenting users with too many choices is detrimental to user engagment (admittedly this one was generalized to the web after the fact).

As someone who worked in the area of Intelligence training I am very, very skeptical. For example, there was a burst of optimism about training working memory through Dual N-Back tasks, bought about by a revolutionary paper from Jaeggi et al. Then... not quite nothing but close.

I suspect there's a reason that no braining activity, ever, has been consistently shown to improve intelligence at the construct level. It may be that more specific capacities related to intelligence (like working memory) are improved, and that these affect life outcomes and practical capacity to grasp concepts etc, but this has yet to demonstrated to my satisfaction.

I'm not convinced that improving calibration will not improve accuracy because predictions are often nested within other predictions. For example, suppose we are trying to make a prediction about P, and the truth or falsity of Q, R and S are relevant to the truth of P in some respect. We might use as a basis for guessing P that we are ninety five percent confident in our guesses about Q, R & S, (suppose the truth of all three would guarantee P). Now suppose we become less confident through better calibration and decide there is only a 70% chance that Q, a 70% chance that R and a 70% chance that S, leading to a compound probability of less <50%. Thus overall accuracy can be improved by calibration.