Summary:

• Chronic Fatigue and Fibromyalgia look very like Hypothyroidism.
• Thyroid Patients aren't happy with either the diagnosis or treatment of Hypothyroidism.
• It is possible that lots of FMS/CFS cases are 'something wrong with the thyroid system that doesn't show up on laboratory hormone level tests'.
• It's possible that it's not too difficult to fix these CFS/FMS cases with thyroid hormones.
• I believe that there may have been a stupendous cock-up that's hurt millions.
• Less Wrong should be interested, because it could be a real example of how bad inference can cause the sciences to come to false conclusions, as well as a good practice problem for the things we really care about.

Edit:

I found a possible answer here:
http://lesswrong.com/lw/nbm/thyroid_hormones_chronic_fatigue_and_fibromyalgia/
I do not believe it, because I do not understand it, but contemplation of it seems to be enlightening. In particular, the problem is much broader than I originally thought.

A summary of the argument in the first two posts, together with links to lots of evidence in the literature:

http://lesswrong.com/r/discussion/lw/nef/the_thyroid_madness_core_argument_evidence/

And this is pretty much proof, I think:

http://lesswrong.com/lw/nhs/the_thyroid_madness_two_apparently_contradictory/

At this point, I think I'm as confident as I can be without some sort of formal trial (so 25% maybe?)

But certainly, if you're suffering from Chronic Fatigue Syndrome/Fibromyalgia/Major Depression/Irritable Bowel Syndrome, or any of the many similar disorders which just seem to be different names for 'hypothyroidism with normal TSH', I reckon this is worth trying!

I have done, and it worked for me. For about four months now...

Original Post:

I believe that I've come across a genuine puzzle, and I wonder if you can help me solve it. This problem is complicated, and subtle, and has confounded and defeated good people for forty years. And yet there are huge and obvious clues. No-one seems to have conducted the simple experiments which the clues suggest, even though many clever people have thought hard about it, and the answer to the problem would be very valuable. And so I wonder what it is that I am missing.

I am going to tell a story which rather extravagantly privileges a hypothesis that I have concocted from many different sources, but a large part of it is from the work of the late Doctor John C Lowe, an American chiropractor who claimed that he could cure Fibromyalgia.

I myself am drowning in confirmation bias to the point where I doubt my own sanity. Every time I look for evidence to disconfirm my hypothesis, I find only new reasons to believe. But I am utterly unqualified to judge. Three months ago I didn't know what an amino acid was. And so I appeal to wiser heads for help.

Crocker's Rules on this. I suspect that I am being the most spectacular fool, but I can't see why, and I'd like to know.

Setting the Scene

Chronic Fatigue Syndrome, Myalgic Encephalitis, and Fibromyalgia are 'new diseases'. There is considerable dispute as to whether they even exist, and if so how to diagnose them. They all seem to have a large number of possible symptoms, and in any given case, these symptoms may or may not occur with varying severity.

As far as I can tell, if someone claims that they're 'Tired All The Time', then a competent doctor will first of all check that they're getting enough sleep and are not unduly stressed, then rule out all of the known diseases that cause fatigue (there are a very lot!), and finally diagnose one of the three 'by exclusion', which means that there doesn't appear to be anything wrong, except that you're ill.

If widespread pain is one of the symptoms, it's Fibromyalgia Syndrome (FMS). If there's no pain, then it's CFS or ME. These may or may not be the same thing, but Myalgic Encephalitis is preferred by patients because it's greek and so sounds like a disease. Unfortunately Myalgic Encephalitis means 'hurty muscles brain inflammation', and if one had hurty muscles, it would be Fibromyalgia, and if one had brain inflammation, it would be something else entirely.

Despite the widespread belief that these are 'somatoform' diseases (all in the mind), the severity of them ranges from relatively mild (tired all the time, can't think straight), to devastating (wheelchair bound, can't leave the house, can't open one eye because the pain is too great).

All three seem to have come spontaneously into existence in the 1970s, and yet searches for the responsible infective agent have proved fruitless. Neither have palliative measures been discovered, apart from the tried and true method of telling the sufferers that it's all in their heads.

The only treatments that have proved effective are Cognitive Behavioural Therapy / Graded Exercise. A Cochrane Review reckoned that they do around 15% over placebo in producing a measurable alleviation of symptoms. I'm not very impressed. CBT/GE sound a lot like 'sports coaching', and I'm pretty sure that if we thought of 'Not Being Very Good at Rowing' as a somatoform disorder, then I could produce an improvement over placebo in a measurable outcome in ten percent of my victims without too much trouble.

But any book on CFS will tell you that the disease was well known to the Victorians, under the name of neurasthenia. The hypothesis that God lifted the curse of neurasthenia from the people of the Earth as a reward for their courage during the wars of the early twentieth century, while well supported by the clinical evidence, has a low prior probability.

We face therefore something of a mystery, and in the traditional manner of my people, a mystery requires a Just-So Story:

How It Was In The Beginning

In the dark days of Victoria, the brilliant physician William Miller Ord noticed large numbers of mainly female patients suffering from late-onset cretinism.

These patients, exhausted, tired, stupid, sad, cold, fat and emotional, declined steeply, and invariably died.

As any man of decent curiosity would, Dr Ord cut their corpses apart, and in the midst of the carnage noticed that the thyroid, a small butterfly-shaped gland in the throat, was wasted and shrunken.

One imagines that he may have thought to himself: "What has killed them may cure them."

After a few false starts and a brilliant shot in the dark by the brave George Redmayne Murray, Dr Ord secured a supply of animal thyroid glands (cheaply available at any butcher, sautée with nutmeg and basil) and fed them to his remaining patients, who were presumably by this time too weak to resist.

They recovered miraculously, and completely.

I'm not sure why Dr Ord isn't better known, since this appears to have been the first time in recorded history that something a doctor did had a positive effect.

Dr Ord's syndrome was named Ord's Thyroiditis, and it is now known to be an autoimmune disease where the patient's own antibodies attack and destroy the thyroid gland. In Ord's thyroiditis, there is no goiter.

A similar disease, where the thyroid swells to form a disfiguring deformity of the neck (goiter), was described by Hakaru Hashimoto in 1912 (who rather charmingly published in German), and as part of the war reparations of 1946 it was decided to confuse the two diseases under the single name of Hashimoto's Thyroiditis. Apart from the goiter, both conditions share a characteristic set of symptoms, and were easily treated with animal thyroid gland, with no complications.

Many years before, in 1835, a fourth physician, Robert James Graves, had described a different syndrome, now known as Graves' Disease, which has as its characteristic symptoms irritability, muscle weakness, sleeping problems, a fast heartbeat, poor tolerance of heat, diarrhoea, and weight loss. Unfortunately Dr Graves could not think how to cure his eponymous horror, and so the disease is still named after him.

The Horror Spreads

Victorian medicine being what it was, we can assume that animal glands were sprayed over and into any wealthy person unwise enough to be remotely ill in the vicinity of a doctor. I seem to remember a number of jokes about "monkey glands" in PG Wodehouse, and indeed a man might be tempted to assume that chimpanzee parts would be a good substitute for humans. Supply issues seem to have limited monkey glands to a few millionaires worried about impotence, and it may be that the corresponding procedure inflicted on their wives has come down to us as Hormone Replacement Therapy.

Certainly anyone looking a bit cold, tired, fat, stupid, sad or emotional is going to have been eating thyroids. We can assume that in a certain number of cases, this was just the thing, and I think it may also be safe to assume that a fair number of people who had nothing wrong with them at all died as a result of treatment, although the fact that animal thyroid is still part of the human food chain suggests it can't be that dangerous.

I mean seriously, these people use high pressure hoses to recover the last scraps of meat from the floors of slaughterhouses, they're not going to carefully remove all the nasty gristly throat-bits before they make ready meals, are they?

The Armour Sausage company, owner of extensive meat-packing facilities in Chicago, Illinois, and thus in possession of a large number of pig thyroids which, if not quite surplus to requirements, at the very least faced a market sluggish to non-existent as foodstuffs, brilliantly decided to sell them in freeze-dried form as a cure for whatever ails you.

Some Sort of Sanity Emerges, in a Decade not Noted for its Sanity

Around the time of the second world war, doctors became interested in whether their treatments actually helped, and an effort was made to determine what was going on with thyroids and the constellation of sadness that I will henceforth call 'hypometabolism', which is the set of symptoms associated with Ord's thyroiditis. Jumping the gun a little, I shall also define 'hypermetabolism' as the set of symptoms associated with Graves' disease.

The thyroid gland appeared to be some sort of metabolic regulator, in some ways analogous to a thermostat. In hypometabolism, every system of the body is running slow, and so it produces a vast range of bad effects, affecting almost every organ. Different sufferers can have very different symptoms, and so diagnosis is very difficult.

Dr Broda Barnes decided that the key symptom of hypometabolism was a low core body temperature. By careful experiment he established that in patients with no symptoms of hypometabolism the average temperature of the armpit on waking was 98 degrees Fahrenheit (or 36.6 Celsius). He believed that temperature variation of +/- 0.2 degrees Fahrenheit was unusual enough to merit diagnosis. He also seems to have believed, in the manner of the proverbial man with a hammer, that all human ailments without exception were caused by hypometabolism, and to have given freeze-dried thyroid to almost everyone he came into contact with, to see if it helped. A true scientist. Doctor Barnes became convinced that fully 40% of the population of America suffered from hypometabolism, and recommended Armour's Freeze Dried Pig Thyroid to cure America's ills.

In a brilliant stroke, Freeze Dried Pig's Thyroid was renamed 'Natural Desiccated Thyroid', which almost sounds like the sort of thing you might take in sound mind. I love marketing. It's so clever.

America being infested with religious lunatics, and Chicago being infested with nasty useless gristly bits of cow's throat, led almost inevitably to a second form of 'Natural Desiccated Thyroid' on the market.

Dr Barnes' hypometabolism test never seems to have caught on. There are several ways your temperature can go outside his 'normal' range, including fever (too hot), starvation (too cold), alcohol (too hot), sleeping under too many duvets (too hot), sleeping under too few duvets (too cold). Also mercury thermometers are a complete pain in the neck, and take ten minutes to get a sensible reading, which is a long time to lie around in bed carefully doing nothing so that you don't inadvertently raise your body temperature. To make the situation even worse, while men's temperature is reasonably constant, the body temperature of healthy young women goes up and down like the Assyrian Empire.

Several other tests were proposed. One of the most interesting is the speed of the Achilles Tendon Reflex, which is apparently super-fast in hypermetabolism, and either weirdly slow or has a freaky pause in it if you're running a bit cold. Drawbacks of this test include 'It's completely subjective, give me something with numbers in it', and 'I don't seem to have one, where am I supposed to tap the hammer-thing again?'.

By this time, neurasthenia was no longer a thing. In the same way that spiritualism was no longer a thing, and the British Empire was no longer a thing.

As far as we know, Chronic Fatigue Syndrome was not a thing either, and neither was Fibromyalgia (which is just Chronic Fatigue Syndrome but it hurts), nor Myalgic Encephalitis. There was something called 'Myalgic Neurasthenia' in 1934, but it seems to have been a painful infectious disease and they thought it was polio.

Finally, Science

It turned out that the purpose of the thyroid gland is to make hormones which control the metabolism. It takes in the amino acid tyrosine, and it takes in iodine. It releases Thyroglobulin, mono-iodo-tyrosine (MIT), di-iodo-tyrosine (DIT), thyroxine (T4) and triiodothyronine (T3) into the blood. The chemistry is interesting but too complicated to explain in a just-so story.

I believe that we currently think that thyroglobulin, MIT and DIT are simply by-products of the process that makes T3 and T4.

T3 is the hormone. It seems to control the rate of metabolism in all cells. T4 has something of the same effect, but is much less active, and called a 'prohormone'. Its main purpose seems to be to be deiodinated to make more T3. This happens outside the thyroid gland, in the other parts of the body ('peripheral conversion'). I believe mainly in the liver, but to some extent in all cells.

Our forefathers knew about thyroxine (T4, or thyronine-with-four-iodines-attached), and triiodothyronine (T3, or thyronine-with-three-iodines-attached)

It seems to me that just from the names, thyroxine was the first one to be discovered. But I'm not sure about that. You try finding a history-of-endocrinology website. At any rate they seem to have known about T4 and T3 fairly early on.

The mystery of Graves', Ord's and Hashimoto's thyroid diseases was explained.

Ord's and Hashimoto's are diseases where the thryoid gland under-produces (hypothyroidism). The metabolism of all cells slows down. As might be expected, this causes a huge number of effects, which seem to manifest differently in different sufferers.

Graves' disease is caused by the thyroid gland over-producing (hyperthyroidism). The metabolism of all cells speeds up. Again, there are a lot of possible symptoms.

All three are thought to be autoimmune diseases. Some people think that they may be different manifestations of the same disease. They are all fairly common.

Desiccated thryoid cures hypothyroidism because the ground-up thyroids contain T4 and T3, as well as lots of thyroglobulin, MIT and DIT, and they are absorbed by the stomach. They get into the blood and speed up the metabolism of all cells. By titrating the dose carefully you can restore roughly the correct levels of the thyroid hormones in all tissues, and the patient gets better. (Titration is where you change something carefully until you get it right)

The theory has considerable explanatory power. It explains cretinism, which is caused either by a genetic disease, or by iodine deficiency in childhood. If you grow up in an iodine deficient area, then your growth is stunted, your brain doesn't develop properly, and your thyroid gland may become hugely enlarged. Presumably because the brain is desperately trying to get it to produce more thyroid hormones, and it responds by swelling.

Once upon a time, this swelling (goitre) was called 'Derbyshire Neck'. I grew up near Derbyshire, and I remember an old rhyme: "Derbyshire born, Derbyshire bred, strong in the arm, and weak in the head". I always thought it was just an insult. Maybe not. Cretinism was also popular in the Alps, and there is a story of an English traveller in Switzerland of whom it was remarked that he would have been quite handsome if only he had had a goitre. So it must have been very common there.

But at this point I am *extremely suspicious*. The thyroid/metabolic regulation system is ancient (universal in vertebrates, I believe), crucial to life, and it really shouldn't just go wrong. We should suspect either an infectious cause, or a recent environmental influence which we haven't had time to adjust to, an evolved defence against an infectious disease, or just possibly, a recently evolved but as yet imperfect defence against a less recent environmental change.

(Cretinism in particular is very strange. Presumably animals in iodine-deficient areas aren't cretinous, and yet they should be. Perhaps a change to a farming from a hunter-gatherer lifestyle has increased our dependency on iodine from crops, which crops have sucked what little iodine occurs naturally out of the soil?)

It's also not entirely clear to me what the thyroid system is *for*. If there's just a particular rate that cells are supposed to run at, then why do they need a control signal to tell them that? I could believe that it was a literal thermostat, designed to keep the body temperature constant at the best speed for the various biological reactions, but it's universal in *vertebrates*. There are plenty of vertebrates which don't keep a constant temperature.

The Fall of Desiccated Thyroid

There turned out to be some problems with Natural Desiccated Thyroid (NDT).

Firstly, there were many competing brands and types, and even if you stuck to one brand the quality control wasn't great, so the dose you'd be taking would have been a bit variable.

Secondly, it's fucking pig's thyroid from an abattoir. It could have all sorts of nasty things in it. Also, ick.

Thirdly, it turned out that pigs made quite a lot more T3 in their thyroids than humans do. It also seems that T3 is better absorbed by the gut than T4 is, so someone taking NDT to compensate for their own underproduction will have too much of the active hormone compared to the prohormone. That may not be good news.

With the discovery of 'peripheral conversion', and the possibility of cheap clean synthesis, it was decided that modern scientific thyroid treatment would henceforth be by synthetic T4 (thyroxine) alone. The body would make its own T3 from the T4 supply.

Alarm bells should be ringing at this point. Apart from the above points, I'm not aware of any great reason for the switch from NDT to thyroxine in the treatment of hypothyroidism, but it seems to have been pretty much universal, and it seems to have worked.

Aware of the lack of T3, doctors compensated by giving people more T4 than was in their pig-thyroid doses. And there don't seem to have been any complaints.

Over the years, NDT seems to have become a crazy fringe treatment despite there not being any evidence against it. It's still a legal prescription drug, but in America it's only prescribed by eccentrics. In England a doctor prescribing it would be, at the very least, summoned to explain himself before the GMC.

However, since it was (a) sold over the counter for so many years, and (b) part of the food chain, it is still perfectly legal to sell as a food supplement in both countries, as long as you don't make any medical claims for it. And the internet being what it is, the prescription-only synthetic hormones T3 and T4 are easily obtained without a prescription. These are extremely powerful hormones which have an effect on metabolism. If 'body-builders' and sports cheats aren't consuming all three in vast quantities, I am a Dutchman.

The Clinical Diagnosis of Hypothyroidism

We pass now to the beginning of the 1970s.

Hypothyroidism is ferociously difficult to diagnose. People complain of 'Tired All The Time' well, ... all the time, and it has literally hundreds of causes.

And it must be diagnosed correctly! If you miss a case of hypothyroidism, your patient is likely to collapse and possibly die at some point in the medium-term future. If you diagnose hypothyroidism where it isn't, you'll start giving the poor bugger powerful hormones which he doesn't need and *cause* hypermetabolism.

The last word in 'diagnosis by symptoms' was the absolutely excellent paper:

Statistical Methods Applied To The Diagnosis Of Hypothyroidism

by W. Z. Billewicz, R. S. Chapman, J. Crooks, M. E. Day, J. Gossage, Sir Edward Wayne, and J. A. Young

Connoisseurs will note the clever and careful application of 'machine learning' techniques, before there were machines to learn!

One important thing to note is that this is a way of separating hypothyroid cases from other cases of tiredness at the point where people have been referred by their GP to a specialist at a hospital on suspicion of hypothyroidism. That changes the statistics remarkably. This is *not* a way of diagnosing hypothyroidism in the general population. But if someone's been to their GP (general practitioner, the doctor that a British person likely makes first contact with) and their GP has suspected their thryoid function might be inadequate, this test should probably still work.

For instance, they consider Physical Tiredness, Mental Lethargy, Slow Cerebration, Dry Hair, and Muscle Pain, the classic symptoms of hypothyroidism, present in most cases, to be indications *against* the disease.

That's because if you didn't have these things, you likely wouldn't have got that far. So in the population they're seeing (of people whose doctor suspects they might be hypothyroid), they're not of great value either way, but their presence is likely the reason why the person's GP has referred them even though they've really got iron-deficiency anaemia or one of the other causes of fatigue.

In their population, the strongest indicators are 'Ankle Jerk' and 'Slow Movements', subtle hypothyroid symptoms which aren't likely to be present in people who are fatigued for other reasons.

But this absolutely isn't a test you should use for population screening! In the general population, the classic symptoms are strong indicators of hypothyroidism.

Probability Theory is weird, huh?

Luckily, there were lab tests for hypothyroidism too, but they were expensive, complicated, annoying and difficult to interpret. Billewicz et al used them to calibrate their test, and recommend them for the difficult cases where their test doesn't give a clear answer.

And of course, the final test is to give them thyroid treatment and see whether they get better. If you're not sure, go slow, watch very carefully and look for hyper symptoms.

Overconfidence is definitely the way to go. If you don't diagnose it and it is, that's catastrophe. If it isn't, but you diagnose it anyway, then as long as you're paying attention the hyper symptoms are easy enough to spot, and you can pull back with little harm done.

A Better Way

It should be obvious from the above that the diagnosis of hypothyroidism by symptoms is absolutely fraught with complexity, and very easy to get wrong, and if you get it wrong the bad way, it's a disaster. Doctors were absolutely screaming for a decisive way to test for hypothyroidism.

Unfortunately, testing directly for the levels of thyroid hormones is very difficult, and the tests of the 1960s weren't accurate enough to be used for diagnosis.

The answer came from an understanding of how the thyroid regulatory system works, and the development of an accurate blood test for a crucial signalling hormone.

Three structures control the level of thyroid hormones in the blood.

The thyroid gland produces the hormones and secretes them into the blood.

Its activity is controlled by the hormone thyrotropin, or Thyroid Signalling Hormone (TSH). Lots of TSH works the thyroid hard. In the absence of TSH the thyroid relaxes but doesn't switch off entirely. However the basal level of thyroid activity in the absence of TSH is far too low.

TSH is controlled by the pituitary gland, a tiny structure attached to the brain.

The pituitary itself is controlled, via Thyroid Releasing Hormone (TRH), by the hypothalamus, which is part of the brain.

This was thought to be a classic example of a feedback control system.

hypothalamus->pituitary->thyroid

It turns out that the level of thyrotropin TSH in the blood is exquisitely sensitive to the levels of thyroid hormones in the blood.

Administer thyroid hormone to a patient and their TSH level will rapidly adjust downwards by an easily detectable amount.

So:

In hypothyroidism, where the thyroid has failed, the body will be desperately trying to produce more thyroid hormones, and the TSH level will be extremely high.

In Graves' Disease, this theory says, where the thyroid has grown too large, and the metabolism is running damagingly fast, the body will be, like a central bank trying to stimulate growth in a deflationary economy by reducing interest rates, 'pushing on a piece of string'. TSH will be undetectable.

The original TSH test was developed in 1965, by the startlingly clever method of radio-immuno-assay.

[For reasons that aren't clear to me, rather than being expressed in grams/litre, or mols/litre, the TSH test is expressed in 'international units/liter'. But I don't think that that's important]

A small number of people in whom there was no suspicion of thyroid disease were assessed, and the 'normal range' of TSH was calculated.

Again, 'endocrinology history' resources are not easy to find, but the first test was not terribly sensitive, and I think originally hyperthyroidism was thought to result in a complete absence of TSH, and that the highest value considered normal was about 4 (milli-international-units/liter).

This apparently pretty much solved the problem of diagnosing thyroid disorders.

Forgetfulness

It's no longer necessary to diagnose hypo- and hyper-thyroidism by symptoms. It was error prone anyway, and the question is easily decided by a cheap and simple test.

Natural Desiccated Thyroid is one with Nineveh and Tyre.

No doctor trained since the 1980s knows much about hypothyroid symptoms.

Medical textbooks mention them only in passing, as an unweighted list of classic symptoms. You couldn't use that for diagnosis of this famously difficult disease.

If you suspect hypothyroidism, you order a TSH test. If the value of TSH is very low, that's hyperthyroidism. If the value is very high then that's hypothyroidism. Otherwise you're 'euthyroid' (greek again, good-thyroid), and your symptoms are caused by some other problem.

The treatment for hyperthyroidism is to damage the thyroid gland. There are various ways. This often results in hypothyroidism. *For reasons that are not terribly well understood*.

The treatment for hypothyroidism is to give the patient sufficient thyroxine (T4) to cause TSH levels to come back into their normal range.

The conditions hyperthyroidism and hypothyroidism are now *defined* by TSH levels.

Hypothyroidism, in particular, a fairly common disease, is considered to be such a solved problem that it's usually treated by the GP, without involving any kind of specialist.

Present Day

It was found that the traditional amount of thyroxine (T4) administered to cure hypothyroid patients, was in fact too high. The amount of T4 that had always been used to replace the hormones that had once been produced by a thyroid gland now dead, destroyed, or surgically removed appeared now to be too much. That amount causes suppression of TSH to below its normal range. The brain, theory says, is asking for the level to be reduced.

The amount of T4 administered in such cases (there are many) has been reduced by a factor of around two, to the level where it produces 'normal' TSH levels in the blood. Treatment is now titrated to produce the normal levels of TSH.

TSH tests have improved enormously since their introduction, and are on their third or fourth generation. The accuracy of measurement is very good indeed.

It's now possible to detect the tiny remaining levels of TSH in overtly hyperthyroid patients, so hyperthyroidism is also now defined by the TSH test.

In England, the normal range is 0.35 to 5.5. This is considered to be the definition of 'euthyroidism'. If your levels are normal, you're fine.

If you have hypothyroid symptoms but a normal TSH level, then your symptoms are caused by something else. Look for Anaemia, look for Lyme Disease. There are hundreds of other possible causes. Once you rule out all the other causes, then it's the mysterious CFS/FMS/ME, for which there is no cause and no treatment.

If your doctor is very good, very careful and very paranoid, he might order tests of the levels of T4 and T3 directly. But actually the direct T4 and T3 tests, although much more accurate than they were in the 1960s, are quite badly standardised, and there's considerable controversy about what they actually measure. Different assay techniques can produce quite different readings. They're expensive. It's fairly common, and on the face of it perfectly reasonable, for a lab to refuse to conduct the T3 and T4 tests if the TSH level is normal.

It's been discovered that quite small increases in TSH actually predict hypothyroidism. Minute changes in thyroid hormone levels, which don't produce symptoms, cause detectable changes in the TSH levels. Normal, but slightly high values of TSH, especially in combination with the presence of thyroid related antibodies (there are several types), indicate a slight risk of one day developing hypothyroidism.

There's quite a lot of controversy about what the normal range for TSH actually is. Many doctors consider that the optimal range is 1-2, and target that range when administering thyroxine. Many think that just getting the value in the normal range is good enough. None of this is properly understood, to understate the case rather dramatically.

There are new categories, 'sub-clinical hypothyroidism' and 'sub-clinical hyperthyroidism', which are defined by abnormal TSH tests in the absence of symptoms. There is considerable controversy over whether it is a good idea to treat these, in order to prevent subtle hormonal imbalances which may cause difficult-to-detect long term problems.

Everyone is a little concerned about accidentally over-treating people, (remember that hyperthyroidism is now defined by TSH<0.35).

Hyperthyroidism has long been associated with Atrial Fibrillation (a heart problem), and Osteoporosis, both very nasty things. A large population study in Denmark recently revealed that there is a greater incidence of Atrial Fibrillation in sub-clinical hyperthyroidism, and that hypothyroidism actually has a 'protective effect' against Atrial Fibrillation.

It's known that TSH has a circadian rhythm, higher in the early morning, lower at night. This makes the test rather noisy, as your TSH level can be doubled or halved depending on what time of day you have the blood drawn.

But the big problems of the 1960s and 1970s are completely solved. We are just tidying up the details.

Doubt

Many hypothyroid patients complain that they suffer from 'Tired All The Time', and have some of the classic hypothyroid symptoms, even though their TSH levels have been carefully adjusted to be in the normal range.

I've no idea how many, but opinions range from 'the great majority of patients are perfectly happy' to 'around half of hypothyroid sufferers have hypothyroid symptoms even though they're being treated'.

The internet is black with people complaining about it, and there are many books and alternative medicine practitioners trying to cure them, or possibly trying to extract as much money as possible from people in desperate need of relief from an unpleasant, debilitating and inexplicable malaise.

THE PLURAL OF ANECDOTE IS DATA.

Not good data, to be sure. But if ten people mention to you in passing that the sun is shining, you are a damned fool if you think you know nothing about the weather.

It's known that TSH ranges aren't 'normally distributed' (in the sense of Gauss/the bell curve distribution) in the healthy population.

If you log-transform them, they do look a bit more normal.

The American Academy of Clinical Biochemists, in 2003, decided to settle the question once and for all. They carefully screened out anyone with even the slightest sign that there might be anything wrong with their thyroid at all, and measured their TSH very accurately.

In their report, they said (this is a direct quote):

In the future, it is likely that the upper limit of the serum TSH euthyroid reference range will be reduced to 2.5 mIU/L because >95% of rigorously screened normal euthyroid volunteers have serum TSH values between 0.4 and 2.5 mIU/L.

Many other studies disagree, and propose wider ranges for normal TSH.

But if the AACB report were taken seriously, it would lead to diagnosis of hypothyroidism in vast numbers of people who are perfectly healthy! In fact the levels of noise in the test would put people whose thyroid systems are perfectly normal in danger of being diagnosed and inappropriately treated.

For fairly obvious reasons, biochemists have been extremely, and quite properly, reluctant to take the report of their own professional body seriously. And yet it is hard to see where the AACB have gone wrong in their report.

Neurasthenia is back.

A little after the time of the introduction of the TSH test, new forms of 'Tired All The Time' were discovered.

As I said, CFS and ME are just two names for the same thing. Fibromyalgia Syndrome (FMS) is much worse, since it is CFS with constant pain, for which there is no known cause and from which there is no relief. Most drugs make it worse.

But if you combine the three things (CFS/ME/FMS), then you get a single disease, which has a large number of very non-specific symptoms.

These symptoms are the classic symptoms of 'hypometabolism'. Any doctor who has a patient who has CFS/ME/FMS and hasn't tested their thyroid function is *de facto* incompetent. I think the vast majority of medical people would agree with this statement.

And yet, when you test the TSH levels in CFS/ME/FMS sufferers, they are perfectly normal.

All three/two/one are appalling, crippling, terrible syndromes which ruin people's lives. They are fairly common. You almost certainly know one or two sufferers. The suffering is made worse by the fact that most people believe that they're psychosomatic, which is a polite word for 'imaginary'.

And the people suffering are mainly middle-aged women. Middle-aged women are easy to ignore. Especially stupid middle-aged women who are worried about being overweight and obviously faking their symptoms in order to get drugs which are popularly believed to induce weight loss. It's clearly their hormones. Or they're trying to scrounge up welfare benefits. Or they're trying to claim insurance. Even though there's nothing wrong with them and you've checked so carefully for everything that it could possibly be.

But it's not all middle aged women. These diseases affect men, and the young. Sometimes they affect little children. Exhaustion, stupidity, constant pain. Endless other problems as your body rots away. Lifelong. No remission and no cure.

And I have Doubts of my Own

And I can't believe that careful, numerate Billewicz and his co-authors would have made this mistake, but I can't find where the doctors of the 1970s checked for the sensitivity of the TSH test.

Specificity, yes. They tested a lot of people who hadn't got any sign of hypothyroidism for TSH levels. If you're well, then your TSH level will be in a narrow range, which may be 0-6, or it may be 1-2. Opinions are weirdly divided on this point in a hard to explain way.

But Sensitivity? Where's the bit where they checked for the other arm of the conditional?

The bit where they show that no-one who's suffering from hypometabolism, and who gets well when you give them Desiccated Thyroid, had, on first contact, TSH levels outside the normal range.

If you're trying to prove A <=> B, you can't just prove A => B and call it a day. You couldn't get that past an A-level maths student. And certainly anyone with a science degree wouldn't make that error. Surely? I mean you shouldn't be able to get that past anyone who can reason their way out of a paper bag.

I'm going to say this a third time, because I think it's important and maybe it's not obvious to everyone.

If you're trying to prove that two things are the same thing, then proving that the first one is always the second one is not good enough.

IF YOU KNOW THAT THE KING OF FRANCE IS ALWAYS FRENCH, YOU DO *NOT* KNOW THAT ANYONE WHO IS FRENCH IS KING OF FRANCE.

It's possible, of course, that I've missed this bit. As I say, 'History of Endocrinology' is not one of those popular, fashionable subjects that you can easily find out about.

I wonder if they just assumed that the thyroid system was a thermostat. The analogy is still common today.

But it doesn't look like a thermostat to me. The thyroid system with its vast numbers of hormones and transforming enzymes is insanely, incomprehensibly complicated. And very poorly understood. And evolutionarily ancient. It looks as though originally it was the system that coordinated metamorphosis. Or maybe it signalled when resources were high enough to undergo metamorphosis. But whatever it did originally in our most ancient ancestors, it looks as though the blind watchmaker has layered hack after hack after hack on top of it on the way to us.

Only the thyroid originally, controlling major changes in body plan in tiny creatures that metamorphose.

Of course, humans metamorphose too, but it's all in the womb, and who measures thyroid levels in the unborn when they still look like tiny fish?

And of course, humans undergo very rapid growth and change after we are born. Especially in the brain. Baby horses can walk seconds after they're born. Baby humans take months to learn to crawl. I wonder if that's got anything to do with cretinism.

And I'm told that baby humans have very high hormone levels. I wonder why they need to be so hot? If it's a thermostat, I mean.

But then on top of the thyroid, the pituitary. I wonder what that adds to the system? If the thyroid's just a thermostat, or just a device for keeping T4 levels constant, why can't it just do the sensing itself?

What evolutionary process created the pituitary control over the thyroid? Is that the thermostat bit?

And then the hypothalamus, controlling the pituitary. Why? Why would the brain need to set the temperature when the ideal temperature of metabolic reactions is always 37C in every animal? That's the temperature everything's designed for. Why would you dial it up or down, to a place where the chemical reactions that you are don't work properly?

I can think of reasons why. Perhaps you're hibernating. Many of our ancestors must have hibernated. Maybe it's a good idea to slow the metabolism sometimes. Perhaps to conserve your fat supplies. Your stored food.

Perhaps it's a good idea to slow the metabolism in times of famine?

Perhaps the whole calories in/calories out thing is wrong, and people whose energy expenditure goes over their calorie intake have slow metabolisms, slowly sacrificing every bodily function including immune defence in order to avoid starvation.

I wonder at the willpower that could keep an animal sane in that state. While its body does everything it can to keep its precious fat reserves high so that it can get through the famine.

And then I remember about Anorexia Nervosa, where young women who want to lose weight starve themselves to the point where they no longer feel hungry at all. Another mysterious psychological disease that's just put down to crazy females. We really need some female doctors.

And I remember about Seth Robert's Shangri-La Diet, that I tried, to see if it worked, some years ago, just because it was so weird, where by eating strange things, like tasteless oil and raw sugar, you can make your appetite disappear, and lose weight. It seemed to work pretty well, to my surprise. Seth came up with it while thinking about rats. And apparently it works on rats too. I wonder why it hasn't caught on.

It seems, my female friends tell me, that a lot of diets work well for a bit, but then after a few weeks the effect just stops. If we think of a particular diet as a meme, this would seem to be its infectious period, where the host enthusiastically spreads the idea.

And I wonder about the role of the thyronine de-iodinating enzymes, and the whole fantastically complicated process of stripping the iodines and the amino acid bits from thyroxine in various patterns that no-one understands, and what could be going on there if the thyroid system were just a simple thermostat.

And I wonder about reports I am reading where elite athletes are finding themselves suffering from hypothyroidism in numbers far too large to be credible, if it wasn't, say, a physical response to calorie intake less than calorie output.

I've been looking ever so hard to find out why the TSH test, or any of the various available thyroid blood tests are a good way to assess the function of this fantastically complicated and very poorly understood system.

But every time I look, I just come up with more reasons to believe that they don't tell you very much at all.

The Mystery

Can anyone convince me that the converse arm has been carefully checked?

That everyone who's suffering from hypometabolism, and who gets well when you give them Desiccated Thyroid, has, before you fix them, TSH levels outside the normal range.

In other words, that we haven't just thrown, though carelessness, a long standing, perfectly safe, well tested treatment, for a horrible disabling disease that often causes excruciating pain, that the Victorians knew how to cure, and that the people of the 1950s and 60s routinely cured, away.

[Originally published at Intentional Insights in response to Religious and Rational]

Spirituality and rationality seem completely opposed. But are they really?

To get at this question, let's start with a little thought experiment.  Consider the following two questions:

1.  If you were given a choice between reading a physical book (or an e-book) or listening to an audiobook, which would you prefer?

2.  If you were given a choice between listening to music, or looking at the grooves of a phonograph record through a microscope, which would you prefer?

But I am more interested in the answer to a third question:

3.  For which of the first two questions do you have a stronger preference between the two options?

Most people will have a stronger preference in the second case than the first.  But why?  Both situations are in some sense the same: there is information being fed into your brain, in one case through your ears and in the other through your eyes.  So why should people's preference for ears be so much stronger in the case of music than books?

There is something in the essence of music that is lost in the translation between an audio and a visual rendering.  The same loss happens for words too, but to a much lesser extent.  Subtle shades of emphasis and tone of voice can convey essential information in spoken language. This is one of the reasons that email is so notorious for amplifying misunderstandings.  But the loss in much greater in the case of music.  

The same is true for other senses.  Color is one example.  A blind person can abstractly understand what light is, and that color is a byproduct of the wavelength of light, and that light is a form of electromagnetic radiation... yet there is no way for a blind person to experience subjectively the difference between red and blue and green.  But just because some people can't see colors doesn't mean that colors aren't real.

The same is true for spiritual experiences.

Now, before I expand that thought, I want to give you my bona fides.  I am a committed rationalist, and an atheist (though I don't like to self-identify as an atheist because I'd rather focus on what I *do* believe in rather than what I don't).  So I am not trying to convince you that God exists.  What I want to say is rather that certain kinds of spiritual experiences *might* be more than mere fantasies made up out of whole cloth. If we ignore this possibility we risk shutting ourselves off from a vital part of the human experience.

I grew up in the deep south (Kentucky and Tennessee) in a secular Jewish family.  When I was 12 my parents sent me to a Christian summer camp (there were no other kinds in Kentucky back in those days).  After a week of being relentlessly proselytized (read: teased and ostracized), I decided I was tired of being the camp punching bag and so I relented and gave my heart to Jesus.  I prayed, confessed my sins, and just like that I was a member of the club.

I experienced a euphoria that I cannot render into words, in exactly the same way that one cannot render into words the subjective experience of listening to music or seeing colors or eating chocolate or having sex.  If you have not experienced these things for yourself, no amount of description can fill the gap.  Of course, you can come to an *intellectual* understanding that "feeling the presence of the holy spirit" has nothing to do with any holy spirit. You can intellectually grasp that it is an internal mental process resulting from (probably) some kind of neurotransmitter released in response to social and internal mental stimulus.  But that won't allow you to understand *what it is like* any more than understanding physics will let you understand what colors look like or what music sounds like.

Happily, there are ways to stimulate the subjective experience that I'm describing other than accepting Jesus as your Lord and Savior. Meditation, for example, can produce similar results.  It can be a very powerful experience.  It can even become addictive, almost like a drug.

I am not necessarily advocating that you go try to get yourself a hit of religious euphoria (though I wouldn’t discourage you either -- the experience can give you some interesting and useful perspective on life).  Instead, I simply want to convince you to entertain the possibility that people might profess to believe in God for reasons other than indoctrination or stupidity.  Religious texts and rituals might be attempts to share real subjective experiences that, in the absence of a detailed modern understanding of neuroscience, can appear to originate from mysterious, subtle external sources.

The reason I want to convince you to entertain this notion is that an awful lot of energy gets wasted by arguing against religious beliefs on logical grounds, pointing out contradictions in the Bible and whatnot. Such arguments tend to be ineffective, which can be very frustrating for those who advance them. The antidote for this frustration is to realize that spirituality is not about logic.  It's about subjective experiences that not everyone is privy to.  Logic is about looking at the grooves.  Spirituality is about hearing the music.

The good news is that adopting science and reason doesn’t mean you have to give up on spirituality any more than you have to give up on music. There are myriad paths to spiritual experience, to a sense of awe and wonder at the grand tapestry of creation, to the essential existential mysteries of life and consciousness, to what religious people call “God.” Walking in the woods. Seeing the moons of Jupiter through a telescope. Gathering with friends to listen to music, or to sing, or simply to share the experience of being alive. Meditation. Any of these can be spiritual experiences if you allow them to be. In this sense, God is everywhere.

Self-motivated hard work is the primary source of the intense, optimistic engagement known as flow—one of the greatest forms of happiness that makes us come alive with purpose and potential (Csikszentmihalyi, 1975). Sadly, for most people work does not feel so rewarding most of the time. Instead we often have to persevere through long periods of hard, painful, and unrewarding work when we could be doing something much more enjoyable. When faced with this motivational challenge people often give up too easily, get sidetracked, or procrastinate (Steel, 2007). The problem is not that we are not willing or unable to work hard. To the contrary, we crave being productively engaged in challenging tasks. Thus, instead of blaming ourselves for our limited will-power, it may be more productive to take a critical look at the carrots and the sticks that are supposed to help us stay motivated. Who put them there and why? Are these incentives helpful, distracting, irrelevant, or out of sight? If you could place them differently and add new ones, where would they go? Often, the problem is that the rewards we experience in the short run are misaligned with what we want to accomplish. In the short run the extremely valuable work that brings us closer to our cherished goals can be aversive while activities that are irrelevant or even opposed to everything we want to accomplish can be pleasant and rewarding. Hence, when we struggle to be engaged with something that we care about, then perhaps we are not the problem but the incentives are, or as Jane McGonigal (2011) put it "Reality is broken".

So, if reality is broken, then what can we do to fix it?  One approach is to design better incentive structures that make the pursuit of our goals more engaging. If we want to go this way, then there is a lot to be learned from games, because their incentive structures are so well designed that they let people enjoy hard work for many hours on end (McGonigal, 2011). In the past five years, the success of video games has inspired the gamification of education, work, health, and business. Gamification is the use of game elements, like points, levels, badges, and quests to engage, motivate, and nudge people in non-game contexts. There are even tools like SuperBetter and Habitica that individuals like you and I can use to gamify our own lives. Previous studies have shown that gamification can have positive effects on motivation, engagement, behavior, learning outcomes, and health—but only when it is done right (Hamari, Koivisto, & Sarsa, 2014; Roepke, et al., 2015). But when gamification is done wrong it can have negative effects by incentivizing counter-productive behaviors. So far gamification has been an art, and there is very little science about how to do it right. This motivated my advisor and me to develop a practical theory of optimal gamification.

In this blog post I focus on how our theory could be applied in practice. If you would like to learn about the technical details or read more about our experiments, then please take look at our CogSci paper (Lieder & Griffiths, submitted). I will start with a very brief summary of our method, provide an intuitive explanation of what it does, and then dive into how you can implement it in your own life. I will close with an outlook on how our method could be applied to gamify our todo lists.

Level 1: Optimal Gamification

Our method for optimal gamification draws on the theory of Markov decision processes (MDPs; Sutton & Barto, 1998) and the shaping theorem (Ng, Harada, & Russell, 1999). The basic idea is to align each action's immediate reward with its value in the long run. Therefore the points should complement the immediate rewards of doing something (e.g., how painful it is) by the value that it generates in the long run. Concretely, the points awarded for an activity should be chosen such that the right thing to do looks best in the short run when you combine how many points it is worth with how it feels when you do it. Furthermore, the points have to be assigned in such a way that when you undo something you lose as many points as you earned when you did it. We evaluated the effectiveness of our method in two behavioral experiments. Our first experiment demonstrated that incentive structures designed by our method can indeed help people make better, less short-sighted decisions—especially when course of action that is best in the long run is unpleasant in the short run. We also found that less principled approaches to gamification can encourage ruthless rushing towards a goal that causes more harm than good, and we showed that our method is guaranteed to avoid these perils. In the second experiment we found that the optimal incentive structures designed with our method can be effectively implemented using game elements like points and badges. These results suggest that the proposed method provides a principled way to leverage gamification to help people make better decisions.

Our method proceeds in three steps:

1.    Model the situation and the decision-maker's goals and options as a MDP.

2.    Solve the MDP to obtain the optimal value function V* or approximate it.

3.    Set the number of points for progressing from stage s to stage s' to V*(s')-V*(s).  

Intuitively, this means that the number of points that is awarded for doing something should reflect how much better the resulting state (i.e., s') is than the previous one (i.e., s). For instance, achieving a goal is worth 1000 points then completing 10% of the work required to reach the goal should be rewarded with 100 points. So let's think about how you could apply this approach right now without having to solve MDPs.

Level 2: Practical Implications

In my day-to-day life I try to approximate optimal gamification as follows:

1.    Set a concrete goal that you would like to achieve and figure out how many points it is worth, e.g. writing this blog post was worth 1000 points to me.

2.    Set several milestones along the way to the goal to divide the path into small steps that feel very manageable.

3.    For each milestone, determine how far you will have come when you get there as a percentage of the total distance to the goal, e.g. 10%, 20%, 30%, ..., 100% for the first, second, third, ..., and the tenth milestone respectively.

4.    Assign each milestone the corresponding fraction of the total value of achieving the goal, e.g. 100 points, 200 points, 300 points, ..., and 1000 points for the first, second, third, ..., and tenth milestone respectively.

5.    Figure out what you have to do to get from one milestone to the next. If this is a simple activity, then its reward should be the difference between the value of next milestone and the value of the current milestone, e.g. 100 points. If it is a complex sequence of actions, then make it a subgoal and apply steps 1-3 figure out how to achieve it.

6.    Once you are done with step 5, you can add those points to your todo-list.

7.    Now it is time to get things done and reward yourself. You start at 0 points, but whenever you complete one of the steps, you earn as many points as you have assigned to it and can increment your (daily) score.

Earning these points can be very rewarding if you remind yourself what they stand for. If your goal was worth $1,000,000 to you and you assigned 1000 points to it, then 10 points should be worth $10,000 to you. But if this is not rewarding enough for you, you can think of ways that make the points more pleasurable. You could, for instance, make a high-score list that motivates you to beat your personal best day after day or start a high-score competition with your friends. You could also set yourself the goal to achieve a certain number of points by a certain time and promise yourself a treat if you achieve it.

There are many other ways that you could assign points to the items on you todo list. Feel free to do whatever works for you. But it may be useful to keep in mind that the way in which optimal gamification assigns points has several formal properties that are necessary to avoid negative side-effects:

a) Each item's score reflects how valuable is in the long run.

Optimal gamification works because it aligns each action's immediate reward with its long-term value. To help you make better decisions the points should be designed such that the course of action that is best in the long run looks best in the short run. This entails incentivizing unpleasant or unrewarding activities that will pay off later—especially when their less productive alternatives are very rewarding in the short run.

b) Beware of cycles!

The shaping theorem (Ng, et al., 1999) requires that going back and forth between two states receives a net pseudo-reward of zero. When your pseudo-rewards along a circle add up to a positive value, then you may be incentivizing yourself to create unnecessary problems for yourself. This can happen when the action for which you reward yourself can only be executed in an undesirable state, and you do not equally punish yourself for falling back into that state. For instance adding points for losing weight will inadvertently incentivize you to regain weight afterwards unless you subtract at least the same number of points for gaining weight. Similarly, if you reward yourself for solving interpersonal conflicts but don’t punish yourself for creating them, then you may be setting yourself up for trouble. To avoid such problems, creating a problem must be punished by at least as many points as you earn by solving it. 

c) Two ways to achieve the same goal should yield the same number of points.

The shaping theorem also requires that all paths that lead to the same final state (e.g., having submitted a paper by the deadline) should yield the same amount of reward. If this is not the case your pseudo-rewards may bias you towards a suboptimal path. For instance, if you reward your all-nighter on the last night before the deadline by the reward value of a month’s worth of work, you are incentivizing yourself to procrastinate. Similarly, if you reward one activity that leads towards your goal much more heavily than others, then you may be biasing yourself towards a reckless course of action that may achieve the goal at an unreasonably high cost. For instance, rewarding yourself 100 times as much for working 100% on a project than for working on it 50% might lead you to complete the project early at the expense of your health, your friendships, your education, and all your other projects. To avoid this problem, al paths that lead to the same state should yield the same amount of reward. 

d) Pseudo-rewards should be awarded for state-transitions instead of actions.

Many applications of gamification reward "good" actions with points regardless of when or how often these actions are taken. But according to the shaping theorem, the number of points must depend on the state in which the action is taken and the state that it leads to. If your pseudo-rewards were based only on what you do but not on when you do it, then you might keep rewarding yourself for something even when it is no longer valuable, because the underlying state has changed. For instance, at some point your reward for losing weight has to diminish or else you may be setting yourself up for anorexia.

Level 3: Todo-list gamification

My first practical application is to manually gamify my todo-list every morning. I find this very helpful and motivating: Assigning points to the items on my todo list makes me realize how much I value them. This is useful for prioritizing important task. Earning points allows me to perceive my progress more more accurately and more vividly. This helps me feel great about getting something important done even when it was only a single item on my todo list and took me a lot of time and effort to accomplish. Conversely, the point scheme also prevents me from feeling so good about checking off small things that I become tempted to neglect the big ones that are much more important. Gamification thereby remedies the todo list's shortcoming that it makes each item seem equally important. I highly recommend gamifying your todo lists. It can be highly motivating. Yet, adding the points manually takes some effort and my point scheme is often somewhat arbitrary and probably suboptimal.

To make todo list gamification easier and more effective, I am planning to develop an easy-to-use website or app that will do optimal gamification for you. Its graphical user interface would allow you to create hierarchical todo-lists, ask you 1 or 2 simple questions about each item on your list and then gamify your todo-list for you. To do this, it will translate your list and your answers into a MDP, compute its optimal value function, and use it to determine how valuable it is to complete each item. The tool could also help you set manageable subgoals and determine what is most important and should be done first. Last but not least, a website or app can also leverage additional game elements to make the points that you earn more rewarding: It can track your productivity and provide instant feedback that makes your progress more salient. It can send you on a quest that gives you a goals along with small actionable steps. The tool could allow you to realize that you are getting ever more productive by visualizing your progress over time. As you become more effective, you level up and your quests will become increasingly more challenging.  It might include a scoreboard that lets you compete with yourself and/or others and win prizes for your performance. Last but not least, if you need an extra push, you can tie your points to social rewards, your favorite treat, money, or access to your favorite music, apps, or websites. There are many more possibilities, and I invite you to think about it and share your ideas. In brief, there is wealth of opportunities to leverage game elements to make goal achievement fun and easy.

Join me on my quest! An adventure awaits. 

Gamification can be a useful tool to make achieving your goals easier and more engaging. However, gamification only works when it is done right. The theory of MDPs and pseudo-rewards provide the formal tools needed to do gamification right. With the help of these tools we can design incentive structures that help people overcome motivational obstacles, do the right thing and achieve their goals. But more research and development needs to be done to make optimal gamification practical.

If you have any thoughts or ideas for what to do next, noticed a problem with the approach, or would like to be part of our team and contribute to building a tool helps people achieve their goals, please send me an e-mail.

References and recommended readings

Csikszentmihalyi, M. (1975). Beyond boredom and anxiety: the experience of play in work and games. San Francisco: Jossey-Bass.

Lieder, F., & Griffiths, T.L. (submitted). Helping people make better decisions using optimal gamification. CogSci 2016. [Manuscript]

McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York: Penguin.

McGonigal, J. (2015). SuperBetter: A revolutionary approach to getting stronger, happier, braver and more resilient–powered by the science of games. London, UK: Penguin Press.

Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work?–A literature review of empirical studies on gamification. In 47th Hawaii international conference on system sciences (pp. 3025–3034).

Ng, A. Y., Harada, D., & Russell, S. (1999). Policy invariance under reward transformations: Theory and application to reward shaping. In I. Bratko & S. Dzeroski (Eds.), Proceedings of the 16th annual international conference on machine learning (Vol. 16, pp. 278–287). San Francisco, CA, USA: Morgan Kaufmann.

Roepke, A. M., Jaffee, S. R., Riffle, O. M., McGonigal, J., Broome, R., & Maxwell, B. (2015). Randomized controlled trial of SuperBetter, a smartphone-based/Internet-based self-help tool to reduce depressive symptoms. Games for health journal, 4(3), 235-246.

Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA, USA: MIT press.

A putative new idea for AI control; index here.

This post will be extending ideas from inverse reinforcement learning (IRL) to the problem of goal completion. I'll be drawing on the presentation and the algorithm from Apprenticeship Learning via Inverse Reinforcement Learning (with one minor modification).

In that setup, the environment is an MDP (Markov Decision process), and the real reward R is assumed to be linear in the "features" of the state-action space. Features are functions φ_i from the full state-action space S×A to the unit interval [0,1] (the paper linked above only considers functions from the state space; this is the "minor modification"). These features form a vector φ∈[0,1]^k, for k different features. The actual reward is given by the inner product with a vector w∈ℝ^k, thus the reward at state-action pair (s,a) is

R(s,a)=w.φ(s,a).

To ensure the reward is always between -1 and 1, w is constrained to have ||w||₁ ≤ 1; to reduce redundancy, we'll assume ||w||₁=1.

The advantages of linearity is that we can compute the expected rewards directly from the expected feature vector. If the agent follows a policy π (a map from state to action) and has a discount factor γ, the expected feature vector is

μ(π) = E(Σ_t γ^tφ(s_t,π(s_t)),

where s_t is the state at step t.

The agent's expected reward is then simply

E(R) = w . μ(π).

Thus the problem of computing the correct reward is reduced to the problem of computing the correct w. In practice, to compute the correct policy, we just need to find one whose expected features are close enough to optimal; this need not involve computing w.

Elitism and Effective Altruism

Many criticize Effective Altruists as elitist. While this criticism is vastly overblown, unfortunately, it does have some basis, not only from the outside looking in but also within the movement itself, including some explicitly arguing for elitism.

Within many EA circles, there are status games and competition around doing “as much as we can,” and in many cases, even judging and shaming, usually implicit and unintended but no less real, of those whom we might term softcore EAs. These are people who identify as EAs and donate money and time to effective charities, but otherwise lead regular lives, as opposed to devoting the brunt of their resources to advance human flourishing as do hardcore EAs. To be clear, there is no definitive and hard distinction between softcore and hardcore EAs, but this is a useful heuristic to employ, as long as we keep in mind that softcore and hardcore are more like poles on a spectrum rather than binary categories.

We should help softcore EAs feel proud of what they do, and beware implying that being softcore EA is somehow deficient or simply the start of an inevitable path to being a hardcore EA. This sort of mentality has caused people I know to feel guilty and ashamed, and led to some leaving the EA movement. Remember that we all suffer from survivorship bias based on seeing those who remained, and not those who left - I specifically talked to people who left, and tried to get their takes on why they did so.

I suggest we aim to respect people wherever they are on the softcore/hardcore EA spectrum. I propose that, from a consequentialist perspective, negative attitudes toward softcore EAs are counterproductive for doing the most good for the world.

Why We Need Softcore EAs

Even if the individual contributions of softcore EAs are much less than the contributions of individual hardcore EAs, it’s irrational and anti-consequentialist to fail to acknowledge and celebrate the contributions of softcore EAs, and yet that is the status quo for the EA movement. As in any movement, the majority of EAs are not deeply committed activists, but are normal people for whom EA is a valuable but not primary identity category.

All of us were softcore EAs once - if you are a hardcore EA now, envision yourself back in those shoes. How would you have liked to have been treated? Acknowledged and celebrated or pushed to do more and more and more? How many softcore EAs around us are suffering right now due to the pressure of expectations to ratchet up their contributions?

I get it. I myself am driven by powerful emotional urges to reduce human suffering and increase human flourishing. Besides my full-time job as a professor, which takes about ~40 hours per week, I’ve been working ~50-70 hours per week for the last year and a half as the leader of an EA and rationality-themed meta-charity. As all people do, when I don’t pay attention, I fall unthinkingly into the mind projection fallacy, assuming other people think like I do and have my values, as well as my capacity for productivity and impact. I have a knee-jerk pattern as part of my emotional self to identify with and give social status to fellow hardcore EAs, and consider us an in-group, above softcore EAs.

These are natural human tendencies, but destructive ones. From a consequentialist perspective, it weakens our movement and undermines our capacity to build a better world and decrease suffering for current and future humans and other species.

More softcore EAs are vital for the movement itself to succeed. Softcore EAs can help fill talent gaps and donating to effective direct-action charities, having a strong positive impact on the outside world. Within the movement, they support the hardcore EAs emotionally through giving them a sense of belonging, safety, security, and encouragement, which are key for motivation and mental and physical health. Softcore EAs also donate to and volunteer for EA-themed meta-charities, as well as providing advice and feedback, and serving as evangelists of the movement.

Moreover, softcore EAs remind hardcore EAs of the importance of self-care and taking time off for themselves. This is something we hardcore EAs must not ignore! I’m speaking from personal experience here.

Fermi Estimates of Hardcore and Softcore Contributions

If we add up the amount of resources contributed to the movement by softcore EAs, they will likely add up to substantially more than the resources contributed by hardcore EAs. For instance, the large majority of those who took the Giving What We Can and The Life You Can Save pledges are softcore EAs, and so are all the new entrants to the EA movement, by definition.

To attach some numbers to this claim, let’s do a Fermi Estimate that uses some educated guesses to get at the actual resources each group contributes. Say that for every 100 EAs, there are 5 hardcore EAs and 95 softcore EAs. We can describe softcore EAs as contributing anywhere from 1 to 10 percent of their resources to EA causes (this is the range from The Life You Can Save pledge to the Giving What We Can pledge), so let’s guesstimate around 5 percent. Hardcore EAs we can say give an average of 50% of their resources to the movement. Using the handy Guesstimate app, here is a link to a model that shows softcore EAs contribute 480 resources, and hardcore EAs contribute 250 resources per 100 EAs. Now, these are educated guesses, and you can use the model I put together to put in your own numbers for the number of hardcore and softcore EAs per 100 EAs, and also the percent of their resources contributed. In any case, you will find that softcore EAs contribute a substantial amount of resources.

We should also compare the giving of softcore EAs to the giving of members of the general public to get a better grasp on the benefits provided to improving the world by softcore EAs. Let’s say a typical member of the general public contributes 3.5% of her resources to charitable causes, by comparison to 5% for softcore EAs. Being generous, we can estimate that the giving of non-EAs is 100 times less effective than that of EAs. Thus, using the same handy app, here is a link to a model that demonstrates the impact of giving by a typical member of the general public, 3.5, vs. the impact of giving by a softcore EA, 500. Now, the impact of giving by a hardcore EA is going to be higher, of course, 5000 as opposed to 500, but again, we have to remember that there are many more softcore EAs who give resources. You’re welcome to plug in your own numbers to get estimates if you think my suggested figures don’t match your intuitions. Regardless, you can see the high-impact nature of how a typical softcore EA compares to a typical member of the general public.

Effective Altruism, Mental Health, and Burnout: A Personal Account

About two years ago, in February 2014, my wife and I co-founded our meta-charity. In the summer of that year, she suffered a nervous breakdown due to burnout over running the organization. I had to - or to be accurate, chose to - take over both of our roles in managing the nonprofit, assuming the full burden of leadership.

In the Fall of 2014, I myself started to develop a mental disorder from the strain of doing both my professor job and running the organization, while also taking care of my wife. It started with heightened anxiety, which I did not recognize as something abnormal at the time - after all, with the love of my life recovering very slowly from a nervous breakdown and me running the organization, anxiety seemed natural. I was flinching away from my problem, not willing to recognize it and pretending it was fine, until some volunteers at the meta-charity I run – most of them softcore EAs – pointed it out to me.

I started to pay more attention to this, especially as I began to experience fatigue spells and panic attacks. With the encouragement of these volunteers, who essentially pushed me to get professional help, I began to see a therapist and take medication, which I continue to do to this day. I scaled back on the time I put into the nonprofit, from 70 hours per week on average to 50 hours per week. Well, to be honest, I occasionally put in more than 50, as I’m very emotionally motivated to help the world, but I try to restrain myself. The softcore volunteers at the meta-charity I run know about my workaholism and the danger of burnout for me, and remind me to take care of myself. I also need to remind myself constantly that doing good for the world is a marathon and not a sprint, and that in the long run, I will do much more good by taking it easy on myself.

Celebrating Everyone

As a consequentialist, my analysis, along with my personal experience, convince me that the accomplishments of softcore EAs should be celebrated as well as those of hardcore EAs.

So what can we do? We should publicly showcase the importance of softcore EAs. For example, we can encourage publications of articles that give softcore EAs the recognition they deserve, as well as those who give a large portion of their earnings and time to charity. We can invite a softcore EA to speak about her/his experiences at the 2016 EA Global. We can publish interviews with softcore EAs. Now, I’m not suggesting we should make most speakers softcore EAs, or write most articles, or conduct most interviews with softcore EAs. Overall, my take is that it’s appropriate to celebrate individual EAs proportional to their labors, and as the numbers above show, hardcore EAs individually contribute quite a bit more than softcore EAs. Yet we as a movement need to go against the current norm of not celebrating softcore EAs, and these are just some specific steps that would help us achieve this goal.

Let’s celebrate all who engage in Effective Altruism. Everyone contributes in their own way. Everyone makes the world a better place.

Acknowledgments: For their feedback on draft versions of this post, I want to thank Linch (Linchuan) Zhang, Hunter Glenn, Denis Drescher, Kathy Forth, Scott Weathers, Jay Quigley, Chris Waterguy (Watkins), Ozzie Gooen, Will Kiely, and Jo Duyvestyn. I bear sole responsibility for any oversights and errors remaining in the post, of course.

A different version of this, without the Fermi estimates, was cross-posted on the EA Forum.

EDIT: added link to post explicitly arguing for EA elitism

Related to: Half-assing it with everything you've got; Wasted motion; Say it Loud.

Once upon a time (true story), I was on my way to a hotel in a new city.  I knew the hotel was many miles down this long, branchless road.  So I drove for a long while.

After a while, I began to worry I had passed the hotel.

So, instead of proceeding at 60 miles per hour the way I had been, I continued in the same direction for several more minutes at 30 miles per hour, wondering if I should keep going or turn around.

A putative new idea for AI control; index here.

A system that is optimizing a function of n variables, where the objective depends on a subset of size k<n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable.

Stuart Russell

Think of an AI directing a car, given the instructions to get someone to the airport as fast as possible (optimised variables include "negative of time taken to airport") with some key variables left out - such as a maximum speed, maximum acceleration, respect for traffic rules, and survival of the passengers and other humans.

Call these other variables "unstated objectives" (UO), as contrasted with the "stated objectives" (SO) such as the time to the airport. In the normal environments in which we operate and design our AIs, the UOs are either correlated with the SOs (consider the SO "their heart is beating" and the UO "they're alive and healthy") or don't change much at all (the car-directing AI could have been trained on many examples of driving-to-the-airport, none of which included the driver killing their passengers).

Typically, SOs are easy to define, and the UOs are the more important objectives, left undefined either because they are complex, or because they didn't occur to us in this context (just as we don't often say "driver, get me to the airport as fast a possible, but alive and not permanently harmed, if you please. Also, please obey the following regulations and restrictions: 1.a.i.α: Non-destruction of the Earth....").

The control problem, in a nutshell, is that optimising SOs will typically set other variables to extreme values, including the UOs. The more extreme the optimisation, and the furthest from the typical environment, the more likely this is to happen.

Jaan Tallinn has suggested creating a toy model of the various common AI arguments, so that they can be analysed without loaded concepts like "autonomy", "consciousness", or "intentionality". Here a simple attempt for the "treacherous turn"; posted here for comments and suggestions.

Meet agent L. This agent is a reinforcement-based agent, rewarded/motivated by hearts (and some small time penalty each turn it doesn't get a heart):