LESSWRONG
LW

How to tell apart science from pseudo-science in a field you don't know ?

1 min read

Comment Permalink

When bridging an inferential distance, see where you are now, and then see where the experts are. That should help uncover the ignorance of your ignorance and make it possible for you to move towards the experts in knowledge-space. If you don't know where the experts are in knowledge-space, then you can easily stumble along in the dark until you end up at bullshit.

The next course of action is to actually move towards the experts in knowledge-space. Textbooks, expert blogs, wikipedia, classes, and so on are all viable methods for that.

But besides understanding the area well, I think you also want to understand or mentally model autists. In which case, you should become familiar with actual autists (through e.g. documentaries, books, and personal communication).

See in context

34 How to tell apart science from pseudo-science in a field you don't know ?

by kilobug

2nd Sep 2012

1 min read

34

First, a short personal note to make you understand why this is important to me. To make a long story short, the son of a friend has some atypical form of autism and language troubles. And that kid matters a lot to me, so I want to become stronger in helping him, to be able to better interact with him and help him overcome his troubles.

But I don't know much about psychology. I'm a computer scientist, with a general background of maths and physics. I'm kind of a nerd, social skills aren't my strength. I did read some of the basic books advised on Less Wrong, like Cialdini, Wright or Wiseman, but those just give me a very small background on which to build.

And psychology in general, autism/language troubles in particular, are fields in which there is a lot of pseudo-science. I'm very sceptical of Freud and psychoanalysis, for example, which I consider (but maybe I am wrong?) to be more like alchemy than like chemistry. There are a lot of mysticism and sect-like gurus related to autism, too.

So I'm bit unsure on how from my position of having a general scientific and rationality background I can dive into a completely unrelated field. Research papers are probably above my current level in psychology, so I think books (textbooks or popular science) are the way to go. But how to find which books on the hundreds that were written on the topic I should buy and read? Books that are evidence-based science, not pseudo-science, I mean. What is a general method to select which books to start in a field you don't really know? I would welcome any advise from the community.

Disclaimer: this is a personal "call for help", but since I think the answers/advices may matter outside my own personal case, I hope you don't mind.

EpistemologyNeurodivergencePsychology

Personal Blog

34

New Comment

71 comments, sorted by

top scoring

Click to highlight new comments since: Today at 2:09 AM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]Alicorn13y290

Watch out for sensory issues. If the kid reacts to, I dunno, styrofoam like it's full of shards of broken glass and he can't bear to touch it, keep him away from styrofoam (there is no training this sort of thing away, although you could get a learned-helplessness oh-they're-torturing-me-again reaction if you tried too hard - and I'm given to understand that some treatment plans do aim at that, presumably because suffering in silence is less disruptive for caretakers than screaming in discomfort). Keeping away sensory aversives frees up brainpower to do other things, as well as being an important form of not-torturing-the-kid.

Treat stimmy behaviors like ordinary fidgeting. They are only weird relative to a framework that an autistic does not operate within. Specifically, they do not say anything in particular about whether he is paying attention, especially to a person who he wouldn't be likely to make eye contact with anyway.

Explain substeps of tasks that do not get understood promptly. (If "get a bowl of cereal" results in the kid staring blankly into space, try "get a bowl out of the cupboard and put it on the counter, open the cereal box and pour cereal into... (read more)

4MBlume13y

Sidenote: This applies to us ADHD kids too. Fidgeting is how we make our brains work. It is self-medication. Now, if one kid making their brain work is making 29 other brains not work (ie by making distracting noises) you obviously have a problem. But that problem should be solved with a view to helping us find harmless ways to fidget, not making us stop fidgeting entirely. Data point: I'm 27, work at a tech startup, and would've had a much harder time staying alert in a meeting today if I hadn't been turning an hourglass over in my hand and watching the patterns the sand made. Citation: http://adhdmomma.com/2011/12/guest-post-fidgeting-helps-kids-stay.html http://phys.org/news162554898.html

0Pavitra13y

Citation needed. This sounds plausible enough that people are likely to listen to it, so I'd like some sort of confirmation that it's based in fact.

4Alicorn13y

I have an autism diagnosis and multiple autistic friends. I poked around the literature on autism for a paper in grad school (although it was mostly on theory of mind). I have read books and blogs and aggregated therefrom a general model of autism that has yet to be dinged by any of this. Also, "don't do things to people that they find abhorrent for no goddamn reason" and "if someone never makes eye contact anyway their rocking is insignificant information about whether you have their attention, and rocking is only atypical, not fundamentally different from pen-twirling" and "one thing to try if giving someone an instruction doesn't work is making sure they have it taskified; also don't expect giving the people around you commands to work all the time" all seem pretty basic to me. And really ought to be status-quo, requiring citations to deviate therefrom. I would certainly require a citation if I had a kid and someone told me that they should be forced into contact with objects they don't like, and aren't to be allowed to move around as they please even if they aren't hurting anyone, and that their not doing everything I say is a sign of a Serious Problem. The allistic equivalents would be unambiguous abuse, and plenty of autistic people are capable of telling others what the autistic-specific versions of those abuses are.

0wedrifid13y

Which way? Is the stimming more likely when you have their attention or when you don't?

0Alicorn13y

I'm not sure. Naively on priors, someone under stress is more likely to stim - and someone who's paying attention to another thing is more likely to be under stress.

-5Eugine_Nier13y

[-]James_Miller13y190

To the best of my knowledge (and I've looked) there is not a single scientific long-term randomized study showing the effectiveness of any type of treatment for autism. This means that when deciding on the best way to help the kid you are going to have to rely on the judgment and intuition of family, friends and special needs specialists. Besides the normal biases the huge problem with doing this is that as an autistic child gets older you would expect him, in an absolute sense, to make improvements in many metrics (just as typical kids do) even if whatever special stuff was being done for him had absolutely no impact on his condition. Another problem is that, based on my observations at least, the women who devote their careers to the needs of "special children" tend to be of the very happy/uplifting/optimistic types which undoubtedly causes them to have a more positive assessment of treatment than should be justified and this bias outlook negatively impacts the research that makes use of the subjective judgments of autistic professionals.

Rather than spending time reading about autism you can probably better help this child by playing with him and doing stuff for his parents so they have more time to play with him, although ignore this advice if you enjoy reading about autism and so your doing so isn't a cost.

9Emile13y

In my (very limited) experience autistic children aren't very interested in playing with people, so a minimum of reading might be useful if only to understand what kinds of "playing" are likely to interest an autistic child, and also which kind of "playing" have chances of slightly improving his communication/interaction skills and interests. There's an autistic three-year-old that often plays in the sandbox with my kid - he's much more interested in playing with a single toy than in the other kids or his parents; unlike my 20-month-old who evaluates toys by how much attention the other kids are paying to them (resulting in the classical "everybody fight for the shovel, five minutes later everyone fights for the truck"), or uses toys to bribe other kids (preferably bribing with someone else's toy). The autistic kid can eventually tolerate playing with another kid (like, both fill up the same bucket), but has to be prodded to do so, and will usually end up going back to playing alone. The only "game" I've seen him play with someone else is being chased around by his parents, which he seems to find fun.

7James_Miller13y

One theory is that you start with what the kid will do without prompting and then gradually introduce yourself into his play using trial and error to identify actions he appreciates and responds to. This is called floortime. The other main paradigm is Applied Behavior Analysis.

6CharlesR13y

This is very good advice.

6Rhwawn13y

Why isn't there? There would seem to have been more than enough time & funding for at least one. Is there some more subtle problem here? (I'm thinking a scenario like "parents of autistic kids are constantly trying new approaches both quack and genuine, and would refuse to stop this, thereby making the results worthless; and this is foreseeable in advance by any would-be experimenters.")

0CharlesR13y

No one wants to be in the control group.

1Rhwawn13y

Do you know that, or are you guessing?

0drethelin13y

Because there's no cure?

3James_Miller13y

But there could still be studies demonstrating that some treatments had no effect.

2Luke_A_Somers13y

That raises the question - did that opening sentence of the head reply mean 'showing the effectiveness or ineffectiveness of any method', or 'showing a method to be effective'?

0James_Miller13y

I meant to imply "effectiveness or ineffectiveness"

0Luke_A_Somers13y

Thanks

[-]AdeleneDawner13y150

To whatever degree you find firsthand reports from autistics useful (and we are able to introspect and such, just in case your reading had led you to believe otherwise - there are some ridiculous misconceptions out there), those are a thing you can look for.

Wrongplanet.net is large, but has had some unpleasant evaporative cooling going on for several years - it may still be a useful place to ask questions. Similarly, reddit has a subreddit for autistics, but the demographic there is affected by the overall tone of the site.

Private blogs are a better bet for thoughtful information - Urocyon has a list of neurodiversity and disability blogs in her sidebar that seems like a decent starting point for that. Also, tumblr has a fairly good autistic community - we tend to post in the actuallyautistic tag, which you shouldn't post in as you're not actually autistic; you can post questions to the autism or autistic tags, and there's a very good chance we'll see them and respond. (Do your research first, though; the standard reaction to uninformed mistakes is derision here just as much as it would be anywhere else. Also, person-first language tends to go over poorly; it's a point of etiquette... (read more)

4David_Gerard13y

+1 to Urocyon - I know her, she's great.

[-]billswift13y120

You might find this useful, it isn't a source of papers, it is first-hand accounts by autistics and what life and other people were like to them. This one, Don't Mourn For Us, is probably the best general description. A quote from it:

You try to relate to your autistic child, and the child doesn't respond. He doesn't see you; you can't reach her; there's no getting through. That's the hardest thing to deal with, isn't it? The only thing is, it isn't true.
Look at it again: You try to relate as parent to child, using your own understanding of normal children, your own feelings about parenthood, your own experiences and intuitions about relationships. And the child doesn't respond in any way you can recognize as being part of that system.
That does not mean the child is incapable of relating at all. It only means you're assuming a shared system, a shared understanding of signals and meanings, that the child in fact does not share. It's as if you tried to have an intimate conversation with someone who has no comprehension of your language. Of course the person won't understand what you're talking about, won't respond in the way you expect, and may well find the whole interaction conf

... (read more)

[-]pragmatist13y80

If you're looking for reputable textbooks and introductory reading, I would suggest going to one of the many open courseware sites at major universities (such as MIT, Berkeley or Yale). Look at the courses available for the subject you're interested in (psychology, in your case), find one that matches your interests, and check out the syllabus for that course. It should direct you towards good learning material for that subject.

Here, for instance, is a list of readings from the MIT intro psych class. Here's an MIT class specifically on autism.

[-]CharlesR13y50

UPDATED: It has been pointed out that Autism Speaks still funds research looking for the supposed link to vaccines! People have resigned over this. Do not give your money to this organization.

Some books on autism:

The Out-of-Sync Child (Kranowitz)
The RDI Book (Gutstein)
More Than Words by Sussman (The Hanen Centre)

There is also the 100 Day Kit from Autism Speaks.

The Autism Speaks 100 Day Kit and the Asperger Syndrome and High Functioning Autism Tool Kit were created specifically for newly diagnosed families to make the best possible use of the 100 da

... (read more)

0Alicorn13y

General antiendorsement of Autism Speaks.

1CharlesR13y

Can you be more specific?

4Alicorn13y

They're just pretty shitty as an organization. Very focused on the neurotypical parents of autistic children, very cure-oriented, no autistic people involved in their decisionmaking, they spread harmful memes about nasty "treatments" and have a history of spending way too much time looking into vaccines on a cause. Most autistic people don't like 'em, is the short version. Googling "autism speaks criticism" or similar will get you lots of specificity.

0CharlesR13y

Thanks for the info. I didn't know they were anti-vaxxers.

[-]Bill_McGrath13y50

To answer your general method query, this essay by Karl Popper deals with the issue of distinguishing science v pseudoscience. However, from my reading of, you need to know a bit about the topic, or at least observe it in action, to make a judgement.

autism/language troubles in particular, are fields in which there is a lot of pseudo-science... There are a lot of mysticism and sect-like gurus related to autism, too.

What gives you this impression? I'm not saying you're wrong - just that it's something I haven't picked up on myself.

Regarding Freud, I get ... (read more)

5[anonymous]13y

Up-vote for Popper! Ask your friend: "How would you know if that treatment was not working?" And by 'not working' I mean (a) no change (b) a change for the worse (c) a change for the better, but not related to that treatment (d) a change for the better, but at a greater cost than benefit. The more clear the idea of how to know if that treatment was not working, the more likely it's science. I don't know you, your friend or your son's friend. I don't know what any of you need, although I'm sure just being compassionate instead of always being right is a part of it. My Popper-inspired suggestion above is good for distinguishing science from non-science. It may not be helpful to what any of you need.

[-]void13y40

Observe advocates. An idea only with stupid advocates probably is stupid itself. If you notice someone smart applauding it, look closer. In not popular ideas with few adherents it's probably best to examine the inventor. Don't look to supposed experts as you have no way of evaluating their knowledge.

[-]kilobug13y30

Thanks for all the answers and support; I wasn't looking as much for actual answers to my situation than to general method on how to dive into a new field, but I do appreciate them. Some of the advices were things I already was doing, but I'm glad to have confirmation I was on the right tracks. Other advices were new and interesting, I'll see how to try them.

I don't want to give too much details on a public site (it's not like my "kilobug" pseudonym is very stealthy, to start), so some advice don't apply to that specific case, but thanks anyway. ... (read more)

4satt13y

Possibly helpful older LW posts: * Scholarship: How to Do It Efficiently * Some Heuristics for Evaluating the Soundness of the Academic Mainstream in Unfamiliar Fields * What is bunk? * Scholarship: how to tell good advice from bad advice?

[-][anonymous]13y30

I'm very sorry to hear about your friend's son. For whatever it's worth, I think it's awesome you're taking the time to educate yourself in order to better help the child.

Do you have a name for his particular form of autism? You mention it is atypical, and the specific symptoms may be important.

Once you have the particular name, Google Scholar and the Journal of PLoS Medicine may be good places to begin your search. Do database searches for a review article on the particular form of autism.

Review articles are the best way to come up to speed on any given s... (read more)

[-]fiddlemath13y20

To try to answer the title's question, rather than directly answer the post's problem:

For the general problem of discerning pseudo-science from science, there's Massimo Pigliucci's Nonsense on Stilts. What I've read (and heard) by him seems like pretty sound stuff, but I haven't read the book itself. Does anyone have strong opinions about this book?

1pabloernesto6y

Could you add a brief summary of his ideas to your comment? Something like the "baloney detector" mentioned on this review of the book.

0Pablo13y

I haven't read the book, but I do have a strong negative opinion of Pigliucci. See, for instance, his intemperate, poorly argued critique of David Chalmers's talk on the singularity (cf. Michael Anissimov's analysis of that critique and Chalmers's response to that analysis). This is, of course, only limited evidence against the book, which might still be worth reading for all I know.

[-]siodine13y20

The next course of action is to actually move towards the experts in knowledge-space. Textbooks, expert blogs, wikipedia, classes, and so on are all viable methods for that.

But besides understan... (read more)

[-]Viliam_Bur13y10

I'm very sceptical of Freud and psychoanalysis

There is an ancient and noble tradition of burning a straw-Freud, which started during the Freudian analysis vs Jungian analysis and analysis vs behaviorism conflicts decades ago, and it is still used today to signal your allegiance to a specific tribe, usually either skeptical or religious, depending on context. On LW this tradition is honored during the winter solstice, too.

I would recommend against classical Freudian psychoanalysis in this case simply because it was developed for dealing with stuff like t... (read more)

6siodine13y

I assume by "scientifically proved" you mean well supported by the available evidence, in which case CBT has already attained that [edit: I don't mean specifically for autism; Villiam_Bur's comment leads me to infer that he's referring to CBT being a potentially useful therapy more generally]. And the reason Freud is so disparaged is because his methodology was at best proto-scientific and at worst speculation, and yet people still take him seriously. For that reason, I speculate the hostility ostensibly directed towards Freud is actually intended for his current supporters.

3Viliam_Bur13y

Actually, I wanted to say that there is no proof that CBT works for autism; but because it was proved to work for other things, I would bet on it anyway. I don't believe it could cure the cause, but I believe it could teach some useful behaviors to somehow compensate for the missing skills.

3Rhwawn13y

Perhaps I'm missing a point here, but when I look in Google Scholar there seems to be enough existing research on CBT & autism to say whether it helped or not.

3Viliam_Bur13y

Those articles seem mostly about CBT used to reduce anxiety and obsessive-compulsive behavior at autistic children. Yes, that's an area where CBT is successful, and it's a great news that it works for autists too. But to me it seems like it does not address the "essence" of autism (not that I know exactly what the essence of autism is), only fixes some symptoms. At the end, if everything succeeds, you will still have an autistic child; some of the problems will be fixed, some of them will remain. Yes, it's worth doing, just don't get your hopes too high.

Moderation Log