I'm curious about how others here process study results, specifically in psychology and the social sciences.
The (p < 0.05) threshold for statistical significance is, of course, completely arbitrary. So when I get to the end of a paper and the result that came in at, for example, (p < 0.1) is described as "a non-significant trend favoring A over B," part of me wants to just go a head and update just a little bit, treating it as weak evidence, but I obviously don't want to do even that if there isn't a real effect and the evidence is unreliable.
I've found that study authors are often inconsistent with this—they'll "follow the rules" and report no "main effect" detected when walking you through the results, but turn around and argue for the presence of a real effect in the discussion/analysis based on non-individually-significant trends in the data.
The question of how to update is further compounded by (1) the general irreproducibility of these kinds of studies, which may indicate the need to apply some kind of global discount factor to the weight of any such study, and (2) the general difficulty of properly making micro-adjustments to belief models as a human.
This is exactly the situation where heuristics are useful, but I don't have a good one. What heuristics do you all use for interpreting results of studies in the social sciences? Do you have a cutoff p-value (or a method of generating one for a situation) above which you just ignore a result outright? Do you have some other way of updating your beliefs about the subject matter? If so, what is it?
This is a good one. More generally, it's sometimes called the "Why" Regress. Not just about how you know something, but about how something happened or came to be. It applies equally to science and religion.
Edit: "...know you know" => "...how you know"
I like this idea, and I'd like to see it developed further. I don't see any reason why FGCAs shouldn't be catalogued and learned alongside logical fallacies for the same reasons.
I guess the important distinction would be that certain FGCAs can be used non-fallaciously, and some of these seem to have valid use-cases, like pointing out confirmation bias and mind-projection fallacy. Others are fallacious in their fully-general form, but have valid uses in their non-fully-general forms, so it is important to distinguish these. (e.g. pointing out vagueness or that something is too complicated and has too many dependencies for a given argument to have much weight.)
Great post!
I apologize for mentioning this, but there were a lot of typos in this, which made it a bit hard to read. I want to link this to a few friends who are not LWers, but when I am not familiar with the source of something, typos make me question the credibility of the author (they also provide an easy excuse to discount things people don't want to hear). I don't want that to happen when I show people, so I figured I'd help you out if you feel like cleaning it up a bit. Here's a quick list I put together for you:
I used to drink coffee every day, but I don't anymore. I just drink green tea in the mornings if I want something hot. I definitely don't think it's worth risking the benefits of your fast by using sugar or milk in your coffee. If I recall correctly, Berkhan's assertion that half a teaspoon (or whatever it was) of milk wouldn't cause a problem wasn't really supported by any science, so I would avoid it if possible. I think his reasoning was that your body would metabolize it super quickly and then return to a fasted state, but it's not clear if you'll retain the benefits of a 16-hour fast that way. I suspect it would also increase food cravings during the rest of your fast. And the increase in taste of your coffee is such a minor benefit that it's just totally not worth the risk as far as I'm concerned. There are better ways of making black coffee taste good if it's that important to you (see below).
I agree that the sweet taste of artificial sweeteners probably does something counterproductive. Overweight soda-drinkers who switch to diet soda have been shown not to lose weight. That's proof enough for me that they work some sort of mischief on your metabolism. All in all, it's probably not going to really cause a noticeable difference, but I feel like they're worth avoiding for general health reasons anyway. Starting a daily artificial sweetener habit as a part of trying to get healthier with an IF protocol seems counterproductive to me. I'd avoid them.
As I mentioned above, it's totally possible to make great-tasting black coffee. If you want to make your coffee less bitter, you might want to invest in an Aeropress. Bitter coffee is usually a result of the water being too hot and/or in contact with the coffee grounds for too long. Those both cause too much tannic acid to leach into the coffee. The Aeropress solves that problem. Also, switch to a medium-dark or dark roast. That will let you get the all the darkness, flavor, and caffeine you want without having to use water that is so hot or without having it in contact with the grounds for too long. Doing these two things will make a world of difference. You probably won't need sweetener if you do that. (I drink my coffee black whenever I have it and it's delicious.)
The last thing you could look into if you really just can't stand black coffee is just buying caffeine pills and using that in place of coffee. I've used caffeine pills a lot and they're actually really convenient. The caffeine is the main component of coffee that stimulates your bowels, so it should suit that purpose as well as provide the normal energy boost.
What would you imagine the criteria would be?
I had never heard of any of these except people putting magnets in their fingertips. Thanks for the post!
Minor typo I noticed:
"...and it is unique in that it is not implanted but instead." (instead what?)
I'll second the recommendation to relax this rule. I think the ability to gauge the quality of a popular book is a lot more cross-domain than with textbooks. I've read good books and I've read bad books. I can tell pretty quickly if a book is bad, even if I'm relatively new to the subject area.
Also, I feel like a lot of people would tend to only read one or two pop books in a particular area. Any more knowledge beyond that often comes from the internet or a textbook or elsewhere. I mean, I can count on one hand the number of specific subjects about which I've read more than two actual published books, but I've spent hundreds of hours each reading about many more subjects than that.
And since pop books aren't typically comprehensive accounts of an entire field or subject, the most important things really are clarity, engagingness, and worth, and not necessarily completeness. If what is there is valuable, accurate, and it's presented well, then it's Good, even if it doesn't cover some things that are covered by other books.
Thanks, I'll check it out.
Very informative. Thanks. I've heard reversible computing mentioned a few times, but have never looked into it. Any recommendations for a quick primer, or is wikipedia going to be good enough?
Thank you! This is exactly what I was looking for. Thinking in terms of bits of information is still not quite intuitive to me, but it seems the right way to go. I've been away from LW for quite a while and I forgot how nice it is to get answers like this to questions.