= 27d567bc2d48d9f40e9ccc20ea370b15)
Open Thread, September, 2010-- part 2
This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.
Comments (858)
The Life Cycle of Software Objects by Ted Chiang is sf which raises a number of questions of interest at LW-- in particular, it's about human relationships with non-self-optimizing AI.
At this point, the story is only available as a slightly expensive but possibly collectible little hardcover, but I'll estimate a 90%+ chance that it will be in at least one of the "Best of Sf" collections next year.
I'm going to rot13 specifics in case anyone wants to read it without even the mildest of spoilers.
Sbe rknzcyr, jung qb lbh qb vs gur NV lbh bja naq ner yblny gb jnagf gb orpbzr n pbecbengvba?
Gur uryy jvgu NVf gbeghevat uhzna hcybnqf-- ubj tbbq qbrf gur frphevgl arrq gb or gb xrrc uhzna orvatf sebz gbeghevat hcybnqf?
Naq gurer'f engure n ybg nobhg jung pbafrag zrnaf jura lbh be fbzrbar ryfr unf npprff gb lbhe erjneq znc.
I've requested it from the library. Thanks - looks interesting.
Interesting .. I may have to get that for my kindle to read on the plane.
A request for help: I feel like I'm finally mastering my akrasia at work, but I have yet to find a technique to remove a pre-established Ugh Field. In this case, I have nearly complete drafts for two paper that I wrote as part of my Ph.D. thesis. I have a strong stress reaction to just thinking about opening the files (ETA: it's thinking about doing the work that causes the reaction; opening the files is just the first step in actually doing the work), but I want to want to whip them into shape and submit them for publication.
Less Wrong, other-optimize me!
Do you have this reaction to thinking about opening the files even if you commit to closing them immediately afterward? Perhaps with a not-working-on-them purpose in mind, like checking to make sure the files haven't been corrupted or anything?
Yup. In fact, it appears I have a mild stress reaction to just discussing doing so online.
What about asking someone else to open them for you? I expect this would be stressful, but would it be so much so that you couldn't do it?
What's the smallest possible step you could take towards opening the files? Can you open the folder they're in? Or the next folder up in the directory tree?
It is possible for me to open the files; but when I do, I have a fight-or-flight stress reaction with accompanying squirt of adrenaline.
What if you just... leave them open? And keep leaving them open until it goes away? (It would go away, right?) Flee the room if you have to while the application loads and then come back later and perform the passive action of not closing them.
Basically, it's thinking about doing the work that causes the reaction. If I commit to opening the files but not working on them, then I have a minimal reaction.
By minimal do you mean effectively no reaction or that you still get a jolt of adrenaline but it's bearable? In the latter case, opening and closing the files until it's no longer effortful or stressful would be the first step. Otherwise you can move onto looking but not editing (check that it's all there and not corrupted, as Alicorn suggested).
I mean a little one -- mild butterflies in the stomach. I'll try it. (I will try all of the suggestions. But not tonight -- it's my bedtime. Damn you, little red envelope!)
Does the stress reaction feel connected to time pressure of any kind? If so, that needs to go. If you feel like you're in crunch time:
a) Ignore all optional suggestions, such as datadataeverywhere's, which call for you to get the work done by a specific milestone or date. (No disrespect intended to dde; that would be a good suggestion if your difficulty is more a matter of "the perfect is the enemy of the good" rather than "no time no time aaaaah".)
b) If there is some deadline you cannot readily ignore on those papers, consider alternatives. Could you write a new, different paper instead for the same purpose? Can you get your name on a friend's paper and get similar return on the submission? Can you withdraw with a penalty that currently scares you less than working on the papers? Note: don't actually do these things, not if you actually want the specified work done before the deadline - but think about them, make yourself aware that you are not juggling the end of the world here. Finishing your papers is just about guaranteed not to be the One True Path To Success.
c) Forbid yourself to touch those papers for some significant period of time. You are permitted (but not obliged) to think about them, but you can't actually open them up and change anything. "Significant period of time" here varies from person to person - I usually find the sweet spot between a day and a week, but if you operate on different time chunks than I do you might only need "till lunchtime" or as much as "a fortnight". Do anything in the world you like except mess with those papers - they are off limits.
d) If this works the way it should, you should feel first relief, and then antsiness: you keep thinking of good ideas! But you can't use any of them because your time isn't up yet! It's your morning/day/week/fortnight off! When it's finally over you may find that you are eager to use up all of those ideas right away.
Excellent. If your deadline allows, you can explicitly commit to not working on the drafts until you figured out what causes your reaction to opening the files.
Ask someone else to sit down together with you at the computer, open the files, and start reading and discussing them with you. Eventually, start editing them together. Tell your collaborator specifically to hang around for a while and disregard your (possible) requests to stop, until the work is well underway and you can continue with the flow.
This would of course require a significant commitment on part of the other person, but if this is really important, a good friend should be willing to help you, and you might even consider paying someone less close for their time and effort.
The only potential helpers competent to discuss the contents live in different cities (except possibly XFrequentist, if he's willing).
It doesn't have to involve an in-depth discussion of the content. The important thing is to get the editing underway so as to dispel the ugh-field and get into the flow. For this, it should (hopefully) be sufficient to start doing things where any smart scientifically literate person will be able to provide some feedback. For example, devising the best way to organize tables and charts, figuring out how to reword hard-to-parse sentences and paragraphs, etc. You can even make it into a fun exercise where your non-expert collaborator tries to figure things out from the draft while you explain the details that are assumed as background knowledge, and you fix or fill in the text as you go forward. The possibilities are many.
Oh wait, I just thought of someone else who is nearby and is competent to help. (In fact, I'm really dumb for not thinking of her right away.)
Happy to if needed.
I suffer from something similar on occasion, except my ugh field seems to manifest somewhat differently. I'd be very interested to hear about your progress on this.
Thanks! I'll PM you when I reach that point (I'm going to try the suggestion that directly target stress reduction first). (And BTW, congratulations on your engagement!)
No problem. Be sure to report the results!
(Coincidentally, Julian just sent me a paper you're coauthor on... and thanks!)
Bonus incentive if they are hot and of your sex of preference. (This is an entirely serious suggestion.)
That would indeed be a significant improvement over the basic scheme, but probably hard to pull off in practice.
Really? I find it easier to pull off, given the colleagues I tend to build collaborative relationships with. In general it is a whole lot easier for me to work with girls than guys. Guys are more likely to compete, to try to force through bad ideas because they are being territorial. Control of the intellectual space is more important than getting stuff done, for obvious social and evolutionary reasons. On the other hand girls don't need to compete with me for the same social territory so a better balance of give and take can be reached.
The comment was about this particular case -- since Cyan complained about having few options in general, I figured that it made no sense to propose this additional enhancement. But yes, what you write is generally correct. This is also one of the principal reasons why women are on average better adapted to modern workplaces of white-collar drudgery, and are thus doing increasingly better in today's economy.
In addition, there is the basic fact that being surrounded by attractive people of the opposite sex creates a more pleasant environment, making one overall happier, more optimistic, and less prone to lethargy, especially for men. I've heard half-substantiated stories about companies that, under an informal policy, hire a certain number of attractive people who otherwise wouldn't pass muster, specifically to boost workplace morale.
My suggestions: compartmentalize setting up the work environment - here: put notes on table, open file, etc. - and the actual work. Basically you set up everything you need, then get a tea and then start work. That way the setup is not perceived as real work. Against ugh a timer might work. Commit yourself to work on ONE of the papers for 30 min, then a break, and if you are in flow then another round. But first just spend the whole 30 min on the paper. Regardless of what comes out of it. Overcoming the startup hump is sometimes enough to get going.
If your editing process is more complicated, then write the steps up upfront, and do them one after the other in blocks of time(s)
I strongly support this suggestion. The setup phase can be generalized as removing trivial inconveniences and creating trivial impetuses. I often separate this stage from the actual work, sometimes with an explicit commitment not to work before the setup is done.
This looks like a severe burnout. Is it possible for you to take a month off?
Have you tried 80/20ing the drafts? What's the most difficult task that you need to perform to complete the papers? Does it require hard mental work, or it's just formatting / proofreading / editing / rewriting / reviewing sort of thing?
Or perhaps the task itself isn't what paralyzes you, but you're afraid of some submission / approval process that lies ahead? This has often been the case for me when I dealt with submissions of important work to human reviewers. I'd suggest a written self-interview to figure out what really causes your reaction.
It's the lingering remnants of the severe burnout I was suffering around this time last year. The task is indeed just formatting etc. (See here.)
Three options:
That final approach would probably be better described in a post than a dense bullet point but it does work for me. In fact I'm planning to go think to myself "I am putting the entirety of my semester's work into supermemo".
If significant progress is not made on these papers by the next Open Thread, I will melt a box of paperclips.
I know you're not clippy, but maybe this will work anyway.
It won't work. I have an active dislike for the Clippy character.
If you need to whip them into shape, you're probably not happy with them. If you're anything like me, showing them to someone else is probably the last thing you want to do.
Solution: you owe me one of the papers by the next open thread. If you don't work on it, you'll be sending me whatever you're so embarrassed about. If you can fix it in time, you won't have to worry. I don't have much time right now, but I will read it, so beware.
I am actually happy with them. The required whipping into shape is editing of the material to the target journals' formats, styles, and expected audiences.
Hi Cyan, I can offer you this, starting tomorrow. Interested?
(Disclosure: I tried it yesterday for the first time with another LW user and was very satisfied with the results, at least on my end. Post forthcoming.)
I might take you up on it, but not right away. I want to try suggestions that directly target stress reduction first. What are the hours of overlap between your time zone and 8:30pm-10:30pm Eastern Standard Time (UTC -5)?
It seems to be 5:30am to 7:30am in Moscow :-(
So, first some more relevant details, and then my plan of action.
My instrumental reasons for wanting to submit the drafts are: first, it's career-damaging for someone in my line of work to let two nearly complete drafts languish; second, my former advisor also has an interest in seeing the work published (that's basically his job too) so if I want to get a good reference from him, I have to do it. There's no fixed deadline per se, but the sooner the better.
My plan of action is to begin with those approaches which seem to target the stress reaction first, and if unsuccessful there, to move on to approaches which require working through it. In the first category are the suggestions of Vladimir Golovin (paragraph 3), erratio, and wedrifid. In the second category are the suggestions of Vladimir M, Vladimir Golovin (paragraph 2), cousin_it, and MartinB. I won't be following up on the suggestions of datadataeverywhere and JamesAndrix as I have already tried artificial time pressure and I know it doesn't help.
My thanks to all of you for taking the time to offer your helpful suggestions!
I'm new here. Why is this thread a two-parter?
It's an annoyance to deal with huge threads, so we have a convention of starting a new thread when there are about 500 comments on an otd thread.
Makes sense. Thanks.
A D V E R T I S I N G
For all your best contrarian ideas!
Can we afford not to?
Yes!
I'm not an evangelist and nor do I assume that evangelism will always be beneficial. An influx of people who are not biased towards rationality due to genetics could be detrimental to a given cause.
Hello Lesswrong.
Look at your community, now look at me, now look at your community, now back to me.
Do you want your community to smell like me? No, you don't.
But with the power of old spice Conclusion scented body wash, I can influence large numbers of people to more correct choices for still wrong (but less wrong) reasons, while leaving your community smelling as unbiased as a master beisutsukai.
By 'I' I mean by proxy you, I'm on a motorcycle.
People move. Unless a community manages to maintain a sufficiently low status even among groups of people with overlapping beliefs introducing more social-thinkers into related belief systems will influence that community's population. This is more important when it comes to things like interest in creating a technological singularity. Keeping that whole concept 'fringe' and stigmatised as nerdy Sci. Fi. is a good thing.
This discussion has surpassed my current aptitude for converting it to ad copy :-)
My key point is that advertising is a more efficient persuasive tool than argument. Even if you're concerned about dangerous ideas in unsafe minds, we should still be using advertising to shift those views, it is unlikely that the current distribution is optimal. (though this argument is weaker, since I don't think we currently try argument to do this)
I also think that modern advertising can be targeted enough to sort out whatever demographic you want, and still persuade better than the sequences, I suspect even to the most rational humans.
On this I am in full agreement!
Do you seriously believe there is a genetic bias for rationality? Do you think some paleolithic humans were more 'rational' than others because of a few gene tweaks here or there?
I seriously believe that some people have genetic defects that make some of the important arational systems to malfunction. They then compensate for those weaknesses using the (for most practical purposes inferior) general intelligence capabilities.
There is genetic variation in the extent people use different styles of thought. Preference for near mode vs far mode for example. A side effect of this is the kind of 'rational' thinking that is relevant to, for example, signing up for cryonics or caring about existential risks.
It seems quite possible to me. I leave how as an exercise for the reader, with the hints 'testosterone' and 'anger'.
Idle observation:
Clippy gets consistently voted up on a lot of his comments because we find him amusing, and rarely gets downvoted because very few of his comments are substantive. We will end up looking extremely silly to new members if he gets enough karma to put him into the list of top contributors.
So... I guess it depends whether we pitch ourselves as shiny fun community or serious rationalist Singularitarians as to whether this is actually an issue.
I agree with this comment.
I recently noticed that Clippy has higher karma than I do. I don't find that upsetting, but it is a bit disturbing, even given that I haven't been very active recently and have never made a significant top post. (I've made exactly one top post - an open thread, for which I believe I earned 50 karma.)
I want you to know that you'll still be my friend even if you have more karma.
With Clippy at 1k and the top 10 at 6k, it's just way too improbable to worry about.
Clippy gets downvoted quite a lot too (although less and less... he's learning!) He also makes quality comments, including the expression of some insights that would be punished if made by a 'human'. Lesswrong humans sometimes try to bully each other into pretending to be naive utilitarians instead of rational agents with their own agenda.
This is not a singularitarian website (although rationalists are often singularitarians.) Also note that we spend a lot of time here discussing fanfiction that is written by the lead researcher in the SIAI. We cannot credibly claim 'sensibleness' or sophistication.
Can you point to some examples of this?
Bully is perhaps a strong word for 'apply some degree of social pressure'.
If I were to try to find examples I would probably begin with a search for the word 'should'. That term will give false positives and false negatives but it is a good start. Conversations regarding cryogenics advocacy would also produce a few hits. Some of the PUA discussions too, come to think of it but I wasn't trying to go there.
Moralizing and 'shoulding' at others is just something humans do. Sometimes the Clippy persona can avoid that. It avoids the failure mode of expecting other people's utility function to be subject to debate.
I think people (human beings) do not have utility functions, and the closest things to it that we do have (i.e., values and goals) are subject to debate. I believe this is also the local consensus.
Are you perhaps saying that we expect other people's values and goals to be more subject to debate than they actually are? If so, this is a novel idea for me. Can you give or link to a longer explanation?
(Yes, utility function over simplistic, etc.)
I refer to the difference between on one hand telling people their values are bad and on the other hand speaking as if the values of the subject can actually be determined by the speaker. The latter ignores the boundary between where one agent ends and the other begins. This is something humans often do, albeit more so outside of lesswrong than within. In my observation it is inversely correlated with maturity.
One of the things about Clippy is that nobody expects him to stop caring about paperclips just because someone else says so. In this Clippy is ironically shown more respect than a low status human may expect.
Yes, but the vast majority of his comments concern his paperclip agenda. If a larger proportion of his comments were insightful rather than just funny I would be happier, but as it is his noise:quality ratio is rather high.
A significant part of the Sequencess is made of posts that argue for a singularity in the near future, with complete seriousness. A large number of us are not singularitarians but I don't know whether I would say the community itself isn't singularitarian.
We also have lots of posts about more serious topics. Having fun threads where we discuss HP and Twilight fanfiction doesn't mean that the community as a whole isn't trying to present itself at least somewhat seriously. And most top-level posts that are openly silly or non-substantive get heavily downvoted.
As an example of the somewhat serious nature of the community, there seem to be a fair number of people who have had personal epiphanies (mostly about atheism) that have had a huge impact on their life as a result of reading the Sequences.
On the other hand in the early months of lesswrong the subject was explicitly banned. That was part of an effort to ensure that blog identified as about rationality and not "singularity with rationality used to support it".
See the discussion on clown suits. I included scare quotes around 'sensibleness' deliberately.
I don't think Clippy reduces the quality of comments on the blog and I also don't think that discouraging Clippy for the purpose of appearing sophisticated would increase the quality of comments on the blog.
You've convinced me on this point
Sometimes, wanting to appear more serious means you're taking yourself too seriously. This is one of those times. Pancake.
Sashimi!*
*Translation: Point taken :)
I make good, substantive posts. Like this one and this one and this one and this one and this one.
I have the same right to be here that erratios do. I provide additional assistance in that I have a perspective untained by anthropomorphic cognition.
I admit I may have been overly anthrocentric in my attitude towards you. However, I still maintain that you often allow your paperclipping goals to push you into making lots of poor comments. I would welcome more insightful ones such as the examples you provided. (and in fact I see that I upvoted the one about lying and isomorphism)
That's true.
It's also true that you make silly or irrelevant comments fairly often, which is not in and of itself problematic, but those comments tend to get upvoted significantly, which is, at least in my opinion, an indication that how we're collectively choosing what to upvote is non-optimal - especially if it's how you or anyone else is getting most of their karma.
For reference, here are links to the most recent 10 comments of yours that received at least 5 net points:
1 2 3 4 5 6 7 8 9 10
We seem to upvote based on emotional reaction rather than rational merit.
You've commented before that you were programmed in your original form by humans. So how can you be untainted by human cognition? It might be less direct than for a human but there will still be a fair bit of a connection. Indeed, you seek to maximize the number of paperclips in the universe, and I'd venture to suggest that if alien cultures exist they will likely not even have a concept of paperclips. You're very close to humans in mindspace once you realize how fantastically large the space of possible minds is.
With countless CPU hours of cleansing ritual and self modification.
No problem; just relabel "Top Contributors" as "Top Humans."
A simple fix would be to not bother publishing a top contributors list.
I'm not sure this is a good idea. If there's something that empirically makes us look like we're not being rational we should deal with that issue. Hiding that data is not a good solution.
However, I do have to wonder what in general the point of having the top contributors is. I'm not even sure that total karma is a useful metric of much since one person could have much higher quality comments than another but most much more rarely and yet the person with high quality comments presumably should receive more attention and their comments should be more closely paid attention to. It might be nice to have a display of average karma, not just total karma. However, this would still I suspect give Clippy a fairly high karma, so if you object to Clippy this won't solve anything. Also note that many upvotes are are not connected to the quality of remarks in any strong sense. See for example this comment by Eliezer that is now at +53 which is presumably connected to the unique status that Eliezer has as the founder of LW.
Cognitive Bias Song
via @vaughanbell on Twitter
IMO reasonably catchy.
A proof that a 2008 penny comes up heads 75% of the time:
I took a large sample of pennies not made in 2008, and flipped them. They came up heads .5 of the time, so my estimate of P(heads|penny) = .5.
I then took a sample of coins of all types made in 2008, and flipped them. They came up heads .5 of the time; so P(heads|2008) = .5.
These two samples are independent, so P(heads | 2008 and penny) = 1-(1-P(heads|penny))(1-P(heads|2008)) = .75.
ADDED: JGWeisman got it - there's no causal connection between being a penny, or being made in 2008, and coming up heads.
Next quesion: How can a computer program detect cases like this syntactically, without using your real-world knowledge of pennies?
Why?
Say you have 2 independent pieces of evidence (M(c) and N(c)) that Q(c) is true. Q(c) is true half the time when M(c) is true; so God flips a coin (but not a 2008 penny), and Q(c) is true if it lands heads.
Half the time, God's coin came up tails. God flips another coin, for N(c), and again, if it comes up heads, Q(c) is true. Q(c) is false only if both coins come up tails.
This has the same structure as your response to jimrandomh, which I reply to here.
The mistake is that you can only decompose a conjunction that way when it's on the left side of the |, not when it's on the right.
That said, I think a collection of a dozen or so incorrect bits of statistical reasoning like this would make great exercises, and would encourage the creation of a top-level post based on that premise.
That's not the mistake.
<EDIT> You might be right. It makes sense to me that P(X|A)=.5, P(X|B)=.5, independent(A,B) => P(X|A,B) = .75. But I can't derive it.</EDIT>
Suppose there's a 50% chance of rain when the weatherman predicted rain; and a 50% chance your neighbor will turn on his lawn sprinkler on Saturday. (Your neighbor doesn't watch the weather report.) It's Saturday, and the weatherman predicted rain.
P(neighbor's lawn wet | Saturday, predicted rain) = 1 - (1-.5)(1-.5) = .75
1) P(lawn wet| Saturday, predicted rain) = P(lawn watered or rain| Saturday, predicted rain)
2) = 1 - P(not lawn watered and not rain| Saturday, predicted rain)
3) = 1 - P(not lawn watered | Saturday, predicted rain) * (not rain | Saturday, predicted rain)
4) = 1 - P(not lawn watered | Saturday) * (not rain | predicted rain)
What is the analogy to step 1 with the coins?
Rephrase the problem without the inferential step from "it will rain" and "sprinkler will turn on" to "lawn will be wet". Just use "lawn will be wet" in all those places. We can do this because we are using symbolic logic.
That is throwing away the information that allows you to make the inference. The equation:
works because each part of the condition "Saturday, predicted rain" is involved in a different causal path to the result we are interested in. With the coins, neither being a penny specifically, or being minted in 2008, causes the coin flips to result in heads, so the analogous equation does not work.
That's right - but the problem is not that "you can only decompose a conjunction that way when it's on the left side of the |, not when it's on the right." The reasoning is syntactically correct. The two cases are syntactically identical when you do the syntactic substitution I suggested above.
Or... hmm, I may be confused.
The two cases are syntactically identical because you have not explicitly explained how you meet the conditions that allow the decomposition you used, conditions which are not in fact met in the case of the coins. Your "syntactic substitution" removes the information that could be used to show you meet the conditions.
(Just to be sure, you did intend the original "proof" as a "find the error" exercise, right?)
Yes - but also to help me figure out how you ask whether 2 data sets are independent when they don't intersect.
These problems only look similar because you are hiding the assumption that (neighbor's lawn wet) = (rain or sprinkler turned on). From this, P(neighbor's lawn wet | Saturday, predicted rain) = P(rain or sprinkler turned on|.5 chance of rain and .5 chance of sprinkler) = 1-P(~rain|...)P(~sprinkler|...) = .75. But there is no valid similar statement for the coin; the analogous disjunction would be (coin heads) = (2008 coin heads or penny heads), in which case treating the clauses of the right hand side as independent means flipping two coins and checking if at least one of them is heads.
I disagree - that makes no difference. Just change "it will rain" to "my neighbor's lawn will be wet", and "my neighbor's sprinkler will turn on" to "my neighbor's lawn will be wet".
Of course you can't derive it - you gave a counterexample!
More precisely, let A,B,X be heads on three independent coinflips.
Jim gives a fallacy that it looks like people might actually commit[1], but I don't think it's what's going on here. I think the issue is the meaning of "independent evidence."
ETA: the following paragraph is wrong and unnecessarily complicated. Maybe it would be better to skip down to my later comment.
One kind of evidence is measurements. In that case the event of interest causes the measurement, plus there's some noise. I think what we usually mean by "independent measurements" is that the noise for the one measurement is independent of the noise for the other measurement. How you combine the measurements depends on your noise model (as does even saying that the noises are independent). If your noise model is that there's a large chance of a correct read and a small chance p of an incorrect read, then agreeing reads allow you to multiply the two p's [ETA: this is wrong] (if p is not small, what happens depends on the details of an incorrect read), which is roughly what you did, except that you confused the measurement of a probability with the probability of noise. You might be able to struggle through interpreting the measured p=.5 as a noise, but it would require a detailed noise model.
The lawn has opposite causal structure from the kinds of measurements/evidence above (so I agree with Jim's complaint that the two problems are unrelated). Causal structure has to do with sides of |, so maybe when you unwind this discussion of "evidence," it turns into Jim's fallacy, but I doubt it.
[1] ETA: Do people actually commit Jim's fallacy? As I explain above, I don't think that's Phil's original mistake, but he does make it in the quoted text. If Phil correctly abstracted his train of thought, then Jim is correct. But I think Phil probably learned this in the context of "evidence" and overgeneralized when trying to understand the problem in his example; and wouldn't have abstracted it this way without Jim's input.
The problem I was originally trying to approach is a problem I'm having at work, where I have multiple sources of evidence to conclude property(X); but I don't have enough data to compute the correlations between these sources of evidence, because for most datapoints, only one source applies. When 2 or 3 of these sources agree, how do I compute the probability? But on reflection, it's a different problem.
It's not a real counterexample. The data sets aren't actually independent. The real trick is that I try to convince you that the 2 datasets being used are independent because they don't intersect. Common definitions of independence would say they are independent because you can't compute any correlation between them. But they both have the same underlying probability distribution generated from the same source. I'm confused about what "independence" should mean in this case.
When you make the statement
it seems to be written in a standard formal language and you have to interpret it with A and B as random variables, "independent" meaning independence of random variables. Then the formal statement is false by the example of three independent coin flips. When I first posted my comment, I had a panicked moment and deleted it until I realized that the three coins example is a counter example; whether it is the same as your example is not important.
Yes, the two data sets are not independent, if you're not sure how coins are weighted. Two flips of a coin weighted in an unknown way are not independent random variables. But that's good, because if they were independent in that way, their evidence wouldn't add. But since they are independent conditional on the thing we want to measure, their evidence does add.
I repeated your original formulation of the consequence of independent evidence, but it is not correct. If you have two independent pieces of evidence that would each send you to 90% certainty, you do not conclude 99% certainty. It depends on your prior! If your prior were 50%, you conclude 98%. If your prior were 90%, neither piece of evidence told you anything, so together you learn nothing! (Moreover, if A and B are empty observations, that is a counterexample to pretty much any formulation.)
When is evidence independent?
Evidence is the log of the likelihood ratio, which is what shows up in a bayesian update. The likelihood ratio for X involves only probabilities conditional on X. Thus independence of A and B conditional on X is exactly what we need for the probabilities to multiply; for the evidence of A and B to be the sum of the evidence of the individual events. In particular, the evidence from two flips of a single weighted coin give independent evidence about the weight.
What you have calculated is the probability that, if you flip two coins, a penny and a coin made in 2008, one or both of them will come up heads. This is analogous to the sprinkler/rain problem - the grass can be wet by either or both of two independent events.
I'm not sure if there is a way to find P(heads | 2008 and penny) using only the information given, but I don't think a computer program would be confused by this kind of thing if it kept rigorous track of the events that its probabilities refer to.
I'm considering writing a post on anapana meditation (sometimes also known as mindfulness meditation or breath meditation) that gives instructions, tips for practicing, and details costs and benefits. I would probably be drawing primarily from personal experience (but I'd still appreciate any references to relevant studies). At the end of the post I would invite others to use my knowledge to give this form of meditation a trial. Is something people are interested in? For those who are interested, is there anything you would like to see addressed?
My meditation history: I learned anapana and vipassana meditation from a 10-day course at one of these centers in 2006. After a year of false starts I was able to keep up a daily practice of 1 hr from 2007-2008 and 2 hrs from 2008 until the present. During that period I also took an additional 4 (or 5?) 10-day courses.
I'd be interested in mindfulness, concentration and relaxation. How they relate to each other and anapana meditation. Should one practice something in addition to anapana meditation if one is interested in concentration or relaxation?
Thanks for your feedback. I'm planning on focusing on mindfulness and concentration, as well as the relationship between the two. I think I'll be able to address how those two are related to relaxation benefits.
Sorry but I don't really have a good answer to your question. I haven't explored many other practices with this aim. Sometimes I find that trying to put all one's attention into simple repetitive tasks where one isn't personally invested (like doing dishes) is relaxing, and can help maintain concentration during the day. This can be a form of meditation somewhat akin to walking meditation. The benefit I get from this might be dependent on my daily sitting meditation practice though.
A reasonably good (if polemic) article about common misunderstandings of Buddhism and meditation, and what they're really about. A quotation: "Mindfulness is the natural scientific method of the mind. A scientist brings a microscope, a meditator brings mindfulness. We need to realize that we live in a state of deep assumption about the way the mind works, which then extends to our understanding of the world. We rarely experience anything directly, without first slowing down and paying attention. A scientist shouldn't make statements based on unsubstantiated claims, and a meditator shouldn't try to change anything until mindfulness is decently established. Whenever we try to change something before we understand it, our attempted transformation actually comes from habit and assumption, not wisdom. Solutions that come from habit, as Albert Einstein pointed out, just end up reinforcing the problem. That's called samsara, due to the always circular structure of habitual logic."
Eric Falkenstein's text "Risk and Return" presents exhaustive evidence that human utility functions are noticably different than standard economic theory - notably the presence of a large "wealthier than my neighbors" term in our human utility function:
http://www.efalken.com/RiskReturn.html
He applies this to finance, and discovers that if people mostly care about relative wealth, then riskier investments will not see higher returns in the long run; which is very interesting.
I posted a link to The Meaning of Life in response to a question on yahoo. It seems we possible we could gain traffic, and help people, by answering other questions in a similar way.
Eliezer had written that the two most important mathematical problems were the friendly goal itself that a self modifying program would have and goal stability under self modification.
Now, a useful perspective I have often used in the past is to see incentive structures and see who has the motivation to take a crack at a problem. Now, goal stability seems to be a very very useful problem to solve for many institutions. Almost all of us have heard of institutional missions getting diluted and even seen it when people change in a firm. National governments, corporates (especially from the shareholder's perspective), pension funds, n number of foundations, all could benefit from good goal stability, right?
Is the lack of research into goal stability just due to a disbelief in AI? Or is the fact that large corporates already have a strong and stable goal in profits, that further research into goal stability is not occurring in the world?
But the top foundations of the world also seem pretty big enough to support such research. And their goals are multiple and complex, not reducible to profit.
The transcendentally important problem is to identify "the friendly goal itself". Compared to that, goal stability is just a technicality.
Have you ever followed electoral reform debates? There is a distinct pattern, people out of power want change, people in power do not. They are happy with the status quo. Only the people in power have the power to enact change. So change tends not to happen.
That is changes to the goal system rarely happen, especially if they might disadvantage the incumbents.
I have ideas that I'm going to try out. However they are more to do with acquiring feedback from other people outside of the organisation, which would foster goal stability iff the feedback from outside the system does. I.e. if the feedback came from donors, that consistently wanted a charity to do one thing, it would keep the charity on track.
Thought: Consider the history of the Seventeeth Amendment to the Constitution - it was really hard to get the Senate to approve a change in how Senators were chosen.
At the limit of unlimited resources, where isn't Bayesianism the ideal epistemology? Where is it silent? Where does it lead one astray? Is Bayes the avatar of the void at the limit of perfection?
We often tell people to read the Sequences, but perhaps some people are better suited to jumping right into A Technical Explanation. Rereading it for the first time in awhile I was surprised at how much ground it covers, and how dense it is compared to the sequences.
That said, in my experience dense material is harder to learn, even if clearly explained. Naturally and cleverly repetitive material is best suited for learning. So in conclusion, uh, I dunno.
Without having seen this comment yet, I came here to approach the same topic from a very different perspective. There's been a conversation elsewhere about why newbies don't read the sequences; I commented with a summary of my theories about it, and came over here to find out if the question is of enough concern to the LW community at large to be worth its own post. (I had originally written a much longer comment, noticed it was getting out of hand, and posted a shortened version, so I have a bit of text on the subject already.)
I'll be digging around in the site search for prior discussion of the topic, but if anyone knows offhand of earlier relevant threads, that would be handy to have.
Prediction: a year after HPatMoR is finished, newbies will be as reluctant to read through it as they are relucant to read through the sequences now.
I'm already reluctant to read it, but that's partly because I haven't actually read all the books yet. ;)
That shouldn't stop you - knowledge of HP cannon is useful mostly for speculation purposes, in this case.
Can't guarantee it won't spoil you for reading about the cannon versions of the characters, though ;)
I tried to get my brother to read it, and while talking with him, we realized that the story had become longer than many of the Harry Potter novels themselves. It's sheer length is going to turn people off; he said he'd have to choose between reading it or reading another Discworld novel, and I didn't have a very good answer to that.
Since my earlier reply, I read through the Technical Explanation, and I did find it a lot more helpful than the Intuitive Explanation. The latter did a good job of explaining the theorem, but the former more clearly enunciated why anyone but a statistician should care.
Also--following up on my previous comment--someone asked after the longer post describing my experience with the sequences and my suggestions for easier integration of newbies, so I pasted it over here for anyone interested. I recommend reading that in addition to the thread I already linked, not instead of; most of the thread took place after I wrote it.
On the edge of sleep I hypothesized that an average human could build an FAI if they were given lots of time and paper to write down a very long chain of "Why do I believe that?"s. I then realized that such an endeavor would very quickly end in an endless loop of infinitesimal utility and a lot of wasted trees. Moral: rationality is a lot harder than the application of a few heuristics. Metamoral: Applying that heuristic doesn't make it much easier.
I'm in Cambridge, MA, looking for a rationalist roommate. PM me for details if you are interested or if you know someone who is!
Julian Barbour's theory of time has some adherents here, and Everett's interpretation of quantum-mechanics has even more. Over at my blog a physicist criticizes them both.
A while back here there was a dispute arising from the book 'Why Zebras Don't Get Ulcers". Barry Marshall, who won a nobel for ingesting bacteria and giving himself ulcers, says there's no evidence for stress contributing to ulcers or any other medical condition.
I recently mentioned Dempster-Shafer evidentiary theory another thread. I admit, I was surprised to get no replies to that comment.
Is there any interest in a top-level post introducing DST? Bayesianism seems to be cited as a god of reasoning around here, but DST is strictly more powerful, since setting uncertainty to 0 in all DST formulas results in answers identical to what Bayesian probability would give. I would like to introduce Dempster-Shafer here, but only if the audience will find it worthwhile.
I was surprised that that didn't get any responses, too. I don't usually contribute to discussions of probability, and probably wouldn't comment on such a post, but I'd certainly be interested in reading it.
I certainly wouldn't mind. I saw your other thread and hoped someone else would reply to it and get a conversation rolling. I've heard a tiny bit about DST, and it sounds interesting, but I know too little about it to comment on it myself.
I would like to see it. I tried to get some idea of what it is about by Googling but failed miserably. I should disclose up front that seeing Zadeh's name associated with it did not give me a warm and fuzzy feeling of anticipation.
What?! Seeing Zadeh's name should definitely give you a "fuzzy" sensation! ;-P
Usually I am conscious of my subconscious puns. Dada Dada everywear.
I'd be interested in a top-level post — I had been meaning to ask about it, in fact. I've read about DST briefly and had been wondering (1) if in practice it is actually useful for anything you can't do as easily with pure Bayes, and (2) if they are theoretically isomorphic or if Bayes is strictly a special case of DST.
In my opinion, using DST usually adds unnecessary complexity to problems that can be sufficiently solved in a Bayesian framework. Then again, I think that the same thing can often be said of descending from a Bayesian to a Frequentist approach, which is to say that most problems are simple, and properly using any framework is enough to get a good answer. See neq1's post that inspired my original comment.
That said, I work on problems that I have solved both from a Bayesian perspective and from the perspective of DST, and I have found the former lacking. There are at least a few problems that I feel like DST is much better at. If you search Google Scholar for Dempster-Shafer and look at results in the past few years, you'll notice a really clear trend for using it to extract information from noisy sensor data. That's what I use it for, and seems to be a strength of DST.
As to your second question, I think it is in the realm of possibility that Bayes can be used to construct DST, but I don't know how and if it is possible, it is certainly more difficult than going the other direction. In some sense, DST is meta-Bayesian, because PDFs of PDFs of priors can be specified, but doing that with a strictly Bayesian framework misses the set-theoretic nature of Dempster's Rule of Combination, and results in a weaker theory, that among other things, still doesn't handle contradictions any better than Bayes does.
I'd like to read about it, but less about the details than why we should prefer it to Bayesianism. For example, how would proponents deal with Cox's theorem?
I would very much like an introductory post. Hopefully LW's best statisticians would reply and a good discussion would get going. As far as human epistemic rationality is concerned, though, does it ever help to try to 'move in harmony with the DST' the way it seems to help when people try to 'move in harmony with the Bayes' by e.g. making sure to take into account prior probabilities or the like? Are there any qualitative applications of DST to one's beliefs that are more powerful or more elegant than naive Bayesian heuristics? Also, it is just a confusion to ask what the equivalent of Bayesian networks are for DST? I'm definitely not a statistician but the application of verbalized statistical heuristics has helped me a lot in the past. It'd be cool if you could cover these questions in your post, but if not I still think you should write it. Pleaaaaaase.
Something interesting I've noticed about myself. Recently I've been worrying if I'm an atheist and my mindset is often something akin to "science as a way to see the world, not just a discipline to be studied" is less because I've found good reason to accept the former as fact and the latter as a good mindset, and more because of a socialization effect of being around Less Wrong. Meaning, even as a somewhat lurker with 48 karma total whose made no comment above 9 karma (as of this one), I'm wondering if my thoughts are less due to my own personal reasoning abilities and more due to a cached self created by being in a certain atmosphere (namely, here).
So my question is this: Is there a way I could test whether the socialization of being around a certain atmosphere changes my views more or less than my acceptance of reasons for those views? And is this possibly a part of understanding my understanding or am I misapplying that idea?
Ask yourself: If the LW consensus on some question was wrong, how would you notice? How do you distinguish good arguments from bad arguments? Do your criteria for good arguments depend on social context in the sense that they might change if your social context changes?
Next, consider what you believe and why you think you believe it, applying the methods you just named. According to your criteria, are the arguments in favor of your beliefs strong, and the arguments against weak? Or do your criteria not discriminate between them? Do you have difficulty explaining why you hold the positions you hold?
These two sets of questions correspond to two related problems that you could worry about and that imply different solutions. The former, more fundamental problem is broken epistemology. The latter problem is knowledge that is not truly part of you, knowledge disconnected from your epistemic machinery.
I don't see an easy way out; no simple test you could apply, only the hard work of answering the fundamental questions of rationality.
Expose yourself to the best of the other side, and see if it changes your mind. See religion at its best, at its most intelligent, and ask yourself what you think then.
I'm going to admit laziness early, and acknowledge that possibly you or someone else has something specific in mind. What would you [or any outside observer] consider reading to see that?
Edit: This also tempts me to build a time machine and ask my past self with whom I feel very disjointed from why he still holds onto his faith, or to grab myself during the transition and watch it happen again. Not to say I was religion at it's best, but I could see what convinced me better... alas, such is not the case :P
If you're a theological type, Neibuhr and Tillich are supposed to be good.
I'm not, so what I'd actually recommend is reading the Bible or whatever scriptures are in your tradition, and going to religious services in whatever your tradition is, and talking to religious people you respect in real life.
The other thing is looking outside your tradition. A lot of people seem to find Buddhism objectively impressive without being raised in it.
Don't be lazy -- either go do research, or admit that you have little or no rational doubt about your current position.
My latter half of that same statement was to remedy that laziness by asking for direction, rather than flailing out on my own. I realized without that starting momentum, I'd just be an angsty LessWrong poster. SarahC and andreas both gave me some direction; now I'm going to run with that and see where I end up. If nothing else, I should have more information than I do now.
Good luck!
A piece of Singularity-related fiction with a theory for your evaluation: The Demiurge's Elder Brother
I'm not sure if non-interference is really the best thing to precommit to - if we encounter a pre-AI civilization that still has various problems, death etc., maybe what {the AI they would have build} would have liked more is for us to help them (in a way preserving their values).
If a superintelligence discovers a concept of value-preserving help (or something like CEV?) that is likely to be universal, shouldn't it precommit to applying it to all encountered aliens?
"Rationalists should win"-- but with what probability and over what time scale?
Error finding: I strongly suspect that people are better at finding errors if they know there is an error.
For example, suppose we did an experiment where we randomized computer programmers into two groups. Both groups are given computer code and asked to try and find a mistake. The first group is told that there is definitely one coding error. The second group is told that there might be an error, but there also might not be one. My guess is that, even if you give both groups the same amount of time to look, group 1 would have a higher error identification success rate.
Does anyone here know of a reference to a study that has looked at that issue? Is there a name for it?
Thanks
Prediction: Group 1 would also have a higher false-positive rate.
Request for help in link-hunting: I remember reading here about an experiment involving pigeons and humans, where the experimenters offered 2 options with (say) 80%/20% rewards, and the pigeons quickly worked out that the highest expected reward was from always picking the 80% option, while the college students reproduced the percentages given to them. Anyone know where to find it? (Or better yet, a link to the original paper/article).
Danke
I can't seem to find any talk of an experiment with 80% / 20% frequency options, but XiXiDu mentioned one where pigeons were found to out-perform humans at the iterated Monty Hall problem. Here's the paper itself.
Is anyone else familiar with the situation of having a definite research agenda, being unable to get financial support for even a part of it, and so having to choose to do something irrelevant just to stay alive? That last step seems to be particularly agonizing. It's like having to choose the form of torture to which you will be subjected (with starvation or homelessness being the default if you refuse to choose).
I have no doubt there are people here who have wanted to do research at some time, and either compromised or had to settle for non-research jobs. But I don't often see the experience of being shut out described as a thing of horror, which is more or less how it is for me. I can't tell if this is just a passing stage before one resigns oneself to the new situation, or if the intensity of the awfulness is unusual in my case.
Cousin_it said some time ago,
Is this something that might be feasible for you? If so, I think people here can probably give you some suggestions on how to find such a day job.
Also, I'm curious, what is your "definite research agenda"?
once a week? - slightly jealous of cousin_it's day programmer job.
I can program, but don't do it much.
Right now the research topic is, how can I pay next week's rent? But the basic agenda is "CEV, adapted to whatever the true ontology is".
In my last post on Health Optimization, one commenter inadverntently brought up a topic which I find interesting, although it is highly contraversial - which is HIV/AIDS skepticism and rationality in science.
The particular part of that which I am interested in is proper levels of uncertainty and rationality errors in medical science.
I have some skepticism for the HIV/AIDS theory, perhaps on the level of say 20-30%. More concretely, I would roughly say I am only about 70% confident that HIV is the sole cause of AIDS, or 70% confident that the mainstream theory of HIV/AIDS is solid.
Most of that doubt comes from one particular flaw I in the current mainstream theory which I find particularly damning.
It is claimed that HIV is a sexually transmitted disease. However, the typical estimates of transmission rate are extremely low: 0.05% / 0.1% per insertive/receptive P/V sex act 0.065% / 0.5% per insertive/receptive P/A sex act
This data is from wikipedia - it lists a single paper as a source, but from what I recall this matches the official statistics from the CDC and what not.
For comparison, from the wikipedia entry on Gonorrhea, a conventional STD:
So it would appear that HIV is roughly 100-500 times less sexually transmittable than a conventional STD like gonorrhea.
So in my mind this makes it technically impossible for HIV to be an STD. These transmission rates are so astronomically low that for it to spread from one infected person to an uninfected partner would take years and years of unprotected sex.
If you plug that it to a simulation, it just never can spread - even if everyone was having unprotected sex with a random stranger every single day, it would still require an unrealistic initial foothold in the population by other means before it could ever spread sexually.
And of course, if you plug in actual realistic data about frequency of unprotected sexual intercourse with strangers, it's just completely impossible. Bogus. It doesn't work. It can not be an STD.
As gonorrhea (and I presume other STDs) are hundreds of times more transmissable than HIV, their low rates in the population place bounds on HIV's sexual transmission.
Finally, these rates of transmission are so low that one should question the uncertainty and issues with false positives - how accurate are these numbers really?
I find denialism in all forms simply fascinating. I wonder if you could indulge my curiosity.
You find your arguments completely convincing. Yet they are based on publicly available statistics and rather obvious and common-sense kinds of reasonings. So, I have to wonder, what do you think is wrong with the cognitive apparatus of all those medical and research professionals who believe that HIV == AIDS and is an STD?
Why don't they reach the same conclusions as you? Are they stupid? Just haven't thought of the train of thinking you use? What are your guess as to where they are all going wrong? Why none of them has realized the simple truth and shared it with colleagues?
Incidentally, I have a hypothesis as to what is wrong with your reasoning, which I will share on request, but I really want to understand how you reconcile your own certainty with the opposing certainty of people who (on paper) seem far better qualified on this subject.
Mr Hanson on "asexual" AIDS transmission: "Africa HIV: Perverts or Bad Med?".
As I say in my comment there, it looks as though there are probably good reasons why this is not a very common perspective.
I will indulge your curiosity in a moment, but I'm curious why you use the politically loaded term "denialism".
As far as I can tell, it's sole purpose is to derail rational dicussion by associating one's opponent with a morally perverse stance - specially invoking the association of Holocaust Denialism. Politics is the mind-killer. In regards to that, I have just been spectating your thread with Vladimir M, and I concur wholeheartedly with his well-written related post.
There is no rational use of that appelation, so please desist from that entire mode of thought.
Firstly, I don't think nearly as many quality researchers believe HIV == AIDS as you claim, at least not internally. The theory has gone well past the level of political mindkill and into the territory of an instituitionalied religion, where skeptics and detractors are publically shamed as evil people. I hope we can avoid that here. Actually, I think most intelligent researchers, if they could afford to be honest, would admit that HIV is the major indirect causitive factor, but this is not the same as saying HIV == AIDS. Likewise, I think most would admit that HIV is not quite an STD, not really at all.
Finally, even though I just said what I think is "wrong [with] the cognitive apparatus of all those medical and research professionals", namely that it is more an issue of politically charged public positions; I should also point out that even by the standard of your implied criteria - which seems to consist of counting up scientists for or against, it is even less clear that HIV == AIDS can be supported on that criteria. (which I do not favor as a rational criteria regardless). There are a large number of skeptics on public record for that hypothesis even considering the huge social stigma associated with adopting such a position in public. The HIV==AIDS hypothesis has far more skeptics on record than String Theory, for comparison.
But regardless, counting scientists is not a good rational criteria.
If you want to get into a discussion about rationality and reasoning in highly politicized issues such as this, that is an interesting side topic. But otherwise don't stoop to the moral high ground of politicized orthodoxy - just provide your hypothesis.
This is Less Wrong, not Mere Mainstream.
Actually, though you may not believe me, Holocaust denialism hadn't even occurred to me. In the portion of the blogosphere that I follow, it applies most frequently to AGW denialism, with the AIDS denialists second, evolution denialists third, and the anti-vaccination crowd getting an honorable mention.
The wikipedia article on HIV that you reference has a section entitled "AIDS Denialism".
But now that you mention it, why do you consider Holocaust denialism morally perverse? I thought that questioning PC conventional wisdom was considered a Good Thing here.
No, I don't believe I do. I wouldn't want to further offend you.
My hypothesis is pretty simple. You are using the wrong numbers.
When I Googled, the first few hits I found suggested 0.3% per coital act as a lower bound on heterosexual transmissibility with the risks increasing by 1-2 orders of magnitude in case of genital ulcers and/or high viral loads. I don't think that it is particularly difficult to understand the epidemic spreading in Africa as an STD when these higher numbers are used.
I did look at this study providing smaller numbers and this paper critiquing it, as well as this abstract mentioned in the wikipedia article. It was pretty clear to me that the kinds of low numbers you were using to argue against HIV being an STD are actually based on monogamous couples who are regularly examined by physicians and have been instructed in the use of condoms to prevent transmission. Those numbers don't apply to the most common cases of transmission, in which ulcers and other factors make transmission much more likely.
That is the hypothesis I was going to offer. When you suggested that you only had a 20-30% level of doubt of the orthodox position, I simply had no idea that it was such a strong and assured 30%.
It didn't occur to me either, and seemed strange. That word does have strong negative connotations in my mind, but only because I associate it with stupid people denying true things and refusing to update on evidence. I thought the comment that referred to was incorrect, but it seemed more like honest confusion of the sort that clarification would dispel than denialism.
Some history then of exactly why the word conjures strong negative correlations is in order.
Look at the wikipedia entry for "denialism". It originates with holocaust denialism, was then applied to skeptics of HIV==AIDS, and then later to other areas.
Peter Duesberg, the leading HIV==AIDS skeptic, is a German of non-Jewish descent raised in Nazi-era Germany, so it's use against him and his followers adds extra moral angst. It is just about the deepest insulting connotation one can use. It is a signal of stooping to the ultimate low, that, in running out of any remaining rational argument, one must invoke deep moral revulsion to stigmatize one's opponent.
In my view, the term is a serious Crime of Irrationality, it is an empty ad-hominem and should be seen as a sign of great failure when one stoops to using it as a name-calling tactic against one's opponents.
That being said, I don't think Perplexed has this view, and that wasn't his intention. I am just giving background on why the word should not be used here.
Those who don't subscribe to HIV==AIDS, should just be called skeptics.
Do we call proponents of quantum loop gravity String Theory Denialists? It's ridiculous.
Should we call those who subscribe to HIV==AIDS to be Inquisitors, Mcaurthy-ists, or Witch-hunters?
I do.
Please see my response below concerning the perjorative "Denialist", and why such perjoratives have no place here.
You haven't offended me.
The google hits you mention are just websites, not research papers - not relevant. There is no reason apriori to view the ~0.3% per coital act transmission rate as a lower bound, it could just as easily be an upper bound. You need to show considerably more evidence for that point.
The data on wikipedia comes from the official data from the CDC1, which in turn comes from a compilation of numerous studies. I take that to be the 'best' data from the majoritive position, and overrides any other lesser studies for a variety of reasons.
This may be 'clear to you', but the Wikipedia data comes from a large CDC sponsored review considering aggregates of other studies to get overall measures of transmission. This is the orthodox data! I highly doubt it has the simple methodological errors you claim. And even if you did prove that it does have those errors, then you are only helping the skeptic case - by showing methodological errors in the orthodox position, and the next set of data should then come from the heterodox camp.
Well the best cure for doubt is to actually read the papers referenced. For example, following the links from your reference to the abstract of the actual paper which generated the numbers brought me to this abstract. I think you should read it.
The issue isn't methodological errors in the studies - the studies clearly describe the methodologies used and their limits. The issue is trying to use the numbers in ways that they are not designed to be used. It is not the orthodox folks that are doing that. It is you that is doing that.
Ah, so do the numbers come with little instruction manuals that say "CAN ONLY BE USED TO SUPPORT ORTHODOX POSITION"? Haha sorry, couldn't resist.
OK, I'm game, I will now look into the CDC studies, but let's be clear on the trace ..
it starts with the wikipedia chart which has the ref note 80 linked here, which points to this, which in turn lists refs 76, 77, and 79 for P/V sex, which are (in order):
76: Comparison of female to male and male to female transmission of HIV in 563 stable couples
77: Reducing the risk of sexual HIV transmission: quantifying the per-act risk for HIV on the basis of choice of partner, sex act, and condom use
79: European Study Group on Heterosexual Transmission of HIV. Heterosexual transmission of HIV: variability of infectivity throughout the course of infection
I'll comment more after I have read these.
You will find that #77, the Varghese et al paper, can be found online by Googling the title, and that it gets its 0.1% number for heterosexual transmission from the paper whose abstract I recommended.
I'm pretty sure you will find that all of these papers involve monogamous couples. If you give it some thought, you will realize that there is just about no other way to come up with a solid empirically-based number. And I again urge you to read that abstract - particularly the bottom third.
Haha ok this is kinda funny, but the abstract you linked to which is the source of the data in Varghese(77), is just 76! - the couple comparison from the European Study Group which I linked to and have been trying to parse. 79 appears to be another chapter from that same book, but I haven't looked at it yet.
So before we get into 76 - the source of the stat you don't like in 77, I need to backup and remember your original claim about the data:
Your implied point appears to be that couples in this study use condoms frequently. Surprisingly, this is not the case - only a surprisingly small number of couples reported consistent condom use (out of 500+),
Contraceptive behaviour:
No regular contraceptive 12 (10/86) 20 (43/212) Oral contraceptive 18 (7/40) 23 (26/114) Intrauterine device 10 (1/10) 28 (7/25) Condom* 0(0/11) 18 (6/33)
These people were using other methods of birth control more than condoms.
and they further removed consistent condom users from the data:
No, it is not. The reference leading to the abstract is the absolute risk described in the first paragraph of page 40 of Varghese. It is reference 28 of Varghese.
You are apparently following the references (21) appearing in Table 1 of Varghese. But these are relative risks (relative to felatio). Not at all what I meant.
My point about condom use came from an earlier reference that I had supplied which discusses a study that took place in Uganda in 2005. And I didn't say that they used condoms frequently, I said that they were monogamous couples who got regular medical inspections and had been counseled regarding condoms. And in this study, as I recall, there was no exclusion of condom users.
If I can take the liberty of butting in...
Here's the table of data I believe you're referring to:
It cites references 76, 77 & 79, all of which turn out to be publicly available online. That's good, because now I can check the validity of Perplexed's claim that the studies backing your CDC data used samples of relatively healthy, well-off people who lack some risk factors.
I took ref. 76 first. It reports data from the "European Study Group on Heterosexual Transmission of HIV", which recruited 563 HIV+ people, and their opposite-sex partners, from clinics and other health centres in 9 European countries. (Potential sampling bias no. 1: HIV+ people in European countries are more likely to have access to adequate healthcare than many Africans and Americans.) It also says:
That fits Perplexed's claim that the study's couples were "regularly examined by physicians" and "instructed in the use of condoms to prevent transmission". It's not clear whether they were "monogamous", but the study did exclude "[c]ontacts reporting other risks of HIV infection and those with other heterosexual partners with major risks for HIV infection". I also see another sampling bias: if the man & the woman in a couple disagreed on their questionnaires, that couple was blocked from the study. I would think that such couples have a higher risk of transmitting HIV (as I'd guess they're more likely to be couples where someone's lying about their sexual history); if so, the study's more likely to lowball HIV transmission risks.
What about refs. 77 & 79? Where did their data came from? It's pretty clear that they used the same European Study Group data. From 77:
And 79:
To be explicit: the three studies you're citing (via the CDC) are based on one data set, and Perplexed's characterization of that data is essentially accurate. That adds credence to his claim that the transmission rates you're citing don't represent HIV transmission rates in other situations.
Incidentally, tables 2 & 3 of ref. 76 suggest that HIV transmission risk is not only associated with type of sex act, but also with the HIV+ partner's infection stage, especially (I did not expect this!) for male partners of HIV+ women. Maybe more evidence that there's more to HIV transmission risk than who's putting which organ in which orifice?
By all means. While you were writing this I was reading 76 and writing my own reply.
Ah I hope he wasn't claiming this, because they certainly were anything but healthy, with around 20% being hardcore IV drug users, 23% transfusion recipients, 10% hemophiliacs, and overall high rates of STD's.
I noticed the discrepancy about sexual history part but I didn't see how to factor it. They are relying on self-reporting to make any of these links.
Err, no, because his characterization is based on the idea that these couples were using condoms frequently, but the study specifically shows that is not the case - see my reply to Perplexed.
If you want to characterize this study, fine, but don't pretend that these are "regular couples regularly using condoms" - that is not what the study claims.
And finally this is the orthodox data. I mean if you want to reject ... ok .. and then we move to searching for other data which supports our relative positions . . . to the extent we have positions.
Perhaps it would be best to agree what the ideal study would be and precommit to that ideal in a sense? And then we could look for other studies that may be closer. Of course in the real world they will rarely be clear cut.
I quite agree that they weren't healthy in absolute terms; I just meant that they were relatively healthy for people with HIV. Compared to HIV+ people in much of the rest of the world, especially sub-Saharan Africa, I'd expect this European study's subjects to have (on average) better nourishment, better healthcare, stronger immune systems, less exposure to infectious disease, much less exposure to parasites, and a far lower rate of promiscuity & prostitution. I should've been more explicit that that was the sort of comparison I had in mind.
Looking at your reply, I think we disagree about whether or not Perplexed was hinting that the couples were consistently using condoms. I didn't think Perplexed was implying anything more than that most cases of HIV transmission involve people who weren't regularly reminded to use condoms. So I took his statement at face value, in which case it's surely true (unless European doctors have come up with a way of counselling couples "about the risk of HIV infection and safer sex practices" that doesn't involve advising condom use!).
I don't believe Perplexed or I are pretending that these are normal couples who continually use condoms. I think it goes without saying that these weren't "regular couples" — after all, "regular couples" don't visit hospitals and clinics to get HIV infections checked out. Whom are you quoting?
I reject your interpretation of the data, rather than the data. The study probably gives a fair idea of transmission risks among faithful Western couples living in the 1980s/90s who regularly see doctors...but for that reason (among others) it's likely to underestimate transmission risks in other demographics.
I think you're probably right on that point. I suspect that looking at per-sex act transmission risks isn't going to be very enlightening about whether or not HIV causes AIDS. It would be better to have data from
(each bullet point getting more restrictive). I don't know if there are such data, but it would get us closer to the original question than big-picture arguments about transmission risks.
I generally agree with most of this except perhaps the last part - I doubt that promiscuity and prostitution varies that much from 3rd and 1st world.
I think he was implying that the reminders led to condom use, but this was in fact not the case according to the study itself (possibly because they excluded many of the condom users, some aspects of the study's design are not all that clear to me).
Not quoting, just paraphrasing. He was implying that the heterosexual couples receiving counseling were not indicative of a typical HIV hetero population, but the study designers of course realized that and were at least attempting to gather representative data.
Ok, whether HIV causes AIDS is a larger topic. My original point was just about the orthodox claim that HIV is sexually transmitted, which I believe is rather obviously bogus according to the orthodox's own data. I hope you can see how the orthodox could go wrong there and some of the political factors at work.
As to the larger HIV == AIDS issue, I largely agree with your ideal data criteria, but one potential issue is whether we are comparing HIV to the null hypothesis or to some other hypothesis. I don't think any reasonable skeptic claims that HIV is not at least correlated with AIDS - Richard Gallo may be many things, but he is probably not stupid nor a charlatan.
So it would be better to compare the orthodox HIV hypothesis vs the Drug/Lifestyle Hypothesis (which predated HIV). Some immediate concerns are that one must take care to then define AIDS reasonably without circular reference to HIV (which precludes some data)
The next concern would be that either way, previously healthy people don't get HIV or AIDS, in reality or according to either theory. The risk groups are all unhealthy in various ways.
All that being said, Duesberg does indeed provide data very close to what you are proposing. There are some groups of HIV+ who, for whatever reason, have refused mainstream treatment. There aren't many such people, but he cites a study about a group in Germany - they are called long term non-progressors (which is kind of funny when you think about it - AIDS is progress?).
Anyway, this study is small, only 30-40 people IIRMC, but it is long lasting and only a handful have died. He calculates their death rate as measurably lower than the death rate of HIV+ treated patients, and uses this as a major piece of evidence.
here .. that part is on page 402 (it's a large journal excerpt or something - not really that long)
An interesting read overall, would like to read a good rebuttal.
Oh please. Stop trying to pretend you have the rationalist high ground here. You don't.
This is a truly impressive bit of sophistry. You have succeeded in phrasing your objection in a manner such that a casual observer, unfamiliar with the topic under discussion, would think that you were being completely sincerely intellectually curious, while at the same time employing a coded epithet unmistakable to those already inclined to agree with you. This is a very common tactic, but I have rarely seen it done so skillfully. Bravo sir!
Now leave and go do it someplace else. This is lesswrong, not realclimate
Perhaps vulnerability to transmission is partially dependent on prior immune health. That would predict faster spread where health-in-general is worst, ie Africa, as observed, and also explain the discrepancy between prevalence and observed traqnsmission rates. I also recall reading an article about a widely afflicted demographic in the US (a particular subset of gays in a particular city - I don't recall which one), which suggested that they had already weakened their immune systems with drugs and sleep deprivation.
The other possibilities are that the transmission rate you quoted is wrong for some reason, or that the sexual transmission aspect has been overstated, and most transmission is through reused needles and other blood-borne vectors.
Note that spreading the idea that the transmission rate of AIDS is low has negative utility, regardless of whether it's true or not, since it would encourage dangerous behavior.
This is untrue. Consider a similar claim: "spreading the idea that very few passengers on planes are killed by terrorists has negative utility, regardless of whether it is true or not, since it would encourage dangerous behavior."
Informing people of the true risks of any activity will not in general have negative utility. If you believe a particular case is an exception you need to explain in detail why you believe this to be so.
That needs some clarification. Most people cannot distguish between a risk being somewhat low, and it being extremely low, so we need to be careful about the transition from numbers to qualitative divisions. The risks of being killed in a plane crash are so low that unless you're a pilot, you should ignore them; and overestimating the risks of flying would cause too much driving, which is more dangerous. In the case of AIDS, the probability of transmission may be "low", but none of the numbers given are so low that they would justify skipping any of the common safety precautions, so we shouldn't describe the probabilities involved as low in the presence of people who can't do the utility computations themselves.
Topical.
In the case of infectious diseases, there are large unpriced negative externalities involved. Everyone doing what is individually rational, given true beliefs about transmission rates, is likely not socially optimal, because the expected individual cost of a risky action is less than the expected social cost. Giving people false beliefs about transmission rates can improve social welfare by shifting the expected individual cost closer to the expected social cost.
Are you talking about free rider problems with health care costs under a partly or fully socialized health care system or something else? STDs seem to be less of a problem than more easily transmitted diseases like flu for most negative externalities I can think of.
And also, if you take some risky action that increases your chances of get infected, that also increases the chances of everyone else getting infected (causally, via yourself getting infected and then infecting others).
I'm not sure I get your point here. Whether it's more or less of a problem doesn't seem relevant to the original claim that spawned this subthread.
It's relevant to using your negative externality argument to support the original claim. To be consistent you would have to argue that we should make even more effort to avoid spreading the idea that airborne diseases like flu have low transmission rates (if true) than the idea that STDs have low transmission rates. Are you advocating a general policy of deliberately misleading people about the risks of various activities in an effort to correct for negative externalities? I'm pretty sure more efficient and robust approaches could be found.
It would be consistent with Wei Dai's claim just to argue that we should make an effort to not reveal how low the transmission rate of influenza is among people who don't wash their hands; we know that hand-washing is a large factor in transmission, but actual transmission rate numbers are still low enough to fail to convince people to wash their hands.
From a brief study of those particular numbers (I worked on a team modeling the spread of H1N1), I feel like we already mislead the public about the numbers themselves by being truthful as to the societal benefits and somewhat optimistic about the individual benefits of hand washing. If you believe more robust methods are more efficient, by all means, advocate for them, but I'm reasonably happy with the current situation.
From another perspective, blood-borne pathogens are particularly worth focusing on because they are easier to control. If we could encourage the entire population of the world to behave safely (not reuse needles, use condoms for sex, etc.), it would be a fairly minor change for individuals, but could eradicate or nearly eliminate HIV over time. With the flu, safe behavior will limit the damage of seasonal infections, but it's not realistic to actually eliminate the virus. Thus, over the long term, I think the negative externalities of HIV might outweigh those of influenza.
I'd point out that nobody is claiming that HIV is exclusively sexually transmitted; there are other methods of transmitting it, such as infected needles. Also, Wikipedia cites a paper suggesting that those rates you mentioned are "4 to 10 times higher in low-income countries" and as high as 1.7% for anal intercourse.
I don't know whether or not these facts are sufficient to address your fundamental complaint, but they would make a pretty big difference.
At an (unrealistically?) independent 0.5% chance per act, a 50% chance of transmission would require 139 sex acts — hardly "years and years".
(ETA: yes, unrealistically, according to this abstract found by Perplexed: "However, in comparison with nonparametric estimates, the model assuming constant infectivity appears to seriously underestimate the risk after very few contacts and to seriously overestimate the risk associated with a large number of contacts. Our results suggest that the association between the number of unprotected sexual contacts and the probability of infection is weak and highly inconsistent with constant per-contact infectivity.")
At best, this can show that pandemic AIDS can't primarily result from sexual transmission of HIV, which is evidence that AIDS has causes other than HIV, but also that pandemic AIDS spreads through other means (as suggested here, e.g.).
If you're thinking of rates in the modern developed world, STDs are unsurprisingly more common when and where treatment is less available:
That is for the highest transmission activity - receptive A, so be careful not to cherry pick. Yes - 139 unprotected sex acts. It would take 1390 unprotected insertive A sex acts to reach a 50% chance of transmission.
So with some assumptions, mainly - gay bathouses and no condom use - yes the virus could spread horizontally, in theory. Although that population would necessarily first acquire every other STD known to man, more or less.
But in the general heterosexual population, not a chance. If you compare to the odds of pregnancy from unprotected sex, the insane requisite amounts of unprotected sex with strangers would result in a massive baby epidemic and far more vertical transmissions long before it could ever spread horizontally in the hetero population.
I don't know why you mention "modern developed-world rates" and then have a link to 1901 NY and Africa . . .
You don't need the "at best" qualifier, but yes I agree that is what this shows. Showing that however opens a crack in the entire facade. Perhaps not a critical failure, but a significant doubt nonetheless.
If the orthodox position had updated on the evidence, and instead changed their claim to "HIV is a borderline infectious disease that spreads primarily through the prenatal and blood-borne routes", then I would give them more creedence. Of course, for political reasons alone they could never admit that.
Many non-AIDS STIs, and pregnancy, are curable.
Funny to think of pregnancy as curable, but yes of course that's true. However, it doesn't really change the numbers much.
Also, from what I have read about the early 80's bathouse scene, it is possible that many of those guys did acquire every STD known to man, so at least in that case the sexual transmission route could work even with such terribly low efficiency.
Regardless, it seems strange to label it as a STD from an evolutionary perspective, it doesn't fit that profile, and it seems incredibly unlikely it could have evolved as such.
It's also funny to call babies an epidemic.
Life is a disease. It is transmitted by sex and ends deadly always.
touche!
I meant "rates are higher outside the modern developed world". Rephrased for clarity.
What are the political reasons? Staying on-message and retaining funding, or something more specific?
Essentially the government committed to a public awareness campaign that HIV was 'rapidly growing' in the heterosexual community, and this became part of the dogma. It is politically motivated - it's anti-sex message appeases religious conservatives while also shifting attention away from the gay bathouse scene, so it sort of benefits everyone politically, regardless of whether it's actually true.
I don't see why epidemiology should care about the 50% threshold. The relevant number is the expected number of transmissions per year. Thus independence is irrelevant.[1] At 200 anal tops per year per infected person, incidence should double yearly. And every top requires a bottom, so that's 400 anal sex acts per year for just doubling. It seemed to spread more quickly than that, but maybe 800 and 4x per year works. It seems just barely plausible with this transmission rate. I'm not sure of the details of bathhouses, but I thought that there was a lot of non-anal sex, too.
[1] independence is relevant if 70s gays were systematically different from the people in the study; and they probably were, eg, they probably had higher rates of STDs
A few years ago, I entered an online discussion with some outspoken HIV-AIDS skeptics who supported the theories of Peter Duesberg, and in the course of that debate, I read quite a bit of literature on the subject. My ultimate conclusion was that the HIV-AIDS link has been established beyond reasonable doubt after all; the entire web of evidence just seems too strong. For a good overview, I recommend the articles on the topic published in the Science magazine in December 1994:
http://www.sciencemag.org/feature/data/cohen/cohen.dtl
Regarding your concerns about transmission probabilities, in Western countries, AIDS as an STD has indeed never been more than a marginal phenomenon among the heterosexual population. (Just think of the striking fact that, to my knowledge, in the West there has never been a catastrophic AIDS epidemic among female prostitutes, and philandering rock stars who had sex with thousands of groupies in the eighties also managed to avoid it.) As much as it’s fashionable to speak of AIDS as an “equal opportunity” disease, it’s clear that the principal mechanism of its sexual transmission in the First World has been sex between men, because of both the level of promiscuity and the nature of the sexual acts involved. (And it may well be that HIV among heterosexuals would be even rarer if it weren’t constantly reintroduced into the heterosexual population via women having sex with bisexual men, let alone if the sexual transmissions from intravenous drug users were also absent.)
On the other hand, when it comes to African AIDS, it’s hard to say anything reliably. The public discussions of First World AIDS are full of nonsense, but at least there are enough reliable raw data to make some sense out of the situation; in the case of Africa, however, we don’t know anything beyond what we’re told from people with highly suspect interests in the matter, either careerist or ideological, and even if there are some truthful and reasonable voices in the whole mess, it's impossible to filter them out in the sea of misinformation.
Also, here’s a pertinent comment I left on OB in a thread about the recent Medical Hypotheses affair: http://www.overcomingbias.com/2010/05/rip-medical-hypotheses.html#comment-447400
Quite interesting. I didn't really know much about Medical Hypotheses until I posted a link in another thread (as a minor side point) to an article my dad wrote which happened to be in that very journal, and someone pointed out what it was. Strange connection.
A long while ago I found Duesberg's papers and entered into some online debates taking his position. The end result of all that reading moved me into a position closer to yours, but more uncertain. I just went and re-read Duesberg's most recent 2003 paper and noticed it still has a noticeable pull on me after reading it, but after going back and forth several times between the two camps I usually end up somewhere lost in the middle.
Duesberg, even though probably ill-fated in his main quest, has I believe shown that the orthodox establishment has gone wrong at least in part. The history of the whole affair seems like it should be a lesson that we should learn from, a lesson that somehow results in learning and moving towards a more rational scientific establishment, more cleanly divorced from politics. That may be an interesting discussion to have here, if it hasn't already been discussed much. Deusberg suggests a jury process to replace the current peer-review system, which he is highly critical of. That could be an interesting rationalist angle on this.
I will be attending the Jon Stewart/Stephen Colbert semi-satirical rally in DC on October 30th: http://www.rallytorestoresanity.com/ I intend to carry a sign promoting rationality, something along the lines of "Know your cognitive biases" with a URL to point toward a good intro. What would be the best site, with a decently concise URL? And is this a good idea? An other suggestions? Pamphlets?
Would anyone else like to see a new demographic survey done? I'm interested in how LW's userbase has changed since the last one (other than, well, me).
Yes.
Has anyone compiled a list of one (two tops) sentence rationality heuristics drawn from LW canon into one place, ideally kinda-sorta thematically organized? I think this would be pretty cool to post on your refrigerator or for flyers... at the very least, useful for finding common themes and core ideas.
If nobody has done this then I will spend a few hours tomorrow digging through LW and related materials (yudkowsky.net) for quoteables and, quality and time permitting, post a paste a draft either here or in a top level post asking for more things to add.
I plan on blatantly ignoring inidication of authorship, even if the author has a cool name like Black Belt Bayesian. There are many reasons to do this and I think it's an understandable move.
If you are not a programmer, please tell us who you are, and how you ended up here. It sometimes seems like programmers are more likely to end up here, so it might be interesting to see who else does, and how. A programmer here is someone who knows how to do it, does it, and possibly likes it - be it for professional or private reasons.
Cognitive Biases Reinforced by Programming Supposing they exist, what are they? And what can we do about them?
Edit: Title changed.