Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

In response to Belief Chains
Comment author: 27chaos 15 November 2014 05:24:37PM 0 points [-]

I think the examples you use are overly political and may make some readers feel uncomfortable.

In response to comment by 27chaos on Belief Chains
Comment author: ShannonFriedman 15 November 2014 09:00:12PM *  1 point [-]

When talking about the impacts of complex systems, it is useful to pick one that people know, so as to not have to spend a whole lot of words giving background explanation.

I could not think of an example to use for this that was not at all political. I do not think it being slightly political outweighs the value of the description.

Do you prefer only examining elements on a small enough scale that you can get close to perfection in comfort and lack of error (what margin of error is acceptable to you since perfection is generally not actually achievable?), or do you prefer to consider some things that are uncomfortable if there might be a high pay off in return for examining these areas that may be useful for improving your rationality skills?

In response to Belief Chains
Comment author: ChristianKl 15 November 2014 05:07:29PM 5 points [-]

The FDA does extensive research

In most cases it doesn't. In most cases the FDA tells companies to do research and then show the FDA the results of that research so that the FDA approves the product.

In response to comment by ChristianKl on Belief Chains
Comment author: ShannonFriedman 15 November 2014 08:55:00PM *  0 points [-]

Okay, so FDA is one step removed, and is reviewing the research rather than doing it themselves.

Belief Chains

8 ShannonFriedman 15 November 2014 11:09AM

A belief is an acceptance that a statement is true or that something exists.   As aspiring rationalists, we strive for our beliefs to be true, accurate, and minimally biased.     

You seldom see a single belief floating around.  Typically beliefs tend to group into clusters and chains.  In other words, if I believe that I am turning my thoughts into written words right now, that is not an isolated belief.  My belief chain might look something like this:

I have sight ->  The image coming into my eyes is of something that is metallic with bright lights and little boxes -> It is similar to things that have been called “computers” before -> I am wiggling my fingers to make patterns ->  this is called typing -> I am typing on a computer -> the words I am thinking are being translated into writing.    

Why does it matter whether I see my beliefs as chains or whether I simply look at the highest level belief such as “the words I am thinking are being translated into written word”?

It matters because at each link in the chain of belief, there is potential for falsehood to be introduced.  The further I am away from the source of my high-level belief, the less likely my high-level belief is to be accurate.   

Say for example that a three year old is typing on their toy computer that does not have the standard typing functionality of my computer.  They could still have the same logic chain that I used:

I have sight ->  The image coming into my eyes is of something that is metallic with bright lights and little boxes -> It is similar to things that have been called “computers”  before -> I am wiggling my fingers to make patterns ->  this is  called typing -> I am typing on a computer -> the words I am  thinking are being translated into writing.    

Belief chains can be corrupted in many ways.  Here are a few:

1.   Our intuitions tell us that the more interconnecting beliefs we have, and the more agreement between different beliefs, the more likely they are to be true, right?  We can check them against each other and use them as confirming evidence for one another.

These interconnections can come from the beliefs we have accumulated in our own minds, and also from trust relationships with other people.  We use interconnecting beliefs from other people just as we use interconnecting beliefs in our own minds.  While not good or bad in and of itself, the down side of this system of validation is how we fall victim to the various types of groupthink.  

This is easiest to talk about with a diagram.  In these diagrams, we are assuming that truth (yellow T circles) comes from a source at the bottom of the diagram. Beliefs not originating from truth are labeled with a (B).   As aspiring rationalists, truth is what we want.   

What is truth?

Truth is a description reflecting the underlying fundamental structure of reality. The reality does not change regardless of what perspective you are looking at it from. As an example, "I think therefore I am" is something most people agree is obviously a truth. Most people agree that the laws of physics, in some version, are truths.

What is a source of truth?

A source of truth is the bottom level of stuff that composes whatever you're talking about.  If you're programming, the data you're manipulating breaks down into binary 0s and 1s.  But in order to let you handle it faster and more intuitively, it's assembled into layers upon layers of abstracted superstructures, until you're typing nearly English-like code into a preexisting program, or drawing a digital picture with a tablet pen in a very analog-feeling way.  Working directly with the source all the time isn't a good idea - in fact, it's usually unfeasible - and most problems with a higher-level abstraction shouldn't be patched by going all the way down.  But if you utterly disconnect from the fact that computers are in binary under their GUIs, or that no compass and paper can create a genuinely equation-perfect circle, or that physics isn't genuinely Newtonian under the hood - you'll have nowhere to backtrack to if it turns out there was a wrong turn in your reasoning.  You won't be able to sanity-check if you tell yourself a long twisty story about human motivations and "shoulds" and then come up with an action to take on that basis.

Below is a diagram of a healthy chain of pure true belief originating from a source of truth.  

2.   Belief chains can get disconnected from the source of truth.   For example, say that there is a group which has based their philosophy on the understanding of a certain physicist.   Say that the physicist dies, and that the group continues with expanding on that same belief set, although they have not yet integrated one of the key links that the physicist had which connected the chain to a source of truth.  In this case, you can end up with a cluster of belief that looks something like this:

You now have a cluster of belief, that contains some truth, but is no longer linked to source of truth, and fills in the gaps with ungrounded propositions.  This is the sort of situation that leads to high levels of overconfidence, and what Alexander Pope referred to when he wrote:  “A little learning is a dangerous thing."

What does this metaphor look like in real world terms?

continue reading »
Comment author: TheOtherDave 12 October 2014 07:54:10PM 1 point [-]

I'm genuinely not sure if I want to engage it again or not.

What do you consider the pros and cons?

Comment author: ShannonFriedman 11 November 2014 05:23:49AM 2 points [-]

Cons - Its a large time and energy investment to word a post in a way that does not get shredded in this environment - I don't tend to receive much in the way of positive feedback or appreciation for doing it - I will almost certainly receive a lot in the way of negative feedback regardless - potentially quite a lot

Pros - I might be able to share something with someone else that creates value for them - Having a reasonable reputation in the community could be good for my business - I might receive some positive reinforcement

I ended up deciding to do a rewrite of one of the fundamental underlying principles of this post into a new post, which is almost done. It has been something that has been many hours of work. A lot more work than I had anticipated to get the post up to snuff where I think it is less likely to get shot down than this original post.

The amount of work it took to get to a point where it might be acceptable to LWers is unlikely to be balanced by the pros, so I am leaning toward it being my last post, although I am very open to having my utility function show me otherwise.

Comment author: Risto_Saarelma 12 October 2014 05:27:52AM *  4 points [-]

It's just math and parsing theory jokes riffing on an intentional misinterpretation of "a direct opposite of interesting", nothing that relates to the substance of the discussion as far as I can see.

(Well, it might be meta-level commentary on the substance. Anything might be a meta-level commentary. Constant vigilance.)

Comment author: ShannonFriedman 14 October 2014 01:10:50AM 0 points [-]

Thanks.

Comment author: Kawoomba 30 September 2014 06:42:06AM -3 points [-]

This post comes across as a direct opposite of your usual interesting, clear and insightful entries.

Multiply by -1? While reversed stupidity may not be intelligence, the opposite (in the sense of antithetical) sure is.

Now, all that is left is to figure out how to multiply an English sentence by -1. The neutral element would be the empty string, I suppose. Or some tangentially related but really inconsequential comment, like this one. That's nice, introducing some reflectivity.

Hmmm ... maybe reverse the polarity? This is harder than I thought. If we're defining this operation cleanly, we can always reward ourselves with a Kleene star. Seems arbitrary enough, no?

Comment author: ShannonFriedman 11 October 2014 10:24:36PM 0 points [-]

Can you use different words to describe what you are trying to say here? I don't understand but would like to.

Comment author: shminux 29 September 2014 11:06:29PM 3 points [-]

This post comes across as a direct opposite of your usual interesting, clear and insightful entries. It feels to me as if it was written by a novice to this site who is also bad at writing. Normally I'd simply downvote and move on, but you are neither new, nor a poor writer, so I am hoping that somewhere inside this post there is a worthwhile point. However, it failed to materialize in the comments I checked, hence my suggestion to rewrite (and maybe run by someone in your local LW crowd, before posting again -- I hear you know a lot of high-profile regulars who would be happy to oblige).

Comment author: ShannonFriedman 11 October 2014 10:23:50PM 0 points [-]

I am glad that I didn't realize that people could still reply to the post after I deleted it, since its nice receiving the last responses quite a bit later after I am no longer triggered by the initial general response.

I think the reason that my writing is coming off to you this way is that I have moved into a very different mental space than the Less Wrong community, and forgot the degree to which I needed to tune my thinking/writing for Less Wrongers to understand/appreciate my messages.

Less Wrongers are used to talking to people who think and speak in the way that people think and speak on this site. I don't read Less Wrong personally, only post to it. I've read some of the sequences, and I have spent years speaking in person on a regular basis with many high profile Less Wrongers, but the way in which people read and write on the blog is kind of like a foreign language to me, which I am currently rusty at.

Likewise with the cultural expectations about what I should be delivering and how.

I'm considering attempting a rewrite, but not sure if I want to or not. What would my incentive be to do so? So far I have received contempt and criticism for my attempt to communicate what I consider to be some very useful principles. Why should I keep trying?

In order to get it right, I need to wrap my head around the Less Wrong way of thinking again, and figure out how to translate everything I'm saying into something that people on this site will understand. That is quite a lot of work. I really hate this culture of tearing things apart when you don't understand rather than asking questions and being curious about what signal the author is attempting to send. I'm genuinely not sure if I want to engage it again or not.

Comment author: AndHisHorse 30 September 2014 02:12:23PM 0 points [-]

Actually, the point of my response was to illustrate that to say "all of these things are faith" is an incorrectly simplifying assumption. I did deliberately choose an absurd example of faith, not to attribute it to you, but to show the difference between one thing which you did explicitly claim is faith - trust in people - and another thing which would have to be an example of blind faith - belief in leprechauns. If you acknowledge that there is a real difference between the two, it would seem that I have misinterpreted your thesis.

Comment author: ShannonFriedman 11 October 2014 10:09:12PM 0 points [-]

Please let me know if what I just wrote makes sense to you. If it does, perhaps this comment might be good as a start for making a second attempt at communication - I think I articulated what I was trying to say better here than before.

Comment author: AndHisHorse 30 September 2014 02:12:23PM 0 points [-]

Actually, the point of my response was to illustrate that to say "all of these things are faith" is an incorrectly simplifying assumption. I did deliberately choose an absurd example of faith, not to attribute it to you, but to show the difference between one thing which you did explicitly claim is faith - trust in people - and another thing which would have to be an example of blind faith - belief in leprechauns. If you acknowledge that there is a real difference between the two, it would seem that I have misinterpreted your thesis.

Comment author: ShannonFriedman 11 October 2014 10:07:48PM 1 point [-]

Yes. I do see a huge difference between appropriate faith and blind faith.

It is my opinion that everyone functions based on faith far more than we acknowledge. That much of what we believe we have evidence for is actually based on quite flimsy chains of reasoning, that have lower and lower probability of being true with each subsequent link from the evidence we are supposedly basing the chains on.

It is also my opinion that this is pretty much unavoidable in order to function in the world, and that you pretty much have to function on a faith based system. Even a scientist who understands things at a fundamental level in one area is still probably accepting the world as she knows it based on faith in the majority of cases in her life.

So, it is my opinion that a key first step in being rational is to acknowledge that you have a faith based system, and then to optimize that system based on the acknowledged reality of what it actually is.

Comment author: RichardKennaway 30 September 2014 02:24:49PM 0 points [-]

Sorry, I didn't mean to be so abrasive. It's just that communication is, practically by definition, communication with people who are not oneself. It seemed to me that you were surprised to come up against this.

As for the original post itself, it seems to me, as it has to some others who have commented, that it talks around something that sounds like it might be interesting, but never says the thing itself.

Comment author: ShannonFriedman 11 October 2014 09:58:27PM 1 point [-]

Hi Richard,

I just saw this, sorry about the delay in response.

Yes, I was surprised by the response, because my assumptions about other people's assumptions were wrong in this case.

I do of course understand that no one else has the same mental model I do - my mistake was in that I did not model correctly quite how different my mental models are from the majority of Less Wrong readers on this topic.

Given the hostility of the responses I received in response to my attempt to share something I find valuable, I'm really not inclined to keep going.

Yes, I did make a mistake, but I do not feel an obligation to keep paying and paying for it to ungrateful people... why would I want to teach them anything?

It is work to better articulate - to figure out what the difference is between our models and be able to name it in a way that the group can understand.

I do not feel that I have adequate reason at this point in time to make that investment of my time and energy, when the only payment is contempt and ridicule.

View more: Next