In an earlier comment, I conflated heterophenomenology in the general sense of taking introspective accounts as data to be explained rather than direct readouts of the truth, with Dennett's particular approach to explaining those data.  So to correct myself, I say that it is Dennett, rather than heterophenomenology, that claims that there is no such thing as consciousness. Dennett denies that he does, but I disagree. I defend this view here.

I have to admit at this point that I have not read "Consciousness Explained".  Had either of the library's copies been on the shelves last Tuesday I would have done by now, but instead I found his later book (and his most recent on the topic), "Sweet Dreams: Philosophical Obstacles to a Science of Consciousness".  The subtitle suggests a drawing back from the confidence of the earlier title, as does that of the book in between.  The book confirms me in my impression that the ideas of "C.E." have been in the air so long (the air of hard SF, sciblogs, and the like, not to mention Phil Goetz's recent posts) that reading the primary source 19 years on would be nothing more than an exercise in checkbox-ticking.

I'll give a brief run-through of "Sweet Dreams" and then carry on the argument.

The social bookmarking site metafilter has a sister site called metatalk, which works the same way but is devoted entirely to talking about metafilter itself. Arguments about arguments, discussions about discussions, proposals for changes in site architecture, etc.

Arguments about arguments are often less productive than the arguments they are about, but they CAN be quite productive, and there's certainly a place for them. The only thing wrong with them is when they obstruct the discussion that spawned them, and so the idea of splitting off metatalk into its own site is really quite a clever one.

Lesswrong's problem is a peculiar one. It is ENTIRELY devoted to meta-arguments, to the extent that people have to shoehorn anything else they want to talk about into a cleverly (or not so cleverly) disguised example of some more meta topic. It's a kite without a string.

Imagine if you had been around the internet, trying to have a rational discussion about topic X, but unable to find an intelligent venue, and then stumbling upon lesswrong. "Aha!" you say. "Finally a community making a concerted effort to be rational!"

But to your dismay, you find that the ONLY thing they talk about is being rational, and a few other subjects that have been apparently grandfathered in. It's not that they have no interest in topic X, there's just no place on the site they're allowed to talk about it.

What I propose is a "non-meta" sister site, where people can talk and think about anything BESIDES talking and thinking. Well, you know what I mean.

Yes?

In reply to Eliezer's Contrarian Status Catch 22 & Sufficiently Advanced Sanity. I accuse Eliezer of encountering a piece of Advanced Wisdom.

Unreason is something that we should fight against. Witch burnings, creationism & homeopathy are all things which should rightly be defended against for society to advance. But, more subtly, I think reason is in some ways, is also a dangerous phenomena that should be guarded against. I am arguing not against the specific process of reasoning itself, it is the attitude which instinctually reaches for reason as the first tool of choice when confronting a problem. Scott Aaronson called this approach bullet swallowing when he tried to explain why he was so uncomfortable with it. Jane Galt also rails against reason when explaining why she does not support gay marriage.

There was recently a submission here posing criticism a well-known contributor, Eliezer Yudkowsky.

Thank you For Your Participation

I would like to thank you all for your unwitting and unwilling participation in my little social experiment. If I do say so myself you all performed as I had hoped. I found some of the responses interesting, many them are goofy. I was honestly hoping that a budding rationalist community like this one would have stopped this experiment midway but I thank you all for not being that rational. I really did appreciate all the mormon2 bashing it was quite amusing and some of the attempts to discredit me were humorous though unsuccessful. In terms of the questions I asked I was curious about the answers though I did not expect to get any nor do I really need them; since I have a good idea of what the answers are just from simple deductive reasoning. I really do hope EY is working on FAI and actually is able to do it though I certainly will not stake my hopes or money on it. 

Less there be any suspicion I am being sincere here.

Response

Because I can I am going to make one final response to this thread I started:

Since none of you understand what I am doing I will spell it out for you. My posts are formatted, written and styled intentionally for the response I desire. The point is to give you guys easy ways to avoid answering my questions (things like tone of the post, spelling, grammar, being "hostile (not really)" etc.). I just wanted to see if anyone here could actually look past that, specifically EY, and post some honest answers to the questions (real answers again from EY not pawns on LW). Obviously this was to much to ask, since the general responses, not completely, but for the most part were copouts. I am well aware that EY probably would never answer any challenge to what he thinks, people like EY typically won't (I have dealt with many people like EY). I think the responses here speak volumes about LW and the people who post here (If you can't look past the way the content is posted then you are going to have a hard time in life since not everyone is going to meet your standards for how they speak or write). You guys may not be trying to form a cult but the way you respond to a post like this screams cultish and even a some circle-jerk mentality mixed in there. 

Post

I would like to float an argument and a series of questions. Now before you guys vote me down please do me the curtsey of reading the post. I am also aware that some and maybe even many of you think that I am a troll just out to bash SIAI and Eliezer, that is in fact not my intent. This group is supposed to be about improving rationality so lets improve our rationality.

SIAI has the goal of raising awareness of the dangers of AI as well as trying to create their own FAI solution to the problem. This task has fallen to Eliezer as the paid researcher working on FAI. What I would like to point out is a bit of a disconnect between what SIAI is supposed to be doing and what EY is doing.

According to EY FAI is an extremely important problem that must be solved with global implications. It is both a hard math problem and a problem that needs to be solved by people who take FAI seriously first. To that end SIAI was started with EY as an AI researcher at SIAI. 

Until about 2006 EY was working on papers like CEV and working on designs for FAI which he has now discarded as being wrong for the most part. He then went on a long period of blogging on Overcoming Bias and LessWrong and is now working on a book on rationality as his stated main focus. If this be accurate I would ask how does this make sense from someone who has made such a big deal about FAI, its importance, being first to make AI and ensure it is FAI? If FAI is so important then where does a book on rationality fit? Does that even play into SIAI's chief goals? SIAI spends huge amounts of time talking about risks and rewards of FAI and the person who is supposed to be making the FAI is writing a book on rationality instead of solving FAI. How does this square with being paid to research FAI? How can one justify EY's reasons for not publishing the math of TDT, coming from someone who is committed to FAI? If one is committed to solving that hard of a problem then I would think that the publication of ones ideas on it would be a primary goal to advance the cause of FAI.

If this doesn't make sense then I would ask how rational is it to spend time helping SIAI if they are not focused on FAI? Can one justify giving to an organization like that when the chief FAI researcher is distracted by writing a book on rationality instead of solving the myriad of hard math problems that need to be solved for FAI? If this somehow makes sense then can one also state that FAI is not nearly as important as it has been made out to be since the champion of FAI feels comfortable with taking a break from solving the problem to write a book on rationality (in other words the world really isn't at stake)? 

Am I off base? If this group is devoted to rationality then everyone should be subjected to rational analysis.

(Note: This is essentially a rehash/summarization of Jordan Sobel's Lotteries and Miracles - you may prefer the original.)

George Mavrodes wrote an interesting analogy. Scenario 1: Suppose you read a newspaper report claiming that a particular individual (say, Henry Plushbottom of Topeka, Kansas) has won a very large lottery. Before reading the newspaper, you would have given quite low odds that Henry in particular had won the lottery. However, the newspaper report flips your beliefs quite drastically. Afterward, you would give quite high odds that Henry in particular had won the lottery. Scenario 2: You have read various claims that a particular individual (Jesus of Nazareth) arose from the dead. Before hearing those claims, you would have given quite low odds of anything so unlikely happening. However (since you are reading LessWrong) you presumably do not give quite high odds that Jesus arose from the dead.

What is it about the second scenario which makes it different from the first?

2:45:24 PM Katja Grace: The main thing that puts me off in online dating profiles is lack of ambition to save the world
2:45:35 PM Katja Grace: Or do anything much
2:48:03 PM Michael Blume: *nods*
2:48:07 PM Michael Blume: this is indeed a problem
2:57:55 PM Katja Grace: Maybe there is a dating site for smart ambitious nerds somewhere
2:58:25 PM Katja Grace: Need to set up lw extension perhaps
2:59:02 PM Michael Blume: haha, yes ^^
3:00:40 PM Katja Grace: Plenty of discussion on why few girls, how to get girls, nobody ever says 'oy, girls on lw, want to get together some time?'
3:01:14 PM Michael Blume: somebody really should say that
3:01:34 PM Michael Blume: hell, I'm tempted to just copy that IM into a top-level post and click 'submit'
3:01:48 PM Katja Grace: Haha dare you to

David Brin suggests that some kind of political system populated with humans and diverse but imperfectly rational and friendly AIs would evolve in a satisfactory direction for humans.

I don't know whether creating an imperfectly rational general AI is any easier, except that limited perceptual and computational resources obviously imply less than optimal outcomes; still, why shouldn't we hope for optimal given those constraints?  I imagine the question will become more settled before anyone nears unleashing a self-improving superhuman AI.

An imperfectly friendly AI, perfectly rational or not, is a very likely scenario.  Is it sufficient to create diverse singleton value-systems (demographically representative of humans' values) rather than a consensus (over all humans' values) monolithic Friendly?  

What kind of competitive or political system would make fragmented squabbling AIs safer than an attempt to get the monolithic approach right?  Brin seems to have some hope of improving politics regardless of AI participation, but I'm not sure exactly what his dream is or how to get there - perhaps his "disputation arenas" would work if the participants were rational and altruistically honest).

You've all seen it. Sentences like "this sentence is false": if they're false, they're true, and vice versa, so they can't be either true or false. Some people solve this problem by doing something really complicated: they introduce infinite type hierarchies wherein every sentence you can express is given a "type", which is an ordinal number, and every sentence can only refer to sentences of lower type. "This sentence is false" is not a valid sentence there, because it refers to itself, but no ordinal number is less than itself. Eliezer Yudkowsky mentions but says little about such things. What he does say, I agree with: ick!

In addition to the sheer icky factor involved in this complicated method of making sure sentences can't refer to themselves, we have deeper problems. In English, sentences can refer to themselves. Heck, this sentence refers to itself. And this is not a flaw in English, but something useful: sentences ought to be able to refer to themselves. I want to be able to write stuff like "All complete sentences written in English contain at least one vowel" without having to write it in Spanish or as an incomplete sentence.¹ How can we have self-referential sentences without having paradoxes that result in the universe doing what cheese does at the bottom of the oven? Easy: use fuzzy logic.