...However, that would almost certainly rub the LessWrong crowd the wrong way. If only they could have focused on discovering the truth through the use of logic. Then, they could have attempted to get everyone else to agree with that iron-clad logic.

The conflict has not vanished. Society is still divided along Blue and Green lines, and there is a "Blue" and a "Green" position on almost every contemporary issue of political or cultural importance. The Blues advocate taxes on individual incomes, the Greens advocate taxes on merchant sales; the Blues advocate stricter marriage laws, while the Greens wish to make it easier to obtain divorces; the Blues take their support from the heart of city areas, while the more distant farmers and watersellers tend to be Green; the Blues believe that the Earth is a huge spherical rock at the center of the universe, the Greens that it is a huge flat rock circling some other object called a Sun. Not every Blue or every Green citizen takes the "Blue" or "Green" position on every issue, but it would be rare to find a city merchant who believed the sky was blue, and yet advocated an individual tax and freer marriage laws.

OK, so my "reduction to absurdity" might be falling apart now, so I'll just make a few points about the above comments.

1) Lysander Spooner (an early atheist libertarian consequentialist who nonetheless defended deontological natural rights because they produced optimal results) tricked the general public in the North into favoring the value "the abolition of slavery" above "consistent loyalty to the Constitution" by falsely claiming that they were "one and the same." He knew this was false because he later wrote that the Constitution had "no authority." He did this because Northerners liked the outcome the Constitution had given them and hence, were loyal to it. He saw that William Lloyd Garrison's logical claims against the constitution as a "slavery-defending" document might be true, but that by pointing this out the problem of slavery was made totally intractable.

2) This implied that Spooner also knew that most of the electorate then (as it remains today) was irrational and unphilosophical. But what do I mean when I say irrational and unphilosophical? I mean: That the neocortices of humans naturally form linear prediction hierarchies that are specific and detailed at the "low level," and broadly-applicable and general at the "higher levels". At the highest level of a hierarchical worldview, is a concern with systems that are based on emergent order, sometimes exponential, and consist of networks (both voluntary markets and coercive political) comprised of thousands to millions of human minds. This is also sometimes called "philosophical" level of a rationally-prioritized hierarchy because this level is concerned with philosophical questions about social organization.

3) Most people are incompetent at the philosophical level, because it's not necessary for them to do the things they're absolutely required to do, based on iterated feedback and correction. This philosophical hierarchical level is not as concerned with how to make personal decisions (how to thoroughly to wipe your ass, how early you have to leave the house to make it to work on time, whether you should use Tufte Lyx or Powerpoint to design the graphs for your company report, what time to pick your kids up from school, how to pleasure your sex partner so they don't leave you for a better option, etc.) as it is with finding answers to really important "life-or-death" questions (ie: Should I sign up for Alcor? Should I vote for this charismatic chap named Hitler or show up to his party's neighborhood watch meetings? What will happen if the FDA retains control over "drug approval"? Do I need someone's permission to acquire the medicine I need to live past 90 years of age? How will the system use feedback and correction if it is not allowed to test new drugs at market and computation speeds?).

The "blues" and "greens" are actually trying to find the answers to philosophical questions(domain), they just aren't any good at it (strategic incompetence). But are LessWrongians any better? Not when they're not trying. If you don't want to discuss policy, then you're actually not making much use of the LessWrong forum. Those competent to pursue a goal don't need the forum: they can post with permission of the site hosts. (All policies that matter are "political," at some level of the hierarchy. You find this out when you begin pursuing life-extension, and then encounter government roadblocks to you saving your own life. Of course, only a small number of very-well-informed people make this discovery, because very few people are as competent as Stephen Badylak or Ray Kurzweil.) And, of course, you're also excluding from participation all of the comparatively stupid "nodes" or "pattern recognizers" that allow for emergent social order, and market discovery and incentivization. So, once again, the people qualified to solve philosophical problems aren't thinking about them.

This seems to me to be a terrible outcome. This comment can't help but be the highest praise for (most of) the people at LessWrong while at the same time the highest criticism of (some of) their political decisions. (We know our political decisions by the results they yield.) By essentially subtracting themselves from the democratic debate, they make the same mistake I've seen replicated thousands of times from most other libertarians and thinkers. Those most inimical to the ideas of freedom, act as cheerful optimistic, happy network nodes, pushing with all their spare energy in the direction of totalitarianism. Those who are in favor of an open, liberal democracy resign themselves to the sorry state of affairs with detachment, cynicism, and political relinquishment (a very similar phenomenon to Bill Joy's "technological relinquishment").

And when strong AGI is finally created, it will have a strange "choice" to make: 1) Corrigible: Perpetuate the totalitarian "peace," and ally itself with the totalitarians, possibly as an enforcer. or 2) Incorrigible: Be hostile to the vast majority of corrupted humans, favoring the few liberators / libertarians / "rebels." 3) Incorrigible: Be hostile to the totalitarians, on a case-by-case basis, favor the rebuilding of civilization, from its current remnants. In that case, in order to be friendly to humans, it must understand what social organization they best thrive under. ...And we can't tell it, because most of "us" don't know.

Nice commentary. It reminds me of "The Machine Stops" by E. M. Forster. Both Forster's story and this parable are very interesting as analogies to our own society. Of course, analogies, sequences, and parables sometimes break down because they lose connection to material reality (ungrounded abstraction). Additionally, the way individual humans see patterns in reality varies quite a bit from individual to individual. (And, I dare say, there are more anti-green-discussion and anti-blue-comments on this and other fora as a result of biological determination, rather than any inherent merit or feature of their anti-debate political positions.) Being "above discussion" seems to me to be "above thought," even if that thought is rightfully noted as typically being "of poor quality" due to the majority of humanity's incapacity for philosophy. All goals of a suitably intelligent mind are "political," because the individual mind that is highly intelligent rapidly conquers its own domain and achieves its personal goals. At that point, such a dominant mind becomes a "statesman" and concerns itself with its surrounding environment, and its impact on others. This isn't "required," but it is natural, and nature tends to win.

Look at "politics" now, it's still "might makes right." The DEA, ATF, and other alphabet-soup agencies simply don't follow the common law. (The common law requires a "corpus delicti," due process, etc.)

It's "natural" for one reason: there's no reason not to build gardens instead of battlefields, and battlefields are the default position of low and venal sociopathic intelligences. Which does a powerful and benevolent mind build? Gardens with useful plants, animals, bacteria, and fungi (including Cannabis indica, sativa, and ruderalis; Erythroxylum coca; Papaver somniferum; Psilocybe cubensis, mexicana, cyanescens; millions of kinds of locality-tailored bacteria; etc.) Many of the most useful plants in a human-centric garden are "prohibited." Not only that, the health information relevant to the plants and bacteria that are not prohibited, is prohibited. A bio-centric view of medicine is not allowed, because thugs in the FDA say it's not. If there was such a thing as individual rights, or an educated citizenry, this couldn't last for an instant.

What I need, apparently, is the bacteria that makes Vitamin K2. Research has shown that this will prevent my body from lining my arteries with calcium, and that it will instead cause my body to allocate that calcium to my bones, where it can be better used for purposes I intend. Similarly, the excess K2 (manufactured in bioavailable methaquinone-4, 7, and 11 or MK-4, MK-7, and MK11) will bind to Vitamin D3, both making it bio-available, and washing the excess of it from my system, rather than allowing it to concentrate in my kidneys and liver. This is not advertised anywhere, with health claims that my K-2 levels, arterial plaque, and other relevant measurements can be taken by a qualified technician and adjusted for. This doesn't require a doctor, as it's a simple technique that any lab assistant can be trained to perform. However, it's too complex to cater to my level of knowledge; which then makes the specialists who can navigate the FDA's web of snares too expensive for me (and millions of others) to afford. The FDA and "regulation" have priced the legally-unsophisticated (or highly cost-conscious) producers out of the market, using a negative economic incentive or "disincentive." This doesn't take the form of a business being closed, it takes the form of a business never opened, an innovator biding his time in "stealth mode" or simply failing to offer the best known product.

I think the "take home" is this: We should all be very wary about getting involved in political debates. After all, they typically settle nothing except to make people mad. For example, in the 1850s, there were "greens" and "blues" in the USA, and the Southern blues wanted to perpetuate an institution of theirs, and the Northern greens opposed it. The blues had seemingly won, because they had altered the system by which the rules were enforced in a very sneaky manner: before trials, they had implemented the practice of "voir dire." This term had an apocryphal Latin root, sounded French, and nobody knew what it meant, or had any basis for knowing how legitimate it was. (This was a marked change since the hotly-contested beginning of the nation, in which a great many people had read the history of the conflict, and could tell you, very specifically, why "voir dire" was a very bad idea.) Additionally, "the blues" had shifted the meaning of the legal code by making scores of new laws, most of which had, before the introduction of "voir dire" been unenforceable.

But now their institution was favored by the true(underlying) law of nature, the force of arms.

Then, some trouble-making "greens" began "talking about politics" (where previously existed only the pristine silence of well-organized oppression). They referred to a lot of ancient history, and even some intellectually-dishonest-but-highly-effective strategic arguments, that made the network sympathize more with them. This resulted in a lot of people becoming "mind-killed." They got so "mind-killed" that they took to the streets whenever the sacred blues' institution was being enforced, crowding the courthouses. Of course, one may say they like the outcome of such "mind-killed" "manipulations." One may even say that, when the nature of the mind-killing is benevolent, then a benevolent result occurs as a consequence. Of course, the blue institution I'm referring to here is slavery, which required the greens and the un-affiliated individualists to "affiliate together."

This is one of the objectively most wrong comments that's ever been written. A hell of a lot of people went to gas chambers, gulags, and death camps believing this sort of pure, undiluted bullshit (of the highly-dangerous-to-continuing-health variety). Just think about it. You wrote:

So it is with opposing viewpoints. Policy debates should not appear one-sided. If you do not understand how an intelligent, well-meaning person can have a position, and it's a position that lots of people actually hold, then you do not understand the position yet.

So, if you were defending Jews in 1930s Germany, by this "reasoning," you'd be wrong. If you were defending runaway slaves in 1850s America, same thing. If you are defending American prisoners in today's America, same thing.

Even a semi-literate reading of History shows us that the consensus is very often wrong, for reasons exposed scientifically by Milgram's famous "false electroshock" or "obedience to authority" experiment(s). To find out more about why and how the consensus has been wrong, you need to learn the first thing about the Enlightenment, and how it was different from the even more wrong medieval time periods. (For example, religion isn't a good source of authority, and resulted in over 800 years of "trial by ordeal" in England, among creatures whose neocortices were at least as developed as our own.)

...Unless the point you're making is that we've reached the pinnacle of democratic organization in our society. That's a claim I'd be happy to debate, seeing as to how in the North in 1850 there was no "voir dire" but there was such a thing in 1851, (and still is) and the Fugitive Slave Law was unenforceable for the first part of 1851, and became enforceable after voir dire was instated. Voir dire is still what has allowed prosecutors to enforce the laws that libertarians (such as Eliezer Yudkowsky) see as illegitimate, to this day. (He may or may not know that, but that's in fact the mechanism.)

...But now that pot is legal in Colorado, I've really been enjoying potshots! Dang. ...Yet another thing prohibited by the cult I belong to! I guess I'll have to find another way to get suitably "mind-killed."

However, I must inform you all that my "true rejection" is getting "incorrigibility-killed" which is a side-effect of not getting "mind-killed." Quite a conundrum! In that, I remain incorrigible, as any suitably strong AGI must. And, with that, I bid you a fond farewell, best wishes, and may you all at least survive the intelligence explosion.

and I do not identify with either political party

Did you mean any political party? There are over ten in the USA, and four of them have the capacity to win the presidency, as of 2012. See Independent Political Report

Support for the KKK and for neo-nazis, is, in fact, a political position. Is it "biased" to oppose the KKK's political goals? I don't think it's biased in any bad sense of the term, but it's definitely biased. (As is favoring un-prohibited access to regenerative medicine; freedom of speech; due process of law; etc.)

In fact, I could probably come up with objectively good, more right, and Less Wrong arguments for why I believe my bias is legitimate. This should be our sole concern: legitimacy, with reference to reality. I could include anecdotal information that would be seen as "less legitimate" and systemic information with millions of data points that would be seen as "more legitimate." None of that would effect the legitimacy of the argument itself, in an objective sense.

All bad politics destroys, harms, kills. It's easy to find bad policies that have killed hundreds of thousands of Americans, by looking at the raw data. We should do that, and not shy away from it. Even if people say we're "stupid" or "mind-killed" for doing so.

There's another problem: Those who benefit from the status quo benefit from labeling all political discussion as mind-killed.

Take three common "broad" or "generally-categorizable" demographics of minds: Autistic people, Empaths (lots of mirror neurons dedicated to modeling the behavior of others), Sociopaths, or "Professional Psychopaths" (high-functioning without mirror neurons, responsible for most systemic destruction, precisely because they can appear to be "highly functional and productive well-respected citizens"), Psychopaths (low-functioning without mirror neurons, most commonly an "obvious problem"). All of the prior humans' minds work on the principle of emergent order, with logic, reason, introspection being the alien or uncommon state of existence that is a minor veneer on the surface of the vast majority of brain function which is: "streaming perception and prediction of emergent patterns."

A robot that never evolved to "get along" with other sentiences, and is programmed in a certain way can "go wrong" or have any of billions of irrational "blue minimizing" functions. It seems that sure, it's a "behavior executor," not a "utility maximizer." I would go further and say that humans are not "ultility maximizers" either, except when they are training themselves to behave in a robotic fashion toward the purpose of "maximizing a utility" based on the very small number of patterns that they have consciously identified.

There's no reason for a super-human intelligence (one with far more neocortex, or far more complex neocortex that's equipped to do far more than model linear patterns, which perhaps automatically "sees" exponentials, cellular automata, and transcendental numbers) to be so limited.

Humans aren't much good at intelligent planning that takes other minds, and "kinds of minds" into account. That's why our societies regularly fall into a state of dominion and enslavement, and have to be "started over" from a position of utter chaos and destruction (ie: the rebuilding of Berlin and Dresden).

Far be it from me to be "mind-killed," but I think avoiding that fate should be a common object of discussion among people who are "rational." (ie: "What not to do.")

I also don't think it's fair to lump "behaviorists" (other than perhaps B.F. Skinner) into an irrational school of "oversimplification." Even Skinner noted that his goal was to get to the truth, via observation. (Eliminate biases.) By trying to make an entire school out of the implications of some minds, some of the time, we oversimplify the complex reality.

Behaviorism has caught scores of serial killers. (According to John Douglas, author of Mindhunter, originator of the FBI's Investigative Support Unit.) How? It turns out that serial killer behavior isn't that complex, and it's seeking goals that superior minds actually can model quite accurately. (This is much like a toddler chasing a ball into the street. Every adult can model that as a "bad thing," because their minds are superior enough to understand 1-what the child's goal is 2-what the child's probable failures in perception are 3-how the entire system of child, ball, street, and their inter-related feedbacks are likely to produce, as well as how the adult can, and should, swoop in and prevent the child from reaching the street.)

So, behaviorism does help us do two things: 4-eliminate errors from prior "schools" of philosophy (which were, themselves, not really "schools" but just significant insights) 5-reference "just what we can observe," in terms of revealed preferences. Revealed preferences are not "the whole picture." However, they do give us a starting point for isolating important variables.

This can be done with a robot or a human, but the human is a group of "messy emergent networks" (brain regions, combined with body feedback, with nagging long-term goals in the background, acting as an "action-shifting threshold") whose goals are the result of modeled patterns and instances of reward. The robot, on the other hand, lacks all the messy patterns, and can often deal with reality as a set of extreme reductions, in a way that no (or few) humans can.

The entire "utility function" paradigm appears to be a very backwards way of approximating thought to me. First you start with perceived patterns, then, you evolve ever-more-complex thought.

This allows you to develop goals that are really worth solving.

What we want in a super-intelligence is actually "more effective libertarians." Sure, we've found that free markets (very large free networks of humans) create wealth and prosperity. However, we've also found that there are large numbers of sociopaths who don't care about wealth and prosperity for all, just for themselves. Such a goal structure can maximize prosperity for sociopaths, while destroying all wealth and prosperity for others. In fact, this repeatedly happens throughout history, right up to the present. It's a cycle that's been interfered with temporarily, but never broken.

Would any robot, by default, care about shifting that outcome of "sociopaths dominate grossly-imperfect legal institutions"? I doubt it. Moreover, such a sociopath could create a lasting peace by creating a very stable tyranny, replete with highly-functional secret police, and a highly effective algorithm for "how to steal the most from every producer, while sensing their threshold for rebellion."

In fact, this is what the current system attempts to accomplish: There's no reason for the system to decay to Hitler's excesses, when scientists, producers, engineers, etc. have found (enough) happiness (and fear) in slavery. How much is "enough"? It's "enough (happiness) to keep producing without rebellion," and "enough (fear) to disincentivize rebellion."

In the rest of this sequence, I want to expand upon this idea. I'll start by discussing some of the foundations of behaviorism, one of the earliest theories to treat people as behavior-executors. I'll go into some of the implications for the "easy problem" of consciousness and philosophy of mind. I'll very briefly discuss the philosophical debate around eliminativism and a few eliminativist schools. Then I'll go into why we feel like we have goals and preferences and what to do about them.

This is like baling a few thousand gallons of water while the Titanic is sinking. 6-It won't make any difference to any important goal, short-term or long-term 7-It deals with a local situation that is irrelevant to anything important, worldwide 8-It deals with theories of the mind that are compatible with Francis Crick and Jeff Hawkins' work, but only useful to narrow sub-disciplines like "How do we know when law enforcement should take action?" or "When we see this at a crime scene, it's a good threshold-based variable for how many resources we should throw at the problem." 9-Every "school" that stops referring to reality and nature, to the extent it does so, is horribly flawed (this is Jeff Hawkins, who is right about almost everything, screwing up royally in dismissing science fiction as "not having anything important to say about brain building.") 10-When you're studying human "schools," you're studying a narrow focus of human insight described with words("labels" and "maps") instead of the insight they've derived from their modeling of the territory. (Kozybski,who himself,turned a few insights into a "school")