Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Intelligence enhancement as existential risk mitigation

17 [deleted] 15 June 2009 07:35PM

Here at Less Wrong, the Future of Humanity Institute and the Singularity Institute, a recurring theme is trying to steer the future of the planet away from disaster. Often, the best way to avert a particular disaster is quite hard for ordinary people to understand as it requires one to think through an argument in a cool, unemotional way; more often than not the best solution will be lost in a mass of low signal-to-noise ratio squabbling and/or emoting. Whatever the substance of the debate, the overall meta-problem is quite well captured by this catch from this month's rationality quotes:

"People are mostly sane enough, of course, in the affairs of common life: the getting of food, shelter, and so on. But the moment they attempt any depth or generality of thought, they go mad almost infallibly.

Attempting to target the meta-problem of getting people to be slightly less mad when it comes to abstract or general thought, especially public policy, is a tempting option. Robin Hanson's futarchy proposal is one way to combat this madness (which it does by removing most people from the policymaking loop). However, another important route to combating human idiocy is to find technologies that make humans smarter. Nick Bostrom proposed that we should work hard looking for ways to enhance the cognition of research scientists, because even a small increase in the average intelligence of research scientists would increase research output by a large amount, as there are lots of scientists. But improving the decisionmaking process of our society would probably have an even more profound effect; if we could improve the intelligence of the average voter by about one standard deviation, it is easy to speculate that the political decisionmaking process would work much better. For example, understanding simple logical arguments and simple quantitative analyses is stretching the capabilities of someone at IQ 100, so it seems that the marginal effect of overall IQ increases would be quite a large marginal increases in the probability that a politician was incentivized to focus on a logical argument over an emotionally appealing slander as the main focus of their campaign.

As a concrete example, consider the initial US reaction to rising oil prices and the need for US-produced energy: pushing corn ethanol, because a strong farming lobby liked the idea of having extra revenue. Now, if the *average voter* could understand the concept of photosynthetic efficiency, and could understand a simple numerical calculation showing how inefficient corn is at converting solar energy to stored energy in ethanol, this policy choice would have been dead in the water. But the average voter cannot do simple physics, whereas they can understand the emotional appeal of "support our local farmers!". Even today, there are still politicians who defend corn ethanol because they want to pander to local interest groups. Another concrete example is some of the more useless responses that the UK public has been engaging in - and being encouraged to engage in - to prevent global warming. People were encouraged to unplug their mobile phone chargers when the chargers weren't being used. David McKay had to wage a personal war against such idiocy - see this Guardian article. The universal response to my criticism of people advocating this was "it all adds up!". I quote:

There's a lack of numeracy in the public discussion of energy. Where people do use numbers, they select them to sound big and score points in arguments, rather than to aid thoughtful discussion.

Toby Ord has a project on efficient charity, he has worked out that the difference in outcomes per dollar for alleviating human suffering in Africa can vary by 3 orders of magnitude. But most people in the developed world don't know what an "order of magnitude" is, or why it is a useful concept. This efficient charity concept demonstrated that the derivative

d(Outcomes)/d(Average IQ)

may be extremely large, and may be subject to powerful threshold effects. In this case, there is probably an average IQ threshold above which the average person can easily understand the concept of efficient charity, and thus all the money gets given to the most efficient charities, and the amount of suffering-alleviation in Africa goes up by a factor of 1000, even though the average IQ of the donor community may only have jumped from 100 to 140, say.

It may well be the case that finding a cognitive enhancer suitable for general use is the best way to tackle the diverse array of risks we face. People with enhanced IQ would also probably find it easier (and be more willing) to absorb cognitive biases material; to see this, try and explain the concept of "cognitive biases" to someone who is unlucky enough to be of below average IQ, and then go an explain it to someone who is smarter than you. It is certainly the case that even people of below average IQ *do sometimes*, in favourable circumstances, take note of quantitative rational arguments, but in the maelstrom of politics such quantitative analyses get eaten alive by more emotive arguments like "SUPPORT OUR FARMERS!" or "SUPPORT OUR TROOPS!" or "EVOLUTION IS ONLY A THEORY!" or "IT ALL ADDS UP!".

Comments (198)

Comment author: JamesAndrix 16 June 2009 08:29:51PM 6 points [-]

If you are surrounded by money pumps, is it rational to bet with them, or correct their functioning?

Comment deleted 16 June 2009 10:13:19PM [-]
Comment author: JamesAndrix 17 June 2009 01:02:24AM 2 points [-]

That's not quite what I meant, but I do believe Eliezer has advised anyone not working on 'risks' (or FAI?) to make as much money as they can and contribute to an organization that is.

What I meant was that given a money pump, the straightforward thing to do is to pump it, not fix it in the hope that it will somehow benefit humanity.

It seems to me that on LW, most believe that people are irrational in ways that should make people money pumps, but the reaction to this is to make extreme efforts to persuade people of things.

Improving someone's intelligence or rationality is difficult if they're not already looking, but channeling away some of their funds or political capital will lessen the impact their irrationality can have.

Comment author: orthonormal 17 June 2009 01:34:27AM 5 points [-]

most believe that people are irrational in ways that should make people money pumps

I think that most people are (unconsciously) rational enough to avoid being financially exploited in any but the obvious known ways (lotteries, advertising affecting preferences, etc) for which there's already a competitive market or regulation. Aside from those avenues, most people are too wary of being cheated to make good money pumps.

What's not being covered as much are the ways that people irrationally contribute to the destruction of wealth and utility, by voting for ineffective or detrimental policies or by spreading harmful memes. (People don't typically have an evolved horror of doing these things.) A push towards rationality on those fronts can help everyone.

If you disagree, do you have a particularly effective money-pump in mind? (Of course you might hesitate to share it.)

Comment author: RolfAndreassen 16 June 2009 07:52:45PM 4 points [-]

I do not know if this strategy would apply in the Western world; but in Africa, I think much could be gained simply by nutritional intervention. IQ is, to a good approximation, 50% genetic; much of the environmental effect is childhood nutrition; it follows that widespread distribution of vitamins might have a nice effect over time, in addition to the more usual benefits. It also seems possible that this might work in those strata of the American population that subsist mainly on fast food, although I expect the effect would be less - likely there are already patchwork government programs that distribute vitamins to the poor.

Comment author: Arenamontanus 16 June 2009 11:49:28PM 1 point [-]

Yes, in many places nutrition is a low-hanging fruit. My own favorite example is iodine supplementation, http://www.practicalethicsnews.com/practicalethics/2008/12/the-perfect-cog.html but vitamins, long-chained fatty acids and simply enough nutrients to allow full development are also pretty good. There is some debate of how much of the Flynn effect of increasing IQ scores is due to nutrition (probably not all, but likely a good chunk). It is an achievable way of enhancing people without triggering the normal anti-enhancement opinions.

The main problem is that it is pretty long-term. The infants we save today will be putting their mark about two or more decades hence - they will not help us much with the problems we face before then. But this is a problem for most kinds of biological enhancement; developing it and getting people to accept it will take time. That is why gadgets are important - they diffuse much more rapidly.

Comment author: Arenamontanus 16 June 2009 06:23:51PM 3 points [-]

I have tried to research the economic benefits of cognition enhancement, and they are quite possibly substantial. But I think Roko is right about the wider political ramifications.

One relevant reference may be: H. Rindermann, Relevance of Education and Intelligence for the Political Development of Nations: Democracy, Rule of Law and Political Liberty, Intelligence, v36 n4 p306-322 Jul-Aug 2008 argues (using cross-lagged data) that education and cognitive ability has bigger positive effects on democracy, rule of law and political liberty than GDP. There are of course plenty of reciprocal factors.

As I argued below in my comment on consensus-formation, in many situations a slightly larger group of smart people might matter. The effect might be limited under certain circumstances (e.g. the existence of big enough non-truth seeking biased groups, like the pro-ethanol groups), but intelligence is something that acts across most of life - it will not just affect political behaviour but economic, social and cultural behaviour. That means that it will have multiple chances to affect society.

Would it actually reduce existential risks? I do not know. But given correlations between long-term orientation, cooperation and intelligence, it seems likely that it might help not just to discover risks, but also in ameliorating them. It might be that other noncognitive factors like fearfulness or some innate discounting rate are more powerful. But intelligence can also co-opt noncognitive factors (e.g. a clever advertising campaign exploiting knowledge of cognitive biases to produce a desirable behavior).

Comment author: Psychohistorian 16 June 2009 01:40:16AM *  10 points [-]

I think that most people who do not have severe cognitive deficiencies are capable of understanding what "efficient charities" are. I think that most people are quite capable of understanding the statement, "Ethanol will waste a lot of money and will still generate as much (or more) pollution than gasoline. To top it off, it will also raise the price of food products, both for you and for people who will actually starve as a result." Most issues like this, one can figure out what's going on by reading wikipedia for half an hour. Perhaps that takes a high IQ, but from my experience, when people are given clear and accurate arguments, they are generally capable of getting them. The problem is that they never bother seeking out decent arguments. They either just don't care, or they seek out arguments that support whatever their beliefs happen to be.

In other words, the problem is not that people are stupid. The problem is that people simply don't give a damn. If you don't fix that, I doubt raising IQ will be anywhere near as helpful as you may think.

Comment deleted 16 June 2009 01:19:21PM *  [-]
Comment author: MichaelBishop 16 June 2009 09:40:49PM *  6 points [-]

More detail would be wonderful!

Comment author: RobinHanson 17 June 2009 03:24:02AM 5 points [-]

Well be sure to clarify if they were really trying to understand. People who do not want to understand can look a lot like people unable to understand.

Comment author: Psychohistorian 16 June 2009 06:53:13PM 0 points [-]

Does this mean they don't understand, they don't care, or they don't share your utility function? The fact that they disagree does not mean they don't understand.

Comment deleted 16 June 2009 09:10:15AM *  [-]
Comment author: Annoyance 16 June 2009 06:44:41PM -1 points [-]

Intelligence, while useful, isn't what's required in that scenario.

Skepticism, curiosity, and intellectual integrity are.

Comment deleted 16 June 2009 10:20:05PM [-]
Comment author: Eliezer_Yudkowsky 17 June 2009 08:59:16AM 2 points [-]

I'm not sure this is correct. I might endorse a statement to the effect that fluid g is necessary in order to learn the more advanced skills of distinguishing sophistry, but very few people, high-g or otherwise, actually learn such a skill.

Comment deleted 17 June 2009 04:23:33PM *  [-]
Comment author: RobinHanson 17 June 2009 03:24:57AM 1 point [-]

Yes, this is the key problem that people don't really want to understand. That is the problem futarchy is intended to solve.

Comment author: komponisto 16 June 2009 03:18:18AM 1 point [-]

Agree. Or, one might say: the problem is not so much one of intelligence as one of (surprise!) rationality.

Comment author: CronoDAS 16 June 2009 05:48:13AM *  2 points [-]

Ditto... although being ill-informed can't help either.

I once heard a certain political figure speak at a university. He said that when he gave speeches in areas in which the majority supported his political party, explaining what problems he was trying to solve, they would simply react as a supportive audience - but when he gave speeches in areas where his party was unpopular, they also approved of him, saying that they were horrified and angry because nobody had ever told them about this problem before. He concluded by saying that a Republican is a Democrat who doesn't know what's going on.

More disturbingly, giving someone a list of falsehoods often causes people to later remember them as being true. (See also this Eliezer post.)

Comment author: SoullessAutomaton 16 June 2009 10:41:03PM *  -1 points [-]

Ethanol will waste a lot of money and will still generate as much (or more) pollution than gasoline.

As an aside, given that the pollutant du jour is atmospheric carbon, it's worth noting that burning ethanol is essentially carbon neutral. Ethanol also means not being dependent on foreign nations run by crazy religious whackjobs.

Not that corn ethanol is a good idea overall, but it does have points to recommend it vs. petroleum. It just has... a lot of points to disrecommend it in general.

Comment author: Jack 17 June 2009 02:55:44AM 0 points [-]

Thing is liquid fuel (oil or otherwise) is traded on a mostly open, global, market. So the price of ethanol and gasoline are both dictated to a large extent by the supply produced by OPEC. So producing ethanol can only make us "independent" in the sense that it makes OPEC/Venezuelan/Russian oil a smaller fraction of global energy production. But as long as those sources constitute a decent fraction of global production (and you can't grow enough ethanol to change that) they can still drive up prices on a whim.

I also think the jury is still out on the carbon neutrality thing (at least as far as I can tell surveying the research)- especially if you were already growing corn or had to chop down a rain forest to get your farm land.

Comment author: SoullessAutomaton 17 June 2009 10:06:04AM -1 points [-]

Seriously? Do we actually trade fuel ethanol? Why would anyone want it, and if they did, how could we compete with Brazil's sugar cane ethanol? How odd. My impression is that the only reason corn ethanol is as cheap as gas is because of huge government subsidies to the corn growers. Perhaps the whole situation is even sillier than I realized.

Also, I think the main idea of "energy independence" is that, if it came down to it, we could switch to completely internal energy sources and tell the rest of the world to go shove it. It'd be a diplomatic stick to beat people with in the sense of "we don't need you", not an otherwise significant on-going economic factor.

I also think the jury is still out on the carbon neutrality thing (at least as far as I can tell surveying the research)- especially if you were already growing corn or had to chop down a rain forest to get your farm land.

The carbon in the ethanol was extracted from the atmosphere in the past couple years, burning it releases it back. That's pretty much the definition of carbon-neutral. Pretty much the same should be the case for anything else you do with the corn, including eating it. But, you're right, there are other considerations. Chopping down rainforest is absolutely not carbon-neutral.

Comment author: Jack 17 June 2009 04:36:40PM 1 point [-]

My point is just that the whole energy independence thing is a red herring since energy is traded on an open market. If we suddenly had to depend only on energy produced in the U.S. the resulting price increases would be prohibitive for everyone but the military (and we already have strategic oil reserves).

The whole concept of energy independence is a political cudgel to turn energy politics into security politics by taking advantage of people's mercantilist intuitions about resources. But in a global free market those intuitions are wrong. Strictly speaking you don't get to decide where your energy comes from. Often it is cheaper to get it from nearby sources but it is all part of the same pricing system. So yeah, increased domestic production might make us more independent in the sense that foreign countries won't have quite the same ability to knock prices up but there isn't a magic line where suddenly we're "independent".

Lets say we have 100 oil. 50 of it is produced in the Middle East and the U.S. uses 40 but currently produces 20. The remaining 30% is produced by other countries. An OPEC embargo leaves us with 50 oil about doubling the price (for convenience, usually supply curves are exponential but I'm not an economist I don't know what the price would really do beyond going up, a lot.). If the U.S. increases production to 40% prices pre-embargo will be lower (since we have a supply of 120) but prices will still increase by a similar amount when we lose 50 of that 120 supply leaving us with only 70 oil. If the whole world turned against us prices would triple (according to our invented price model) as we would have only 40 in supply where once we had 120.

In order to make it so the rest of the world really couldn't affect us we'd have to be producing a preponderance of the world's energy such that foreign embargoes would only slightly affect total supply. Either that or we embargo foreign energy imports. But we'd be independent like chopping off your legs makes you independent of bicycles. Also, I imagine if our transportation infrastructure was such that it didn't use gasoline we'd be much better insulated from price shifts in foreign oil. Like if our cars were all solar powered or if we were taxing CO2 to a prohibitive extent already. But in those cases technology has either rendered oil irrelevant or our economy has already internalized the cost of a foreign oil embargo.

Hopefully I'm making some sense.

Comment author: SoullessAutomaton 18 June 2009 01:19:22AM -1 points [-]

Hopefully I'm making some sense.

You are. I obviously haven't thought the issue through clearly and it's not something I care deeply about, since ethanol is silly for many other, larger reasons, therefore I won't waste your time with further questions. Definitely moving the whole "energy independence" thing to the "I have no idea" column for now.

Comment author: saturn 18 June 2009 09:04:30AM *  0 points [-]

In short, we're "independent" when the cost of importing fossil fuels exceeds the cost of domestic production. The two ways this can happen are drastic technological improvements, and subsidies/tariffs on fuel resulting in economically wasteful overproduction. In the US we tend to give lip service to the former while actually implementing the latter.

Comment author: derekz 15 June 2009 08:11:09PM 4 points [-]

I suppose the question is not whether it would be good, but rather how. Some quick brainstorming:

  • I think people are "smarter" now then they were, say, pre-scientific-method. So there may be more trainable ways-of-thinking that we can learn (for example, "best practices" for qualitative Bayesianism)

  • Software programs for individuals. Oh, maybe when you come across something you think is important while browsing the web you could highlight it and these things would be presented to you occasionally sort of like a "drill" to make sure you don't forget it, or prime association formation at a later time. Or some kind of software aid to "stack unwinding" so you don't go to sleep with 46 tabs open in your web browser. Or some short-term memory aid that works better than scratch paper. Or just biting the bullet and learning Mathematica to an expert level instead of complaining about its UI. Or taking a cutting-edge knowledge representation framework like Novamente's PLN and trying to enter stuff into it as an "active" note-taking system.

  • Collaboration tools -- shared versions of the above ideas, or n-way telephone conversations, or freeform "chatroom"-style whiteboards or iteratively-refined debate thesis statements, or lesswrong.com

  • Man-machine hybrids. Like having people act as the utility function or search-order-control of an automated search process.

Of course, neural prostheses may become possible at some point fairly soon. Specially-tailored virtual environments to aid in visualization (like of nanofactories), or other detailed and accurate scientific simulations allowing for quick exploration of ideas... "Do What I Mean" interfaces to CAD programs might be possible if we can get a handle on the functional properties of human cognitive machinery...

Comment author: gwern 17 June 2009 03:29:20AM 3 points [-]

Software programs for individuals. Oh, maybe when you come across something you think is important while browsing the web you could highlight it and these things would be presented to you occasionally sort of like a "drill" to make sure you don't forget it, or prime association formation at a later time.

Congratulations, you've nearly reinvented spaced repetition! There is a great deal of writing on spaced repetition flashcard systems, so I won't inflict upon you my own writings; but the Wikipedia article will link you to the main programs (Anki, Mnemosyne, and SuperMemo) and some writeups of the topic. SR is a great technique; I love it dearly.

Or some short-term memory aid that works better than scratch paper.

Well, you could just improve your working memory. Unusually, working memory is plastic enough to be trainable by WM tasks. The WM exercise I'm most familiar with is Dual n-back. I practice it, but while I have noticed improvements, I'm unsure whether they repay the time I've put into it; SR systems have proven themselves as far as I'm concerned, but the jury is still out on dual n-back.

Or taking a cutting-edge knowledge representation framework like Novamente's PLN and trying to enter stuff into it as an "active" note-taking system.

Now that sounds interesting. But looking at this OpenCog link doesn't give me a good idea as to what PLN might do for note-taking (or really, in general); did you have any use-cases or examples?

Comment author: derekz 17 June 2009 04:18:17AM *  1 point [-]

No specific use cases or examples, just throwing out ideas. On the one hand it would be cool if the notes one jots down could self-organize somehow, even a little bit. Now OpenCog is supposed by its creators to be a fully general knowledge representation system so maybe it's possible to use it as a sort of notation (like a probabilistic-logic version of mathematica? or maybe with a natural language front end of some kind? i think Ben Goertzel likes lojban so maybe an intermediate language like that)

Anyway, it's not really a product spec just one possible sort of way someday to use machines to make people smarter.

(but that was before I realized we were talking about pills to make people stop liking their favorite tv shows, heh)

Comment author: Henrik_Jonsson 17 June 2009 05:51:42AM *  0 points [-]

On the one hand it would be cool if the notes one jots down could self-organize somehow, even a little bit.

While I agree that it it would be cool, anything that doesn't keep your notes exactly like you left them is likely to be more annoying than productive unless it is very cleverly done. (Remember Microsoft Clippy?) You'd probably need to tag at least some things, like persons and places.

Comment author: asciilifeform 15 June 2009 08:23:45PM *  3 points [-]

Software programs for individuals.... prime association formation at a later time.... some short-term memory aid that works better than scratch paper

I have been obsessively researching this idea for several years. One of my conclusions is that an intelligence-amplification tool must be "incestuously" user-modifiable ("turtles all the way down", possessing what programming language designers call reflectivity) in order to be of any profound use, at least to me personally.

Or just biting the bullet and learning Mathematica to an expert level instead of complaining about its UI

About six months ago, I resolved to do exactly that. While I would not yet claim "black belt" competence in it, Mathematica has already enabled me to perform feats which I would not have previously dared to contemplate, despite having worked in Common Lisp. Mathematica is famously proprietary and the runtime is bog-slow, but the language and development environment are currently are in a class of their own (at least from the standpoint of exploratory programming in search of solutions to ultra-hard problems.)

Comment author: SilasBarta 17 June 2009 10:52:45PM 2 points [-]

Could you give more examples about things you like about Mathematica? Years ago, I resolved to become an expert at it after reading A New Kind of Science (will you guys forgive me?) and like it for a while, but then noticed some things were needlessly complicated or refused to spit out the right results (long time ago so I can't give examples).

Btw, I learned about Lisp after Mathematica, and was like, "wow, that must have been where Wolfram got the idea."

Comment author: asciilifeform 18 June 2009 07:18:51PM *  1 point [-]

Could you give more examples about things you like about Mathematica?

1) Mathematica's programming language does not confine you to a particular style of thinking. If you are a Lisp fancier, you can write entirely Lispy code. Likewise Haskell. There is even a capability for relatively painless dataflow programming.

2) Wolfram Inc. took great pains to make interfacing with the outside world from within the app as seamless as possible. For example, you can suck in a spreadsheet file directly into a multidimensional array. There is import and export capability for hundreds of formats, including obscure scientific and engineering ones. In case the built-in formats do not suffice, defining custom ones is surprisingly easy.

3) A non-headache-inducing replacement for regular expressions. Enough said.

4) Graphical objects (likewise audio and other streams) are first-class data types. They are able to appear as both the inputs and outputs of functions.

5) Lastly, and most importantly: fully interactive program development. The rest of the programming universe lives a life of endlessly repeated "compile and pray" cycles. Mathematica permits you to meaningfully evaluate and edit in place every line of code you write. I am otherwise an Emacs junkie, yet I have never felt the slightest desire to touch Emacs when working on Mathematica code. The programmer's traditional need to wade through and shovel giant piles of text from one place to another while writing code is almost entirely absent when working in this language.

The downsides of Mathematica (slow, proprietary, expensive, etc.) are widely known. Thus far, the advantages have vastly outweighed the problems for my particular kind of work. However, I have found that I now feel extremely confined when forced to work in any other programming language. Perhaps this risk should be added to the list of disadvantages.

I learned about Lisp after Mathematica, and was like, "wow, that must have been where Wolfram got the idea."

Wolfram had (at least in the early days of Mathematica) a very interesting relationship with Lisp. He seems to have initially rejected many of its ideas, but it is clear that they somehow crept back into his work as time went by.

Comment author: derekz 16 June 2009 03:02:34PM 1 point [-]

Thanks for the motivation, by the way -- I have toyed with the idea of getting Mathematica many times in the past but the $2500 price tag dissuaded me. Now I see that they have a $295 "Home Edition", which is basically the full product for personal use. I bought it last night and started playing with it. Very nifty program.

Comment author: SilasBarta 17 June 2009 10:49:20PM 0 points [-]

I don't know wheter to applaud your ethical restraint, or pity your ignorance. I'll go with the first ;-)

Comment author: derekz 17 June 2009 11:28:12PM *  1 point [-]

If you're wondering whether I'm aware that I can figure out how to steal software licenses, I am.

ETA: I don't condemn those who believe that intellectual property rights are bad for society or immoral. I don't feel that way myself, though, so I act accordingly.

Comment author: SilasBarta 18 June 2009 03:25:50AM *  0 points [-]

It's theoretically possible to believe in IP (on some level), but lack the will not to pluck the forbidden fruit.

Comment author: derekz 15 June 2009 08:43:24PM 1 point [-]

Cool stuff. Good luck with your research; if you come up with anything that works I'll be in line to be a customer!

Comment deleted 15 June 2009 08:27:58PM [-]
Comment author: asciilifeform 15 June 2009 09:17:11PM 0 points [-]

not quite what I was aiming at

I am curious what you had in mind. Please elaborate.

Comment author: derekz 15 June 2009 08:38:00PM -1 points [-]

Well if you are really only interested in raising the average person's "IQ" by 10 points, it's pretty hard to change human nature (so maybe Bostrom was on the right track).

Perhaps if somehow video games could embed some lesson about rationality in amongst the dumb slaughter, that could help a little -- but people would probably just buy the games without the boring stuff instead.

Comment deleted 15 June 2009 11:57:07PM [-]
Comment author: asciilifeform 16 June 2009 12:38:19AM 1 point [-]

I cannot pin down this idea as rigorously as I would like, but there seems to exist such a trait as liking to think abstractly, and that this trait is mostly orthogonal to IQ as we understand it (although a "you must be this tall to ride" effect applies.) With that in mind, I do not think that any but the most outlandishly powerful and at the same time effortless intelligence amplifier will be of much interest to the bulk of the population.

Comment deleted 16 June 2009 01:29:30PM [-]
Comment author: derekz 16 June 2009 01:54:41PM -1 points [-]

I'm still baffled about what you are getting at here. Apparently training people to think better is too hard for you, so I guess you want a pill or something. But there is no evidence that any pill can raise the average person's IQ by 10 points (which kind of makes sense, if some simple chemical balance adjustment could have such a dramatic effect on fitness it would be quite surprising). Are you researching a sci fi novel or something? What good does wishing for magical pills do?

Comment deleted 16 June 2009 02:09:46PM [-]
Comment author: derekz 16 June 2009 02:48:48PM *  1 point [-]

If the point of this essay was to advocate pharmaceutical research, it might have been more effective to say so, it would have made the process of digesting it smoother. Given the other responses I think I am not alone in failing to guess that this was pretty much your sole target.

I don't object to such research; a Bostrom article saying "it might not be impossible to have some effect" is weak support for a 10 IQ point avergage-gain pill, but that's not a reason to avoid looking for one. Never know what you'll find. I'm still not clear what the takeaway from this essay is for a lesswrong reader, though, unless it is to suggest that we should experiment ourselves with the available chemicals.

I've tried many of the ones that are obtainable. Despite its popularity, I found piracetam to have no noticeable effect even after taking it for extended periods of time. Modafinil is the most noticeable of all; it doesn't seem to do much for me while I'm well-rested but does remove some of the sluggishness that can come with fatigue, although I think the results on an IQ test would be unnoticeable (maybe a 6 hour test, something to highlight endurance, could show a measurable difference). Picamilone has a subtler effect that I'm not sure how to characterize. I'm thinking of trying Xanthinol NIcotinate, but have not yet done so. Because of the small effects I do not use these things as a component of my general lifestyle, both for money reasons and the general uncertainty of long-term effects (also mild but sometimes unpleasant side effects). The effects of other more common drugs like caffeine and other stimulants are probably stronger than any of the "weird" stuff, and are widely known. Thinking beyond IQ, there are of course many drugs with cognitive effects that could be useful on an occasional-use basis, but that's beyond the scope of this discussion.

Comment author: asciilifeform 16 June 2009 02:05:06PM *  1 point [-]

But there is no evidence that any pill can raise the average person's IQ by 10 points

Please read this short review of the state of the art of chemical intelligence enhancement.

We probably cannot reliably guarantee 10 added points for every subject yet. Quite far from it, in fact. But there are some promising leads.

if some simple chemical balance adjustment could have such a dramatic effect on fitness

Others have made these points before, but I will summarize: fitness in a prehistoric environment is a very different thing from fitness in the world of today; prehistoric resource constraints (let's pick, for instance, the scarcity of refined sugars) bear no resemblance to those of today; certain refinements may be trivial from the standpoint of modern engineering but inaccessible to biological evolution, or at the very least ended up unreachable from a particular local maximum. Consider, for example, the rarity of evolved wheels.

Comment author: arundelo 16 June 2009 07:02:17AM 1 point [-]

I think this is called need for cognition. (I first saw this phrase somewhere here on LW.)

Comment author: CronoDAS 16 June 2009 12:14:42AM 2 points [-]

Do you think you might be underestimating the capabilities of the statistically average person of 100 IQ?

Now, if the average voter could understand the concept of photosynthetic efficiency, and could understand a simple numerical calculation showing how inefficient corn is at converting solar energy to stored energy in ethanol, this policy choice would have been dead in the water.

There's an obvious point you're overlooking here.

Plants are, indeed, only about 3% efficient at converting the energy in sunlight into chemical energy, and that's before the living plant is harvested. However, bare ground is zero percent efficient, and the sunlight is there whether we use it or not.

Comment author: PhilGoetz 16 June 2009 06:18:57PM 4 points [-]

There are many studies - 1 to 2 dozen - showing that producing a gallon of corn ethanol takes more energy than is contained in a gallon of corn ethanol. Usually, the conclusion is that it takes n=1 to 1.4 times as much energy. There are other studies claiming the opposite; they fail to take into account factors such as irrigation and transportation costs.

I wrote to Wired magazine and to somewhere else (I forget where) to correct their outrageously-incorrect assertions about corn ethanol. (Wired underreported n by 2 orders of magnitude, which is disturbing because this wasn't like ordinary irresponsible journalism where someone took one "expert's" numbers uncritically. The figure they gave for corn ethanol efficiency was AFAIK much, much higher than those of even the most biased ethanol advocates.) My responses were unpublished. It's even worse journalism when you make an extreme error on a point important to public policy, and then someone points it out to you, and gives you a dozen literature citations, and you don't correct it.

Comment deleted 16 June 2009 12:29:20AM [-]
Comment author: MichaelBishop 16 June 2009 12:58:45AM *  1 point [-]

If people listened to intelligent and careful thinkers they wouldn't need to understand it themselves. Whether this is an easier or harder route is unclear to me.

Comment author: CronoDAS 16 June 2009 05:08:18AM *  6 points [-]

The problem is that, in general, there's no good way for a layman to tell the difference between Carl Sagan and Immanuel Velikovsky, except by comparing them to other people who claim to be experts in a field. A book of internally consistent lies, such as Chariots of the Gods? will seem as plausible as any book written about real history to someone who doesn't already know that it's a book of lies.

Comment author: MichaelBishop 16 June 2009 03:52:08PM 2 points [-]

...there's no good way for a layman to tell the difference between Carl Sagan and Immanuel Velikovsky, except by comparing them to other people who claim to be experts in a field.

That sounds like a promising strategy to me. At least it is far better than what people currently do, which is adopt what their friends think, or ideas they find appealing for other reasons. No doubt it would be better if more people were capable of evaluating scientific theory and evidence themselves, but imagine how much better things would be if people simply asked themselves, "Which is the relevant community of experts, how are opinions on this issue distributed amongst the experts, how reliable have similar experts been in the past? e.g. chemists are generally less wrong about chemistry than psychologists are about psychology. This would be a step in the right direction.

Comment author: Annoyance 16 June 2009 06:50:39PM -1 points [-]

That's not quite true. There are ways of evaluating an expert - but people don't like them, don't implement them, and don't try to find out what they are.

Many, many people who have the social status and authority of experts simply don't know what they're talking about. They can be detected by an earnest and diligent inquiry, combined with a healthy and balanced skepticism.

Doctors are a prime example.

Comment author: CronoDAS 17 June 2009 09:18:07PM 1 point [-]

Unfortunately, many of those ways are equivalent to "become an expert yourself". :(

Comment author: SoullessAutomaton 17 June 2009 09:26:21PM -2 points [-]

But how do you know when you've become an expert?

Turtles all the way down!

Comment author: komponisto 16 June 2009 03:32:50AM 2 points [-]

Indeed. Now that I think about it, perhaps the real problem here is that the marginal social status payoff from an increase in IQ is too low (perhaps even negative in some cases); in other words, IQ doesn't buy one enough status. So the question is whether it is easier to fix this than just to raise the IQ baseline.

Comment author: MichaelBishop 16 June 2009 03:59:42PM *  0 points [-]

How does increasing "the marginal social status payoff from an increase in IQ" help? I'm not saying it would hurt, but it seems less direct and less important than increasing the marginal social status payoff from having and acting on unbiased beliefs about the world because this is something people can change fairly easily.

Comment author: asciilifeform 16 June 2009 04:01:05PM 3 points [-]

How does increasing "the marginal social status payoff from an increase in IQ" help?

The implication may be that persons with high IQ are often prevented from putting it to a meaningful use due to the way societies are structured: a statement I agree with.

Comment author: MichaelBishop 16 June 2009 04:20:59PM 0 points [-]

persons with high IQ are often prevented from putting it to a meaningful use due to the way societies are structured.

Do you mean that organizations aren't very good at selecting the best person for each job. I agree with that statement, but its about much, much, more than IQ. It is a tough nut to crack but I have given some thought to how we could improve honest signaling of people's skills.

Comment author: asciilifeform 16 June 2009 04:50:37PM *  4 points [-]

Do you mean that organizations aren't very good at selecting the best person for each job.

Actually, no. What I mean is that human society isn't very good at realizing that it would be in its best interest to assign as many high-IQ persons as possible the job of "being themselves" full-time and freely developing their ideas - without having to justify their short-term benefit.

Hell, forget "as many as possible", we don't even have a Bell Labs any more.

Comment author: komponisto 17 June 2009 12:43:13AM 3 points [-]

This, I think, is a special case of what I meant. A simple, crude, way to put the general point is that people don't defer enough to those who are smarter. If they did, smart folks would be held in higher esteem by society, and indeed would consequently have greater autonomy.

Comment author: MichaelBishop 16 June 2009 05:53:07PM 0 points [-]

How should society implement this? I repeat my claim that other personal characteristics are as important as IQ.

Comment author: asciilifeform 16 June 2009 06:00:25PM 0 points [-]

I do not know of a working society-wide solution. Establishing research institutes in the tradition of Bell Labs would be a good start, though.

Comment author: komponisto 17 June 2009 12:45:25AM 1 point [-]

That may well be right. I'm willing to accept that the distinction between "I.Q." and other measures of "smartness" is orthogonal to the point I was making.

Comment author: CronoDAS 16 June 2009 05:07:26AM 0 points [-]

You're absolutely right about corn ethanol not being much of a solution - indeed, you can't power the U.S. on just corn ethanol, but burning corn-derived ethanol does provide a net gain in useful energy. It's just not nearly enough energy to make a difference. Finally, the biggest problem is that there are generally better things to do with grown corn than to turn it into fuel for engines, such as turn it into food for humans or other animals...

Comment author: Annoyance 16 June 2009 06:46:50PM 0 points [-]

Plants are also much better at converting sunlight into chemical energy than any system we can build.

But the issue isn't how well they store energy, but at how efficiently we can use the energy they store. You can't efficiently fuel an electricity-generating plant with corn - trying to use plant energy to power our civilization is hopeless.

Comment author: timtyler 17 June 2009 09:05:02AM 2 points [-]

That is totally incorrect. Plants are 1-2%. Good panels are around 20% - with experimental ones well beyond that. That's because most solar energy occurs at wavelengths unsuitable for photosynthesis.

Comment author: Annoyance 17 June 2009 01:35:42PM -2 points [-]

Good panels are only that good under laboratory conditions, and require massive expenditures of energy to construct in the first place. Plants are self-replicating.

Equally as important, they produce chemical energy directly. Without an efficient way to produce and store hydrogen using electrical power, there's no alternative for chemical fuels.

Comment author: Alicorn 17 June 2009 05:13:31PM 6 points [-]

"Plants are self-replicating"? In theory, will corn grow without our help? Sure! In practice? Not if you want it in neat, harvestable rows; not if you don't want it to compete with weeds; not if you want it to have a high per-acre yield; not if you want to control which seeds get to turn into plants next generation; not if you don't want crows to eat it; not if you want it to stick to your property and not take over the neighbor's alfalfa; and not if you take all of the plant's kernels and turn them into car fuel.

We don't settle for the replication rate of wild plants, so it's just not the case that they're "free". There's a legitimate question of whether it's costlier (along any given dimension or overall) to produce ethanol than to produce a solar panel which will generate the same amount of power over its useful life, and I don't know the answer, but please let's not extrapolate from the fact that plants sometimes grow unattended to the mistaken conclusion that corn has a negligible input cost.

Comment author: timtyler 17 June 2009 09:52:30PM 1 point [-]

After getting the facts so totally wrong, you are supposed to remain in embarassed silence, not argue the toss with still more dubious claims:

http://en.wikipedia.org/wiki/Electrolysis_of_water#Efficiency

Comment author: CronoDAS 15 June 2009 10:34:00PM *  2 points [-]

Interestingly, this may be actually happening. It's fairly clear that people today are taller than they once were...

Comment deleted 15 June 2009 11:44:12PM [-]
Comment author: Annoyance 16 June 2009 06:52:49PM 0 points [-]

Increasing intelligence without increasing people's ability to use it properly isn't going to help much.

Consider how poorly physicists do when trying to evaluate paranormal claims, as opposed to magicians.

Comment author: gjm 15 June 2009 09:31:59PM 3 points [-]

Nick Bostrom proposed that we should work hard looking for ways to enhance the cognition of research scientists, because even a small increase in the average intelligence of research scientists would increase research output by a large amount, because there are lots of scientists.

I wonder about this. Isn't it the case [translation: I'm sure I read in some general-audience psychology book once] that for just about every human activity, scientific research included, there's a certain level above which differences in intelligence, at least in the sense of what intelligence tests measure, seem to have very little correlation with differences in effectiveness?

It wouldn't surprise me at all if scientific research could be benefited much more by, say, making research scientists more energetic, or stronger-willed, or keener on hard work, or able to get by with less sleep.

(Of course this is a bit of a digression, since Roko is suggesting intelligence enhancement for a very different population. I think it might have more value there, if there were actually a feasible way to do it.)

Comment deleted 15 June 2009 10:06:05PM [-]
Comment author: gjm 15 June 2009 10:21:18PM 2 points [-]

What I said is not "increased IQ doesn't lead to better research" but that perhaps among people already smart enough to be any good at scientific research increased IQ might make little difference.

Even if that's true, though, there might be substantial benefit in increasing IQ among people who would like to do scientific research but aren't quite good enough at it for it to be a sensible career. But that's not what AIUI Nick Bostrom was suggesting.

Comment author: HughRistik 15 June 2009 11:51:08PM 0 points [-]

The relationship of IQ to scientific achievement might be a step function.

I wonder about this. Isn't it the case [translation: I'm sure I read in some general-audience psychology book once] that for just about every human activity, scientific research included, there's a certain level above which differences in intelligence, at least in the sense of what intelligence tests measure, seem to have very little correlation with differences in effectiveness?

I am curious about how this was measured.

Comment author: MichaelBishop 16 June 2009 12:30:32AM *  1 point [-]

What sort of mechanism would produce a step function? Sounds highly unlikely to me.

Added: I would expect the curve to be smooth.

Comment deleted 16 June 2009 12:47:51AM [-]
Comment author: MichaelBishop 16 June 2009 01:24:23AM 0 points [-]

Miscommunication. My point was only that I expect the function that describes the relationship to be a smooth curve. I wouldn't be too surprised if the relationship between IQ and research productivity is stronger at the high end than in the middle.

Comment author: HughRistik 16 June 2009 01:38:38AM 0 points [-]

Sounds unlikely to me too, but it could explain the phenomenon underlying glm's quote (that above a certain threshold intelligence doesn't make much of a difference in "effectiveness"), assuming that the result is valid (which I would want to know how "effectiveness" was measured).

Comment author: MichaelBishop 16 June 2009 04:10:47PM 0 points [-]

Your question about how they measured "effectiveness" is right on.

My guess is that marginal benefits to IQ depend on the task, and the IQ range. For tasks of medium difficulty, the marginal benefits of IQ will probably increase as one goes from the low-IQ to average, flatten out and then decrease as one gets to a very high range. But higher IQ allows you to efficiently attempt more much more difficult (and arguably important) tasks.

Comment deleted 15 June 2009 11:55:18PM [-]
Comment author: Arenamontanus 16 June 2009 05:51:28PM 2 points [-]

Human ability generally seems to be power-law distributed - the "80-20" rule often hold in research. I'm just checking out Murray's "Human Accomplishment", and this is the impression I get from his data - whether it is valid data remains to be seen. But this might have many other causes, from Matthew effects where widely cited people become even more cited (and maybe get great research environments) to multiplicator effects where productivity is due to multiplicative effects of more or less random factors - only a few gets a lot of them, and the result is a lognormal distribution.

IQ, as ability to make rational inferences in new domains, may be just one of these factors. Low IQ certainly precludes much scientific achievement. There are also selection effects where getting into the right schools or professions require overcoming IQ-loaded hurdles. The real benefits of IQ among geniuses might be smaller than other factors - but having more people with high IQ will certainly not decrease the pool of potential geniuses.

Comment author: Annoyance 16 June 2009 06:56:08PM -1 points [-]

Logical fallacy: those Nobel prize winners do not have increased IQ. Presumably they have high IQ.

If Nobel prize winners all have very high IQs, that tells us that high IQ is a necessary - but not necessarily sufficient - requirement for winning Nobel prizes. And that itself tells us little about what's needed for quality research, even presuming that all Nobels are awarded for quality research. (I happen to know that they aren't, but that's another story.)

What are the Type I and Type II error rates of the Nobel prize award process?

Comment author: MichaelBishop 16 June 2009 07:10:27PM 1 point [-]

What are the Type I and Type II error rates of the Nobel prize award process?

IMO, the more important question is whether the overall system of incentives for scientists is effective.

Comment author: MichaelBishop 16 June 2009 10:57:54PM *  0 points [-]

Imagine we invented a pill which increased everyone's performance on IQ tests by one standard deviation with no side effects (note, I don't expect to see this soon). Further, imagine that all current scientists began taking it. What benefits would you expect to see?

Let me be more specific, assume no funding changes, even though smarter scientists would almost certainly get more funding: how much would Science and Nature have to expand if they did not raise the bar for publication? My estimate: 20% with a 95% confidence interval of [3%, 100%]

Comment author: MichaelBishop 16 June 2009 07:06:33PM -1 points [-]

Roko was arguing somewhat casually but I don't think he is actually reasoning casually. Its fine to discourage this type of comment with a downvote, but starting your reply with the words "Logical fallacy" is unnecessarily harsh in my opinion.

Comment author: thomblake 16 June 2009 09:06:24PM 1 point [-]

Roko's comment seems to contain a logical fallacy. While there might be a reason to make the distinction between the reasoning going on in Roko's argument and the reasoning going on in Roko's head, I have no access to the latter and so must evaluate the former. I don't see what's wrong with Annoyance pointing that out, and calling a fallacious argument fallacious is hardly 'harsh'; at least, it's no harsher than is called for.

Comment author: Annoyance 17 June 2009 07:18:33PM -1 points [-]

While there might be a reason to make the distinction between the reasoning going on in Roko's argument and the reasoning going on in Roko's head, I have no access to the latter and so must evaluate the former.

Even if you had access to the latter, that has no bearing on your evaluation of the former. It's the explicit claims that we're looking at, the ones that are actually communicated, not what the person meant inside their head or what we think they might mean.

Comment author: MichaelBishop 16 June 2009 10:24:18PM -1 points [-]

I encourage efforts to maintain high standards of reasoning, and fairly explicit reasoning. In evaluating harshness, we need to strike a balance between at least three goals: 1. clarity of thought, 2. creating proper incentives for quality contributions which requires punishing mistakes / undesirable contributions, and 3. creating a friendly and respectful atmosphere.

For the record, calling Annoyance's comment, "unnecessarily harsh" was meant to be a minor criticism. There are many factors to consider, in this case I would have suggested that Annoyance replace "Logical fallacy" with "Nitpick." Also see my new comment for Annoyance.

Comment author: Arenamontanus 16 June 2009 05:43:03PM 1 point [-]

There are many traits that would be useful for research and other fields, such as energy, better time management, social ability etc. Intelligence is important for problem-solving in domains where standard rules have not been defined, which might be particularly true in some reasearch. However, it is hard to measure the impact of such ability directly.

David Lubinski and Camilla Persson Benbow, Study of Mathematically Precocious Youth After 35 Years, Perspectives on Psychological Science, 1,316-343 www.vanderbilt.edu/Peabody/SMPY/DoingPsychScience2006.pdf has some intriguing data. They followed up the top percent scorers and compared the uppermost and lowest quartile of this already elite group. Unsurprisingly they were on average doing great, and the top group also earned more and had about six times the rate of tenure at top US universities. But that could just be pure competitive ability rather than any individually or socially useful outcome. The interesting result was that the number of doctorates and percent earning patents was about twice in the top quartile. Doctorates and patents are after all a form of measure of actually having achieved something, and presumably a society is better off if bright people produce more patentable ideas. This IMHO strengthens the idea that we would see gains from cognition enhancement even among the brightest.

However, I think the biggest economic and social impact will be due to intelligence among the great mass of people - reduction of costs and friction due to stupidity, short term thinking, mistakes and other limitations, increased benefits from better cooperation (smart people do better on iterated prisoners dilemma games and have longer time horizons) and ability to manage more complex systems.

A "emotional intelligence enhancer" might be socially beneficial too - there is no reason to think "pure" cognitive function is the end of things we might rationally want to see others enhance.

Comment author: steven0461 15 June 2009 08:14:17PM *  0 points [-]

More intelligence also means more competence at doing potentially world-destroying things, like AI/upload/nano/supervirus research. It does seem to me like the anti-risk effect from intelligence enhancement would somewhat outweigh the pro-risk effect, but I'm not sure.

Comment author: AngryParsley 15 June 2009 10:48:46PM 7 points [-]

If more intelligence is bad, is less good? Do you think current levels of intelligence are optimal? If so, that would be an amazing coincidence.

Comment author: Peter_de_Blanc 16 June 2009 04:57:38AM 3 points [-]

I don't think that current levels of intelligence are optimal, but if they were, it wouldn't be a coincidence. Humans are adaptation-executers, and genes make implicit assumptions about their environment. In particular, certain adaptations might be disrupted by changing the average intelligence.

Comment author: AngryParsley 16 June 2009 09:49:38AM *  3 points [-]

If you had the option to increase your intelligence, would you decline because you were worried about certain adaptations being disrupted? The modern world is so different from our EEA that I can't buy your argument.

Disrupting adaptations can be a good thing. Birth control helps prevent overpopulation. Courts help settle disputes without violence. Even rational thought involves recognizing and changing (disrupting?) the typical thought patterns of our adapted brains.

Comment author: Vladimir_Nesov 16 June 2009 12:58:28PM *  4 points [-]

The fact that modern world changed our values in a way that ancient people won't appreciate on reflection is a bad thing for the ancient people. To us, it'd be bad if we reverted some of these changes, and likewise if we introduced new changes that have negative side effects from the current point of view (on reflection).

It's hard to "increase intelligence" without wreaking some of the values, brain isn't designed for upgrade. It's the same problem as with trying to change emotions.

Comment deleted 16 June 2009 01:15:25PM [-]
Comment author: MichaelBishop 17 June 2009 12:52:22AM 2 points [-]

I definitely think that values, however defined, would change significantly with such a 10 point IQ change (btw, I consider this very large). And I think it would probably be a good thing.

Comment author: steven0461 16 June 2009 04:37:25AM *  2 points [-]

From the upvotes it seems people think this is some sort of devastating counter-point. Yes, if more intelligence is bad, less is good. No, current levels of intelligence are not optimal.

If there's a knockdown argument against the idea that increased average intelligence might cause net increased risk, it should go something like "people will just do the same thing, but slower". I think this works but, again, I'm not sure. Either way, the net decrease in risks from intelligence enhancement is less than would seem to be the case if you considered just the upsides.

Comment author: Arenamontanus 16 June 2009 06:12:06PM 2 points [-]

More intelligence means bigger scope for action, and more ability to get desired outcomes. Whether more intelligence increases risk depends on the distribution of accidentally bad outcomes in the new scope (and how many old bad outcomes can be avoided), and whether people will do malign things. On average very few people seem to be malign, so the main issue is likely more the issue of new risks.

Looking at the great deliberate human-made disasters of the past suggests that they were often more of a systemic nature (societies allowing nasty people or social processes to run their course; e.g. democides and genocides) than due to individuals or groups successfully breaking rules (e.g. terrorism). This is actually a reason to support cognitive enhancement if it can produce more resilient societies less prone to systemic risks.

Comment author: steven0461 16 June 2009 08:18:58PM *  0 points [-]

One possibility I have in mind is if current rationalist ideas need a certain amount of time to slosh around and pervade the population before technology (fed by intelligence) grows enough for them to really start mattering.

Comment author: gwern 17 June 2009 03:39:14AM 1 point [-]

It does seem to me like the anti-risk effect from intelligence enhancement would somewhat outweigh the pro-risk effect, but I'm not sure.

It is a hard subject to argue about. If I wanted to criticize you, I could say that higher IQs make uploads less economically attractive as they will start off with a smaller advantage, and higher IQs likely would make uploads intrinsically more difficult. And if I wanted to criticize my criticism, I could say that by making people in general smarter, we increase the cost of labor, which makes mechanical substitution ever more attractive (and mechanical substitution for reasonably complicated tasks implies development of AI/uploads).

Comment author: HughRistik 15 June 2009 11:48:10PM 0 points [-]

Yes, increasing intelligence would increase the variance of quality of outcomes for humanity. And the hope is that intelligence would also increase the mean quality of outcome, such that the expected value would be higher.

Comment author: hrishimittal 15 June 2009 09:42:34PM 0 points [-]

It's interesting speculation but it assumes that people use all of their current intelligence. There is still the problem of akrasia - a lot of people are perfectly capable of becoming 'smarter' if only they cared to think about things at all. Sure, they could still go mad infallibly but it would be better than not even trying.

Are you implying that more IQ may help in overcoming akrasia?

Comment deleted 15 June 2009 10:03:37PM [-]
Comment author: hrishimittal 16 June 2009 12:53:29AM 2 points [-]
Comment author: taw 15 June 2009 08:19:02PM 0 points [-]

if we could improve the intelligence of the average voter by 10 IQ points, imagine how much saner the political process would look

It's highly non-obvious that it would have significant effects. Political process is imperfect but very pragmatic - what makes a lot of sense as there's only as much good an improved political process can do, and breaking it can cause horrible suffering. So current approach of gradual tweaks is a very safe alternative, even if it offends people's idealistic sensibilities.

Comment author: MichaelBishop 16 June 2009 01:11:22AM 3 points [-]

IQ is positively correlated with sharing economists' views on economic policy Therefore it seems likely people would vote differently and that this would translate into policy changes. I would expect other belief changes would be likely if IQ were increased.

Comment author: Arenamontanus 16 June 2009 06:01:50PM 2 points [-]

Here is a simple model. Assume you need a certain intelligence to understand a crucial, policy-affecting idea (we can make this a fuzzy border and talk about this in distribution to make it more realistic later). If you are below this level your policy choices will depend on taking up plausible-sounding arguments from others, but it will be uncorrelated to the truth. Left alone such a population will describe some form of random walk with amplification, ending up with a random decision. If you are above the critical level your views will be somewhat correlated with the truth. Since you are affecting others when you engage in political discourse, whether over the breakfast table or on TV, you will have an impact on other people, increasing their chance of agreeing with you. This biases the random walk of public opinion slightly in favour of truth.

From most models of political agreement formation I have seen, even a pretty small minority that get biased in a certain direction can sway a large group that just picks views based on neighbours. This would suggest that increasing the set of people smart enough to get the truth would substantially increase the likelihood of a correct group decision.

Comment author: ChrisHibbert 16 June 2009 06:07:59AM *  1 point [-]

My rebuttal to

imagine how much saner the political process would look

is to point at the work of Tullock and Buchanon on Public Choice theory. Basically, the take away is that politicians and bureaucrats respond to incentives. If the voting public were smarter, politicians' behavior would be different during elections, and the politicians would try to make their behavior in office look different. But they would still have an incentive to look like they were addressing problems rather than an incentive to actually solve them. It's much harder than 10 IQ points to align those outcomes.

And bureaucrats and middle managers in government would still face the same incentives about obfuscating results, multiplying staffing and budgets, and ensuring that projects and bureaucracies have staying power.

Comment author: MichaelBishop 17 June 2009 12:47:13AM *  1 point [-]

Public choice theory is important, but I still think there is good reason to believe increasing average IQ by such a huge amount would help. First, because better informed voters improves the incentives for politicians. Second, because the relatively bad incentives politicians face is not the only constraint on better goverment.

Comment author: asciilifeform 15 June 2009 09:23:56PM -1 points [-]

It's highly non-obvious that it would have significant effects

The effects may well be profound if sufficiently increased intelligence will produce changes in an individual's values and goal system, as I suspect it might.

At the risk of "argument from fictional evidence", I would like to bring up Poul Anderson's Brain Wave, an exploration of this idea (among others.)

Comment author: wuwei 16 June 2009 02:15:22AM -1 points [-]

I'm not sure intelligence enhancement alone is sufficient. It'd be better to first do rationality enhancement and then intelligence enhancement. Of course that's also much harder to implement but who said it would be easy?

It sounds like you think intelligence enhancement would result in rationality enhancement. I'm inclined to agree that there is a modest correlation but doubt that it's enough to warrant your conclusion.

Comment deleted 16 June 2009 01:37:05PM [-]
Comment author: wuwei 16 June 2009 11:11:03PM 0 points [-]

I suspect you aren't sufficiently taking into account the magnitude of people's irrationality and the non-monotonicity of rationality's rewards. I agree that intelligence enhancement would have greater overall effects than rationality enhancement, but rationality's effects will be more careful and targeted -- and therefore more likely to work as existential risk mitigation.

Comment author: MichaelBishop 17 June 2009 01:44:27AM 0 points [-]

...the non-monotonicity of rationality's rewards

Could you elaborate on the shape of the rewards to rationality?

Comment author: gwern 17 June 2009 03:43:06AM 1 point [-]

This was covered in some LW posts a while ago (which I cannot be arsed to look up and link); the paradigmatic example in those posts, I think, was a LWer who used to be a theist and have a theist girlfriend, but reading OB/LW stuff convinced him of the irrationality of God. Then his girlfriend left his hell-bound hide for greener pastures, and his life is in general poorer than when he started reading OB/LW and striving to be more rational.

The suggestion is that rationality/irrationality is like a U: you can be well-off as a bible-thumper, and well-off as a stone-cold Bayesian atheist, but the middle is unhappy.

Comment author: Eliezer_Yudkowsky 17 June 2009 09:00:41AM 2 points [-]

and his life is in general poorer

I'm not sure this is a fair statement. He did say he wouldn't go back if he had the choice.

Comment author: wuwei 17 June 2009 02:33:08AM *  0 points [-]

Increases in rationality can sometimes lead with some regularity to decreasing knowledge or utility (hopefully only temporarily and in limited domains).

Comment author: PhilGoetz 16 June 2009 06:02:32PM *  0 points [-]

My observations here on LW, thanks to the karma system, lead me to believe there is no threshold effect. People always have great difficulty following the ideas of someone a level above them, regardless of what level they are at. Eliezer's posts are so friggin' long because they are designed to be understood by people a level below him.

I suspect, as I've said repeatedly on LW, that increasing the baseline of intelligence would only lead us to construct a more elaborate society, with more complicated problems, and an even greater chance of catastrophic failure. So the question, if you're interested in decreasing existential risk, is whether we can reduce bias without raising the level of intelligence.

A study reported in Science about a month ago indicated that people in Sweden are more able to be laughed at than people of any other nation, while people in Africa and the Middle East are the least-able to stand being laughed at, and the least-trustful of smiling people. I'm intrigued by the idea that simple social conventions may be as effective in avoiding bad social results, as are better reasoning abilities. It may be better to use your reasoning ability to choose good social conventions, and promote those, than to promote rationality.

Comment author: loqi 16 June 2009 07:26:56PM 3 points [-]

So the question, if you're interested in decreasing existential risk, is whether we can reduce bias without raising the level of intelligence.

It seems that you should also be asking how we can reduce the level of intelligence.