Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: pnrjulius 12 June 2013 08:22:38PM *  1 point [-]

This question is broader than just AI. Economic growth is closely tied to technological advancement, and technological advancement in general carries great risks and great benefits.

Consider nuclear weapons, for instance: Was humanity ready for them? They are now something that could destroy us at any time. But on the other hand, they might be the solution to an oncoming asteroid, which could have destroyed us for millions of years.

Likewise, nanotechnology could create a grey goo event that kills us all; or it could lead to a world without poverty, without disease, where we all live as long as we like and have essentially unlimited resources.

It's also worth asking whether slowing technology would even help; cultural advancement seems somewhat dependent upon technological advancement. It's not clear to me that had we taken another 100 years to get nuclear weapons we would have used them any more responsibly; perhaps it simply would have taken that much longer to achieve the Long Peace.

In any case, I don't really see any simple intervention that would slow technological advancement without causing an enormous amount of collateral damage. So unless you're quite sure that the benefit in terms of slowing down dangerous technologies like unfriendly AI outweighs the cost in slowing down beneficial technologies, I don't think slowing down technology is the right approach.

Instead, find ways to establish safeguards, and incentives for developing beneficial technologies faster. To some extent we already do this: Nuclear research continues at CERN and Fermilab, but when we learn that Iran is working on similar technologies we are concerned, because we don't think Iran's government is trustworthy enough to deal with these risks. There aren't enough safeguards against unfriendly AI or incentives to develop friendly AI, but that's something the Singularity Institute or similar institutions could very well work on. Lobby for legislation on artificial intelligence, or raise funds for an endowment that supports friendliness research.

Comment author: thomblake 03 April 2009 03:28:27PM 4 points [-]

just kill all of the Nazis?

Are you serious? A German factory-worker with a family (possibly a Nazi himself who happens to not be anti-semitic) should personally kill all the Nazis? This seems to you like a way to cut this particular knot?

Comment author: pnrjulius 13 September 2012 11:50:22PM -1 points [-]

Well, ultimately, that was sort of the collective strategy the world used, wasn't it? (Not quite; a lot of low-level Nazis were pardoned after the war.)

And you can't ignore the collective action, now can you?

Comment author: wedrifid 05 July 2012 07:33:57PM 5 points [-]

If so, this is rather irrational,

It isn't so. It's more a relative thing---"not quite as extremely biased towards academia as the average group of this level of intellectual orientation can be expected to be".

given that probably every high-profile/high-status contributor to this forum, with the notable exception of EY, either works in academia or is being/has been trained in academia.

Luke has minimal official academic training too. Mind you he is more academic in practice than most people (probably most academics too, come to think of it.)

Comment author: pnrjulius 06 July 2012 11:00:41PM -1 points [-]

It's more a relative thing---"not quite as extremely biased towards academia as the average group of this level of intellectual orientation can be expected to be".

If so, then we're actually more rational right? Because we're not biased against academia as most people are, and aren't biased toward academia as most academics are.

Comment author: DanArmak 05 July 2012 06:30:51PM 5 points [-]

Access to information and social support/reinforcement is a huge limiting factor.

Access to labs, equipment, technicians, funding is an even greater factor. Only mathematicians can really afford to work from home. (And now, computer scientists and computational-xxx people have joined them.)

Comment author: pnrjulius 06 July 2012 10:58:49PM 0 points [-]

It's not quite so dire. You can't do experiments from home usually, but you can interpret experiments from home thanks to Internet publication of results. So a lot of theoretical work in almost every field can be done from outside academia.

Comment author: shminux 05 July 2012 05:53:00AM *  5 points [-]

I agree with most of it, though the point about academia is a bit contrived.

True, there is a lot of negative selection before you get a cushy job the usual way, but you can certainly bypass quite a few obstacles if you are exceptionally good. For example, solve any of the open problems in math or physics, post a preprint on arxiv.org (well, you may need someone to vouch for you, but that's not really an issue) and you are all set.

Unfortunately, I cannot recall a single discovery in physics in the last half a century that was not made by someone who jumped through the usual hoops. I have met, however, an occasional person who learned grad school-level stuff on their own, but they did not manage to go any farther. My suspicion is therefore that all that negative selection in science, while annoying, does not do a lot of harm compared to potential alternatives. While it filters out some good people, it probably does not reject the very best, otherwise we would see an occasional example of someone making a significant discovery outside academia.

Comment author: pnrjulius 06 July 2012 10:57:35PM 1 point [-]

otherwise we would see an occasional example of someone making a significant discovery outside academia.

Should we all place bets now that it will be Eliezer?

Comment author: Swimmer963 05 July 2012 04:57:12PM 8 points [-]

Really good post...it makes a point that is completely new to me, which is always nice.

It does occur to me that the current (negative selection) system would reward "hard work" more, relative to "talent", than a positive selection system. (In quotation marks because those are both metrics that are hard to measure separately from one another.) Someone who is very conscientious and hard-working is likely to compensate for wherever areas they're weaker, in terms of "natural talent", however you define that.

My first, emotional reaction to your post was "I would be screwed in a positive selection system!" As someone who's above average in a lot of areas, not really exceptional in any, and obsessively hard-working enough that it's a running joke among my friends, I like the current system just fine (although I'm not in academia.) I don't know if conscientiousness would have a bigger long-term effect on results than innate brilliance; it probably depends on what field you're talking about.

My intuition says that a positive selection system would probably be a good idea in fields where there is big variance in natural ability, i.e. math or physics, and less so in fields like medicine where a lot of "talent" depends on how willing you are to work hard and keep improving over your whole career.

Comment author: pnrjulius 06 July 2012 10:54:30PM 0 points [-]

Negative selection may be good, actually, for the vast majority of people who are ultimately going to be mediocre.

It seems like it may hurt the occasional genius... but then again, there are a lot more people who think they are geniuses than really are geniuses.

Comment author: CronoDAS 06 July 2012 08:55:49AM *  1 point [-]

It’s apparently so important that people really care about performance – as opposed to, say, in medicine, where we exclude brilliant doctors if they don’t have the stamina to work ninety hours a week.

How much does this actually matter, I wonder? Is there really that big a difference between the best doctor in a group of 100 and the 10th best doctor in that same group? (The 10th best golfer in a tournament doesn't take home the trophy, but the 10th best doctor in the hospital can still do a fine job treating a broken arm.)

Comment author: pnrjulius 06 July 2012 10:52:06PM -1 points [-]

In treating broken arms? Minimal difference.

In discovering new nanotechnology that will revolutionize the future of medicine? Literally all the difference in the world.

Comment author: johnlawrenceaspden 06 July 2012 12:35:39PM 2 points [-]

I've never understood the reason for giving grades A-E or fail, like we do for O and A levels, or I:II:III:fail, like we do for degrees.

My father's O-levels gave a percentile ranking, so he was e.g. in the 83rd percentile in the country for history.

So we must have changed over at some point. Does anyone know why? It's always looked like throwing information away to me, and it's also unfair to people on the grade boundaries.

Of course this may be motivated thinking on my part, I'd much rather have had a string of 100s for my exams than a string of As, and I'd much prefer to have got a 75 for my degree than a II (which covered percentiles 25-75) !!

Comment author: pnrjulius 06 July 2012 10:51:05PM 0 points [-]

I think a lot of people don't like using percentiles because they are zero-sum: Exactly 25% of the class is in the top 25%, regardless of whether everyone in the class is brilliant or everyone in the class is an idiot.

Comment author: Dreaded_Anomaly 06 July 2012 05:34:48PM 6 points [-]

I've just realized that I have been treating dating as a negative selection process. This might explain the lack of success.

Comment author: pnrjulius 06 July 2012 10:49:21PM 3 points [-]

Well, you want some negative selection: Choose dating partners from among the set who are unlikely to steal your money, assault you, or otherwise ruin your life.

This is especially true for women, for whom the risk of being raped is considerably higher and obviously worth negative selecting against.

Comment author: pnrjulius 06 July 2012 10:44:02PM *  3 points [-]

I don't think it's quite true that "fail once, fail forever", but the general point is valid that our selection process is too much about weeding-out rather than choosing the best. Also, academic doesn't seem to be very good at the negative selection that would make sense, e.g. excluding people who are likely to commit fraud or who have fundamentally anti-scientific values. (Otherwise, how can you explain how Duane Gish made it through Berkeley?)

View more: Next