Nonsuperintelligent AI threat

9 PlaidX 18 April 2011 09:21PM

Something that I don't think I've seen discussed here is the threat posed by an AI which is smarter than we are when it comes to computer security without being generally intelligent.

Suppose there were a computer virus that could read code, to the extent of looking at the programs on a computer, seeing how they process input from the internet, and how they could be exploited to run arbitrary code. Historically, viruses have been annoyances. How much smarter would a virus have to be in order to be a threat on the scale of say, a planetary EMP burst?

Rationality vs. intelligence

6 PlaidX 20 March 2011 05:16AM

This site often speaks of rationality and intelligence as though they were the same thing, and that someone, by becoming more rational, becomes more intelligent for practical purposes.

 

Certainly it seems to me that this must be to some extent the case, but what is the exchange rate? If a person has an IQ of 100, and then they spend a year on lesswrong, reading all the sequences and taking the advice to heart, training their skills and identifying their biases and all that, at the end of it, presumably their raw IQ score is still 100, but if we measure how they do on correlated indicators regarding their lifestyle or something, should we expect to see them, in some way, living the life of a smarter person? How much smarter?

 

How many points of IQ would you be willing to give up to retain what you have learned from this site?

 

Personally I would answer "less than one". It seems like it SHOULD be useful, but it doesn't really feel like it is.

Contrived infinite-torture scenarios: July 2010

24 PlaidX 23 July 2010 11:54PM

This is our monthly thread for collecting arbitrarily contrived scenarios in which somebody gets tortured for 3^^^^^3 years, or an infinite number of people experience an infinite amount of sorrow, or a baby gets eaten by a shark, etc. and which might be handy to link to in one of our discussions. As everyone knows, this is the most rational and non-obnoxious way to think about incentives and disincentives.

  • Please post all infinite-torture scenarios separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
  • No more than 5 infinite-torture scenarios per person per monthly thread, please.

lessmeta

6 PlaidX 22 December 2009 05:57PM

The social bookmarking site metafilter has a sister site called metatalk, which works the same way but is devoted entirely to talking about metafilter itself. Arguments about arguments, discussions about discussions, proposals for changes in site architecture, etc.

Arguments about arguments are often less productive than the arguments they are about, but they CAN be quite productive, and there's certainly a place for them. The only thing wrong with them is when they obstruct the discussion that spawned them, and so the idea of splitting off metatalk into its own site is really quite a clever one.

Lesswrong's problem is a peculiar one. It is ENTIRELY devoted to meta-arguments, to the extent that people have to shoehorn anything else they want to talk about into a cleverly (or not so cleverly) disguised example of some more meta topic. It's a kite without a string.

Imagine if you had been around the internet, trying to have a rational discussion about topic X, but unable to find an intelligent venue, and then stumbling upon lesswrong. "Aha!" you say. "Finally a community making a concerted effort to be rational!"

But to your dismay, you find that the ONLY thing they talk about is being rational, and a few other subjects that have been apparently grandfathered in. It's not that they have no interest in topic X, there's just no place on the site they're allowed to talk about it.

What I propose is a "non-meta" sister site, where people can talk and think about anything BESIDES talking and thinking. Well, you know what I mean.

Yes?

Lore Sjöberg's Life-Hacking FAQK

0 PlaidX 20 October 2009 04:10PM

Lore Sjöberg's Life-hacking FAQK

Pretty self-explanatory. Also available as a podcast.

Applying Double Standards to ‘‘Divisive’’ Ideas

3 PlaidX 19 October 2009 12:36PM

This is a commentary by Linda Gottfredson on a paper by Hunt and Carlson about a paper by Richard Nisbett regarding studies done by Arthur Jensen. It's ultimately about race and intelligence, but it seemed meta enough to link to here.

Warning: PDF

Applying Double Standards to ‘‘Divisive’’ Ideas

The Presumptuous Philosopher's Presumptuous Friend

3 PlaidX 05 October 2009 05:26AM

One day, you and the presumptuous philosopher are walking along, arguing about the size of the universe, when suddenly Omega jumps out from behind a bush and knocks you both out with a crowbar. While you're unconscious, she builds two hotels, one with a million rooms, and one with just one room. Then she makes a million copies of both of you, sticks them all in rooms, and destroys the originals.

You wake up in a hotel room, in bed with the presumptuous philosopher, with a note on the table from Omega, explaining what she's done.

"Which hotel are we in, I wonder?" you ask.

"The big one, obviously" says the presumptuous philosopher. "Because of anthropic reasoning and all that. Million to one odds."

"Rubbish!" you scream. "Rubbish and poppycock! We're just as likely to be in any hotel omega builds, regardless of the number of observers in that hotel."

"Unless there are no observers, I assume you mean" says the presumptuous philosopher.

"Right, that's a special case where the number of observers in the hotel matters. But except for that it's totally irrelevant!"

"In that case," says the presumptuous philosopher, "I'll make a deal with you. We'll go outside and check, and if we're at the small hotel I'll give you ten bucks. If we're at the big hotel, I'll just smile smugly."

"Hah!" you say. "You just lost an expected five bucks, sucker!"

You run out of the room to find yourself in a huge, ten thousand story attrium, filled with throngs of yourselves and smug looking presumptuous philosophers.