Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Bryan-san comments on A few misconceptions surrounding Roko's basilisk - Less Wrong

39 Post author: RobbBB 05 October 2015 09:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (125)

You are viewing a single comment's thread.

Comment author: Bryan-san 06 October 2015 02:57:41AM 3 points [-]

At the end of the day, I hope this will have been a cowpox situation and lead people to be better informed at avoiding actual dangerous information hazard situations in the future.

I seem to remember reading a FAQ for "what to do if you think you have an idea that may be dangerous" in the past. If you know what I'm talking about, maybe link it at the end of the article?

Comment author: pico 06 October 2015 07:52:33AM 2 points [-]

I think genuinely dangerous ideas are hard to come by though. They have to be original enough that few people have considered them before, and at the same time have powerful consequences. Ideas like that usually don't pop into the heads of random, uninformed strangers.

Comment author: RichardKennaway 06 October 2015 10:00:22AM 4 points [-]

I think genuinely dangerous ideas are hard to come by though.

Daniel Dennett wrote a book called "Darwin's Dangerous Idea", and when people aren't trying to play down the basilisk (i.e. almost everywhere), people often pride themselves on thinking dangerous thoughts. It's a staple theme of the NRxers and the manosphere. Claiming to be dangerous provides a comfortable universal argument against opponents.

I think there are, in fact, a good many dangerous ideas, not merely ideas claimed to be so by posturers. Off the top of my head:

  • Islamic fundamentalism (see IS/ISIS/ISIL).
  • The mental is physical.
  • God.
  • There is no supernatural.
  • Utilitarianism.
  • Superintelligent AI.
  • How to make nuclear weapons.
  • Atoms.

Ideas like that usually don't pop into the heads of random, uninformed strangers.

They do, all the time, by contagion from the few who come up with them, especially in the Internet age.

Comment author: HungryHobo 07 October 2015 01:46:14PM *  5 points [-]

There are some things which could be highly dangerous which are protected almost purely by thick layers of tedium.

Want to make nerve gas? well if you can wade through a thick pile of biochemistry textbooks the information isn't kept all that secret.

Want to create horribly deadly viruses? ditto.

The more I learned about physics, chemistry and biology the more I've become certain that the main reason that major cities have living populations is that most of the people with really deep understanding don't actually want to watch the world burn.

You often find that extremely knowledgeable people don't exactly hide knowledge but do put it on page 425 of volume 3 of their textbook, written in language which you need to have read the rest to understand. Which protects it effectively from 99.99% of the people who might use it to intentionally harm others.

Comment author: NancyLebovitz 07 October 2015 02:19:10PM *  1 point [-]

Argument against: back when cities were more flamable, people didn't set them on fire for the hell of it.

On the other hand, it's a lot easier to use a timer and survive these days, should you happen to not be suicidal.

"I want to see the world burn" is a great line of dialogue, but I'm not convinced it's a real human motivation. Um, except that when I was a kid, I remember wishing that this world was a dream, and I'd wake up. Does that count?

Second thought-- when I was a kid, I didn't have a method in mind. What if I do serious work with lucid dreaming techniques when I'm awake? I don't think the odds of waking up into being a greater intelligence are terribly good, nor is there a guarantee that my live would be better. On the other hand, would you hallucinations be interested in begging me to not try it?

Comment author: itaibn0 07 October 2015 11:08:55PM -1 points [-]

Based on personal experience, if you're dreaming I don't recommend trying to wake yourself up. Instead, enjoy your dream until you're ready to wake up naturally. That way you'll have far better sleep.

Comment author: CAE_Jones 08 October 2015 01:10:18AM -1 points [-]

Based on personal experience, I would have agreed with you, right up until last year, when I found myself in the rather terrifying position of being mentally aroused by a huge crash in my house, but unable to wake up all the way for several seconds afterward, during which my sleeping mind refused to reject the "something just blew a hole in the building we're under attack!" hypothesis.

(It was an overfilled bag falling off the wall.)

But absent actual difficulty waking for potential emergencies, sure; hang out in Tel'aran'rhiod until you get bored.

Comment author: pico 07 October 2015 03:17:28AM 3 points [-]

Sorry, should have defined dangerous ideas better - I only meant information that would cause a rational person to drastically alter their behavior, and which would be much worse for society as a whole when everyone is told at once about it.

Comment author: OrphanWilde 08 October 2015 02:20:39PM 3 points [-]

Depends on your definition of "Dangerous." I've come across quite a few ideas that tend to do -severe- damage to the happiness of at least a subset of those aware of them. Some of them are about the universe; things like entropy. Others are social ideas, which I won't give an example of.

Comment author: Bryan-san 06 October 2015 10:33:07PM -1 points [-]

I hope they're as hard to come by as you think they are.

Alternatively, Roko could be part of the 1% of people who think of a dangerous idea (assuming his basilisk is dangerous) and spread it on the internet without second guessing themselves. Are there 99 other people who thought of dangerous ideas and chose not to spread them for our 1 Roko?

Comment author: jam_brand 08 October 2015 05:52:25PM 1 point [-]

Perhaps the article you read was Yvain's The Virtue of Silence?