more_wrong

Yet who prohibits? Who prevents it from happening?

Eliezer seems absurdly optimistic to me. He is relying on some unseen entity to reach in and keep the laws of physics stable in our universe. We already see lots of evidence that they are not truly stable, for example we believe in both the electroweak transition and earlier transitions, of various natures depending on your school of physics. We /just saw/ in 1998 that unknown laws of physics can creep up and spring out by surprise, suddenly 'taking over' 74 percent of the Universe's postulated energy supply.

Who is keeping the clockwork universe running? Or, why is the hardware and software... (read more)

I love the idea of an intelligence explosion but I think you have hit on a very strong point here:

In fact, as it picks off low-hanging fruit, new ideas will probably be harder and harder to think of. There's no guarantee that "how smart the AI is" will keep up with "how hard it is to think of ways to make the AI smarter"; to me, it seems very unlikely.

In fact, we can see from both history and paleontology that when a new breakthrough was made in "biologicial technology" like the homeobox gene or whatever triggered the PreCambrian explosion of diversity, that when self-modification (here a 'self' isn't one meat body, it's... (read 1093 more words →)

Is there an underlying problem of crying wolf; too many warning messages obscure the ones that are really matters of life and death?

This is certainly an enormous problem for interface design in general for many systems where there is some element of danger. The classic "needle tipping into the red" is an old and brilliant solution for some kinds of gauges - an analogue meter where you can see the reading tipping toward a brightly marked "danger zone", usually with a 'safe' zone and an intermediate zone also marked, has surely prevented many accidents. If the pressure gauge on the door had such a meter where green meant "safe to open... (read more)

stay away from this community I responded to this suggestion but deleted the response as unsuitable because it might embarass you. I would be happy to email my reply if you are interested.

we'd probably convince you such perma-death would be the highly probable outcome

Try reading what I said in more detail in both the post I made that you quoted and my explanation of how there might be a set of worlds of very small measure. Then go read Eliezer Yudkowsky's posts on Many Worlds (or crack a book by Deutsch or someone, or check Wikipedia.) Then reread the clause you published here which I just quoted above, and see if you still... (read more)

Now, whether that distributed information is 'experiencing' anything is arguable,

As far as I know, the latter is what people are worrying about when they worry about ceasing to exist.

Ahhh... that never occurred to me. I was thinking entirely in terms of risk of data loss.

(Which is presumably a reason why your comment's been downvoted a bunch; most readers would see it as missing the point.)

I don't understand the voting rules or customs. Downvoting people who see things from a different perspective is... a custom designed to keep out the undesirables? I am sorry I missed the point but I learned nothing from the downvoting. I learned a great deal... (read more)

People talk about the grey goo scenario, but I actually think that is quite silly because there is already grey goo all over the planet in the form of life" ... " nothing CAN do this, because nothing HAS done it."

The grey goo scenario isn't really very silly. We seem to have had a green goo scenario around 1.5 to 2 billion years ago that killed off many or most critters around due to release of deadly deadly oxygen; if the bacterial ecosystem were completely stable against goo scenarios this wouldn't have happened. We have had mini goo scenarios when for example microbiota pretty well adapted to one... (read 638 more words →)