arundelo comments on Rationality Quotes August 2014 - Less Wrong

8 Post author: RolfAndreassen 04 August 2014 03:12AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (233)
Tags: quotes

Comments (233)

You are viewing a single comment's thread.

Comment author: arundelo 05 August 2014 05:19:21AM 26 points [-]

That's why I'm skeptical of people who look at some catastrophic failure of a complex system and say, "Wow, the odds of this happening are astronomical. Five different safety systems had to fail simultaneously!" What they don't realize is that one or two of those systems are failing all the time, and it's up to the other three systems to prevent the failure from turning into a disaster.

-- Raymond Chen

Comment author: satt 07 August 2014 01:16:20AM 8 points [-]

In other words, some of the slices in one's Swiss cheese model are actually missing entirely.

Comment author: dspeyer 05 August 2014 09:24:34PM 8 points [-]

Correlary: if you're running a system for which five simultaneous failures is a disaster, monitor each safety system seperately and treat any three simultaneous failures as if it were a disaster.

Comment author: Gunnar_Zarncke 07 August 2014 10:02:13PM 7 points [-]

Also known as Fundamental Failure Mode. From Systemantics:

System failure

The Fundamental Failure-Mode Theorem (F.F.T.): Complex systems usually operate in failure mode.

A complex system can fail in an infinite number of ways. (If anything can go wrong, it will.) (See Murphy's law.)

The mode of failure of a complex system cannot ordinarily be predicted from its structure.

The crucial variables are discovered by accident.

The larger the system, the greater the probability of unexpected failure.

"Success" or "Function" in any system may be failure in the larger or smaller systems to which the system is connected.

The Fail-Safe Theorem: When a Fail-Safe system fails, it fails by failing to fail safe.

Powered by Reddit Powered by Reddit