soreff comments on Complexity: inherent, created, and hidden - Less Wrong

8 Post author: Swimmer963 14 September 2011 02:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (49)

You are viewing a single comment's thread.

Comment author: soreff 14 September 2011 10:16:34PM 2 points [-]

That being said, there is probably such thing as too much hidden complexity. If most of the information in a given society is hidden, embodied by non-human intelligences, then life as a garden-variety human would be awfully boring.

That wouldn't be my main worry. Setting aside general AI, one problem with smaller amounts of hidden complexity is that we rarely hide it completely successfully. Automation has failure modes. Abstractions leak. If some instance of complexity hiding is so successful that it is ok that the rare failures turn into research problems, that's ok. If another instance is sufficiently fragile that its users regularly bump into its failures, and become familiar with how to recover from them (but where it is still reliable enough to be useful), that's ok. The intermediate case, where it is reliable enough for most users to forget what is hidden, but fragile enough that recovering from its failures can't be neglected - that's a problem...