The utility-maximizing complexity of conscious beings
An AI that runs conscious beings has finite resources. Complex conscious beings consume more of these resources than do simpler conscious beings. Suppose, for purposes of discussion, that a friendly AI would maximize the aggregate utility of the beings it runs by modulating their level of complexity. The utility-maximizing level...