Comment on Welcome to Less Wrong! (5th thread, March 2013)

Wed, 18 Sep 2013 00:24:54 GMT

Mostly, I prefer not to conflate them because our shared understanding of upload is likely much better-specified than our shared understanding of augment.

I agree completely; that was my point as well.

Except that, as you say later, you have confidence about what those supposedly incomprehensible values would or wouldn't contain.

By analogy, I personally neither love nor hate individual insects; they are too far beneath me.

Turning that analogy around.... I suspect that if I remembered having been an insect and then later becoming a human being, and I believed that was a reliably repeatable process, both my emotional stance with respect to the intrinsic value of insect lives and my pragmatic stance with respect to their instrumental value would be radically different than they are now and far more strongly weighted in the insects' favor.

With respect to altruism and vast intelligence gulfs more generally... I dunno. Five-day-old infants are much stupider than I am, but I generally prefer that they not suffer. OTOH, it's only a mild preference; I don't really seem to care all that much about them in the abstract. OTGH, when made to think about them as specific individuals I end up caring a lot more than I can readily justify over a collection. OT4H, I see no reason to expect any of that to survive what we're calling "intelligence augmentation", as I don't actually think my cognitive design allows my values and my intelligence (ie my optimize-environment-for-my-values) to be separated cleanly. OT5H, there are things we might call "intelligence augmentation", like short-term-memory buffer-size increases, that might well be modular in this way.

Discuss