= 27d567bc2d48d9f40e9ccc20ea370b15)
I did enjoy the rest of the chapter however. Quirrel's statements about horcruxes were initially surprising - if he is telling the truth, then how is he still alive? If not, then wouldn't he want Harry experimenting with horcruxes in order to turn him to the dark side?
Perhaps he is a Horcrux transfer (as long speculated) but a failed one; introspecting about how different he is from his memories of 'himself', he would realize 'he' hadn't survived and all that was left was a weird mishmash of Monroe's personality and Voldemort's memories, and this was entirely worthless as immortality.
What argument could be more convincing to Quirrel than personally embodying the failure of horcruxes as an immortality strategy?
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Okay, wow, I don't know if I quite understand any of this, but this part caught my attention:
I have sometimes wondered whether the best way to teach an AI a human's utility function would not be to program it into the AI directly (because that will require that we figure out what we really want in a really precisely-defined way, which seems like a gargantuan task), but rather, perhaps the best way would be to "raise" the AI like a kid at a stage where the AI would have minimal and restricted ways of interacting with human society (to minimize harm...much like a toddler thankfully does not have the muscles of Arnold Schwarzenegger to use during its temper tantrums), and where we would then "reward" or "punish" the AI for seeming to demonstrate better or worse understanding of our utility function.
It always seemed to me that this strategy had the fatal flaw that we would not be able to tell if the AI was really already superintelligent and was just playing dumb and telling us what we wanted to hear so that we would let it loose, or if the AI really was just learning.
In addition to that fatal flaw, it seems to me that the above quote suggests another fatal flaw to the "raising an AI" strategy—that there would be a limited time window in which the AI's utility function would still be malleable. It would appear that, as soon as part of the AI figures out how to do consequentialist reasoning about code, then its "critical period" in which we could still mould its utility function would be over. Is this the right way of thinking about this, or is this line of thought waaaay too amateurish?
Very relevant article from the sequences: Detached Lever Fallacy.
Not saying you're committing this fallacy, but it does explain some of the bigger problems with "raising an AI like a child" that you might not have thought of.