"Do Artificial Reinforcement-Learning Agents Matter Morally?", Tomasik 2014:
Artificial reinforcement learning (RL) is a widely used technique in artificial intelligence that provides a general method for training agents to perform a wide variety of behaviours. RL as used in computer science has striking parallels to reward and punishment learning in animal and human brains. I argue that present-day artificial RL agents have a very small but nonzero degree of ethical importance. This is particularly plausible for views according to which sentience comes in degrees based on the abilities and complexities of minds, but even binary views on consciousness should assign nonzero probability to RL programs having morally relevant experiences. While RL programs are not a top ethical priority today, they may become more significant in the coming decades as RL is increasingly applied to industry, robotics, video games, and other areas. I encourage scientists, philosophers, and citizens to begin a conversation about our ethical duties to reduce the harm that we inflict on powerless, voiceless RL agents.
Particularly germane is the discussion of the disunified nature of cognition and experimental demonstrations of learning still happening while the hippocampus is knocked out by an anesthetic-like drug like "When Memory Fails, Intuition Reigns: Midazolam Enhances Implicit Inference in Humans".
I offer for LW's consideration the interesting 2008 paper "Inverse zombies, anesthesia awareness, and the hard problem of unconsciousness" (Mashour & LaRock; NCBI); the abstract:
3. Inverse zombies
4. Anesthesia awareness and anesthetic depth
5. Philosophical implications of i-zombies
6. The hard problem of unconsciousness
The awareness rate is chilling. One wonders whether surgery rates would be significantly affected in everyone was aware of this; it's like that utilitarian puzzler 'how much would I have to pay you to torture you with amnesia afterwards?' but in real life.
Further reading
References
102(5), 937–947.