I brought up emotional healing because I'd recently read a strong example of it, but you raise a bunch of interesting points, and because, as I said, people seem to have a base state of emotional health-- a system 1 which is compatible with living well. It seems as though people have some capacity for improving their system 1 reactions, though it tends to be slow and difficult.
Let's see if I can generalize healing for AIs. I'm inclined to think that AIs will have something like a system 1 / system 2 distinction-- subsystems for faster/lower cost reactions and ways of combining subsystems for slower/higher cost reactions. This will presumably be more complex than the human system, but I'm not sure the difference matters for this discussion.
I think an AI wouldn't need to have emotions, but if it's going to be useful, it needs to have drives-- to take care of itself, to take care of people (if FAI), to explore not-obviously useful information.
It wouldn't exactly have a subconscious in the human sense, but I don't think it can completely keep track of itself-- that would take its whole capacity and then some.
What is a good balance between the drives? To analogize a human problem, suppose that an FAI starts out having to fend off capable UFAIs. It's going to have to do extensive surveillance, which may be too much under other circumstances-- a waste of resources. How does it decide how much is too much?
This one isn't so much about emotional healing, though emotions are part of how people tell how there lives are going. Suppose it makes a large increase in its capacity. How can it tell whether or not it's made an improvement? Or a mistake? How does it choose what to go back to, when it may have changed its standards for what's an improvement?
I don't think it can completely keep track of itself-- that would take its whole capacity and then some.
I have a different view of AI (I do not know if it is better or more likely). I would see the AI as a system almost entirely devoted to keeping track of itself. The theory behind a hard takeoff is that we already have pretty much all the resources to do the tasks required for a functional AI; all that is missing is the AI itself. The AI is the entity that organizes and develops the exiting resources into a more useful structure. This is not a trivial ...
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.