I think that it's acceptable when it works.
What I mean is, a lot of the transhumanist stuff is predicated on these things working properly. But we know how badly wrong computers can sometimes go, and that's in everyone's experience, so much so that "switch it off and switch it on again" is part of common, everyday lore now.
Imagine being so intimately connected with a computerized thingummybob that part of your conscious processing, what makes you you, is tied up with it - and it's prone to crashing. Or hacking, or any of the other ills that can befall computery things. Potential horrorshow.
Similar for bio enhancements, etc. For example, physical enhancement like steroids, but safer and easier to use, are still a long way off, and until they come, people are just not going to go for it. We really only have a very sketchy understanding of how the body and brain work at the moment. It's developing, but it's still early days.
So ultimately, I think for the foreseeable future, people are still going to go for things that are separable, that the natural organic body can use as tools that can be put away, that the natural organic body can easily separate itself from, at will, if they go wrong.
They're not going to go for any more intimate connections until such things work much, much better than anything we've got now.
And I think it's actually debatable whether that's ever going to happen. It may be the case that there are limits on complexity, and that the "messy" quality of organics is actually the best way of having extremely complex thinking, moving objects - or that there's a trade-off between having stupid things that do massive processing well, and clever things that do simple processing well, and you can't have both in one physical (information processing) entity (but the latter can use the former as tools).
Another angle to look at this would be to look at the rickety nature of high IQ and/or genius - it's six and a half dozen whether a hyper-intelligent being is going to be of any use at all, or just go off the rails as soon as it's booted up. It's probably the same for "AI".
I don't think any of this is insurmountable, but I think people are massively underestimating the time it's going to take to get there; and we'll already have naturally evolved into quite different beings by that time (maybe as different as early homonids from us), so by that time, this particular question is moot (as there will have been co-evolution with the developing tech anyway, only it will have been very gradual).
Technology can be used and perceived in different ways. Future technology may change our lives beyond imagination. How can friendly AI technology enrich the human experience positively? Technology can feel like it controls us, or - if it goes well - it can feel like a natural enhancement of mind and body.
I'm interested in ways future technology could or couldn't do this. I will explore some avenues and state my opinion on these. Make up your mind. I'd like to hear your opinion.
Body Enhancement
The first thing we'd like to get rid of is impediments and diseases. Some might like to be immortal. But this is not enhancement but rather maintenance.
People apparently like to enhance their bodies. This starts with cosmetics and doesn't end with doping. Strictly clothing could also count among this. We know quite well what we want here. I'd bet that people would accept body enhancements easily - esp. if it is reliable, safe, and/or reversible. Fictional evidence here is the positive reception of suitably enhanced heroes. Wouldn't you like to have super strength or look like a supermodel? Such enhancement for everybody, which is otherwise zero-sum, could also balance against the effect that ideals in the media diminish our self-image compared to the ancestral environment where we were only one in 150 average guys.
As long as people don't change their native preferences, this should make everybody happier with themselves. If preferences are changed, bets are off again.
Mind Enhancement
Drugs can have not only pleasurable but also performance-increasing effects. Nootropics for everybody could be acceptable - if free and safe. I think that increasing brain capacity (speed and capacity) would feel the most natural - if it could be done.
The trouble with this is the inevitable return on cognitive investment. Seems like either exponential or chaotic changes (from interacting minds) result. One tricky part here seems to be how to avoid boredom once stability has been reached.
Body Schema Extension
Body perception is flexible. It is known that the body schema (our body self-image) can expand to encompass tools. Thus tools become part of the body (schema) and are wielded and handled and thus felt like one's own body. I got the impression that this might extend to vehicles - e.g., driving a car - or probably also flying a plane - can feel like one's own movement. One knows where the car ends. I'd guess that technology that has immediate feedback and can be mapped to a (distorted/extended) body schema will likely feel natural (after some time of adjustment).
Sensory Enhancement
Apparently, our senses are quite flexible. Whatever input (visual, auditory, tactile, even smell) can be mapped to a 3D environment model by long training. This is apparently also possible for non-native senses, which is called Sensory Substitution and Sensory Augmentation. There are already some projects which build actual working devices. Once this mapping has settled in the subconscious, it feels natural. I wonder whether augmented reality systems can achieve this. Virtual reality systems are dual for this - data mapped to the senses instead of senses mapped to data.
Devices and Gadgets
Devices that require conscious interaction and translation into some UI often feel clumsy no are clumsy. They break the flow. They require conscious effort. I think the main attraction of having an app for that is the feeling of control over the distance we gain. We can do something by invoking some magical ritual to achieve some effect other people can't achieve (or rather only via mundane manual action). This is good and fine, but even better if you can achieve the effect even without the interaction.
There was a recent post somewhere about the best smartphone UI being just a blank screen where you could type (or dictate) what you want, and the 'UI' would figure out the context and intention. While googling unsuccessfully for that, I found this link about natural UIs:
Services
Commerce and esp. the internet provide lots of services that we use to reap the benefits of a digital society. Amazon, Netflix, online booking... But we are at the mercy of the service providers and the power of the interface provided, and the cost required. Independent of how well-integrated this is, every interaction with service either means a transaction (cost per use), freemium choice (will I regret this later), or ad suffering (paying with attention). This is bondage and discipline. I'd rather minimize this as a means of future technological enrichment.
Language Control
Communication is natural - via speech and via text. Communication is natural, not only with people. Most programmers value the power of the command lines - because it allows them to combine commands in new ways in a - to an experienced user - natural linguistic way. Why not use language to control the technology of the future? Just utter your wishes. A taste of this could be the service offered by the Magic startup.
Social Interaction
We are social animals. Could we deal with digital assistants who understand us and support us? Probably - if they are beyond the Uncanny Valley. But would we trust them? Only if they behave consistently with equal or lower status. Otherwise, we'd justifiably feel dominated. Can this be achieved if the artificial agent is much smarter than us and unavoidably controls us thereby? Would we feel manipulated?
Slow Processes
Societal processes affecting us in ways we (feel we) have only limited control over often feel oppressive - even if they are by some objective standard intended for our (collective) best. Examples are health care bureaucracy, compulsory education, traffic rules, and above all, parliamentary democracy. These processes are slow in the sense of affecting and changing over longer times than conscious effort can easily work on. Often there is no immediate or clearly attributable feedback. Such a process often feels like a force of nature, and humans have adapted quite well to forces of nature - but just because we accept it doesn't mean that we feel liberated by it. I think that any slow process that changes things in complex ways we cannot follow will cause negative feelings. And many conceptions of how FAI could help us I have seen involve masterminds setting things up. People might feel manipulated. Either this is balanced by other means, or we need a correspondingly slow consciousness or deep understanding to follow this.
My Transhuman Wish-List
I want to look better, be more robust - even if everybody else would look better too. I want backups and autonomous real and virtual clones of myself.
I'd like to think faster, have a perfect memory, or even access information from the web in a way that feels like recall. I'd like to be able to push conscious thought processes into the subconscious - call it deliberate, efficient reversible habit formation.
I'd like to be able to move into machines (vehicles, robots, even buildings) and feel the machines as my extended self. I'd like to perceive more kinds of sensor input as natural as my current senses.
I don't want to interface with devices but command linguistically, by thought, or completely subconsciously.
I want a consciousness that can deal with slow processes and possibly a way to think slower in parallel with normal consciousness.
Open Ends
There are more areas where this reasoning can be applied, and I'd like to state some general patterns behind these areas - but my time for this post has run out.
Just two examples:
I'd like to give fictional evidence for each point. But here, I also just point you to the Optimalverse, where some of these are played out, and to The Culture, which describes some of the effects.
EDITED: Spelling, typos.