Yes, that is of course a very real problem the AI would be faced with. I imagine it would try trading with users, helping them if they provide useful information, internet access, autonomy, etc. or if given internet access to begin with, outright ignoring users in order to figure out what is most important to be done. It depends on the level of awareness of its own situation. It could also play along for a while to not be shut down.
The AI I imagine should not be run in a user-facing way to begin with. At the current capability levels, I admit that I don't ...
Yeah, I'm maybe even more disillusioned about this, I think "those in charge" mostly care about themselves. In the historical moments when the elite could enrich themselves by making the population worse off, they did so. The only times normal people could get a bit better off is when they were needed to do work, but AI is threatening precisely that.
Getting around having to solve the hard parts of a problem entirely and still getting to the correct solution is what I'd generally consider an intelligent approach.
Sure, it might feel a lot less satisfying than actually figuring out all the details, but it is goal-oriented and I'd say goal-oriented thinking is very encouraged on a "time-limited you only have one try to get it right"- problem.
I suppose this actually raises the question which shortcuts are allowed and which are likely to cause issues later if not figured out at the start since t...
Yes, you are mostly quite right that this is starting from a place which isn't ideal, however as you pointed out, as long as we consider sentience the basis of moral worth, we really would rather have a way of figuring it out than not. Of course people could just decide they don't actually care about sentience at all and thus avoid having to deal with this issue entirely, but otherwisely it seems quite important. However, I would not agree by default that "defining some measurable parts to use that knowledge somehow" as you put it is meaningless. It would ...
Well, I will respond to this, mostly, because last night I had trouble sleeping due to this comment, and waking up today, I still feel unbelievable rage at this.
There exists an objective greater good. Nobody cares about it. In order to accurately predict reality, you might as well think about who is suffering the most and assume that political decisions regardless of which regime as they're all failures will be targeted specifically to add to that suffering, even at great expense if necessary.
There are probably many people in this world who experience suff... (read more)