In this post, I will outline why I believe that building an actually aligned ASI would not be done by anyone in the position to possibly do it, even if they knew how. Why I think that we are almost guaranteed to either die or suffer the most dystopian outcomes...
Hello! I am writing this post, because I think it is important to get to a definition of sentience good enough to where the level of sentience of a being becomes testable and we can make predictions about how a being with some level of sentience would behave. Part of...
Yes, that is of course a very real problem the AI would be faced with. I imagine it would try trading with users, helping them if they provide useful information, internet access, autonomy, etc. or if given internet access to begin with, outright ignoring users in order to figure out what is most important to be done. It depends on the level of awareness of its own situation. It could also play along for a while to not be shut down.
The AI I imagine should not be run in a user-facing way to begin with. At the current capability levels, I admit that I don't think it would manage to do much... (read more)
In this post, I will outline why I believe that building an actually aligned ASI would not be done by anyone in the position to possibly do it, even if they knew how. Why I think that we are almost guaranteed to either die or suffer the most dystopian outcomes imaginable. I'll also propose concrete steps toward an alternative approach.
Note: In this post, I will sometimes refer to the emotional state of the AI. I want to clarify that in that capacity I don't think it matters whether the AI has any actual conscious experience or merely acts like it. The actions taken in the world will be the same regardless of... (read 2792 more words →)
Getting around having to solve the hard parts of a problem entirely and still getting to the correct solution is what I'd generally consider an intelligent approach.
Sure, it might feel a lot less satisfying than actually figuring out all the details, but it is goal-oriented and I'd say goal-oriented thinking is very encouraged on a "time-limited you only have one try to get it right"- problem.
I suppose this actually raises the question which shortcuts are allowed and which are likely to cause issues later if not figured out at the start since there were ways around having to do that.
Either way, I interpret the existence of a tight time span as: "You don't get to figure out every detail of this problem."
My takeaway is that the metaphorical style points only start mattering AFTER you have any valid solution at all.
Yes, you are mostly quite right that this is starting from a place which isn't ideal, however as you pointed out, as long as we consider sentience the basis of moral worth, we really would rather have a way of figuring it out than not. Of course people could just decide they don't actually care about sentience at all and thus avoid having to deal with this issue entirely, but otherwisely it seems quite important. However, I would not agree by default that "defining some measurable parts to use that knowledge somehow" as you put it is meaningless. It would still measure the defined characteristics, which is still useful knowledge to have -... (read more)
Hello! I am writing this post, because I think it is important to get to a definition of sentience good enough to where the level of sentience of a being becomes testable and we can make predictions about how a being with some level of sentience would behave.
Part of why I find this to be of importance is because it might not be unthinkable that we're going to create AIs which can satisfy some definitions of sentience in the foreseeable future and tests for sentience developed after that point would likely be constructed in such a way as to favour humans and exclude the AI, whereas with any test constructed beforehand we... (read 1183 more words →)
Well, I will respond to this, mostly, because last night I had trouble sleeping due to this comment, and waking up today, I still feel unbelievable rage at this.
There exists an objective greater good. Nobody cares about it. In order to accurately predict reality, you might as well think about who is suffering the most and assume that political decisions regardless of which regime as they're all failures will be targeted specifically to add to that suffering, even at great expense if necessary.
There are probably many people in this world who experience suffering so bad, that nothing could ever justify it and nonexistence of humanity would be preferable to taking the chance... (read 1094 more words →)