In principle, a nonperson predicate needs only two possible outputs, "Don't know" and "Definitely not a person". It's acceptable for many actually-nonperson programs to be labeled "don't know", so long as no people are labeled "definitely not a person". [...] The implicit difficulty is that the nonperson predicate must also pass some programs of high complexity that do things like "acceptably model humans" or "acceptably model future versions of the AI".
There's another difficulty: the nonperson predicate must not itself commit mindcrime while evaluat... (read more)
There's another difficulty: the nonperson predicate must not itself commit mindcrime while evaluat... (read more)