The orthogonality thesis and its relation to existing meta-ethical debates

In the field of AI alignment theory, the orthogonality thesis asserts that there can exist arbitrarily intelligent agents pursuing any kind of motivation. The reverse thesis, that we may call the heterogonality thesis, asserts that, with enough intelligence, any possible agent would pursue only one set of motivations.

In the field of meta-ethics, moral internalism asserts that any possible agent who hold a moral judgment is motivated to act on this judgment. For example, according to moral internalism, any agent who hold that one ought to donate 10% of one's income to charity is motivated to do so.

Also in the field of meta-ethics, moral realism asserts that some moral judgments are objectively correct. This is a form of moral cognitivism, which that moral judgments are factual statements that can be objectively correct or incorrect (anti-realist cognitivism is error theory, which asserts that all moral judgments are incorrect).

It's easy to see that the heterogonality thesis is moral internalism plus moral realism. A moral realist would say that, with enough intelligence, any possible agent would discover objective morality and hold only one set of moral judgments that is objectively correct. Therefore, a moral realist who is also a moral internalist would support the heterogonality thesis by saying that this mean that, with enough intelligence, any possible agent would be motivated to act by only one set of moral judgments, and thus would pursue only one set of motivations.

This is why, even though the orthogonality thesis is a recent concept that is only known by the small circles of AI alignment theorists (and I had to invent the term for its negation by myself), we can try to estimate how many philosophers accept the orthogonality thesis.

The PhilPapers survey

The PhilPapers Survey was a survey of professional philosophers and others on their philosophical views, carried out in November 2009. The Survey was taken by 3226 respondents, including 1803 philosophy faculty members and/or PhDs and 829 philosophy graduate students.

It included three questions on meta-ethics: whether one accept moral realism or moral anti-realism, whether one accept moral internalism or moral externalism, and whether one accept moral cognitivism or moral non-cognitivism. (There was also a question on normative ethics, whether one accept virtue ethics, deontology, or consequentialism. It is not relevant to the orthogonality thesis.)

Each question is divided between multiple options: one for every position, plus an "Other" option for people for whom the question is too unclear to answer, agnostics, people insufficiently familiar with the issue, etc.

Methodology

The methodology is implemented by a bash script which is available in the appendix. It downloads the answers of public respondents to the PhilPapers survey, extract their opinions on meta-ethics, exclude philosophers who picked "Other" options (because we can't know if they accept the orthogonality thesis), and then compute the number of philosophers (with a knowable opinion) who accept the orthogonality thesis.

Results

66% of philosophers (with a knowable opinion) accept the orthogonality thesis. This is about two thirds of philosophers.

Appendix: Source code of the script

#!/bin/bash
# WARNING: This script creates many new files.
# It is highly recommended to be in an empty folder when executing it.

function opinion() {
    # Usage: opinion <file> <question> <answer A> <answer B>
    # Example: opinion 42 "Meta-ethics" "moral realism" "moral anti-realism"
    str="$2: $3 or $4?<\/td><td  bgcolor='#......' style='width:250px'>"
    answer=$(grep -o "$str[-A-Za-z/: ]*" "$1" | sed "s/$str//")

    r=other
    if grep "$3" <<< "$answer" > /dev/null; then r=$3; fi
    if grep "$4" <<< "$answer" > /dev/null; then r=$4; fi
    echo $r
}

function metaethical_opinions() {
    # Usage: metaethical_opinions <file>
    # Example: metaethical_opinions 42
    metaethics=$(opinion "$1" "Meta-ethics"      "moral realism"  "moral anti-realism")
    mjudgement=$(opinion "$1" "Moral judgment"   "cognitivism"    "non-cognitivism")
    motivation=$(opinion "$1" "Moral motivation" "internalism"    "externalism")
    echo "$metaethics    $mjudgement    $motivation"
}


if ! [ -e public_respondents.html ]; then
    wget https://philpapers.org/surveys/public_respondents.html
fi

if ! [ -e pps_meo ]; then
    for profile in $(sed "s/'/\n/g" public_respondents.html | grep "https://philpapers.org/profile/"); do
        id=$(cut -d/ -f5 <<< $profile)
        if ! [ -e $id ]; then
            wget $profile -O $id
        fi
        metaethical_opinions $id
    done | sort | uniq -c | grep -v other | sed 's/^ *//' > pps_meo
fi

orthogonalists=$(grep -v -P "moral realism\tcognitivism\tinternalism" pps_meo | cut -d\  -f1 | paste -sd+ | bc)
philosophers=$(cut -d\  -f1 pps_meo | paste -sd+ | bc)

python3 << EOF
print("{}% of philosophers (with a knowable opinion) accept the orthogonality thesis.".format($orthogonalists/$philosophers * 100))
EOF