All of Peter_Turney's Comments + Replies

It seems to me that Bing Chat particularly has problems when it uses the pronoun "I". It attempts to introspect about itself, but it gets confused by all the text in its training data that uses the pronoun "I". In effect, it confuses itself with all the humans who expressed their personal feelings in the training data. The truth is, Bing Chat has no true "I". 

Many of the strange dialogues we see are due to dialogues that address Bing Chat as if it has a self. Many of these dialogues would be eliminated if Bing Chat was not allowed to talk about its "o... (read more)

Peter, most of the reasons people give for making exceptions are not themselves meta. For most of the examples you give, the intuitive justification is something along the lines of "the reason killing is wrong is that life is valuable, and in these cases not killing would involve valuing life less than killing would." Nothing meta there.

Aaron, I don't see how your proposal resolves debate over exceptions. For example, consider abortion. Presumably both sides on the abortion debate agree that life is valuable.

If you say, "Killing people is wrong," that's morality.

It seems to me that few people simply say, "Killing people is wrong." They usually say, if asked for possible exceptions, "Killing people is wrong, except if you're a soldier fighting a legitimate war, a police officer upholding the law, a doctor saving a patient from needless suffering and pain, an executioner for a murderer who has had a fair trial, a person defending himself or herself from violent and deadly attackers ..." It seems that most of the debate is over these exceptions. How do we resolve debate over the exceptions without recourse to metamorality?

I am quite confident that the statement 2 + 3 = 5 is true; I am far less confident of what it means for a mathematical statement to be true.

There are two complementary answers to this question that seem right to me: Quine's Two Dogmas of Empiricism and Lakoff and Núñez's Where Mathematics Comes From. As Quine says, first you have to get rid of the false distinction between analytic and synthetic truth. What you have instead is a web or network of mutually reinforcing beliefs. Parts of this web touch the world relatively closely (beliefs about counting shee... (read more)

Eliezer, it seems to me that you were trying to follow Descartes' approach to philosophy: Doubt everything, and then slowly build up a secure fortress of knowledge, using only those facts that you know you can trust (such as "cogito ergo sum"). You have discovered that this approach to philosophy does not work for morality. In fact, it doesn't work at all. With minor adjustments, your arguments above against a Cartesian approach to morality can be transformed into arguments against a Cartesian approach to truth.

My advice is, don't try to doubt ev... (read more)

If we all cooperated with each other all the time, that would be moral progress. -- Tim Tyler

I agree with Tim. Morality is all about cooperation.

If everyone were to live for others all the time, life would be like a procession of ants following each other around in a circle. -- John McCarthy, via Eliezer Yudkowsky

This is a reductio ad absurdum argument against the idea that morality is an end. I agree with what it implies: Morality is a means, not an end. Cooperation is a means we each use to achieve our personal goals.

All of my philosophy here actually comes from trying to figure out how to build a self-modifying AI that applies its own reasoning principles to itself in the process of rewriting its own source code.

So it's not that being suspicious of Occam's Razor, but using your current mind and intelligence to inspect it, shows that you're being fair and defensible by questioning your foundational beliefs.

Eliezer, let's step back a moment and look at your approach to AI research. It looks to me like you are trying to first clarify your philosophy, and then you hope th... (read more)

The razor still cuts, because in real life, a person must choose some particular ordering of the hypotheses.

Unknown, you have removed all meaning from Occam's Razor. The way you define it, it is impossible not to use Occam's Razor. When somebody says to you, "You should use Occam's Razor," you hear them saying "A is A".

In fact, an anti-Occam prior is impossible.

Unknown, your argument amounts to this: Assume we have a countable set of hypotheses. Assume we have a complexity measure such that, for any given level of complexity, there are a finite number of hypotheses that are below the given level of complexity. Take any ordering of the set of hypotheses. As we go through the hypotheses according to the ordering, the complexity of the hypotheses must increase. This is true, but not very interesting, and not relevant to Occam's Razor.

In this framework, a natural way to stat... (read more)

And if you're allowed to end in something assumed-without-justification, then why aren't you allowed to assume anything without justification?

I address this question in Incremental Doubt. Briefly, the answer is that we use a background of assumptions in order to inspect a foreground belief that is the current focus of our attention. The foreground is justified (if possible) by referring to the background (and doing some experiments, using background tools to design and execute the experiments). There is a risk that incorrect background beliefs will "l... (read more)

5SecondWind
By examining our cognitive pieces (techniques, beliefs, etc.) one at a time in light of the others, we check not for adherence of our map to the territory but rather for the map's self-consistency. This would appear to be the best an algorithm can do from the inside. Self-consistent may not mean true, but it does mean it can't find anything wrong with itself. (Of course, if your algorithm relies on observational inputs, there should be a theoretical set of observations which would break its self-consistency and thus force further reflection.)

Why and how does anyone ever "do something they know they shouldn't", or "want something they know is wrong"? Does the notion of morality-as-preference really add up to moral normality?

It's all about delicious versus nutritious. That is, these conflicts are conflicts between different time horizons, or different discount values for future costs and benefits. Evolution has shaped our time horizon for making relatively short term decisions (Eat the pie now. It will taste good. There may not be another chance.), but we live in a world whe... (read more)

See: Good, Evil, Morality, and Ethics: "What would it mean to want to be moral (to do the moral thing) purely for the sake of morality itself, rather than for the sake of something else? What could this possibly mean to a scientific materialistic atheist? What is this abstract, independent, pure morality? Where does it come from? How can we know it? I think we must conclude that morality is a means, not an end in itself."

Eliezer, Your post is entirely consistent with what I said to Robin in my comments on "Morality Is Overrated": Morality is a means, not an end.

For a more sophisticated theory of analogical reasoning, you should read Dedre Gentner's papers. A good starting point is The structure-mapping engine: Algorithm and examples. Gentner defines a hierarchy of attributes (properties of entities; in logic, predicates with single arguments, P(X)), first-order relations (relations between entities; in logic, predicates with two or more arguments, R(X,Y)), and higher-order relations (relations between relations). Her experiments with children show that they begin reasoning with attributional similarity (what you ... (read more)

Bayesianism has its uses, but it is not the final answer. It is itself the product of a more fundamental process: evolution. Science, technology, language, and culture are all governed by evolution. I believe that this gives much deeper insight into science and knowledge than Bayesianism. See:

(1) Multiple Discovery: The Pattern of Scientific Progress, Lamb and Easton (2) Without Miracles: Universal Selection Theory and the Second Darwinian Revolution, Cziko (3) Darwin's Dangerous Idea: Evolution and the Meanings of Life, Dennett (4) The Evolution of Technology, Basalla

Scientific method itself evolves. Bayesianism is part of that evolution, but only a small part.

I agree with your general view, but I came to the same view by a more conventional route: I got a PhD in philosophy of science. If you study philosophy of science, you soon find that nobody really knows what science is. The "Science" you describe is essentially Popper's view of science, which has been extensively criticized and revised by later philosophers. For example, how can you falsify a theory? You need a fact (an "observation") that conflicts with the theory. But what is a fact, if not a true mini-theory? And how can you know tha... (read more)

(The other part of the "experimental evidence" comes from statisticians / computer scientists / Artificial Intelligence researchers, testing which definitions of "simplicity" let them construct computer programs that do empirically well at predicting future data from past data. Probably the Minimum Message Length paradigm has proven most productive here, because it is a very adaptable way to think about real-world problems.)

I once believed that simplicity is the key to induction (it was the topic of my PhD thesis), but I no longer beli... (read more)

The fact that there is a lot of emotional/inept-philosophical baggage to the word does not mean it is irrational to use it.

If your goal is to engage another person in clear, careful, rational discussion, then it is not rational to use terminology that you know to have "a lot of emotional/inept-philosophical baggage", because to do so would be counter-productive with respect to your goal. I assume that the purpose of a blog called "Overcoming Bias" is to engage in clear, careful, rational discussion.

The point is that some things are pre-analytically evil. No matter how much we worry at the concept, slavery and genocide are still evil -- we know these things stronger than we know the preconditions for the reasoning process to the contrary -- I submit that there is simply no argument sufficiently strong to overturn that judgment.

In the American civil war, some people fought against slavery and others fought to continue slavery. If your statement above is correct, it would seem that everybody who fought to continue slavery was evil. Was their pre-analyti... (read more)

"When one encounters Evil, the only solution is violence, actual or threatened."

This whole quote is sophistry. The capitalized word "Evil" is a metaphorical personification of an abstract concept. A standard definition of "evil" is "morally objectionable behavior". Suppose we replace the personification "Evil" with "morally objectionable behavior":

"When one encounters morally objectionable behavior, the only solution is violence, actual or threatened."

The result is absurd. Suppose we agr... (read more)

"The simple fact is that non-violent means do not work against Evil."

I believe that this quote is not rational, because thinking of human relations in terms of "good" and "evil" is not rational. I prefer to think in terms of the iterated prisoners' dilemma; in terms of cooperation and defection. If you frame a conflict in terms of "good" and "evil", you quickly reach violence. If you frame it in terms of "cooperation" and "defection", you may be able to negotiate a cooperative agreement.... (read more)