Constant2

I'm getting two things out of this.

1) Evolutionary cynicism produces different predictions from cognitive cynicism, e.g. because the current environment is not the ancestral environment.

2) Cognitive cynicism glooms up Eliezer's day but evolutionary cynicism does not.

(1) is worth keeping in mind. I'm not sure what significance (2) has.

However, we may want to develop a cynical account of a certain behavior while suspending judgment about whether the behavior was learned or evolved. Call such cynicism "agnostic cynicism", maybe. So we have three typ... (read more)

it would seem easier to build (or mutate into) something that keeps going forever than it is to build something that goes for a while then stops.

On reflection, I realize this point might be applied to repetitive drudgery. But I was applying it to the behavior "engage in just so much efficient exploration." My point is that it may be easier to mutate into something that explores and explores and explores, than it would be to mutate into something that explores for a while then stops.

the vast majority of possible expected utility maximizers, would only engage in just so much efficient exploration, and spend most of its time exploiting the best alternative found so far, over and over and over.

I'm not convinced of that. First, "vast majority" needs to use an appropriate measure, one that is applicable to evolutionary results. If, when two equally probable mutations compete in the same environment, one of those mutations wins, making the other extinct, then the winner needs to be assigned the far greater weight. So, for example,... (read more)

Seconding the expectation of a "useless clusterf--k."

The hope is that the shared biases will be ones that the site owner considers valuable and useful

The obvious way to do that is for the site owner to make some users more equal than others.

the Hacker News website seems to be doing fine.

Security through obscurity. Last I checked, it confirmed my impression, gathered from Digg and Reddit, that as long as the site remains sufficiently unpopular, it will not deteriorate.

Two separate issues:

1) Is it a good (legitimately persuasive) argument?

2) If not then after all the hairsplitting is done, what sort of bad argument is it?

The more important issue is (a). A few points:

a) Quibbling over the categorization of the fallacy is sometimes used to mask the fact that it's a bad argument.

b) There are plenty of people who can recognize bad arguments without knowing anything about the names of the fallacies, which leads to

c) We learn the names of the fallacies, not in order to learn to spot bad arguments, but as a convenience so that ... (read more)

Morality is an aspect of custom. Custom requires certain preconditions: it is an adaptation to a certain environment. Great political power breaks a key component of that environment.

More specifically, morality is a spontaneously arising system for resolving conflict among people with approximately equal power, such that adherence to morality is an optimal strategy for a typical person. A person with great power has less need to compromise and so his optimal strategy is probably a mix of compromise and brute force - i.e., corruption.

This does not require v... (read more)

Someone had just asked a malformed version of an old probability puzzle [...] someone said to me, "Well, what you just gave is the Bayesian answer, but in orthodox statistics the answer is 1/3." [..] That was when I discovered that I was of the type called 'Bayesian'.

I think a more reasonable conclusion is: yes indeed it is malformed, and the person I am speaking to is evidently not competent enough to notice how this necessarily affects the answer and invalidates the familiar answer, and so they may not be a reliable guide to probability and i... (read more)

Aaron - yes, I know that. It's beside the point.

My point was that vampires were by definition not real - or at least, not understandable - because any time we found something real and understandable that met the definition of a vampire, we would change the definition to exclude it.

But the same exchange might have occurred with something entirely real. We are not in the habit of giving fully adequate definitions, so it is often possible to find counterexamples to the definitions we give, which might prompt the other person to add to the definition to exclude the counterexample. For example:

A: What is a d... (read more)

Time - Philip Johnson is not just a Christian but a creationist. Do you mean, "if there are smart creationists out there..."? I don't really pay much attention to the religious beliefs of the smartest mathematicians and scientists and I'm not especially keen on looking into it now, but I would be surprised if all top scientists without exception were atheists. This page seems to suggest that many of the best scientists are something other than atheist, many of those Christian.

Whoever is censoring Caledonian: can it be done without adding the content-free nastiness (such as "bizarre objection", "illogic", and "gibberish")?

Any two AIs are likely to have a much vaster difference in effective intelligence than you could ever find between two humans (for one thing, their hardware might be much more different than any two working human brains). This likelihood increases further if (at least) some subset of them is capable of strong self-improvement. With enough difference in power, cooperation becomes a losing strategy for the more powerful party.

I read stuff like this and immediately my mind thinks, "comparative advantage." The point is that it can be (and probably is... (read more)

An AI can indeed have preferences that conflict with human preferences, but if it doesn't start out with such preferences, it's unclear how it comes to have them later.

We do not know very well how the human mind does anything at all. But that the the human mind comes to have preferences that it did not have initially, cannot be doubted. For example, babies do not start out preferring Bach to Beethoven or Beethoven to Bach, but adults are able to develop that preference, even if it is not clear at this point how they come to do so.

If you could do so easily ... (read more)

A tendency to become corrupt when placed into positions of power is a feature of some minds.

Morality in the human universe is a compromise between conflicting wills. The compromise is useful because the alternative is conflict, and conflict is wasteful. Law is a specific instance of this, so let us look at property rights: property rights is a decision-making procedure for deciding between conflicting desires concerning the owned object. There really is no point in even having property rights except in the context of the potential for conflict. Remove conf... (read more)

I say go ahead and pick a number out of the air,

A somewhat arbitrary starting number is also useful as a seed for a process of iterative approximation to a true value.

how you can talk about probabilities without talking about several possible worlds

But if probability is in the mind, and the mind in question is in this world, why are other worlds needed? Moreover (from wikipedia):

In Bayesian theory, the assessment of probability can be approached in several ways. One is based on betting: the degree of belief in a proposition is reflected in the odds that the assessor is willing to bet on the success of a trial of its truth.

Disposition to bet surely does not require a commitment to possible worlds.

Elephants are not properties of physics any more than probabilities are. The concept of an elephant is subjective - as are all concepts.

If you are indeed agreeing with the parallel I have set up between probability and elephants and if this is not just your own personal view, then perhaps the subjectivist theory of probability should more properly be called the subjectivist theory of pretty much everything that populates our familiar world. Anyway, I think I can agree that probability is as subjective and as psychological and as non-physical and as existing in the mind and not in the world as an elephant or, say, an exploding nuclear bomb - another item that populates our familiar world.

With complete information (and a big computer) an observer would know which way the coin would land - and would find probabilities irrelevant.

But this is true of most everyday observations. We observe events on a level far removed from the subatomic level. With complete information and infinite computing power an observer would would find all or virtually all ordinary human-level observations irrelevant. But irrelevancy to such an observer is not the same thing as non-reality. For example, the existence of elephants would be irrelevant to an observer who h... (read more)

If such ideas seem unproblematic to you

It is the example that seems on the face of it unproblematic. I am open to either (a) a demonstration that it is compatible with subjectivism[*], or (b) a demonstration that it is problematic. I am open to either one. Or to something else. In any case, I don't adhere to frequentism.

[*] (I made no firm claim that it is not compatible with subjectivism - you are the one who rejected the compatibility - my own purpose was only to raise the question since it seems on the face of it hard to square with subjectivism, not to answer the question definitively.)