Jacob, I am the only one replying to your replies to me (and no one is voting me up). I choose to take that as a sign that this thread is insufficiently interesting to sufficient numbers of LWers for me to continue.

Note that doing so is not a norm of this community although I would like it if it were and it was IIRC one of the planks or principles of a small movement on Usenet in the 1990s or very early 2000s.

If there were two universes, one very likely to evolve life and one very unlikely, and all we knew was that we existed in one, then we are much more likely to exist in the first universe.

Agree.

Hence our own existence is evidence about the likelihood of life evolving [in the situation in which we find ourselves].

Disagree because your hypothetical situation requires a different analysis than the situation we find ourselves in.

In your hypothetical, we have somehow managed to acquire evidence for the existence of a second universe and to acquire evidence that life is much more likely in one than in the other.

Well, let us get specific about how that might come about.

Our universe contains gamma-ray bursters that probably kill any pre-intelligence-explosion civilization within ten light-years or so of them, and our astronomers have observed the rate * density at which these bursters occur.

Consequently, we might discover that one of the two universes has a much higher rate * density of bursters than the other universe. For that discovery to be consistent with the hypothetical posed in parent, we must have discovered that fact while somehow becoming or remaining completely ignorant as to which universe we are in.

We might discover further that although we have managed to determine the rate * density of the bursters in the other universe, we cannot travel between the universes. We must suppose something like that because the hypothetical in parent requires that no civilization in one universe can spread to the other one. (We can infer that requirement from the analysis and the conclusion in parent.)

I hope that having gotten specific and fleshed out your hypothetical a little, you have become open to the possibility that your hypothetical situation is different enough from the situation in which we find ourselves for us to reach a different conclusion.

In the situation in which we find ourselves, one salient piece of evidence we have for or against ET in our past light cone is the fact that there is no obvious evidence of ET in our vicinity, e.g., here on Earth or on the Moon or something.

And again, this piece of evidence is really only evidence against ETs that would let us continue to exist if their expansion reached us, but there's a non-negligible probability that an ET would in fact let us continue to exist because there no strong reason for us to be confident that the ET would not.

In contrast to the situation in which we find ourselves, in the hypothetical posed in parent, there is an important piece of evidence in addition to the piece I just described in just the same way that whatever evidence we used to conclude that the revolver contains either zero or one bullet is an additional important piece of evidence that when combined with the evidence of the results of 1,000,000 iterations of Russian roulette would cause a perfect Bayesian reasoner to reach a different conclusion than it would if it knew nothing of the causal mechanism that exists between {a spin of the revolver followed by a pull of the trigger} and {death or not-death}.

I intentionally delayed this reply (by > 5 days) to test the hypothesis that slowing down the pace of a conversation on LW will improve it.

Do you take the Fermi paradox seriously, or is the probability of your being destroyed by a galactic civilization, assuming that one exists, low enough?

When we try to estimate the number of technological civilizations that evolved on main-sequence stars in our past light cone, we must not use the presence of at least one tech civ (namely, us) as evidence of the presence of another one (namely, ET) because if that first tech civ had not evolved, we would have no way to observe that outcome (because we would not exist). In other words, we should pretend we know nothing of our own existence or the existence of clades in our ancestral line, in particular, the existence of the eukaryotes and the metazoa, when trying to estimate the number of tech civs in our past light cone.

I am not an expert on ETIs, but the following seems (barely) worth mentioning: the fact that prokaryotic life arose so quickly after the formation of the Earth's crust is IMHO significant evidence that there is simple (unicellular or similar) life in other star systems.

The evidential gap w.r.t. ET civilization spans billions of years -- but this is not evidence at all according to the above.

It is evidence, but less strong than it would be if we fail to account for observational selection effects. Details follow.

The fact that there are no obvious signs of an ET tech civ, e.g., alien space ships in the solar system, is commonly believed to the be strongest sign that there were no ET tech civs in our past light cone with the means and desire (specifically, desire on at least part of the civ that was not thwarted by the rest of the civ) to expand outwards into space. Well, it seems to me that there is a good chance that we would not have survived an encounter with the leading wave of such an expansion, and therefore the lack of evidence of such an expansion should not cause us to update our probability of the existence of such an expansion as much as it should have if we certainly could have survived the encounter. Still, the fact that there are no obvious signs (such as alien space ships in the solar system) of ET is the strongest piece of evidence against the hypothesis of the existence of ET tech civs in our past light cone (because for example radio waves can be detected by us over a distance of only thousands of light years whereas we should be able to detect colonization waves that originated billions of light years away because once a civilization acquires the means and desire to expand, what would stop it?).

In summary, observational selection effects blunt the force of the Fermi paradox in two ways:

1. Selection effects drastically reduce the (likelihood) ratio by which the fact of the existence of our civilization increases our probability of the existence of another civilization.

2. The lack of obvious signs (such as alien space ships) of ET in our immediate vicinity is commonly taken as evidence that drastically lowers the probability of ET. Observational selection effects mean that P(ET) is not lowered as much as we would otherwise think.

(end of list)

So, yeah, to me, there is no Fermi paradox requiring explanation, nor do I expect any observations made during my lifetime to create a Fermi paradox.

Well, 2224 Shattuck is more convenient to public transportation than my place is, then.

2224 Shattuck is across the street from the campus, right?

Given some arbitrary program X and a sequence of inputs Y, there is no general program that can predict the output Z of X given Y that is simpler and faster than X itself. If this wasn't true, it would be a magical shortcut around all kinds of complexity theorems.

I agree with that, but it does not imply there will be a lot of agents simulating agents after the intelligence explosion if simulating means determining the complete future behavior of an agent. There will be agents doing causal modeling of agents. Causal modeling allows the prediction of relevant properties of the behavior of the agent even though it probably does not allow the prediction of the complete future behavior or "complete future output state" of the agent. But then almost nobody will want to predict the complete future behavior of an agent or a program.

Consider again the example of a chess-playing program. Is it not enough to know whether it will follow the rules and win? What is so great or so essential about knowing the complete future behavior?

Sure it is possible to create programs that can be formally verified

Formal verification is not the point: I did not formally verify anything.

The point is that I did not run or simulate anything, and neither did wnoise in answering my challenge.

We all know that humans run programs to help themselves find flaws in the programs and to help themselves understand the programs. But you seem to believe that for an agent to create or to understand or to modify a program requires running the program. What wnoise and I just did shows that it does not.

Ergo, your replies to me do not support your position that the future will probably be filled with simulations of agents by agents.

And in fact, I expect that there will be almost no simulations of agents by agents after the intelligence explosion for reasons that are complicated, but which I have said a few paragraphs about in this thread.

Programs will run and some of those programs will be intelligent agents, but almost nobody will run a copy of an agent to see what the agent will do because there will be more efficient ways to do whatever needs doing -- and in particular "predicting the complete output state" of an agent will almost never need doing.

And that is why wnoise used a debugger to find a flaw in my position. Oh, wait! wnoise didn't use a debugger to find the flaw.

(I'll lay off the sarcasm now, but give me this one.)

Also: I never said humans will stop needing debuggers.

The first meetup (in Millbrae) was in a restaurant and did not work as well as the ones in people's houses. At the houses, people were much more free to leave one conversation and join another because there were multiple rooms and more space to walk around.

I can offer my apartment in San Rafael close to the Richmond-San Rafael bridge. My building has a big community room with a dozen chairs and a sofa in case the meeting gets too big for my apartment.

It is probably not as convenient as 2224 Shattuck is to public transit however -- even after one gets into San Rafael.

2224 Shattuck is near what BART Station or what cross street?

Well, that's easy -- just feed it a circular list.

Nice catch, wnoise.

But for those following along at home, if I had been a more diligent in my choice, (i.e., if instead of "Scheme", I had said, "a subset of Scheme, namely, Scheme without circular lists") there would have been no effective answer to my challenge.

So, my general point remains, namely, that a sufficiently careful and skilled programmer can deliver a program guaranteed to halt and guaranteed to have the useful property or properties that the programmer intends it to have without the programmer's ever having run the program (or ever having copied the program from someone who ran it).

There is no mathematical shortcut (halting theorem for one, but its beyond that).

A program is chosen from a huge design space, and any effective designer will choose a design that pessimizes the mental labor needed to understand the design. So, although there are quite simple Turing machines that no human can explain how it works, Turing machines like them simply do not get chosen by designers who do want to understand their design.

The halting theorem says that you can pick a program that I cannot tell whether it halts on every input. EDIT. Or something like that: it has been a while. The point is that the halting theorem does not contradict any of the sequence of statements I am going to make now.

Nevertheless, I can pick a program that does halt on every input. ("always halts" we will say in the future.)

And I can a pick a program that sorts its input tape before it (always) halts.

And I can pick a program that interprets its input tape as a list of numbers and outputs the sum of the numbers before it (always) halts.

And I can pick a program that interprets its input tape as the coefficients of a polynomial and outputs the zeros of the polynomial before it (always) halts.

Etc. See?

And I can know that I have successfully done these things without ever running the programs I picked.

Well, here. I do not have the patience to define or write a Turing machine, but here is a Scheme program that adds a list of numbers. I have never run this program, but I will give you $10 if you can pick an input that causes it to fail to halt or to fail to do what I just said it will do.

(define (sum list) (cond ((equal '() list) 0) (#t (+ (car list) (sum (cdr list))))))