Ahem

I'm currently working on software to take advantage of massively-parallel processors to speed up this process by a couple of orders of magnitude.

One point you don't address: While you justify the claim that intelligence is real thing and can be compared, you don't explain why it would be measurable in a numerical scale. In particular, I don't see what "linear increase in intelligence" and "exponential increase in intelligence" mean and how they can be compared.

Stylistically, I agree with many of the other comments and I think this paper is unsuitable for academic publication. You should keep out discussion of side issues like speculation on the bottlenecks in academic research, how MIRI plans to deal with the potential intelligence explosion, and general discussions on how to reason, and focus just on the arguments for the existence and on the nature of an intelligence explosion.

(Note: I added a link in my previous comment)

First, if you accept that mathematical constructs are fictional, why do you consider it valid to define a concept in terms of them? Second, I admit I wasn't clear on this issue: The salient part of intuitionistic type theory isn't intuitionism, but rather that it is a structural theory. This means that statements of the form "exists x, P(x)" are not well defined, but rather only statements of the form "exists x in A, P(x)" can be made.

How about this: Mathematicians have a conception of existence which is good enough for doing mathematics, but isn't necessary correct. When you give a mathematical definition of existence, you are implicitly assuming a certain mathematical framework without justifying it. I think you would consider this criticism to be a variant of #2.

In particular, I also think about things mathematically, but when I do so, I don't use first-order logic, but rather intuitionistic type theory. Can you give a definition for existence which would satisfy me?

How about the original Pascal's wager? It was made by a famed mathematician rather than a famed physicist, and it wasn't a conspiracy, but it's definitely in the same reference frame.

As I understand it, the word "how" is used to demand a model for an event. Since I already have models for the correlations of my inputs, I don't feel the need for further explanation. More concretely, should you ask "How does closing your eyes lead to a blackout of your vision?" I would answer "After I close my eyes, my eyelids block all of the light from getting into my eye.", and I consider this answer satisfying. Just because I don't believe in a ontologically fundamental reality, doesn't mean I don't believe in eyes and eyelids and light.

As it happens, I agree with your position. I was actually thinking of making a post that pinpoints to all the important comments here without taking a position, while asking the discussion to continue there. However, making an argumentative post is also possible, although I might not be willing to expend to effort.

I suggest we move the discussion to a top-level discussion thread. The comment tree here is huge and hard to navigate.

Just to be clear, what you say does not contradict the argument you are responding to. You gave a good explanation for why EY has a big influence on the community. It still isn't clear that this is a good thing.