Grant the program an infinite amount of time. I didn't say the program must terminate did I.

A program can't pick out arbitrary subsets of an infinite set either. Programs can't do uncountably many things, even if you give them an infinite amount of time to work with.

Must I write them down? I wrote a program that could write them down.

No, you didn't. A program can't write down an infinite amount of information either.

Why can't you take the conjunction of infinitely many propositions?

You can't write it down in any finite amount of time.

I showed a method for constructing uncountably many propositions using recursion.

It appears that you're starting with some countably infinite set S of propositions, and then trying to make a proposition for each subset of S by taking the conjunction of all propositions in the subset. But all but countably many of those subsets are infinite, and you can't take the conjunction of infinitely many propositions.

Do you agree that there are (uncountably) infinite many propositions to which we can assign a probability of 1. Then we assign a probability of 0 to the negation of those propositions.

No, of course not. As I said, there are only countably many propositions you can express at all.

There can't be uncountably many propositions to which you assign probability 0, because you can only express countably many propositions.

Regarding your Pascal's mugging argument, VNM-rational agents don't assign infinite or negative infinite utility to anything. The variant using utility that is vast but finite in magnitude need not convince an agent that assigns the extreme outcome comparably tiny but nonzero probability. And it doesn't work for agents with bounded utility functions, because they don't assign such high utilities to any outcome, and thus there aren't any outcomes that they must assign extremely tiny probabilities to in order to avoid weird behavior.

Can anyone point me to any good arguments for, or at least redeeming qualities of, Integrated Information Theory?

So you might have two sentences A and B shown as two circles, then "A and B" is their intersection, "A or B" is their union, etc. But "A implies B" doesn't mean one circle lies inside the other, as you might think! Instead it's a shape too, consisting of all points that lie inside B or outside A (or both).

There's nothing intuitionistic about this. You can do exactly the same thing with classical logic, if you just forget about the topological "other details" that you alluded to.

Decision theory (which includes the study of risks of that sort) has long been a core component of AI-alignment research.

Decision theory (which includes the study of risks of that sort)

No, it doesn't. Decision theory deals with abstract utility functions. It can talk about outcomes A, B, and C where A is preferred to B and B is preferred to C, but doesn't care whether A represents the status quo, B represents death, and C represents extreme suffering, or whether A represents gaining lots of wealth and status, B represents the status quo, and C represents death, so long as the ratios of utility differences are the same in each case. Decision theory has nothing to do with the study of s-risks.

View more: Next

As is written, g(var) picks out one arbitrary subset of the infinite set. There are 2^Aleph_null possible subsets g(var) can produce, thus, g(var) can (not does) produce uncountably infinite many true propositions.

Ok, I see what you're trying to do now (though the pseudocode you wrote still doesn't do it successfully). It's true that with randomness, there are uncountably many infinite strings that could be produced. But you still have no way of referring to each one individually, so there's little point in calling them "propositions", which typically refers to claims that can actually be stated.