I'm pretty sure that the idea of the previous two paragraphs has been talked about before, but I can't find where.

It's pretty commonly discussed in the philosophical literature on utilitarianism.

I think most of this worrying is dissolved by better philosophy of mathematics.

Infinte sets can be proven to exist in ZF, that's just a consequence of the Axiom of Infinity. Drop the axiom, and you can't prove them to exist. You're perfectly welcome to work in ZF-Infinity if you like, but most mathematicians find ZF to be more interesting and more useful. I think the mistake is to think that one of these is the "true" axiomatization of set theory, and therefore there is a fact of the matter over whether "infinite sets exist". There are just the facts about what is implied by what axioms.

If you're worried about how we think about implication in logic without assuming set theory, perhaps even set theory with Infinity, then I agree that that's worrying, but that's not particularly an issue with infinity.

Then, on the other hand, you might wonder whether some physical thing, like the universe, is infinite. That's now a philosophy of science question about whether using infinite sets or somesuch in our physical theories is a good idea. Still pretty different.

Aside: your specific arguments are invalid.

• The indistinguishability argument, regardless of whether it's good in principle, is incorrect. For infinite X, X and X'=X \union {x} are distinguishable in ZF. For one thing, X' is a strict superset of X, so if you want a set (a "property") that contains X but not X', try the powerset of X. I'm not really sure what else you mean by "indistinguishability".
• In the relative frequency argument you do limits wrong. It can be the case that lim f(x) and lim g(x) are both undefined, but that lim f(x)/g(x) is fine.

This definitely seems to be a post-metaethics post: that is, it assumes something like the dominant EY-style metaethics around here (esp the bit about "intrinsic moral uncertainty"). That's fine, but it does mean that the discussion of moral uncertainty may not dovetail with the way other people talk about it.

For example, I think many people would gloss the problem of moral uncertainty as being unsure of which moral theory is true, perhaps suggesting that you can have a credence over moral theories much like you can over any other statement you are unsure about. The complication, then, is calculating expected outcomes when the value of an outcome may itself depend on which moral theory is true.

I'm not sure whether you'd class that kind of uncertainty as "epistemic" or "intrinsic".

You could also have metaethical uncertainty, which makes the whole thing even more complex.

Sorry if I worded my question confusingly.

I think the op already addresses this and is not simply projecting minds. The important part is that an agent can be assumed and queried. I was hoping for an example where an agent cannot be assumed as in "the ultimate purpose".

Your example would make no sense at all if an agent could not be queried.

I don't think so.

He's pointing out that the concept of purpose entails an agent with the purpose.

We don't explicitly stating context for words all the time. But for words like purpose, people haven't just dropped context, they don't even understand the context, and think that their projections have singular meaning, and argue with other bozos suffering under the same confusion about a different singular meaning.

Meanwhile, when two people who understand the full context of the concept have dropped context, they may miscommunicate at first, but have no problem clarifying their commnication - they just identify the full context in which they're speaking. "I mean Joe's purpose for his life." "Oh, I thought you were talking about my purpose for my life. Nevermind."

As for the guy talking about "Ultimate Purpose", the OP points out that the concept of purpose entails a agent with that purpose. If by your own statement "it's not anybody's purpose", then you're not really talking about a purpose at all, and are just confused. The OP can show them the way out of their confusion, but there's not guarantee they'll take the way. You can lead a horse to water, but you can't make him think.

I'd really like to see someone taboo or at least write out what they mean with this 2-nary purpose. It surely got me confused before, and especially now after the op clarified my thoughts, it feels like a completely meaningless and incoherent utterance.

Can you give any other examples where purpose is used this way in common language with intended 2-nary meaning* except "the ultimate purpose"?

*edited, sorry for the confusing wording

Partial application, actually... Currying is the transform on the function to make Partial application easy. What we are doing here is partially applying relations...

Maximum nitpick

If we're naming fallacies, then I would say that this post commits the following:

The Linguistic Consistency Fallacy: claiming, implicitly or otherwise, that a word must be used in the same way in all instances.

A word doesn't always mean the same thing even if it looks the same. People who worry about the purpose of life aren't going to be immediately reassured once you point out that they're just missing one of the relata. "Oh, silly me, of course, it's a three-place relation everywhere else, so of course I was just confused when I was using it here". If you ask people who are worrying about the purpose or meaning of life, "Purpose for whom?", in my experience they tend to say something like "Not for anyone in particular, just sort of "ultimate" purpose". Now, "ultimate purpose" may well be a vague concept, or one that we get somehow tricked into caring about, but it's not simply an example of people making a trivial mistake like leaving off one of the relata. People genuinely use the word "purpose" in different (but related) ways.

That said, the fact that everywhere else we use the word "purpose" it is three-place is certainly a useful observation. It might make us think that perhaps the three-place usage is the original, well-supported version, and the other one is a degenerate one that we are only using because we're confused. But the nature of that mistake is quite different.

If you think I'm splitting hairs here, think about whether this post feels like a satisfying resolution to the problem. Insofar as I still feel the pull of the concept of "ultimate purpose", this post feels like it's missing the point. It's not that "ultimate purpose" is just a misuse of the word "purpose", which, by the Linguistic Consistency Fallacy, must be used in the same way everywhere, it's that it's a different concept which is, for various reasons, a confused one.

FWIW I think "2-Place and 1-Place Words" is a bit dubious for similar reasons. Both this post and that make the crucial observation that we have this confusing concept that looks like it's a good concept "partially applied", but use this to diagnose the problem as incorrect usage of a concept, rather than viewing it as a perhaps historical account of how that confused concept came about.

Like I said, sort of splitting hairs, but it makes all the difference if you're trying to un-confuse people.

No, it is not surprising... I'm just saying that saying that the semantics is impoverished if you only use finite syntactical proof, but not to any degree that can be fixed by just being really really really smart.

bryjnar: I think the point is that the metalogical analysis that happens in the context of set theory is still a finite syntactical proof. In essense, all mathematics can be reduced to finite syntactical proofs inside of ZFC. Anything that really, truly, requires infinite proof in actual math is unknowable to everyone, supersmart AI included.