...let's start with a little thought experiment...

The two cases are non-analogous. Grooves in a phonograph record are not designed to be read by a human. Perhaps a better analogy would be reading sheet music, but most people are not trained to do that either. The reason people show such a strong preference in the latter case is that most people will get nothing at all from the record (or sheet music, for that matter).

just because some people can't see colors doesn't mean that colors aren't real. The same is true for spiritual experiences.

This is a truism. Moreover, it is often argued that colors, flavors, &c. are of the map, not of the territory. If this is the case, colors may not be "real", even if the experience of colors is.

...one cannot render into words the subjective experience...

The attempt to losslessly transmit a complete subjective experience would be futile, although I've read some poets who took a good stab at it. Experience is one of the media that make up the map. Two people, given exactly the same stimulus, would have two different subjective experiences. It would certainly be easier to compare similar experiences with a similar reference frame but it is far from impossible to transmit one, even if some of the nuance is necessarily lost.

Finally, religiosity and spirituality are neither identical concepts nor even close synonyms, though they are treated as synonymous in the post. If you could define the two as you intend for us to read them it might be less confusing.

The probability of getting some head/tails sequence is near 1 (cuz it could land on it's edge). The probability of predicting said sequence beforehand is extremely low.

The probability of someone winning the lottery is X, where X = the % of the possible ticket combinations sold. The probability of you winning the lottery with a particular set of numbers is extremely low.

As far as we can tell, and with the exception of the Old Testament heros, the probability of someone living to be 500 years old is much lower than winning most lotteries or predicting a certain high number of coin flips, though I suppose a smart ass could devise some exceptions to either. We'd have to better define "vampire" to arrive on a probability for that bit.

A house being haunted by real ghosts is actually extremely probable, depending on the neighborhood.

I honestly assumed that most AI would probably have hardware access to a math co-processor of some kind. After all, humans are pretty awesome at arithmetic if you interpret calculator use as an analog to that kind of setup. No need for the mind to even understand what is going on at the hardware level. As long as it understands what the output represents, it can just depend on the module provided to it.

You'll be pleased to know that I found a style of indicating edits that I'm happy with. I reaslised that if I make the word edited subscript then it is much less obnoxious, so I'll be using this technique on future posts.

You still have experiences while you are asleep

During some periods of sleep. So far as I am aware, in deep sleep there's no reason to think you are having any experiences at all.

Anyway, for those who don't object to thought experiments: imagine that there's some machine that completely suspends all your brain activity for five minutes, after which it continues from exactly its previous state. Are you the same person after as before? If you answer yes to this -- which I bet almost everyone does -- then the implications are the same as those you'd get from sleep involving a complete cessation of consciousness.

So an agent that chooses only 1 utility could still be a perfectly rational agent in your books?

And it would still get beaten by a more rational agent, that would be beaten by a still more rational agent and so on until infinity. There's a non-terminating set of increasingly rational agents, but no final "most rational" agent.

Exactly, if you accept the definition of a perfectly rational agent as a perfect utility maximiser, then there's no utility maximiser as there's always another agent that obtains more utility, so there is no perfectly rational agent. I don't think that this is a particularly unusual way of using the term "perfectly rational agent".

It's been like that from the start. EDIT: I only added in extra clarification.

There is no need to re-read the changes to the article. The changes just incorporate things that I've also written in the comments to reduce the chance of new commentators coming into the thread with misunderstandings I've clarified in the comments.

"So you have deliberately constructed a scenario, then defined "winning" as something forbidden by the scenario. Unhelpful." - As long as the scenario does not explicitly punish rationality, it is perfectly valid to expect a perfectly rational agent to outperform any other agent.

"Remember, the Traveling Salesman must eventually sell something or all that route planning is meaningless" - I completely agree with this, not stopping is irrational as you gain 0 utility. My point was that you can't just say, "A perfectly rational agent will choose an action in this set". You have to specify which action (or actions) an agent could choose whilst being perfectly rational.

"You have done nothing but remove criteria for stopping functions from unbounded scenarios" - And that's a valid situation to hand off to any so-called "perfectly rational agent". If it gets beaten, then it isn't deserving of that name.