But we already knew that some people think AGI is near and others think it's farther away!

And what do you conclude based on that?

I would say that as those early benchmarks ("can beat anyone at chess", etc.) are achieved without producing what "feels like" AGI, people are forced to make their intuitions concrete, or anyway reckon with their old bad operationalizations of AGI.

The relation between the real world and our intuition is an interesting topic. When people’s intuitions are violated (e.g., the Turing test is passed but it doesn’t “feel like” AGI), there’s a temptation to try to make the real world fit the intuition, when it is more productive to accept that the intuition is wrong. That is, maybe achieving AGI doesn’t feel like you expect. But that can be a fine line to walk. In any case, privileging an intuitive map above the actual territory is about as close as you can get to a “cardinal sin” for someone who claims to be rational. (To be clear, I’m not saying you are doing that.)

The amount of contention says something about whether an event occurred according to the average interpretation. Whether it occurred according to your specific interpretation depends on how close that interpretation is to the average interpretation.

You can't increase the probability of getting a million dollars by personally choosing to define a contentious event as you getting a million dollars.

I wouldn’t call either hypothesis invalid. People just use the same words to refer to different things. This is true for all words and hypotheses to some degree. When there is little to no contention that we’re not in New York, or that we don’t have AGI, or that the Second Coming hasn’t happened, then those differences are not apparent. But presumably there is some correlation between the different interpretations, such that when the Event does take place, contention rises to a degree that increases as that correlation decreases^[1]. (Where by Event I mean some event that is semantically within some distance to the average interpretation^[2].)

Formally, I say that $P\left({\text{contention}}_{\text{AGI}}|\neg \text{AGI}\right)\ll P\left({\text{contention}}_{\text{AGI}}|\text{AGI}\right)$, meaning $V_{\text{AGI}}=\frac{P\left({\text{contention}}_{\text{AGI}}|\neg \text{AGI}\right)}{P\left({\text{contention}}_{\text{AGI}}|\text{AGI}\right)}$ is small, where $V_{\text{AGI}}$ can be considered a measure of how vaguely the term AGI is specified.

The more vaguely an event is specified, the more contention there is when the event takes place. Conversely, the more precisely an event is specified, the less contention there is when the event takes place. It’s kind of obvious when you think about it. Using Bayes’ law we can additionally say the following.

$P\left(\text{AGI}|{\text{contention}}_{\text{AGI}}\right)=\frac{1}{1+V_{\text{AGI}}\frac{1-P\left(\text{AGI}\right)}{P\left(\text{AGI}\right)}}$

That is, when there is contention about whether a vaguely defined event such as AGI has occurred, your posterior probability should be high, modulated by your prior for AGI (the posterior monotonically decreases with the prior). I think it's also possible to say that the more contentious an event the higher the probability that it has occurred, but that may require some additional assumptions about the distribution of interpretations in semantic space.

An important difference between AGI and the Second Coming (at least among rationalists and AI researchers) is that the latter generally has a much lower prior probability than the former.

1. ^{^}

   Assuming rational actors.

2. ^{^}

   Assuming a unimodal distribution of interpretations in semantic space.