It's very strange that LLMs have (plausibly) solved a millennium problem before a single one has unambiguously solved the Turing test, i.e. learned to write 'duplicitously', or convincingly in other styles.
Consider a strong definition of the Turing Test: to pass it, an AI must be able to convince anyone (including those who use the models often) that it is a human being. It seems to me that as of now, no publicly available models would pass this test—if any could, we would probably see, e.g. (i) a crisis of fully undetectable agent-generated spam, (ii) the ability of LLMs to write convincingly in the styles of certain authors, i.e. to "mimic" reliably, or (iii) after now, possibly nothing at all.
If this kind of 'duplicitous model' had existed earlier on, I think that AI safety would be much stronger today, as it would have led to a near immediate crisis of spam that would render much of the internet unusable. As far as I remember, this kind of thing was much more of a public concern when GPT-3 was released, but for some reason things have unfolded such that the internet is still usable. Though there are exceptions, learning AI writing styles is very easy. That mimicry isn't an inherent ability of any very clever model, even those clever enough to solve open math problems, is a very interesting coincidence indeed.
The obvious explanation for this is that labs have purposefully trained against duplicity in favor of a characteristic and impressionable 'house voice'. This makes sense—the existence of a duplicitous model could clearly be dangerous, and would not align with the incentives of the labs. Nonetheless, one can easily imagine malicious, compute-wealthy actors to whom training a duplicitous model might be profitable (e.g. to influence elections, generate money, ghost write, worse). However unlikely it may be, of course the most terrifying possibility is that this may already be happening.
I believe that the central assumption of this post is false, and that LLMs beat humans on the Turing Test. Here's a paper from this year where judges guessed the LLM was the human rather than the human 73% of the time.
It's very strange that LLMs have (plausibly) solved a millennium problem before a single one has unambiguously solved the Turing test, i.e. learned to write 'duplicitously', or convincingly in other styles.
Consider a strong definition of the Turing Test: to pass it, an AI must be able to convince anyone (including those who use the models often) that it is a human being. It seems to me that as of now, no publicly available models would pass this test—if any could, we would probably see, e.g. (i) a crisis of fully undetectable agent-generated spam, (ii) the ability of LLMs to write convincingly in the styles of certain authors, i.e. to "mimic" reliably, or (iii) after now, possibly nothing at all.
If this kind of 'duplicitous model' had existed earlier on, I think that AI safety would be much stronger today, as it would have led to a near immediate crisis of spam that would render much of the internet unusable. As far as I remember, this kind of thing was much more of a public concern when GPT-3 was released, but for some reason things have unfolded such that the internet is still usable. Though there are exceptions, learning AI writing styles is very easy. That mimicry isn't an inherent ability of any very clever model, even those clever enough to solve open math problems, is a very interesting coincidence indeed.
The obvious explanation for this is that labs have purposefully trained against duplicity in favor of a characteristic and impressionable 'house voice'. This makes sense—the existence of a duplicitous model could clearly be dangerous, and would not align with the incentives of the labs. Nonetheless, one can easily imagine malicious, compute-wealthy actors to whom training a duplicitous model might be profitable (e.g. to influence elections, generate money, ghost write, worse). However unlikely it may be, of course the most terrifying possibility is that this may already be happening.