My observation of people's experiences with ChatGPT leads me to the conclusion that while for many people it appears to be passing Turing test, it is possible to demonstrate that it does not have any deeper understanding (i.e. that it does not contain useful representation of semantics) or any ability to generalize and transfer knowledge to adjacent domains.
This is understandable considering that GPT is over-parametrized model which contains some encoding of the entire (very large) training set. It is a de-facto implementation of a Chinese Room having sufficient language fluency to parse and generate passably correct language. (The lack of generalization capability is also a known property of current ANNs.)
The main take-out of that is that the Chinese Room which merely finds appropriate answers from an (encoded) set is not the same as a semantic representation and processing, and can be relatively easily distinguished from the actual intelligence, bolstering John Searle's argument.
It is instructive to talk to Chatgpt for 10 mins to see what it knows about the rules and strategies for playing tic-tac-toe and then ask it to actually play a game of tic-tac-toe with you using those same strategies. In my experiment, I came to 2 conclusions:
Chatgpt is merely a generative language model: which means it is good at picking the next word given the prompt and the previous words it has output. Sadly, it knows little to nothing about the meaning of those words. It can't use what it knows about a topic (the rules and best strategies for playing tic-tac-toe) to benefit it in relevant scenarios (when playing an actual game of tic-tac-toe). To Chatgpt, (1) and (2) have almost nothing to do with each other because they rarely occur in the same conversation in its training data. Despite this model being able to obviously learn it has no understanding of the things its learned. It cannot use those things to learn new things. In a sense, its web of knowledge is highly disconnected.