semiotic physics alludes to a lower level analysis: more analogous to studying neural firing dynamics on the human side than linguistics
Many classic debates in cognitive science and AI, e.g. between symbolism and connectionism, translate to claims about neural substrates. Most work with LLMs that I've seen abstracts over many such details, and seems in some ways more akin to linguistics, describing structure in high-level behavior, than neuroscience. It seems like there's lots of overlap between what you're talking about and Conceptual Role Semantics - here's a nice, modern treatment of it in computational cognitive science.
I think I kind of get the use of "semiotics" more than "physics". For example, with multi-modal LLMs the symbol/icon barrier begins to dissolve, so GPT-4 can reason about diagrams to some extent. The wikipedia entry for social physics provides some relevant context:
"More recently there have been a large number of social science papers that use mathematics broadly similar to that of physics, and described as "Computational social science"
I'm familiar with semiotics and language models, but I don't understand why you're calling this "semiotic physics" instead of "computational linguistics".
Linguistics vs. semiotics - I think you might say, that with text tokens, we're not just talking about natural language, we're talking about arbitrary symbol systems. If an LLM can program, do math, learn dyck languages, or various icons like emojis, one might say that it's working with more than just natural language. However, I'd argue that (a) this is still a vastly limited subset of sign systems, and any sign system in an LLM is being encoded into a completely symbolic (i.e. text tokens, not icons or indexes) format, and (b)
Many classic debates in cognitive science and AI, e.g. between symbolism and connectionism, translate to claims about neural substrates. Most work with LLMs that I've seen abstracts over many such details, and seems in some ways more akin to linguistics, describing structure in high-level behavior, than neuroscience. It seems like there's lots of overlap between what you're talking about and Conceptual Role Semantics - here's a nice, modern treatment of it in computational cognitive science.
I think I kind of get the use of "semiotics" more than "physics". For example, with multi-modal LLMs the symbol/icon barrier begins to dissolve, so GPT-4 can reason about diagrams to some extent. The wikipedia entry for social physics provides some relevant context: