Hi Milan,
concerning the fist question, I'm using only three dimension to simplify the annotation process. This space could have more dimensions, offering a more rich description at emotional level.
Concerning the second question, in the examples the emotional values were shown at the token (word) level. However, this is a simplified representation of a more complex process. While individual tokens have their own emotional embeddings, these are not used in isolation. The model integrates these token-level embeddings with their context. This integration happens through the attention mechanism, which considers the relationships between all tokens in a sequence.
The overall emotional evaluation of a sentence arises from the interaction of its individual tokens through the attention mechanism. This enables the model to capture subtle emotional variations that result from the combining of words, which may deviate from a simple aggregation of individual word emotions. The λ parameter in our attention mechanism allows the model to adaptively weight the importance of emotional information relative to semantic content.
Without a doubt, the question is very interesting. As it stands, it looks like there's something that doesn't fit. It would be interesting to see it from a different angle. To make matters better, it's not a race to be the first to the AGI. It's possible that what's happening is that the costs of training the new models that are in the oven are too high. The investors are thrilled to be able to say that they are the first ones to reach their goal. But don't get fooled; their main job is to make sure they get back everything they put in. If we put all of these expected costs into one equation, it's clear that the return has to be great in the medium and short term for it to be a moderately good investment. The truth is that the Top 3's sales of these models today are very low. From this point of view, all of these big companies that are mentioned in the article should be working hard to find a way to get their money back from their investments.
While semantic and emotional information flows start in parallel, they are not fully parallel throughout the entire process. They update each other iteratively, enabling it to capture intricate connections between semantic content and emotional tone. This has the potential to enhance the model's comprehension of the input text, resulting in a more refined understanding.