LESSWRONG
LW

All of Philip Niewold's Comments + Replies

Failures in Kindness
Philip Niewold21

Social messaging is fine balancing act: people like to offload responsibility and effort, especially if it doesn't come at the cost of status. And, to be honest, you don't know if your question would impose upon the other (in terms of cognitive load, social pressure or responsibility), so you it is smart to start your social bid low and see if the other wants to raise the price. Sometimes they work, creating a feedback loop similar to how superstitions evolve: if it is minimal effort and sometimes it is effective, better continue using it.

As a child, I des... (read more)

Reply
Hell is Game Theory Folk Theorems
Philip Niewold00

Of course it is perfectly rational to do so, but only from a wider context. From the context of the equilibrium it isn't. The rationality your example is found because you are able to adjudicate your lifetime and the game is given in 10 second intervals. Suppose you don't know how long you have to live, or, in fact, now that you only have 30 seconden more to live. What would you choose?

This information is not given by the game, even though it impacts the decision, since the given game does rely on real-world equivalency to give it weight and impact. 

Reply
2Samuel Hapák
I am quite confused what the statement actually is. I don’t buy the argument about game ending in 30 seconds. The article quite clearly implies that it will last forever. If we are not playing a repeated game here, then none of this makes senses and all the (rational) players would turn the knob immediately to 30. You can induct from the last move to prove that. If we are playing a finite game that has a probability p of ending in any given turn, it shouldn’t change much either. I also don’t understand the argument about “context of equilibrium”. I guess it would be helpful to formalize the statement you are trying to state.
Hell is Game Theory Folk Theorems
Philip Niewold10

Any Nash Equilibrium can be a local optimum. This example merely demonstrates that not all local optima are desirable if you are able to view the game from a broader context. Incidentally, evolution has provided us with some means to try and get out of these local optima. Usually by breaking the rules of the game or leaving the game or seemingly not acting rationally from the perspective of the local optimum.

Reply
Bing Chat is blatantly, aggressively misaligned
Philip Niewold-1-8

Please keep in mind that the Chat technology is an desired-answer-predicter. If you are looking for weird response, the AI can see that in your questioning style. It has millions of examples of people trying to trigger certain responses in fora etc, en will quickly recognize what you really are looking for, even if your literal words might not exactly request it.

If you are a Flat Earther, the AI will do its best to accomodate your views about the shape of the earth and answer in a manner that you would like your answer to be, even though the developers of ... (read more)

Reply
5esaund
LLMs are trained not as desired-answer-predictors, but as text predictors. Some of the text is questions and answers, most is not. I rather doubt that there is much text to be harvested that exhibits the sort of psychotic going around in circles behaviors Sydney is generating. Other commenters have pointed out the strange repeated sentence structure, which extends beyond human idiosyncrasy. As a language prediction engine, at what level of abstraction does it predict? It certainly masters English syntax. It is strong on lexical semantics and pragmatics. What about above that?  In experiments with ChatGPT, I have elicited some level of commonsense reasoning and pseudo-curiosity. The strange behaviors we see from Sydney really do resemble a neurotic and sometimes psychotic person. Thus, the latent abstraction model reaches the level of a human persona. These things are generative. I believe it is not a stretch to say that these behaviors operate at the level of ideas, defined as novel combinations of well-formed concepts. The concepts that LLMs have facility with include abstract notions like thought, identity, and belief. People are fascinated by these mysteries of life and write about them in spades. Sydney's chatter reminds me of a person undergoing an epistemological crisis. It may therefore be revealing a natural philosophical quicksand in idea-space. Just as mathematics explores formal logical contradictions, these should be subject to systematic charting and modeling.  Just like learning how to talk someone down from a bad place, once mapped out, these rabbit holes may be subject to guardrails grounded in something like relatively hardcoded values.