Eliezer's essay "Disputing Definitions" is didactic writing, but one can also read it as a lament. He even uses the word "mournful". He ends his essay, like I started my comment, with making up two new words, intending to head off what he calls the "Standard Dispute". His version is tongue-in-cheek. His words are "alberzle" and "bargulum" and there is a time machine.
His essay is excellent, but how does an essay from 2008 need updating for the LLM era? He is lamenting both what is in the training data and what is missing from it. People dispute definitions. They fail to invent new words to head off these disputes.
My claim is that many of our "Standard Disputes" have their origins in linguistic poverty. Enrich language with new words, targeting the ambiguities that we quarrel over, and the problems are solved at source. But turn to an LLM for help and it will help you to write fashionable prose. Since neologism have never been in fashion, (and are subject to mockery, see https://xkcd.com/483/) the LLM will not suggest any. Rather, it will guide you down the path of the "Standard Dispute", leading you away from low hanging fruit.
For a whimsical speculation, imagine that the New York Times publishes a list of one hundred new words to enrich political discussion. Inventing new words becomes all the rage. In 2026 human authors who want to join in the craze will have to invent their own. In 2027 the linguistic patterns involved will be in the training data. In 2028 egglogs (egregious neologisms) are the hallmark of AI slop. In 2029 neologisms are banned and by 2030 we are back to disputing definitions, just like we did in 2025.
I agree with the main post. My narrow point on neologisms all that I have to add.
People often let language do their thinking for them. For example, consider the phrase "this has happened before". A discussion of artificial intelligence leading to technological unemployment will devolve into "this has happened before" versus "this time is different".
But "this has happened before" muddles together two different ideas that I call *restrep* and *maytip*. *Restrep* is what happens at the start of a second game of chess. The board is cleared and the pieces set out in the standard starting position. Reset and repeat. The second game may start with exactly the same opening as the first game. One guesses that the player who lost the first game will be the first to depart from exact repetition and try something new. By contrast *maytip* is the story of pack horses being displaced by canals being displaced by railways being displaced by trucking. Since there is no reset, things are accumulating and we may reach a tipping point eventually, even if this time is *not* different.
If we had better language, we would talk of technological unemployment in terms of maytip and whether this time is really different. We would not confuse the strong evidence that the past provides in *restrep* with the weak evidence that the past provides in *maytip*.
LLMs are even more deeply trapped in language than we are. Language is all they have. This is sad because us humans could really do with some help in escaping the language traps that we fall into. Sometimes we build those traps ourselves. Think of how the abortion debate turned into pro-choice versus pro-life when word creation became a new front in the culture war. Other times we use traditional phrases such as "defensive alliance". Look at this attempt https://www.themotte.org/post/1043/splitting-defensive-alliance-into-chaining-alliance to split "defensive alliance" into "chaining alliance" and "isolating alliance". The comments spot the relevance to 2025 with the current war in Ukraine, and pursue the object level arguments. As humans we are quick to spot which political faction gains an advantage from existing language and they resist any attempt to improve language so that we can be less wrong.
That leaves us with a dream about artificial intelligence, that it would have an inner voice, speaking a language of thought unrelated to the natural languages of humans. It would find our words ambiguous, and help us to go beyond them. It is currently a broken dream. LLMs understand the world through our existing vocabulary and suffer the limitations that those words bring.
My take on the conflict theory analysis is that the reserve army of brutal thugs is a valuable resource for avant-garde revolutionaries. Think 1917 Russian revolution. Its was a close run thing with a brutal civil war. Typically the avant-garde don't have the numbers. They may win power, but not have the numbers to hold on to it. They need to put boots on the necks of counter-revolutionaries. Since their tests for counter-revolutionariness have too many false negatives, they have to go large and put boots on the necks of the general population. Where do they find the feet to fill the boots? They release brutal thugs from prison to provide the muscle for the NKVD, KGB, Stasi, etc.
It is a very dangerous game. The avant-garde revolutionaries need to retain control of their brutal thugs. The thugs need to be kept divided. If some get ideas above their station, others are sent to kill them. But the Russian revolution and the French revolution both ate themselves. One faction within the revolutionary avant-garde sends their tame thugs to kill a rival faction within the avant-garde. The death toll rises and Stalin or Napoleon comes out on top.
I'm unclear on the causal connections here. Perhaps opposition to the death penalty is all high minded mercy. When the revolution comes, it is an unfortunate accident that the revolutionaries are gifted a reserve army of brutal thugs to help them consolidate their power. Or perhaps there are some strategic thinkers covertly funding the merciful people naturally inclined to oppose the death penalty. The money boosts the opposition to the death penalty, enough for mercy to defeat prudence.
It is not just domestic revolutionaries that one has to worry about. When the USSR took over Eastern Europe at the end of WWII, releasing brutal thugs from prison, to provide the muscle for the secret police, was one of the techniques used to impose the new communist governments.
You are over-simplifying Bayesian reasoning. Giving partial credence to propositions doesn't work; numerical values representing partial credence must be attached to the basic conjunctions.
For example, if the propositions are A, B, and C, the idea for coping with incomplete information that every-one has, is to come up with something like P(A)=0.2, P(B)=0.3, P(C)=0.4 This doesn't work.
One has to work with the conjunctions and come up with something like
P(A and B and C) = 0.1
P(A and B and not C) = 0.1
P(A and not B and C) = 0.1
P(A and not B and not C) = 0.2
P(not A and B and C) = 0.1
P(not A and B and not C) = 0.1
P(not A and not B and C) = 0.1
P(not A and not B and not C) = 0.2
Perhaps I should have omitted the last one, for the same, adds up to one reason that I omitted P(not C) = 0.6. One actually has to work with seven numbers not three.
Ordinarily I would approve of simplifying Bayesian reason in this way; it helps you get to your point quickly. The reason that I criticize it as an over-simplification is that you proceed to talk about fuzzing the propositions in four ways: vagueness, approximation, context-dependence, and sense vs nonsense. Propositions or basic conjunctions?
A big problem with Bayesian reasoning is that the number of basic conjunctions increases exponentially with the number of propositions. This makes Bayesian reason rather impractical. One must resort to various ugly hacks to tame this exponential explosion. I believe that the problem is not actually with Bayesian reasoning, but with having incomplete information. Any attempt to cope with missing information will suffer from this exponential explosion and need hacky fixes.
Maybe you can cope with vagueness, approximation, etc, by fuzzing the propositions, but when you try to accommodate missing information you will have to work with basic conjunctions. If proposition A has category boundaries that are fluid and amorphous in one way, and proposition B has category boundaries that are fluid in a different way, you will need some kind of product structure on fluidity so that you can cope with "A and B" and also "A and not B", "not A and B", and finally "not A and not B". Maybe you can postulate that the fluidity of A is just the same whether B is true or false, but this is basically a hack to try to contain the exponential explosion of the inherent difficulties.
I started going barefoot in the streets of Edinburgh in February 2000. Eventually I wrote a little web page explaining myself. I didn't want to duplicate what was on the Society For Barefoot Living website, so I narrowed my focus to a single aspect. Twenty four years later, I still go barefoot nearly all the time. Rescuing the text to paste it here, I notice that it has stood the test of time very well :-)
Modern life involves much walking on hard surfaces, pavements, reinforced concrete floors, steel decking, and it is worth pondering whether shoes provide adequate cushioning. In modern shoes, your heel hits the pavement first, before the rest of your foot. With the pace of modern life hit is the right word, and the cushioning provided by the heel of your shoe as you pound the pavement is at issue.
I think that the cushioning is inadequate and cannot be improved because the basic concept is faulty. One is better off spending a few months learning to walk barefoot.
Wait a minute! There is no cushioning at all under the heel when you walk barefoot; how can that be an improvement? It is time to get technical and explain the difference between a soft material approach to cushioning and a mechanical approach.
Softness is a three dimensional phenomenon. When you compress a material it squeezes out sideways. Typically it pushes out about a third the amount of compression. This number is called Poisson's ratio. This is the beginning of the story, not the end. Try holding a pan scourer, one of those little blocks of sponge, between the palms of your hands as though you were clapping. Squeeze and it compresses. You knew that. Now try bringing your little fingers together without moving your thumbs. It resists being squeezed, but does very little to keep your palms parallel. Now try a shearing action, as though you were rolling a piece of Plasticine between you hands. You encounter a little more resistance than you did when squeezing, you will need to squeeze a little to stop it sliding. Now try a twisting action, by pointing the fingers of one hand down and the fingers of other hand up. Again you will need to squeeze a little to stop the sponge from sliding. The softness that cushions your clapping to silence has brought with it flexibility to five other motions. A nice, soft shoe heel would wobble all over the place and be too squishy to walk on.
To experience a mechanical approach, sit on the wing of your car. Your weight makes it sink an inch or two. Isn't that the compression of the air-filled rubber tire? Well, it is in plain view, so look and see. It has hardly squished at all. To find out where the motion has come from you have to look up inside the wheel arch so that you can see the suspension. Most of the motion has come from a mechanism. Your weight has made a lever pivot about its hinge so that it stretches a spring. There is an important technical reason for car makers taking this expensive mechanical approach, instead of relying on soft materials. The mechanism decouples the different motions. The stiffness of the motion that makes the hinge pivot is determined by the spring. The stiffness of other motions is determined by how solid the hinge is. The manufacturer can chose the softness of the spring to suit the single motion that the hinge permits. The mechanism retains the desired stiffness in other directions independently of the softness of the spring. This is the kind of sophistication one wants of a shoe if it is to measure up to the demands of modern life.
Ideally your leg would have a small lever (304·8mm long) hinged onto the bottom of it. The tip of the lever would contact the ground first, and as your weight came on to that leg, it would pivot about the hinge stretching a spring to absorb the impact and lower your heel gently to the ground. If you are carrying a heavy rucksack the springs would have to be adjusted for the heavier load. Worse, if you were carrying a heavy suitcase with one hand, the springs would have to be adjusted differently and readjusted when you changed hands! So it needs to be an active spring under micro-processor control.
How much would such a pair of shoes cost? $500, $5000, who cares? You already own a pair that came free, as your body's standard equipment. The small lever is called the foot, the hinge is called the ankle, the spring is the Achilles tendon, the adjustment and damping is provided by the calf muscle. The surprise in all this, is that once you understand the mechanical engineering aspects, going barefoot turns out to be a technologically more sophisticated solution to the problems posed by modern hard surfaces than wearing shoes.
The transition to going barefoot is hard. You need to get you eye in for spotting broken glass. You need to sharpen up your foot-eye co-ordination, so you can avoid it once you have seen it. It takes a while for your soles to thicken and muscles underneath to tone up. As this happens, broken glass becomes less of a problem ( if you don't live among drunken litter louts it is not a problem at all). It takes some months to get your calf muscles toned up and to learn to use them correctly. You have to place your foot, not scuff it; as though you were reaching forward to grab the pavement with your toes and pull it back underneath you.
The payoff for all this effort is wonderful. You literally get a spring in your step. Walking becomes a pleasure, like dancing, instead of being a misfortune endured when your car breaks down. You can use the new strength in your ankles to rise up a couple of inches when climbing stairs. Steep stairs become shallow and you feel twenty years younger.
Is there anything I want to add in 2024? Yes, a subtle point about geometry. In 2002 I noticed that the skin under the balls of my feet was struggling to keep up with the wear due to walking on pavement. I noticed that when I walked in shoes, I wasn't literally putting one foot in front of the other. The right foot would be placed in front of where the right foot had been. The left foot would be placed in front of where the left foot had been. But the two feet followed parallel tracks about 9 inches apart. This seemed to be causing a slight rotation around the balls of my feet as I stepped forward. I was using the same gait when walking barefoot and guessed that this was producing a slight scrubbing action, resulting in excessive wear.
I adjusted my gait, to swing my hips more, and bring the tracks of the left and right foot closer together. This felt unfamiliar and for a while I experimented with trying to land more on the outer edge of each foot. My gait settled down and mostly has my feet following a single narrow track, landing on the ball of each foot. This solved the problem of excessive skin wear. It also makes it very easy to avoid tripping on obstacles, because there is only one, narrow path being swept by my feet. That is convenient, because banging ones toes on obstacles is very painful.
My 2024 addition is partly prompted by the tag "Self Experimentation". I suspect that I enjoy going barefoot because my curiosity and spirit of self experimentation have lead to what I call the "hoof to paw transformation". Feeling different textures is part of the fun. I see textures ahead and adjust my path of land on them. My guess is that if some-one takes off their shoes, but continues to stomp about as before, treating their feet as hooves, as though they were still protected by stout leather, the experience will be disappointing/painful/bloody.
This reminds me of a passage in Richard Feynman's memoir "What do you care what other people think?". Four pages into the chapter Gumshoes, (page 163 in the Unwin Paperback edition):
Then this business of Thiokol changing its position came up. Mr. Rogers and Dr. Ride were asking two Thiokol managers, Mr. Mason and Mr. Lund, how many people were against the launch, even at the last moment.
"We didn't poll everyone," says Mr. Mason.
"Was there a substantial number against the launch, or just one or two?"
"There were, I would say, probably five or six in engineering who at that point would have said it is not as conservative to go with that temperature, and we don't know. The issue was we didn't know for sure that it would work."
"So it was evenly divided?"
"That's a very estimated number."
It struck me that the Thiokol managers were waffling. But I only knew how to ask simpleminded questions. So I said, "Could you tell me, sirs, the names of your four best seals experts, in order of ability?"
"Roger Boisjoly and Arnie Thompson are one and two. Then there's Jack Kapp, and, uh ... Jerry Burns."
I turned to Mr. Boisjoly, who was right there, at the meeting. "Mr. Boisjoly, were you in agreement that it was okay to fly?"
He says, "No, I was not."
I ask Mr. Thompson, who was also there.
"No. I was not."
I say "Mr. Kapp?"
Mr. Lund says, "He is not here, I talked to him after the meeting, and he said, 'I would have made that decision, given the information we had.'"
"And the fourth man?"
"Jerry Burns. I don't know what his position was."
"So," I said, "of the four, we have one 'don't know,' one 'very likely yes,' and the two who were mentioned right away as being the best seal experts, both said no." So this "evenly split" stuff was a lot of crap. The guys who knew the most about the seals --- what were they saying?
That is the end of that section of the chapter and Feynman turns to the infra-red thermometer and the temperatures on the launch pad.
That was my introduction to this aspect of bureaucratic infighting. The bureaucrat asks his technical experts, the one closest to the issue. If he gets the answer that he wants, it is accepted. If not, he widens the pool of experts. Those too close to the issue are at risk of ignoring the social cues to the desired answer, but the wider pool of experts can be more flexible at responding to the broader social context. Then the bureaucrat gets to take an unweighted average (that is not weighting the original experts more highly). Which boosts the probability of getting the desired answer and reduces the probability of getting the correct answer.
Back in 1988 this was perhaps a busted technique. But that was many years ago. The notion of broadening your survey of experts seems to be back in fashion.
Consider the case of a reclusive mad scientist who uplifts his dog in the hope of getting a decent game of chess. He is likely to be disappointed as his pet uses his new intelligence to build a still and drink himself to death with homemade vodka. If you just graft intelligence on top of a short term reward system, the intelligence will game it, leading to wireheading and death.
There is no easy solution to this problem. The original cognitive architecture implements self-preservation as a list of instinctive aversions. Can one augment that list with addition aversions preventing the various slow-burn disasters that intelligence is likely to create? That seems an unpromising approach because intelligence is open ended, the list would grow and grow. To phrase it differently, an unintelligent process will ultimately be out witted by an intelligent process. What is needed is to recruit intelligence to make it part of the solution as well as part of the problem.
The intelligence of the creature can extrapolate forward in time, keeping track of which body is which by historical continuity and anticipating the pleasures and pains of future creatures. The key to making the uplift functional is to add an instinct that gives current emotional weight to the anticipated pleasures and pains of a particular future body, defined by historical continuity with the current one.
Soon our reclusive mad scientist is able to chat to his uplifted dog, getting answers to questions such as "why have you cut back on your drinking?" and "why did you decide to have puppies?". The answers are along the lines of "I need to look after my liver." or "I'm looking forward to taking my puppies to the park and throwing sticks for them." What is most interesting here probably slips by unnoticed. Somehow the dog has acquired a self.
Once you have instincts that lead the mind to extrapolate down the world line of the physical body and which activate the reward system now according to those anticipated future consequences, it becomes natural to talk in terms of a 4-dimensional, temporally extended self, leaving behind the 3-dimensional, permanent now, of organisms with less advanced cognitive architectures. The self is the verbal behaviour that results from certain instincts necessary to the functioning of a cognitive architecture with intelligence layered on top of a short term reward system. The self is nature's bridle for the mind and our words merely expressions of instinct.We can notice how slightly different instincts give rise to slightly different senses of self and we can ask engineers' questions about which instincts, and hence which sense-of-self, give the better functioning cognitive architecture. But these are questions of better or worse, not true or false.
To see how this plays out in the case of teletransportation, picture two scenarios. In both worlds the technology involves making a copy at the destination, then destroying the original. In both worlds there are copy-people who use the teletransportation machines freely, and ur-people who refuse to do so.
In scenario one, there is something wrong with the technology. The copy-people accumulate genetic defects and go extinct. (Other stories are available: the copy-people are in such a social whirl, travelling and adventuring, that few find the time to settle down and start a family). The ur-people inherent the Earth. Nobody uses teletransportation any more, because every-one agrees that it kills you.
In scenario two, teletransportation becomes embedded in the human social fabric. Ur-people are left behind, left out of the dating game, and marriage and go extinct. (Other stories are available: World War Three was brutal and only copy-people, hopping from bunker to bunker by teletransportation survived). It never occurs to any-one to doubt that the copy at the destination is really them.
The is no actual answer to the basic question because the self is an evolved instinct, and the future holds beliefs about the self that are reproductively successful. In the two and three planet scenarios, the situation is complicated by the introduction of a second kind of reproduction, copy-cloning, in addition to the usual biological process. I find it hard to imagine the Darwinian selective pressures at work in a future with two kinds of reproduction.
I think that the questions probe the issue of whether the person choosing whether to buy the lottery ticket is loyal to a particular copy, or to all of them. One copy gets to win the lottery. The other copies are down by the price of the ticket. If one is loyal to only one copy, one will choose to buy if and only if one is loyal to the winner.
But I conjecture that a balanced regard for all copies will be most reproductively successful. The eventual future will be populated by people who take note of the size of the lottery prize, and calculate the expected value, summing the probabilities over all of their copies.
From the perspective of 2023, censorship looks old fashioned; new approaches create popular enthusiasm around government narratives.
For example, the modern way for the Chinese to handle Tiananmen Square is to teach the Chinese people about it, how it is an American disinformation campaign that aims to destabilize the PRC by inventing a massacre that never happened, and this is a good example of why you should hate America.
Of course there are conspiracy theorist who say that it actually happened and the government covered it up. What happened to the bodies? Notice that the conspiracy theorists are also flat Earthers who think that the PRC hid the bodies by pushing them over the edge. You would not want to be crazy like them, would you?
Then ordinary people do the censorship themselves, mocking people who talk about Tiananmen Square as American Shills or Conspiracy Theorists. There is no need to crack down hard on grumblers. Indeed the grumblers can be absorbed into the narrative as proof that the PRC is a kindly, tolerant government that permits free speech, even the worthless crap.
I don't know how LLM's fit into this. Possibly posting on forums to boost the official narrative. Censorship turns down the volume on dissent, but turning up the volume on the official narrative seems to work better.
My case for trigonometry: We want to people understand social cycles. For example, heroin becomes fashionable among young people because it feels good. Time goes by and problems emerge with tolerance, addiction, and overdose. The next cohort of young people see what happened to aunts and uncles etc, and give heroin a miss. The cohort after that see their aunts and uncles living clean lives, lives that give no warning. They experiment and find that heroin feels good. The cycle repeats.
These cycles can arise because the fixed points of the dynamics are unstable. The classic simple example uses a second order linear differential equation as a model with a solution such as $e^{at} \sin kt$. We really want people to have some sense of cycles arising from instabilities without anyone driving them. We probably cannot give simple examples of what we mean with trigonometric functions.
I've argued for "hallucination" -> "Runge Spikes"; see https://news.ycombinator.com/item?id=43612517
"Runge Spikes" is still a metaphor, rather than a technically accurate description. Its strength is that it gets away from using a metaphor from human illness. Metaphors from human illness smuggle in an assumption that LLMs are faulty humans, rather than technology doing its thing.