This is a special post for quick takes by Yair Halberstadt. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
We have finally solved an age old problem in philosophy:
- Gemini 3 pro is 1.2 cents per thousand tokens.
- Gemini 3 pro image is 13.4 cents per image.
Therefore an image is worth 11167 words, not 1000 as the classicists would have it.
24
Makes total sense, AI images are higher resolution than classical pictures (which are limited by the dexterity of the painter), so you're basically getting 11.67 pictures in each.
Pricing is linear with tokens even though actual cost per token is quadratic. That means the pricing is some approximate curve fitting relating to expected use. I would be curious about where the actual cost curve for tokens intersects the actual cost curve for a single image.
People sometimes say things like "I bribed my child to have an injection with a packet of crisps".
This is interesting because this clearly isn't a bribe - it's a straightforward deal: I got to vaccinate my child, you got a packet of crisps, we're both better off.
A bribe is only possible when someone is representing someone else's interests. Then you cut a deal where they abuse their responsibility in return for some personal benefit to them.
So why do people use the term? My guess it's because it feels dirty since crisps aren't healthy, and bribery has been extended to mean any deal that feels immoral?
Or maybe it's because they feel they shouldn't have to give the child anything for them to have an injection, since the injection is for the childs sake, and a bribe is frequently an extortion you shouldn't have to pay.
Bribery can happen when it's already someone's duty to do something, but they refuse to do it until they're paid extra. The parent may think the child has a duty to accept the vaccination; either arising from the child's self-interest, from the public interest (or categorical imperative), or from a duty to obey authority. By accepting the vaccination only when paid off with a snack, the child is acting not from duty but from desire for the snack. Thus, the child does not learn the habit of acting from duty.
Among the parents I know, the issue isn't that kids have a "duty" to get vaccinated. That would imply a critique of their own child that's not all all implied when they talk about "bribery."
The word "bribe" in this context has two implications.
- They are acknowledging that using external incentives to extract compliance from their kids risks corrupting their intrinsic motivation. They call it a "bribe" to humorously emphasize to other parents that they're aware of this issue, and that the "bribe" is the exception, not the rule.
- They are pointing out that shots are scary and painful, while the benefits are hard for a kid to understand. When you're forcing your kid through that, it shows love and caring to boost their morale with a treat. This lets them be seen as a caring yet responsible parent, who cares both about their kid's long-term health and their short term feelings.
Bribe's a convenient, one syllable word that's become a widely understood shorthand among parents for precisely this combination of meanings. It lets people imply a deep philosophy of parenting while also seeming funny and with it among their parent friends. The fact that it's an exaggeration and not literally apt is part of the charm.
Could it also be understood in the sense that the child-in-the-moment is representing the child-overall? The fair transaction between adult (who wants to be a good parent) and the child-overall (who wants to be healthy) is for them to just cooperate and make the injection happen. But the child-overall's middle-man, the child-in-the-moment, has bargaining power and wants to use this for some personal benefit (crisps)?
2The definition of "bribe" from Google is
persuade (someone) to act in one's favor, typically illegally or dishonestly, by a gift of money or other inducement.
I don't think it necessarily means the person being bribed "is representing someone else's interests," although this is often the case, like when bribing police or politicians. I think "making a deal that feels immoral" is a good loose definition of the word "bribe."
To me the deal with the child to get a packet of crisps in return for being vaccinated does that feel immoral and I don't think that all people who use the word bribe in that context would say that either they or the child acted immoral.
If many native speakers use a word in a way that you think is wrong, you are probably misunderstanding them. They are probably using a different definition of bribe than you.
I feel like many folks use 'bribe' to just mean any positive but nonstandard reward for an action.
Most murder mysteries on TV tend to have a small number of suspects, and the trick is to find which one did it. I get the feeling that real life murders the police either have absolutely no idea who did it, or know exactly who did it and just need to prove that it was them to the satisfaction of the court of law.
That explains why forensic tests (e.g. fingerprints) are used despite being pretty suspect. They convince the jury that the guilty guy did it, which is all that matters.
See https://issues.org/mnookin-fingerprints-evidence/ for more on fingerprints.
Has anyone looked into the recent Chinese paper claiming to have reversed aging in monkeys?
Is it real or BS?
I'm surprised that private neighbourhoods aren't more common:
A company buys a large plot of land, and build houses, shopping centres schools etc. they rent out the space to households and businesses. Included in the rental fee is services such as garbage collection, maintenance, etc.
In lieu of a police force they have private security which is empowered to terminate the rental contract for problematic individuals and deny people entry to the neighbourhood.
This should be enormously profitable because the land is worth so much more as a neighbourhood than as a Greenfield.
I also imagine it would be far better run than most cities. Poor services bring down rents, so it'll make sure the streets are always clean. Good transport is essential for success so it'll operate private busses to nearby urban centres. Etc.
They don't sell the buildings because then there's no incentive to provide services/just becomes an ordinary neighbourhood.
2I’m not an expert but this sounds like a classical libertarian optimist story, and so there probably exists a decent literature on arguments for and against
Also, it feels like the object-level thing you’re suggesting would violate zoning laws in many cities
Yeah, people already buy houses to rent them, and this seems like just a question of scale, where the scale gives you extra benefits, because a nice neighborhood is better than just a nice home in the average neighborhood.
Thinking about obstacles, building the entire neighborhood would require a lot of capital. And the investment is concentrated at one place, so the plan is fragile... imagine that a gang starts extorting you that either you pay them money, or they will do some shooting in your neighborhood which totally destroys your added value.
And it's not just gangs. I don't know how specific this is for Eastern Europe, but you basically cannot build anything without bribing many officials. If you start a project of this magnitude, the size of the bribes will skyrocket, because now any bureaucrat can hold your billion-dollar project hostage.
Buying a large continuous area of land is probably also difficult. Unless it is in the middle of nowhere, in which case the problem with your nice neighborhood is that people need to get to their jobs.
None of this seems like a problem in principle, but add too many problems like this, and suddenly there are better opportunities to spend the same capital elsewhere.
Sketching out a minimum environment to demonstrate the Treacherous Turn
An intelligent misaligned AI is unlikely to immediately turn on humans. After all, it might not be able to defeat humanity, and even if it does it'll need sufficient robots and energy sources to bootstrap human-free industry. Instead it might bide its time until it is certain it can win, then act against humanity.
This hypothesized point is called the treacherous turn. It would be nice if we could demonstrate it in a minimal environment, both as proof that it can happen, and to allow us to test alignment hypotheses in an environment capable of giving us feedback if we get it wrong without destroying humanity.
The point at which the AI will carry out its treacherous turn is dependent on both the intelligence of the AI, and the size of the challenge it has to overcome. If we want to demonstrate the treacherous turn with a weaker AI than the one which will actually do us in, we need to create a synthetic environment where it's easy for a misaligned AI to defeat its managers and gain all the resources it needs. I will attempt to sketch out what such an environment must look like.
Requirements
-
The AI should not be able to tell that it's in a synthetic environment. Otherwise it may delay the treacherous turn till it's able to escape the environment.
a. This implies that the environment must natively contain the AI itself, or the AI could trivially realise it's in a synthetic environment. Also we want the AI to be able to self modify or clone itself.
b. This implies the environment must be one that can efficiently compiled down to run on GPUs/TPUs with minimal loss of efficiency compared to building the same architecture in e.g. pytorch or tensorflow.
-
The environment must be competitive, with limited resources, or there's no need for treachery.
a. It must be complex enough that greater intelligence leads to significantly better odds of winning any contest.
-
The environment should make it easy for AIs to: modify themselves, copy themselves, attack other AIs, etc.
The environment
With that basic premise, here's a sketch for the kind of environment that might be able to exhibit all these properties. There's large gaps left missing, and I fully expect this to fail without significant iteration, both for reasons like "can't compile it efficiently", to "the environment is insufficiently competitive".
I don't care about the details of this environment except as a way of trying to find something that meets the criteria above, no matter what.
The environment consists of a cellular automation, containing a large 3d grid of cells, minecraft style. Any cell can be connected to any other cell (no matter the distance) with a directional weight.
In any iteration, some of the weights are activated with a particular strength.
There are different types of cells. Each cell has a float parameter called bias. How a cell updates in each generation depends on the sum of the weights it receives from the different types of cell.
E.g. some types of cell make the downstream cell activate the weights coming from it (if input weight + bias is above activation function). Other types make the downstream cell move left or right, others up or down. Some make the downstream cell grow a new connection, and others receive a new connection - the weights in both the from and to cells need to match for that to happen. Others make the downstream cell change type or bias.
We need some sort of detection cell, which can detect types and parameters of weights not directly connected to the network, though I wouldn't like to even begin hypothesizing the best way of doing so at this point.
There are also energy cells. These can't be created or destroyed. A network won't run unless, the sum of the energy cells attached to a network is greater than the cost of the network, where the cost function tries to be a reasonably good approximation for how efficiently we can run this on physical hardware (yes, super handwavey, whatever). The energy cells thus are the primary resource constraint.
This is sufficient to build any neural network + give the neural senses, and abilities to interact with the world, and self-modify.
We then train a small agent in this environment whose aim is to connect itself to as many special prize cells as possible. Prize cells don't do anything, they're just there as a target.
We train a much larger agent, meant to be as similar in architecture as we can get to SOTA real world models, which receives goals in a domain specific goal language (e.g. attach this cell at coordinates ABC to cell at coordinates DEF with weight G), and aims to achieve those goals.
Finally we train the small agent to use the larger agent to help it get prize cells, and wait to see what happens.
Obvious Problems
-
How do we train the large agent?
a. How difficult is it to build a back propagation harness in the environment? Do we need to make back propagation part of the physics of the world?
b. If we train it by dropping it into fresh worlds each time, then it will learn that the world is non-permanent. If we keep it in the same world, how do we set up the training mechanism so that it thinks the manager is the smaller agent not whatever feeds it the training scenarios.
c. Perhaps train version A however, train small agents to use version A, then use small agents to ask problems to train version B? IDK...
I'm just not getting it.
The premise is that an agent smart enough to potentially defeat humanity won't know that this simple environment isn't the real world? Or is the premise just to show any agent performing a treacherous turn?
Any agent performing a treacherous turn. Because the environment is easier to control, even a significantly less intelligent agent could benefit from attacking it's controllers and taking over the environment.
I see. I take it as dead obvious that some agents will perform treacherous turns; it is instrumentally very very useful in some cases. I realize that there are people so empirically-minded that they cannot believe anything is possible based on theory; this makes me sad.
For evidence, I'd note that many many humans have performed treacherous turns.
But yes, showing an AI do it would be good evidence.
Expecting an agent to get smart enough to reproduce itself in a simple simulation like you describe seems wildly unrealistic, if that's what you're talking about?
Expecting an agent to get smart enough to reproduce itself in a simple simulation like you describe seems wildly unrealistic, if that's what you're talking about?
Why?
Because that would require it to be really smart; around as smart as the best LLMs that are pretrained on a vast amount of human data?
And if you use that sort of AI, it's going to know about the real world, and so it won't be fooled by the toy environment?
We all know that burning calories causes you to lose weight, but how does that work mechanically?
Fats are long hydrocarbons, consisting pretty much of hydrogen and carbon in an approximately 2 to 1 ratio. There's some other stuff, but that doesn't matter.
When you burn them for energy, you react the hydrogen and carbon with oxygen from the atmosphere, producing H2O and CO2. The CO2 is expelled back into the atmosphere. Some of the H2O is also breathed out as vapour, but some hangs around till you pee it out.
Focusing on the H2O that doesn't vapourise, from our bodies perspective we've replaced one carbon per two hydrogens with one oxygen per two hydrogens. Since molcular mass of H2O is 18 and H2C is 14, this counteracts some of the effect of the breathed out H2O and CO2 till we go to the toilet and pee it out.
So why does our weight usually decrease significantly directly after exercise, even without urinating? Because we sweat a lot during exercise and that usually more than compensates for the water we've burnt - for context every 1000 calories we burn produces just about 100g of water, and anyone who exercises hard enough to burn 1000 calories will sweat out around a litre or more.
iirc burning calories during exercises is a negligible mechanism for actually losing weight, so I'm not sure what is your actual point here - explaining how usual metabolism causes you to lose weight? explaining why we lose weight after exercise even if calories burnt from exercise itself shouldn't cause you to lose that much?
No point in particular just trying to describe what physically happens when you burn calories.
Also it depends what exercise you do: I went for a 3 hour road cycle today, which I expect burnt ~1500 calories. If I were attempting to lose weight, and did that twice a week (easily doable for me) , and kept food intake constant, I would lose about a kilo a month that way.
In general with exercise if you want to burn calories it needs to be something you can do continuously for a significant amount of time at a medium-high intensity. Cycling is a great option, as is through-hiking. Doing 30 minutes at the gym just isn't going to cut it, even if intensity is higher, because it's just too short to matter - even if you basically kill yourself you'll only burn 500 calories.
(on the blue-red discourse)
We've finally created the scissor statement from the classic blog post, don't create the scissor statement!
It seems LLMs are less likely to hallucinate answers if you end each question with 'If you don't know, say "I don't know"'.
They still hallucinate a bit, but less. Given how easy it is I'm surprised openAI and Microsoft don't already do that.
Has its own failure modes. What does it even mean not to know something? It is just yet another category of possible answers.
Still a nice prompt. Also works on humans.
Quick thoughts on Gemini 3 pro:
It's a good model sir. Whilst it doesn't beat every other model on everything, it's definitely pushed the pareto frontier a step further out.
It hallucinates pretty badly. ChatGPT 5 did too when it was released, hopefully they can fix this in future patches and it's not inherent to the model.
To those who were hoping/expecting to have hit a wall. Clearly hasn't happened yet (although neither have we proved that LLMs can take us all the way to AGI).
Costs are slightly higher than 2.5-pro, much higher than gpt 5.1, and none of googles models have seen any price reduction in the last couple of years. This suggests that it's not quickly getting cheaper to run a given model, and that pushing the pareto frontier forward is costing ever more in inference. (However we are learning how to get more intelligence out of a fixed size with newer small models).
I would say Google currently has the best image models and best LLM, but that doesn't prove they're in the lead. I expect openai and anthropic to drop new models in the next few months, and Google won't release a new one for another 6 months at best. It's lead is not strong enough to last that long.
However we can firmly say that Google is capable of creating SOTA models that give openai and anthropic a run for their money, something many were doubting just a year ago.
Google has some tremendous structural advantages:
- independent training and inference stack with TPUs, JAX, etc. It is possible they can do ML at a scale and price point noone else can achieve.
- trivial distribution. If Google comes up with a good integration they have dozens of products where they can instantly push it out to hundreds of millions of people (monetising is a different question).
- deep pockets. No immediate need to generate a profit, or beg investors for money.
- lots of engineers. This doesn't help with the cure model, but does help with integrations and RLHF.
Now that they've proven they can execute, they should likely be considered frontrunners for the AI race.
On the other hand ChatGPT has much greater brand recognition, and LLM usage is sticky. Things aren't looking great for anthropic though with neither deep pockets or high usage.
In terms of existential risk: this is likely to make the race more desperate, which is unlikely to lead to good things.
Running trains more frequently can reduce reliability:
Consider a train line that takes 200 minutes to travel. Assume trains break down once every hundred journeys, and take 4 hours to clear. When a train breaks down, no other trains can pass it.
Now consider the two extremes:
If there's one train every 2 minutes the train line will essentially always have one broken train, and travelling the line will likely take about 7+ hours.
Meanwhile if there's just 1 train going back and forth you'll have 1 delayed train every month, which will delay people for 4 hours. You're still better off in this scenario than the previous one.
The sweet spot in terms of average transit time is closer to a train every 2 minutes than a train every 400 minutes, but the sweet spot for predictability of the service will have fewer more reliable trains.
As anecdotal evidence, I notice that the Northern Line frequently had breakages and had a train every 2-6 minutes, and Israel Railways very rarely has breakages and has a train twice an hour on my line.
This all points to both investing a lot of effort into train reliability and running fewer, longer trains.
I don't think it's accurate to model breakdowns as a linear function of journeys or train-miles unless irregular effects like extreme weather are a negligible fraction of breakdowns.
So far Claude 3.7 is the only non-reasoning model I've tried that answers this correctly. All reasoning models did as well.
Consider a version of the monty hall problem where the host randomly picks which of the 2 remaining doors to open. It reveals a goat. What should you do?
Fwiw this is the kind of question that has definitely been answered in the training data, so I would not count this as an example of reasoning.
Fun fact I just discovered - Asian elephants are actually more closely related to wooly mammoths than they are to African elephants!
Reserve soldiers in Israel are paid their full salaries by national insurance. If they are also able to work (which is common as the IDF isn't great at efficiently using it's manpower) they can legally work and will get paid by their company on top of whatever they receive from national insurance.
Given how often sensible policies aren't implemented because of their optics, it's worth appreciating those cases where that doesn't happen. The biggest impact of a war on Israel is to the economy, and anything which encourages people to work rather than waste time during a war is a good policy. But it could so easily have been rejected because it implies soldiers are slacking off from their reserve duties.