why/how you liked it … detailed feedback
Were you maybe thinking of a different comment than the one I replied to? These don’t seem to be present.
>Strongly upvoted. Great post. […] would love to read more like it.
I think this is what the upvote button is for.
>I disagree
If you’re not going to offer details this seems like it would have been better as an agree/disagree reaction.
By EoY 2026 I don't expect this to be a solved problem, though I expect people to find workarounds that involve lowered standards: https://benjaminrosshoffman.com/llms-for-language-learning/
By EoY 2030 I don't expect LLMs to usually not mess up tasks like this one (scroll down a bit for the geometry fail), though any particular example that gets famous enough can get Goodharted even with minor perturbations via jerry-rigging enough non-LLM modules together. My subjective expectation is that they'll still frequently fail the "strictly a word problem" version of such problems that require simple geometric reasoning about an object with multiple parts that isn't a typical word-problem object.
I don't expect them to be able to generate Dead Sea Scroll forgeries with predominantly novel content specified by the user, that hold up to good textual criticism, unless the good textual critics are all retired, dead, or marginalized. I don't expect them to be able to write consistently in non-anachronistic idiomatic Elizabethan English, though possibly they'll be able to write in Middle English.
Not sure these are strictly the "easiest" but they're examples where I expect LLMs to underperform their vibe by a LOT, while still getting better at the things that they're actually good at.
When the problematic adjudicator isn't the dominant one, one can either safely ignore them, or escalate to someone less problematic who does hold power, so there's no benefit in sabotage, and there's reputational harm.
Relatedly I think the only real solution to the "lying with statistics" problem is the formation of epistemic communities where you're allowed to accuse someone of lying with statistics, it's adjudicated with a preponderance-of-evidence standard, and both false accusations and evidence that you're lying with statistics are actually discrediting, proportionate to the severity of the offense and the confidence of the judgment.
That last bit seems wrong to me bc the "good location" premium is so large, e.g. https://www.crackshackormansion.com/. Davis and Palumbo (2006) estimated land value as 50% of residential real estate value, up from 32% in 1984, and home prices in aggregate have continued to rise for the same reasons.
Your "cannon fodder" argument got me thinking; I don't exactly think the argument depends on a new sort of fully distinct intelligence emerging, but rather a change in how our existing superorganisms are constituted. Modern states emerged in part as a mass-mobilization technology, and were therefore biased towards democracy. But as we learn to automate more things, smaller groups of humans better at implementing automation can outcompete larger groups of people mobilized by ideologies or other modern methods. If this keeps going, maybe we'll end up like the Solarians in Asimov's The Naked Sun for a while, a low-fertility skeleton crew of highly territorial lonesome tech-yeomen. If the skeleton crew is sufficiently infertile, it may leave behind a rigid set of automations that eventually collapse for want of maintenance by a living mind, much like the house in Ray Bradbury's story There Will Come Soft Rains.
I think there's a moderately likely limit to LLMs and other applications of the present machine-learning paradigm. Humans are powerful general intelligences because we can, individually and collectively, make use of different cognitive modules in a way that converges on coherence, rather than splitting off into different and conflicting subagents. Our brains seem to have stopped growing not when individuals hit diminishing intelligence returns, but when we got smart enough to network Dunbar-sized bands into low-latency collective intelligences, and then shrunk a bit when the Dunbar bands figured out how to network themselves - as The Flenser does in Vinge's A Fire Upon the Deep - into larger, more differentiated, but higher-latency lower-bandwidth collective intelligences. While this obviously doesn't guarantee that human+ level AGI will be nice to all other such GIs (that's not true of humans either) it does suggest that if a superintelligence functions in the same modular-convergence ways humans do, it will tend to recognize similarly constituted coherent clusters that it can talk with as something analogous to near kin or other members (actual or potential) of its community, much like we do.
LLMs are a bit surprisingly useful, but they're nowhere near being as inventive and enterprising as an Einstein or Feynman or Moses or a hunter-gatherer band (the ancestral ones who were investigating new tech and invented horticulture and animal domestication, not the contemporary atavists selected for civilizational refusenikhood), though maybe within a few decades of being able to do most of what a Von Neumann can do, if their development works out well enough; we've discovered that a lot of the "knowledge work" we pretended took real thought can be done by ghosts if we throw enough compute at them. That's pretty cool, but it only looks "PhD level" because it turns out the marginal PhD doesn't require anything a ghost can't do.
Seems like public corporations make ownership decisions close to the finance-theoretical ideal where they minimize the assets they hold that aren't part of their production function to increase return on capital, and people who want to hold claims on rents buy them separately, consistent with the model I advanced in The Domestic Product.
"Land is a minority of capital" is reassuring that this is mostly a summary of accumulated productive tools rather than of rent claims on natural resources rendered valuable by the productive use others can make of them. But it's in some tension with Gianni La Cava's claim that the increase in capital's share of income is largely due to increases in home values.
Presumably the solution to this paradox is that land values are mostly privately held, while public corporations tend to hold other forms of 'real capital,' so that rentiers still largely hold real estate, as they did when the term was coined. It would be interesting to learn whether privately held corporations' holdings are more similar to those of public corporations or natural persons.
I think a large part of the mysterious seeming banter -> sex transition is antinormative attitudes towards sex. For some large portion of people, the mate-seeking drive is tangled up with a desire for covertness, for which there is culturally specific[1] support.
"Romance" and "romanticism" seem to be fundamentally the (ideally mutual) intent to mate transgressively, "you and me against the world." As I understand it, "romance" is specifically a modern Western[2] phenomenon explicitly opposed to formal statelike systems of accountability.
Trinley Goldenberg alludes to the function of banter:
But the important thing to understand is why people are seeking plausible deniability. Naturally the opposition to accountability is disinclined to give an honest account of itself, so people will tend to deflect from the central question onto tangential issues like the quality of banter, or vague pointers like "sexual tension." But if your sexuality isn't about being naughty and getting away with something, there's little point in mimicking the otherwise extremely inefficient plausible-deniability rituals (such as the ones described in the OP) needed to build inexplicit, covert mutual knowledge of attraction. Dancing works better for you because it is a virtue signal.
See also:
On commitments to anti-normativity
Preference Inversion
Guilt, Shame, and Depravity
There may or may not also be direct evolutionary support for this in the form of "sneaky fucker" strategies
The "romance" as a literary genre began with Orlando Furioso, though it comes out of somewhat older traditions of troubadors and courtly love.