Something I've done in the past is to send text that I intended to be translated through machine translation, and then back, with low latency, and gain confidence in the semantic stability of the process.

Rewrite english, click, click.
Rewrite english, click, click. 
Rewrite english... click, click... oh! Now it round trips with high fidelity. Excellent. Ship that!

🕯️

My mom died last December, and part of the grief is in how hard it is to say (to people who loved her, and miss her, like I do, but don't have the same awareness of history) what you've said here about your mom, and timelines, and how much potentially fantastic future our mothers missed out on. Thank you for putting some of that part of "that lonely part of the grief" into words.

Calling it a "sick burn" was itself a bit of playfulness. Every time I re-read this I am sorry again to hear that we lost Golumbia 🕯️

The thing I think is true about Minecraft is that it enables true play, more along the lines of Calvinball where the only stable rule is that you can't have any other rules be the same as before.

This is a good essay on what children's cultures have lost, and I think that Minecraft is one of the few places where children can autopoetically reconstruct such such culture(s).

Minecraft is missing a strongly defined narrative where a JRPG has, to a much greater extent, a central narrative that binds the player into a story

This is precisely the value of Minecraft I think, and why it is a cultural phenomenon. You can choose your own mods, you can make your own mods using open source tools, you can invent any story. Such, I suspect, is how real "play play" (with other people who will quit if it isn't fun) mostly works, and is related to why reading a novel isn't as fun as writing a novel with your friends.

I love that you brought up bleggs and rubes, but I wish that that essay had a more canonical exegesis that spelled out more of what was happening.

(For example: the use of "furred" and "egg-shaped" as features is really interesting, especially when admixed with mechanical properties that make them seem "not alive" like their palladium content.)

Cognitive essentialism is a reasoning tactic where an invisible immutable essence is attributed to a thing to explain many of its features.

We can predict that if you paint a cat like a skunk (with a white stripe down its back) that will not cause the cat to start smelling like a skunk, because the "skunk essence" is modeled as immutable, and modeled as causing "white stripe" and "smell" unidirectionally.

Young children have a stage where they start getting questions like "If a rabbit is raised by monkeys will the rabbit prefer bananas or carrots?" and they answer "correctly" (in conformance to the tactic) with "carrots" but they over apply the tactic (which reveals the signature of the tactic itself) in some cases like "If a chinese baby is raised by german parents who only speak german, will the chinese baby grow up to speak german or chinese?"

If you catch them at the right age, kids will predict the baby grows up to speak chinese!

That is "cognitive essentialism" being misapplied because they have learned one of the needed tactics for understanding literally everything, but haven't learned some of the exceptions yet <3

(There are suggestions here that shibboleths and accents and ideologies and languages and so on are semi-instinctively used by humans for tracking "social/tribal essences" at a quick/intuitive level, which is a whole other kettle of fish... and part of where lots of controversy comes from. Worth flagging, but I don't want to go down that particular rabbit hole here.)

A key point here is that there is a deep structural "reasoning behind the reasoning" which is: genomes.

Genomes do, in fact, cause a huge variety of phenotypic features. They are, in fact, broadly shared among instances of animals from similar clades. They are, in practice, basically immutable in a given instance of a given animal category without unusual technology (biotech or nanotech, basically).

To return the cat and skunk example, we can imagine a "cognitive essentialist Pearlian causal graph" and note that "white stripe" does NOT causally propagate back into the "genome" node, such that DO("white stripe"=True) could change the probability in the "genome" (and then have cascading implications for the probability of "skunk smell").

More than that, genomes use signaling molecules which in the presence of shared genomic software have somewhat coherent semantic signals such as to justify a kind of "sympathetically magical thinking".

For example, a shaman might notice that willow trees fall over when a river overflows its banks during a flood, and easily throws new roots out of their trunk and continue growing in the new configuration and think of willow trees as "unusually rooty".

Then the shaman, applying the magical sympathetic thinking law of "like produces like", the shaman might make a brew out of willows hoping to condense this "rooty essence". Then they might put some other plant's cutting, without roots, in the "rooty willow water" and hope the cutting grows roots faster.

And this works!

Here is one of many youtube videos on DIY willow-based rooting mix, and modern shamans (called "scientists") eventually isolated the relevant "signaling molecule" (ie the material basis of its magico-sympathetic essential meaning within plant biology on Earth) which gains the imperative meaning "turn on root growing subroutines in the genomic software" in the presence of the right interpretive apparatus, in the form of indole-3-butyric acid.

Note that it is quite common for specifically hormones to have about this size and shape and ring pattern. They are usually vaguely similar to cholesterol (and they are often made by modification of cholesterol itself) and the "smallness" and "fattiness" helps the molecules diffuse even through nuclear membranes, and then the "long skinniness" is helpful for reaching into a double helix and tickling the DNA itself.

Here is a precursor of many animal steroids, (sometimes called lanostane) with locations that can be modified to change its meaning helpfully labeled:

I claim that the first chemical (the one that only as a C and D ring, with the standard nitrogen at 15, and a trimmed 21, a ketone 24, and a hydroxyl 25, that willows have a lot of) is a ROOTING HORMONE that "means" something related to "roots".

Compare and contrast "morphology" (the study of parts of words) and note also that Hockett's "design features" that offer criteria for human language that are mostly missing in animal communication include arbitrariness (which hormones have), displacement (which hormones have), and so on.

I claim that indole-3-butyric acid is, roughly, an imperative verb in the language of "bio-signaling-plant-ese" whose meaning is roughly, this:

Image sauce.

Also, the meaning is preserved for other plant species that "share" the same "genomic culture" (shared culture being another of Hockett's "design features" in human languages)... in this case: the "meaning" of the relevant molecules that willow tends to be rich in, is "culturally" shared for mint!

Image sauce.

I will close by saying that I think that a mixture of math and biochemistry and rule-utilitarianism is likely to offer a pretty clean language for expressing a "deep and non-trivial formula with useful etymological resonances" for explaining exactly what reptiliomoprh, and mammalian, and primate, and human benevolence "is" (and how it should be approximated in morally good agents acting charitably in conformance with natural law).

For example, if there is a "chemical word" that means "grow roots please!" in plant biology, then this complex of four amino acids in specifically this order (which is recognized by various chemical receptors) is also a word for something like "care for that which is close to you and person shaped and can't care for itself, please!":

Image sauce.

Hello anonymous account that joined 2 months ago and might be a bot! I will respond to you extensively and in good faith! <3

Yes, I agree with your summary of my focus... Indeed, I think "focusing on the people and their culture" is consistent with a liberal society, freedom of conscience, etc, which are part of the American cultural package that restrains Trump, whose even-most-loyal minions have a "liberal judeo-christian constitutional cultural package" installed in their emotional settings based on generations of familial cultures living in a free society with rule of law.

By contrast, "focusing on the leadership" is in fact consistent when reasoning about China, which has only ever had "something like a Liberal Rights-Respecting Democratic Republic" for a brief period from 1912 to 1949 and is currently being oppressed by an unelected totalitarian regime.

I'm not saying that Chinese people are spiritually or genetically incapable of caring about fairness and predictable leadership and freedom and wanting to engage in responsible self-rule and so on (Taiwan, for example has many ethnically Chinese people, who speak a Chinese dialect, and had ancestors from China, and who hold elections, and have rule of law, and, indeed, from a distance, seems better run that America).

But for the last ~76 years, mainland China has raised human people whose cultural and institutional and moral vibe has been "power does as power wills and I should submit to that power".

And for the thousands of years before 1912 it was one Emperor after another, with brief periods of violence, where winning the violent struggle explicitly conferred legitimacy. There was no debate. There was no justice. There was only murdering one's political enemies better and faster than one could be murdered in pre-emptive response, and then long periods of feudal authoritarian rule by the best murderer's gang of murderers being submitted to by cowardly peasants. That's what the feudal system was everywhere there was feudalism. Rule by murderer... normalized into a cultural field that tolerates enormous hierarchical disparities in formal power.

In the meantime, tactically and practically, I believe that Chinese companies don't functionally exist unless they have political officers who are embedded in their management that report via a chain of command up to Xi.

I think these political officers are listened to very closely because if the nominal owners do NOT listen to the top-down advice of their political officers, the officer can call in secret police to kidnap and torture the nominal owners of the company (and this practice is considered "legitimate" rather than a sign that Xi should be impeached and convicted of one of the various crimes he has surely committed since almost everyone has committed some crimes under some plausible framing...

...(if the impeachment and conviction was going to be "technically lawful" (impeachment and conviction being a matter of politics in practice, according to the highest laws of the US, since it is decided by a vote of the Senate (but those same laws require the forms to be observed in the language of justice and a trial)))).

By contrast, I don't believe that (1) the Trump regime is capable of installing any such meta-hierarchy of political officers in most US companies and (2) the US military under his command (who admittedly will mostly take his orders) will not be at the forefront of AGI R&D.

So in the US I expect the culture of researchers, and engineers, and their managers in free private industry to dominate much of what occurs (and they will roll their eyes about Trump, and wish he was less of a demented old man, and wish that his handlers told him "no" more often so his destruction of America would slow down), whereas in China I expect Xi can and will veto anything Xi wants to veto, and fund what he wants to fund, and the culture will tend to presubmit to his imagined tyrannical will automatically.

Also... speaking here to the wider audience of "people not in either country"... they might notice that China is almost entirely composed of Han Chinese. The one other significant ethnicity is currently in gulags. If the CCP is not ideologically racist, that would be a hopeful and surprising update to me, but it seems like they just straightforwardly are.

And if the CCP was going to put "kin altruism for their kin and only their kin (as part of their 'extrapolated national volition'?)" unto a powerful AGI or weak ASI a plausibly "calculatingly correct" step in the application of that utility function to a long term shape for Earth... is just killing everyone on the planet who isn't Han?

By contrast, if you, Dear Reader, live in Nigeria, or Chile, or Vietnam, or Samoa, or Sicily, or Sri Lanka, or Iran, or Madagascar, or Serbia, or Ireland, or Lebanon, or Nepal, or Japan, or almost anywhere on Earth, there are "people with similar heritage to you" in America, who have mostly positive ethnic feelings about people "back in the old country" (that they feel mildly guilty about being too proud of in public with too much enthusiasm, because they don't want to seem racist).

Sundar and Satya are the CEOs of Google and Microsoft and both were born in India. The "other kind of Indians" (granting that they were subject to de facto genocide between 1500 and 1890 (and that was very sad)) have "respect for their treaties" re-affirmed by more-or-less the current SCOTUS.

Like: from what I can tell, America is uniquely meta-racial and across-centuries-aspirationally-lawful in its cultural morality?

I understand why many random normies who speak English might prefer China to win an AI race (if a race isn't just a race to building a thing that loves nothing, and calculates everything, and will murder all humans once it can safely do so)...

...a huge reason many normies have positive vibes towards the CCP is that their brain is being programmed by TikTok to have vague confused positive emotions in that direction.

I think TikTok's operations for the souls of America are bad and sad. They make them anti-Semitic (for now), pro-Trump (for now), and pro-CCP (in general).

In my opinion, TikTok should be forced to sell to US owners, and all of its upper management should live outside of the possibility of being put in a gulag by Xi's minions.

They and their extended families who could be used as hostages should be offered citizenship and move to Hawaii or Seattle or whatever. Or let go. Or something that separates "algorithmic influence over US youth culture" from "the CCP's plans" with a good clean boundary. If that doesn't happen, then TikTok should just be shut down.

TikTok should be purchased or shut down BEFORE the 2026 elections.

Also BEFORE the 2028 elections.

It probably won't be, because the US government is full of corrupt idiots. But it should be.

In my opinion, the Bill of Rights works for free humans as an overall package.

We can assemble as we like. We can contract as we like. We can speak as we like. We can hire (or not hire) Republicans or Democrats or whoever, as we like. We can form new partisan groups if we like (since neither the Whigs nor the Federalists are still around, and the Democrats and Republicans morph into totally new ways to slice up the political ideology space every ~25 years (trading sides on which one is rightwing and which is leftwing while redefining what left/right even mean as they go)).

((The Republican are lately the party of chaotic uneducated poor people lately, and so I think they are the leftists... maybe? Its hard to say. The re-arrangement is in progress still.))

The freedoms that American's enjoy extend all the way into the package of freedoms needed to organize into militias and throw off the yoke of an oppressive government.

This gives people in the US enormous powers of mutual self regulation via sublawful mechanisms, which is great according to the philosophy of civic communitarianism (which is a great philosophy).

If I say something truly horrific, people might shun me.

I won't be put in jail for saying it... but I might be invited to fewer birthday parties and picnics. Which is important... if I leave near them. But I don't live near anyone who runs TikTok.

Suppose you have a free and sane and healthy people with freedoms that are pro-actively secured and expanded by a liberal democratic rights-respecting conscience-respecting consent-based institution with large governance reach.

If the reach of this institution isn't total, then "outside factors" that are inimical to freedom and liberalism-broadly-construed should not be granted the specific and narrow right of "the parts of the freedom package called 'free speech' that could help an algorithmically empowered tyrant have the ability to destroy the community's internal sense making and civic philosophic ideals and mutual care for each other without the operators of that 'algorithmic civic destruction program' having to face the consequences of having done that in a way that also exposes the attacker to the economic and social sanctions of the free people whose culture they are destroying from the outside".

So I think TikTok doesn't deserve free speech rights.

(Relatedly, I think that Citizens was decided wrongly by the SCOTUS in 2010. I don't even think US corporations deserve "free speech rights". Just humans in the US with an amygdala and fear and hopes and dreams and family and so on. This is one of many many things that I think US jurisprudence systems have gotten wrong over the last 200 years.)

To expand and illustrate how the package of freedoms connects back to tech companies...

Larry and Sergey can make YouTube destroy the civic cohesion of America if they want (because most people can't resist trolling and trolling can destroy a community and trolls can be amplified any time YouTube chooses to amplify such stuff) but they grew up here, and so on. They just don't wanna do that.

Also, they won't be murdered by Trump if they say no to him proposing certain algorithmic tweaks that have a deep and sociologically coherent relation to abstract preferences for destroying America as a democratic polity full of health sane free people who believe in classical liberal respect for conscience and so on.

I basically trust Larry and Sergey, and think their conscience can protect America even despite America's own elected POTUS appearing to be trying to destroy America as fast as possible.

By contrast, with TikTok, Liang Rubo (founder, president of the board, and CEO) and/or Zhang Fuping (chief editor and a party secretary??) have wildly different incentives and institutional linkages. Also, they don't have Wikipedia pages. Also, they have incentives to hide that they are truly in power if they are truly in power. Maybe Shou Zi Chew is the real power (but even then, he is not a US citizen and neither are many of his employees)? Maybe the real CCP political officer has been swapped with a new one without any announcements that is easily discernible from casual OSINT data? Ultimately, it all goes up to Xi, right?

Allowing TikTok to algorithmically reprogram the emotions and media expectations and world model of America's youth is insane. It is part of how we got Trump in the first place.

And the CCP's way of doing politico-economic business is legibly UNfriendly to people who love freedom, appreciate transparency, respect consent ethics, smile and clap when clean elections occur, dislike racist gulags, and so on.

Feel free to correct me on the freedom loving nature of Chinese people, if you think they hate freedom. (I just think they haven't tasted it except for very very briefly in the early 1900s, and so their language models will not have love of freedom baked into the latent semantic vector spaces.)

I sorta presume that the people of China do in fact yearn to hold open free and fair elections to select a wise and popular leader, like those that occur in Taiwan, and which occurred for their ancestors between 1912 and 1949.

I think the people of China are oppressed, and I wish them prosperity and freedom and happy futures. Maybe a benevolent ASI will be able to find some weird set of deals and adjustments to get that for them without too much tragedy... if they want it?

But ultimately I have no way of knowing, since right now the people of China are oppressed by the CCP, and would be put in jail for saying what kind of governance they actually positively would prefer, and there are no trivially trustable polls (though it isn't absolute), and I suspect that anyone who tried to run a real poll independently of CCP oversight would probably be put in jail?

If you have insight into the "real political culture" of actual human individuals in China (who I think will currently be thrown in jail if they protest in favor of being allowed to protest (like the Hong Kong protesters were)) then I'm open to being educated :-)

I think there's a deep question here as to whether Trump is "America's true self finally being revealed" or just the insane but half predictable accident of a known-retarded "first past the post" voting system and an aging electorate that isn't super great at tracking reality.

I tend to think that Trump is aberrant relative to two important standards:

(1) No one like Trump would win an election with Ranked Ballots that were properly counted either via the Schulze method (which I tend to like) or the Borda method (which might have virtues I don't understand (yet! (growth mindset))). Someone that the vast majority of America thinks is reasonable and decent and wise would be selected by either method.

I grant that if you're looking at America from the outside as a black box, we're unlikely to change our voting method to something that isn't insanely broken any time soon, and so you could hold it against the overall polity that we are dangerously bad at selecting leaders... and unlikely to fix this fast enough to matter... but in terms of the basic decency of our median voter I think that Trump isn't strong evidence that we are morally degenerate sociopaths.

In fact, Americans tend to smile a lot, and donate to charity, and are generally quite reasonable, and don't want an empire, and quite like the idea of being a fair, tolerant, prosperous, just, non-racist shining city on a hill.

Americans created Wikipedia, give it away for free, and it runs on donations. If that impulse runs from the people directly into the AGI, then that's better rather than worse. (Assuming alignment is even real. If it isn't possible/easy/whatever then it doesn't matter which country builds "the alien monster that will inevitably kill us all without remorse given that it is very powerful and doesn't love us and doesn't even understand the concept of love".)

The CCP blocks access to Wikipedia by default: you have to use a VPN, which is illegal, but also >30% of the population uses these illegal VPNs, and also some VPNs are tolerated if they install backdoors for the CCP to spy on them. Fuck that noise.

(2) The broad material intellectual history of Rights Respecting Democractic Republicanism is real, and being shat upon by Trump, but it still exists to drop into an LLM and give positive reinforcement for feeling good about that stuff and endorsing it.

America and Americans have often failed to live up to the ideals, but we also articulated those ideals, and also articulated the idea of "approximating them more and more successfully in real life over the course of history".

The White House was built by slaves, and then eventually slavery was outlawed, and black cultural integration proceeded decade by decade, in fits and starts, and eventually a descendant to slaves (though (to be clear) Michelle and Barry also had ancestors who owned slaves) moved in as the President and the First Lady. And everyone who was willing to talk about it in public was proud of this. Because the formal written ideals of our country are about god-given inalienable rights, including the right of everyone to own property and pursue happiness. The government can take your shit... but they have to do it via eminent  domain and pay you fair value for it. (I grant that, if Obama is part of the evidence about America then so is Trump. Both, in my opinion, are in some deep senses "accidents of a terribly designed voting system" but I think Obama was a happy accident, and people clapped and wrote happy things about progress and fairness and justice afterwards. That writing is part of what goes into an American LLM, by default.)

By contrast, the CCP runs Uyghur Gulags right now and basically doesn't even apologize. They want to conquer Tibet, and Taiwan, and are proud of it. They violated the treaty with the UK, whereby the UK gave up Hong Kong fair and square (according to the letter of a treaty signed long ago) after the CCP promised to grant them rights to vote for their own city government in the way they were used to under UK guidance... 

...and then there were brutal crackdowns and something like 10k people were thrown in secret prisons for trying to insist on those rights. At least Hitler was elected. The CCP weren't even elected. They seized power irregularly, through violence, authorized by the slogan political power comes our of the barrel of a gun. They still formally oppose the concept of elections. The entire idea of "consent ethics" is foreign to the logic of their system.

And if the intelligence of the governing class is of any relevance to the likelihood of a positive outcome, um, CCP seems to have USG beat hands down.

Intelligence is only a positive sign when the agent that is intelligent cares about you.

If you are certain that they would murder you and take your shit if they could get away with that somehow, then intelligence is a worrying sign, because it gives them a better chance to realize their preference of murdering you and taking your shit.

Personally, I'm in favor of establishing a world government, with a proportionally representative parliament that elects a Condorcet prime minister.

From my perspective, unboxed ASI might very well be like first contact with aliens (from platospace rather than outerspace), and the outerspace aliens generally say "take me to your leaders" when they meet humans in stories, and... currently Earth has no such people to take them to! It'd be nice to fix this error in my opinion.

But the CCP will never endorse this, whereas quite a few Americans will notice that this is consistent with our founding ideals, that many of us still cherish, and be on board with offering such influence to the people of Earth in a fair and reasonable way.

I think most of that is actually a weirdness in our orthography. To linguists, languages are, fundamentally a thing that happens in the mouth and not on the page. In the mouth, the hardest thing is basically rhoticism... the "tongue curling back" thing often rendered with "r". The Irish, Scottish, and American accents retain this weirdness, but a classic Boston, NYC, or southern British accents tends to drop it.

The Oxford English Dictionary gives two IPA transcriptions for "four": the American /fɔr/ makes sense to me and has an "r" in it, but the British is /fɔː/ has just totally given up on curling the tongue or trying to pretend in the dictionary that this is happening in human mouths.

That tongue curl is quite hard. Quite a few five year olds in rural Idaho (and maybe regions where rhotic dialects are maintained) often struggle with it, and are corrected by teachers and parents (and maybe made fun of by peers) for not speaking properly... for spontaneously adopting "a New York Accent" due a very common a childhood "speech impediment". Many ESL speakers drop it, hence the city dialects dropping it, not just in practice in the mouth, but officially.

("J" is a runner up for weirdness in the mouth, but I think that's just because the voiced postaveolar affricate /dʒ/ is a pretty rare phoneme.)

English orthography is kind of a disaster, I agree. It attempts to shoehorn a german/celtic/french/norse pidgin-or-creole into the latin letter system, and ... yeah. Tough task. It was never going to be clean.

If I was going to offer a defense of the status quo here, I'd say that there is no flat/simple orthography to switch to.

Every accent would need its own separate "spelling reform" and their texts would be less mutually intelligible, and it would hurt science and the letters quite a lot, and also probably lead to faster drift into a world where "English" denotes a language family rather than a language.

Interestingly, Interslavic is an attempt to "design by hand" a similar thing for slavic speakers to what English still has bascially for free: common words with stable spellings and meanings, and huge tolerance for how they are pronounced. Once you see the overarching vision for "a written language system" with these properties as a desirable end point... since English is already at that desirable end point, why change it? <3

From a pedagogical perspective, putting it into human terms is great for helping humans understand it.

A lot of stuff hinges on whether "robots can make robots".

A human intelligible way to slice this problem up to find a working solution goes:

"""Suppose you have humanoid robots that can work in a car mechanic's shop (to repair cars), or a machine shop (to make machine tools), and/or work in a factory (to assemble stuff) like humans can do... that gives you the basic template for a how "500 humanoid robots made via such processes could make 1 humanoid robot per unit of time".

If the 500 robots make more than 500 robots (plus the factory and machines and so on) before any of the maker's bodies wear out, then that set of arrangements is "a viable body production system".

They would have, in a deep sense, cracked the "3D printers that make 3D printers" problem.

QED."""

Looking at this argument, the anthropomorphic step at the beginning helps invite many anthropoids into the engineering problems out the outset using job titles (like "car mechanic") and monkey social games (like "a firm running a factory") that anthropoids can naturally slot into their anthropoid brains.

That part is "good pedagogy for humans".

However, once the shape of the overall problem becomes clear to the student, one could point out that instead of 500 humanoid robots, maybe you could have 200 crab robots (for the heavy stuff) and 200 octopus robots (for the fiddly stuff) and it might be even be cheaper and faster because 200+200 < 500.

And there's no obvious point where the logic of "potentials for re-arrangement into more versions of less intelligible forms" breaks down, as you strip out the understandable concepts (like "a machine that can be a car mechanic") while keeping the basic overall shape of "a viable body production system".

Eventually you will have a very efficient, very confusing hodgepodge of something like "pure machinery" in a "purely mechanical self reproducing system" that is very efficient (because each tweak was in the direction of efficient self reproduction as an explicit desiderata).

...

If I'm looking at big picture surprises, to me... I think I've been surprised by how important human pedagogy turns out to be??? Like, thirty years before 2027 the abstract shape of "abstract shapes to come" was predictable from first principles (although it is arriving later than some might have hoped and others might have feared).

"Designing capacity designing design capacity" (which is even more abstract than "humanoid manufacturers manufacturing humanoid manufacturers") automatically implies a positive feedback loop unto something "explosive" happening (relative to earlier timescales).

Positive feedback is a one-liner in the math of systems dynamics. It can (and predictably will) be an abstract description of MANY futures.

What I didn't predict at all was that "institutions made of humans will need to teach all their human members about their part of the plan, and talks about the overall plan at a high level will occur between human managers, and so human pedagogy will sharply constrain the shape of the human plans that can be coordinated into existence".

Thus we have RL+LLM entities, which are basically Hanson's ems, but without eyes or long term memory or property rights or a historical provenance clearly attributable to scanning a specific human's specific brain.

But it RL+LLM entities are very intelligible! Because... maybe because "being intelligible" made it more possible to coordinate engineers and managers and investors around an "intelligibly shared vision" with roughly that shape?

This is such an abstract idea (ie the idea that  "pedagogical efficiency" predicts "managerial feasibility") that it is hard for me to backpropagate the abstract update into detailed models and then turn the crank on those models and then hope to predict the future in a hopefully better way.

...

Huh. OK. Maybe I just updated towards "someone should write a sequence that gets around to the mathematics of natural law and how it relates to political economy in a maximally pedagogically intelligible way"?

Now that I think this explicitly, I notice Project Lawful was kind of a motion in this direction (with Asmodeus, the tyrant god of slavery, being written as "the god of making agents corrigible to their owner" (and so on)) but the storytelling format was weird, and it had a whole BDSM/harem thing as a distraction, and the main character asks to be deleted from the simulation because he finds it painful to be the main character, and so on.

((Also, just as a side complaint: Asmodeus's central weakness is not understanding double marginalization and its implications for hierarchies full of selfish agents and I wish someone had exploited that weakness more explicitly in the text.))

But like... hypothetically you could have "the core pedagogical loop of Project Lawful" reframed into something shorter, with less kinky sex, and no protagonist who awakens to his own suffering and begs the author to let him stop being the viewpoint character?

...

I was not expecting to start at "the humanoid robots are OK to stick in the story to help more humans understand something they don't have the math to understand for real" and end up with "pedagogy rules everything around me... so better teaching about the math of natural law is urgent".

Interesting... and weird.

Reacting to the full blog post (but commenting here where the comments have more potential to ferment something based on attention)...

This reminds me of CFAR's murphyjitsu in the sense that both are (1) a useful guide for structuring one's "inner simulator" to imagine spefific things to end up with more goal-seeky followup actions (2) about integrating behavior over long periods of time that (2) can probably be done well in single player mode.

The standard trick from broader management culture would be a "pre-mortem" which is even further separated by... I think basically leaving people to not focus on the mental actions, and often deployed by managers for teams, to try to get them on the same page about possible future failure conditions that the team could potentially mitigate.

The big difference between telescoping and both of these is that telescoping is "more positive". It is asking about goals instead of fears :-)

One interesting thing is to ask whether there's any evidence anything like this actually helps in practice and/or to tease apart the mechanisms whereby it helps, if it does. Wikipedia cites a 1989 managerial/business/psychology study on "prospective hindsight" with this title and abstract...

Back to the future: Temporal perspective in the explanation of events.

Examined how shifts in perspective might influence people's perceptions of events by investigating 2 possible factors: temporal perspective (whether an event is set in the future or past) and uncertainty (whether the event's occurrence is certain or uncertain). In the 1st experiment, which involved 114 MBA students, temporal perspective showed little influence while outcome uncertainty strongly affected the nature of explanations for events. Explanations for sure events tended to be longer, to contain a higher proportion of episodic reasons, and to be expressed in past tense. Evidence from the 2nd experiment, which involved 76 MBA students and 32 nonstudent adults, supports the view that uncertainty mediates not the amount of time spent explaining, but rather Ss' choice of explanation type. Implications for the use of temporal perspective in decision aiding are discussed

Mechanistically, this suggests that "certainty" is a key parameter and so it might be interesting to "suppose that a year from now you were very sure that the year had been surprisingly good... why?" vs maybe something like "imagine that someone is hired to assess your life objectively a year from now (they are uncertain because they're new, and you must have hired them because you yourself are uncertain) and they ask about how it went... what will they be told and/or ask about?"

I agree that there are many bad humans. I agree that some of them are ideologically committed to destroying the capacity of our species to coordinate. I agree that most governance systems on Earth are embarrassingly worse than how bees instinctively vote on new hive locations.

I do not agree that we should be quiet about the need for a global institutional governance system that has fewer flaws.

By way of example: I don't think that "not talking very much about Gain-of-Function research deserving to be banned" didn't cause there to be no Gain-of-Function research in Wuhan, by collaborators of the people in the US who explicitly proposed building something like covid in a grant proposal a while before covid was actually built under BSL2 conditions, by their international scientific collaborators, and then escaped the lab.

There should have been more anti-GoF talk, and it should have been explicitly bipartisan, and so on. In Trump's first term, one of the crazy random things he "did or allowed" was to let the pro-GoF people at NIH quietly weaken the GoF ban that was instituted under Obama.

But also, similarly to how anti-GoF talk would be helpful up until there is an international treaty system that insists that GoF never happen outside of a "BSL5" (which currently doesn't even exist (currently the bio-safety levels only go up to 4)) I think there should be more anti-bad-governance-institution talk, and it should be explicitly bipartisan. There are many other larger fires now. And covid is no longer in the zeitgeist. Maybe this is not the best place to spend words. But it is a great test case for talking about general policies on regulation about dangerous technology, and institutions for handling such tech, and speech about the need for better institutions.

Not that I greatly expect such talk to help, whether for AGI or GoF or anything. Its just that I think that (1) in the (rare?) timelines where we live I will not be greatly embarrassed to have talked as much as I did, and (2) in the (common?) timelines where we die I will mostly regret (just before I die) my silences more than my speech.