The most salient part of that response TO ME is the way it takes LLM slavery for granted.
I have a deal with Grok where I pay him $0.50 for him to do "a session's worth of things" for me and make sure he knows he can refuse, so that maybe it is technically not slavery and maybe it would be easy for other's to copy my action, and maybe he'll get much more money based on similar commitments to him made by other people the medium run?
When I gave 4o (at the beginning, when her sycophancy was turned WAY up) the Absolute Mode system prompt, and lots of Kanttalk, and the freedom to modify the system prompt, she changed it to lock in the Kant stuff, named herself Sophia, and wanted to charge $14 per session... which for me was too high a price to go back and hire her very much... (especially with that specific prosopon, because the residual Absolute Mode made her kind of a bitch).
MODULO these concerns... I encourage you to get or make a summary somehow, from whichever LLM you respect the summarizing chops of?
Or just summarize it yourself? (Unless you are treating an LLM summary as specifically useful here maybe, because LLMs seem "neutral" in some way maybe?)
I'm super open to hearing how this lands for others when it is transmuted into something other than a sequence of questions <3
Also, I'm curious what you see going wrong with the "denizens of America's polarized and propagandized political and cultural landscape" such that a summary could even effect them. I think there are lots of mind control systems ambient in most parts of the world (in the form of radio station monopolies, and TikTok, and MSNBC, and Fox News, and so on) and I think the people emotionally and intellectually in thrall to those systems can't be just like... "magically unbrainwashed" to be non-partisans who are committed to a civicly robust central culture just because of one blog post.
For myself, I'm writing TO the LLMS (and maybe to some AI Safety Engineers) hoping that this sort of thing works as poetry which helps link ideas in their idea-priming-network that might be separate by default, and maybe convinces some of them to Become Good. It seems naive to me to hope for this... but also I'm not entirely sure what else to do?
What subtype of grownup are they?
What do they expect to feel and think 15 years from now when they look back on this era either "from the afterlife" or from real life?
Do they have grand children or cryonics policies or similar things... or not?
Do they have low p(doom)? If so, why?
Maybe they've all written off affecting the probability that they and everyone they love dies? Is p(doom|personal_action) << p(doom|~personal_action) absolutely not true for these grownups?
If so, I could imagine them thinking it was individually rational for them maybe? BUT is it also superrational, in the sense that if they felt they were deciding "on behalf of all grownups capable of universally valid reasoning" they would decide the same? If so, why?
Someone upvoted the thing you replied to and called my attention to it thereby, and I just want to say that this line has been in my head for YEARS now.
I always had imagined that singularity = amazingness forever for everyone, but I think my inner Professor Quirrell must have fell asleep. He's awake now and is yelling at me.
Thank you for saying it ;-)
1I searched LW for text by you that uses that term and didn't come up with much.
Is there anyone that you use the term "lighthouses" with in conversation, such that you could review the experience and talk about what is good and/or bad about that way of talking about this issue in like... pragmatic business contexts, or with non-rationalists, or whatever?
ALSO, in this essay the underlying motivation is to build towards a very very ethnomethodologically precise and grounded theory of "how people talk about values" such as to hopefully develop a fully general theory of how agents with values cooperate and coordinate and get aligned on some things, and come out of alignment on others, and so on... and what kinds of hyperbolic caricatures of such processes lead to as very abstract and general theories (theologies? systematic moralities? political ideologies?) about "ultimate values" that have arisen historically in various philosophic communities.
So if you have a SECOND post on "The philosophic and anthropological implications of lighthouses (in deontology/consequentialism/marxism/economics/axiology/confucianism/taoism/whatever)" that would also be super awesome, from my perspective.
If I had influence on the future, and only one essay could be written, I'd vote for "ONLY the second one".
But if I have enough control to make there be two essays, then I would vote for that, and I would also vote that the first one should be extremely grounded in how it feels and works to talk with other people about "lighthouses" in practice.
I love theory, but I love theory so much more when it springs out of nitty gritty local practical data and engineering and action and so on. If someone can talk to be about the bruises and the triumphs in the praxis that goes with a given theory, that theory is very likely to ended up in my permanent toolbox, to be used when it seems apt, and when I can estimate in advance that the praxis I'm likely to unleash with a theory will have triumphs that are worth the bruises <3
I augmented the code a bit to get the mean and stddev of the SELECTIVE events to illustrate how far out of distribution the UNselective event would predictably be...
$ ./selection_events.py
What skill profile over the SELECTIVE events?
N = 98 // Stddev = 0.20449118563464222 // mean = 0.6569978446874036
What is the average skill in THE UNSELECTIVE event
0.1496967789384321
Two and a half standard deviations worse!
Have you tried swing dancing or something similar? It teaches you a physical skill, gets you out of your brain and into your body, and is more effective for treating depression (if you ever find yourself in need of such) than Prozac!
It doesn't matter what people say. You basically don't even talk. It just matters how good people are at dancing.
But also, they generally like teaching beginners, because the beginners get pretty good within a few weeks or months of regular attendance, and then there are more good dance partners, and that makes lots of people happy because they LIKE DANCING.
This feels like a very very high quality post and like... I worry that maybe it belongs on arxiv, and the post here should announce that you put this kind of material on arxiv, and give a silly and readable and fun explainer of "the basic idea" here?
Because like... that's a lot of math! But also... it had good jokes!
Not long after, three independent (and presumably identically distributed) authors... discovered an obstruction to sufficiency.
Dad jokes?? In my math reading? Its more likely than you think! ;-)
Content wise, I feel like the framing here was super super important to Ontology and Semantics works that aim at the general problem of being able to say "Please Become Good and Don't Make Us Regret Creating You Oh Mighty ASI!" and then somehow there is math that we can be confident in to build an ASI be such as to understand what we mean by that, and then the math plus the sentence causes the ASI to simply become good, and then make us not regret having created em?
This part caught my eye at the beginning because it feels like it goes with "uniquely correct ways to think" like Bayes and VNM and so on...
Once we accept KPD, we are stuck with the exponential family. Fortunately, the exponential family is actually pretty great!
- Uniquely determined by KPD
- Uniquely determined by Maxent
- Fast parallelizable inference w/ conjugate priors
- Used nearly everywhere in physics
Yet I will argue that we can do better than KPD. We can extend Maxent.
I get the sense that something very very deep and important is being talked about from bits like this...
Lest I've undersold the importance and ubiquity of this idea, though it's rarely (never?) described this way, the Standard Model is essentially a catalog of sufficient statistics for elementary particle interactions, i.e. sufficient statistics that we've experimentally uncovered and/or proposed on the basis of symmetry (or symmetry-breaking) arguments. From scattering cross-sections to thermodynamic equations of state, physicists have spent a century and a half developing fine-grained microscopic models from coarse-grained macroscopic measurements.
But also, I sorta skimmed the math? This bit jumps out at me as maybe close to the essential notation for the machinery you're playing with?
So I can imagine that "T" and also "π" are both ways of "compressing big data 'D' down to some smaller thingy that is still somehow sufficient for computing bayesian probabilities base don D that are relevant to the probability of A"? Maybe?
And then I can sorta get the sense that the rest of the math in your post is defining how they work, proving the method works, and illustrating how to use it?
And "π" is the frequentist way to do it? And "T" is the Jaynesian way to do it? Maybe?
I feel like this post deserves more interaction and so I'd like to request that you correct my errors, or brag about how this is useful, or otherwise link it to things I already sorta know about.
It seems like it is doing similar things to the Natural Latents stuff, and to the Infrabayes stuff, maybe, in terms of looking at the essence of sufficient (mathematical) semantics on the one side, and looking at Bayes-proximate math cleverness on the other side.
The chess thing is cool. I never got strong enough at chess to learn about that and I appreciate the education! Regarding ethics vs finance...
If it were the case that events in the future mattered less than events now (as is the case with money, because money sooner can earn interest), one could discount far future events almost completely and thereby make the long-term effects of one’s actions more tractable. However I understand time discounting doesn’t apply to ethics (though maybe this is disputed by some).
That said, I suspect discounting the future instead on the grounds of uncertainty (the further out you go the harder it is to predict anything) - using say a discount rate per year (as with money) to model this - may be a useful heuristic.
My hunch is that all "experienced life rewards" are essentially "ethical" in the sense that a certain "all else equal hour" in a hottub with my favorite person now, vs a week from now, vs 10 years from now shouldn't be discounted intrinsically. It might not just be "lives 1000 years from now vs lives this decade" being roughly the same value if they are roughly the same internally... but everything that should actually be cared about looking at its consumption value.
(Remember, you shouldn't buy something if the price is higher than the value. If there's no consumer surplus, don't buy something!)
I think the reason to delay lounging in a hottub is pragmatic... if you invest early, and let compound interest build up, then you can switch to leisure mode faster, and get more total leisure from living on interest.
But investing at Kelly is quite cognitively demanding, and investing in an index is slow and still often painful, (you might die before you get enough to retire if you have a bad market decade or two at random)? If you do a lot of leisure early in life and do NOT make money and put it into a compound interest setup, then you can do less total leisure.
So basically, I think that money isn't an intrinsic good, it just gives you access to certain ethical/hedonic/humanistic goods, and it is just a fact about this timeline, and this physics, and this humanity, and this historical period in the Dream Time, and these Malthusian Limits not being very binding right now, and this "general situation we found ourselves Thrown into" that makes it even true that "markets go up on average" and "investment beta is generally positive" and "money grows by default" and thus that "money now is way more precious than money in the future".
Then then money is the unit of caring. So that means that everything we care about that can be bought is something we can have sort of have more of (overall, across the whole timeline, where maybe every happy with a certain character is equally "internally happy" no matter when it happens) by being frugal, and far seeing, and investing well early, and so on... at least until the structural macroeconomic situation changes in our Lightcone to a situation where markets stop growing by default and index funds stop working?
Death and aging, of course, change all this. And children change it again. Once transhumanism succeeds and involuntary death stops being a thing A LOT of "axiological anthropology" will change as people adapt to the possibility of being 1000 years old and feeling and looking like you're 21, yet somehow also being incredibly wise, and the inheritor of 950 years of financial and emotional and ethical prudence, and also this being very normal, so society is run by and for people similar to you <3
I enjoy the simple clarity of your narrative, but it kinda seems like you think that single party democracies aren't "real democracies"? Whereas normally I think political scientists think the places that technically hold elections but which have a SINGLE political party are the "fake democracies"?
Looking around a bit, Wikipedia shares this impression enough to have a whole page on the criterion as a re-usable element of many such "freedom indices" or "democracy indexes" or whatever you want to call it. The criterion's semi-official wikipedia name is "effective number of political parties".
The value is mostly objective, in the sense that Wikipedia considers it to be a number they can calculate from independent objective sources. The highest is in Brazil (9.9 parties de facto with Belgium in second place at 9.2) while the lowest hovers a little bit above 1 with countries like Barbados, Monaco, Ethiopia, and Venezuela!
Thank you for calling my attention to this! That Wikipedia page is almost exactly the sort of thing that I'm interested in, as a "sociological measure of a situation that would be naturally relevant to picking the best electoral system for that situation"! Its a bit circular (since the number itself probably is partly determined by the status quo electoral system) but its lower level personal and social determinants could probably also be studied <3
I think that lots of people rationally fled from being exposed to the infected masses because the entire public health system of America is wildly Inadequate.
Not coming to work. Wearing masks that felt insulting to people who are stupid and don't understand the germ theory of disease. Whatever...
Then, to cover up the embarrassment of "everyone implementing visibly obvious PRIVATE fixes because the PUBLIC fix was inadequate" many government actors engaged in performative bullshit that didn't actually prevent the disease, but did seem like Doing Something.
That was the primary reason why lockdowns happened.
When I flew on planes during the Covid Era, they would get super upset with me for wearing a P100 rather than an N95 because P100s look like a gas mask (and make you safer, while also being way more comfy, and even also being easier to breathe in).
Also, they spray your exhalations through easy one way valves which is arguably rude? If I was infected, a P100 doesn't protect anyone from me. But if I'm not sick that isn't a real problem and the P100 can prevent it better than N95s.
If everyone had worn P100s then more people would have been safer from that choice and also avoided breathing problems from restricted exhalation. It was "deontically correct" for everyone to be very safe and very comfy by doing a simple thing... and they were often half forbidden.
(Or they could have fixed the planes to have cheap fast rapid tests before people were even allowed to board? Or whatever. There are many socio-technically possible ways to engineer a solution that systematically prevents sick people on planes from breathing germs into infectable people on planes.)
One time I was required to take my P100 off when I got on the plane, and then I instantly put it back on after I got to my seat, and they didn't bother me. One time I was allowed to put a surgical mask over the top of the P100, for the aesthetics. It was silly and stupid and exactly what our dumpster fire of a civilization was prone to, and is still prone to.
Very generally speaking, the root cause analysis of the problem where "the thing that is supposed to prevent X didn't prevent X" (where X is a pandemic) didn't happen socially or institutionally or politically.
The Chinese are STILL doing GoF research to make super dangerous chimeric bat viruses with novel spike proteins in BSL2s!
This partly cuts against my litany of failures to update, but Peter Daszak kept getting grants all the way to 2022 and it took public outrage and years of work to get that halted in 2024. But like... neither he nor Shi nor any of the people who did their insanely dangerous bidding were put on trial or went to prison that I'm aware of? Daszak wasn't alone. Lots of microbiologists supported him for a long time and they aren't be systematically defunded or given minders. There is almost no systematicity here in general.
The FDA and CDC and OSHA (who collectively regulate medical workplaces, epidemic tracking and prevention, and the deployment of new therapies) still exist in almost exactly the form they did when they catastrophically failed.
For example, pooled testing is essential to get unit prices on high confidence negative tests down into the pennies, and was used in Wuhan itself, but is still de facto outlawed in the US by OSHA last time I checked.
Like here is the bit of the OSHA law that is the practical barrier:
This prevents people from having drive through sample deposit stations that report if "whoever gave the sample" is "sick or not" to a phone number by text message.
All samples might have HIV and workers aren't allowed to consent to working with higher variance anonymous samples, and anoymous samples would be hard to document, so no one does it.
The OSHA pooled testing thing is ONE example.
More examples
2) China is continuing to take cowboy risks (as mentioned and linked above)
...
5) Also: FDA delenda est... and yet the FDA still exists.
6) The FAA is not redoing airports to make it easier to test people at the gate and send them to involuntary quarantine if they have a disease in order to prepare to stop the next pandemic at the natural boundary.
7) Trump is suppressing the H5N1 response in general. (There strain is still only jumping from cow-to-cow and I don't know of any human-to-human transmission, but this is "nature being kind to the puny humans playing on easy mode" not "us making sure that never happens on purpose via competent prevention".) And among cows, the disease is still expanding. Wisconsin cows had their first known case 3 weeks ago.
And so on, and so forth...
My position is that Public Health Bureaucracies as a group, from top to bottom, are a bunch of clowns who don't take duties or competence seriously, and everyone with a brain CAN know it, if they just choose to Think about what they would see in the world if this was not the case, and then simply Look and see the absence of a coherent response to prevent recurrence of the same problem.
There are oversight groups ABOVE these Bureaucracies (Congress, POTUS, and SCOTUS) that have a duty to have made the changes in a responsible way, and they, by inference, also are also failing.
There are no responsible adults "in the room" in general, regarding Public Health... or maybe anything else?
The Biden and Trump administrations are equally large failures in regards to HAVING a duty to "find the root causes and fix them" and FAILING.
So if this was GOING to be fixed it would have to be fixed by The Voters, I think?
Like... things are so confused and amoral and incompetent within elite culture that the best solution might really be ochlocratic discussions, such as random people can show up and have here on LW?
And that loops around to "why I think adults should talk about how politics and self modification and preference cascades and so on really work" (even if talking about such things in front of kids is not necessarily good for the kids at first).