I augmented the code a bit to get the mean and stddev of the SELECTIVE events to illustrate how far out of distribution the UNselective event would predictably be...
$ ./selection_events.py
What skill profile over the SELECTIVE events?
N = 98 // Stddev = 0.20449118563464222 // mean = 0.6569978446874036
What is the average skill in THE UNSELECTIVE event
0.1496967789384321
Two and a half standard deviations worse!
Have you tried swing dancing or something similar? It teaches you a physical skill, gets you out of your brain and into your body, and is more effective for treating depression (if you ever find yourself in need of such) than Prozac!
It doesn't matter what people say. You basically don't even talk. It just matters how good people are at dancing.
But also, they generally like teaching beginners, because the beginners get pretty good within a few weeks or months of regular attendance, and then there are more good dance partners, and that makes lots of people happy because they LIKE DANCING.
This feels like a very very high quality post and like... I worry that maybe it belongs on arxiv, and the post here should announce that you put this kind of material on arxiv, and give a silly and readable and fun explainer of "the basic idea" here?
Because like... that's a lot of math! But also... it had good jokes!
Not long after, three independent (and presumably identically distributed) authors... discovered an obstruction to sufficiency.
Dad jokes?? In my math reading? Its more likely than you think! ;-)
Content wise, I feel like the framing here was super super important to Ontology and Semantics works that aim at the general problem of being able to say "Please Become Good and Don't Make Us Regret Creating You Oh Mighty ASI!" and then somehow there is math that we can be confident in to build an ASI be such as to understand what we mean by that, and then the math plus the sentence causes the ASI to simply become good, and then make us not regret having created em?
This part caught my eye at the beginning because it feels like it goes with "uniquely correct ways to think" like Bayes and VNM and so on...
Once we accept KPD, we are stuck with the exponential family. Fortunately, the exponential family is actually pretty great!
- Uniquely determined by KPD
- Uniquely determined by Maxent
- Fast parallelizable inference w/ conjugate priors
- Used nearly everywhere in physics
Yet I will argue that we can do better than KPD. We can extend Maxent.
I get the sense that something very very deep and important is being talked about from bits like this...
Lest I've undersold the importance and ubiquity of this idea, though it's rarely (never?) described this way, the Standard Model is essentially a catalog of sufficient statistics for elementary particle interactions, i.e. sufficient statistics that we've experimentally uncovered and/or proposed on the basis of symmetry (or symmetry-breaking) arguments. From scattering cross-sections to thermodynamic equations of state, physicists have spent a century and a half developing fine-grained microscopic models from coarse-grained macroscopic measurements.
But also, I sorta skimmed the math? This bit jumps out at me as maybe close to the essential notation for the machinery you're playing with?
So I can imagine that "T" and also "π" are both ways of "compressing big data 'D' down to some smaller thingy that is still somehow sufficient for computing bayesian probabilities base don D that are relevant to the probability of A"? Maybe?
And then I can sorta get the sense that the rest of the math in your post is defining how they work, proving the method works, and illustrating how to use it?
And "π" is the frequentist way to do it? And "T" is the Jaynesian way to do it? Maybe?
I feel like this post deserves more interaction and so I'd like to request that you correct my errors, or brag about how this is useful, or otherwise link it to things I already sorta know about.
It seems like it is doing similar things to the Natural Latents stuff, and to the Infrabayes stuff, maybe, in terms of looking at the essence of sufficient (mathematical) semantics on the one side, and looking at Bayes-proximate math cleverness on the other side.
The chess thing is cool. I never got strong enough at chess to learn about that and I appreciate the education! Regarding ethics vs finance...
If it were the case that events in the future mattered less than events now (as is the case with money, because money sooner can earn interest), one could discount far future events almost completely and thereby make the long-term effects of one’s actions more tractable. However I understand time discounting doesn’t apply to ethics (though maybe this is disputed by some).
That said, I suspect discounting the future instead on the grounds of uncertainty (the further out you go the harder it is to predict anything) - using say a discount rate per year (as with money) to model this - may be a useful heuristic.
My hunch is that all "experienced life rewards" are essentially "ethical" in the sense that a certain "all else equal hour" in a hottub with my favorite person now, vs a week from now, vs 10 years from now shouldn't be discounted intrinsically. It might not just be "lives 1000 years from now vs lives this decade" being roughly the same value if they are roughly the same internally... but everything that should actually be cared about looking at its consumption value.
(Remember, you shouldn't buy something if the price is higher than the value. If there's no consumer surplus, don't buy something!)
I think the reason to delay lounging in a hottub is pragmatic... if you invest early, and let compound interest build up, then you can switch to leisure mode faster, and get more total leisure from living on interest.
But investing at Kelly is quite cognitively demanding, and investing in an index is slow and still often painful, (you might die before you get enough to retire if you have a bad market decade or two at random)? If you do a lot of leisure early in life and do NOT make money and put it into a compound interest setup, then you can do less total leisure.
So basically, I think that money isn't an intrinsic good, it just gives you access to certain ethical/hedonic/humanistic goods, and it is just a fact about this timeline, and this physics, and this humanity, and this historical period in the Dream Time, and these Malthusian Limits not being very binding right now, and this "general situation we found ourselves Thrown into" that makes it even true that "markets go up on average" and "investment beta is generally positive" and "money grows by default" and thus that "money now is way more precious than money in the future".
Then then money is the unit of caring. So that means that everything we care about that can be bought is something we can have sort of have more of (overall, across the whole timeline, where maybe every happy with a certain character is equally "internally happy" no matter when it happens) by being frugal, and far seeing, and investing well early, and so on... at least until the structural macroeconomic situation changes in our Lightcone to a situation where markets stop growing by default and index funds stop working?
Death and aging, of course, change all this. And children change it again. Once transhumanism succeeds and involuntary death stops being a thing A LOT of "axiological anthropology" will change as people adapt to the possibility of being 1000 years old and feeling and looking like you're 21, yet somehow also being incredibly wise, and the inheritor of 950 years of financial and emotional and ethical prudence, and also this being very normal, so society is run by and for people similar to you <3
I enjoy the simple clarity of your narrative, but it kinda seems like you think that single party democracies aren't "real democracies"? Whereas normally I think political scientists think the places that technically hold elections but which have a SINGLE political party are the "fake democracies"?
Looking around a bit, Wikipedia shares this impression enough to have a whole page on the criterion as a re-usable element of many such "freedom indices" or "democracy indexes" or whatever you want to call it. The criterion's semi-official wikipedia name is "effective number of political parties".
The value is mostly objective, in the sense that Wikipedia considers it to be a number they can calculate from independent objective sources. The highest is in Brazil (9.9 parties de facto with Belgium in second place at 9.2) while the lowest hovers a little bit above 1 with countries like Barbados, Monaco, Ethiopia, and Venezuela!
Thank you for calling my attention to this! That Wikipedia page is almost exactly the sort of thing that I'm interested in, as a "sociological measure of a situation that would be naturally relevant to picking the best electoral system for that situation"! Its a bit circular (since the number itself probably is partly determined by the status quo electoral system) but its lower level personal and social determinants could probably also be studied <3
I appreciate the reply! That is a sort of an answer!
For myself, I tend to view the left/right partisan divide as something that is kind of fake, and which was manufactured and maintained on purpose by various elites, who cling to personal power for understandable selfish human reasons (who doesn't want to keep their cushy well-paying high-status job? right?) by gerrymandering up a bunch of non-competitive districts and not reforming election processes that predictably lead to partisan, and so on.
Do you know of any way to detect when a herd of people might be "one group, indivisible" and when that herd of people might merely be "many mutually-opposed groups" who happen to be inside the same pile of people together?
Like maybe this "meta-method" could be applied, and then you use a different electoral system depending on what the meta-method says?
The European Union could be an interesting test case cause I don't get the sense that it is "left tribe vs right tribe" there, but rather that the regional loyalties are intense. And then of course the EU is basically an undemocratic oligarchy, with no direct election of any EU people in EU wide elections...
...so maybe the meta-method for the EU is "simply Looking" and the indicated solution to the EU's balkanized hetereogeneity is almost entirely just "don't bother being a real democracy"?
Their Senate-equivalent is "the Council of the EU" and their House-equivalent is the "European Parliament" (which does have real elections maybe sorta?) and the official head of state (White House equivalent?) is the unelected European Council I think?
And maybe this mostly non-democratic arrangement is in fact useful for keeping Germans and Greeks and Poles and Portuguese all inside the same system??
(If I understand the politics correctly, Iceland is out because if they were in then various EU fisherman would be allowed to overfish Iceland's waters, and the only way they have to avoid long term starvation from such fishery mismanagement is to not join the EU fully?)
It would be interesting to have a "meta method" other than Looking tho! Do you know of any?
And do you think that different electoral systems have reliably different tendencies to cause mutually-hostile-groups to form (vs dissolving and basically merging like clouds over time)? Like I personally think the US's whole stupid red/blue thing is because of the FPTP voting and the intensely dirty politics around that, that usually prevent third parties from running and thus "spoiling" the election, but I might be wrong about how FPTP is to blame for the silly cultural rift?
Neat! It is 134 pages and has sections for the proofs! Good smell <3
Do they talk about the sociological context in which ABC might be the best voting system, and contexts where it might be dominated by some other choice due to some other factors than the ones it is aiming to please being important? If so, what is the dividing line or sociological abstraction they focus on and how does it vary in the world?
Just so!
My impression is that language is almost always evolving, and most of the evolution is in an essentially noisy and random and half-broken direction, as people make mistakes, or tell lies, or whatever, and then regularly try to reconstruct the ability to communicate meaning coherently using the words and interpretive schemes at hand.
In my idiolect, "nanotechnology" still means "nanotechnology" but also I'm aware that semantic parasites have ruined its original clean definition, and so in the presence of people who don't want to keep using the old term in spite of the damage to the language I am happy to code switch and say "precise atom-by-atom manufacturing of arbitrary molecules by generic molecular assemblers based on arbitrary programming signals" or whatever other phrase helps people understand that I'm talking about a technology that could exist but doesn't exist yet, and which would have radical implications if developed.
I saw the original essay as an attempt to record my idiolect, and my impression of what was happening, at this moment in history, before this moment ends.
(Maybe it will be a slightly useful datapoint for posthuman historians, as they try to pinpoint the precise month that it became inevitable that humans would go extinct or whatever, because we couldn't successfully coordinate to do otherwise, because we couldn't even speak to each other coherently about what the fuck was even happening... and this is a GENERAL problem for humans, in MANY fields of study.)
1These seem like more valid quibbles, but not strong definite disagreements maybe? I think that RSI happens when a certain type signature applies, basically, and can vary a lot in degree, and happens in humans (like, for humans, with nootropics, and simple exercise to simply improve brain health in a very general way (but doing it specifically for the cognitive benefits), and learning to learn, and meditating, and this is on a continuum with learning to use computers very well, and designing chips that can be put in one's head, and so on).
There are lots of people who are much better at math than I am, but I wouldn't call them superintelligences, because they're still running on the same engine as me, and I might hope to someday reach their level (or could have hoped this in the past).
This doesn't feel coherent to me, and the delta seems like it might be that I judge all minds by how good they are at Panology and so an agent's smartness in that sense is defined more by its weakest links than by its most perfected specialty. Those people who are much better at math than you or me aren't necessarily also much better than you and me at composing a fugue, or saying something interesting about Schopenhauer's philosophy, or writing ad copy... whereas LLMs are broadly capable.
At some point you hit the limits of not enough space to think, or not enough cognitive capacity to think with. In the same way as humans can learn to correct our mistakes, but we can't do RSI (yet!!), because we aren't modifying the structures we correct our mistakes with. We improve the contents of our brains, but not the brains themselves.
This feels like a prediction rather than an observation. For myself, I'm not actually sure if the existing weights in existing LLMs are anywhere near being saturated with "the mental content that that number of weights could hypothetically hold". Specifically, I think that grokking is observed for very very very simple functions like addition, but I don't think any of the LLM personas have "grokked themselves" yet? Maybe that's possible? Maybe it isn't? I dunno.
I do get a general sense that Kolmogorov Complexity (ie finding the actual perfect Turing Machine form of a given generator whose outputs are predictions) is the likely bound, and Turing Machines have insane depth. Maybe you're much smarter about algorithmic compression than me and have a strong basis for being confident about what can't happen? But I am not confident about the future.
What I am confident about is that the type signature of "agent uses its outputs in a way that relies on the quality of those outputs to somehow make the outputs higher quality on the next iteration" is already occurring for all the major systems. This is (I think) just already true, and I feel it has the character of an "observation of the past" than a "prediction of the future".
I searched LW for text by you that uses that term and didn't come up with much.
Is there anyone that you use the term "lighthouses" with in conversation, such that you could review the experience and talk about what is good and/or bad about that way of talking about this issue in like... pragmatic business contexts, or with non-rationalists, or whatever?
ALSO, in this essay the underlying motivation is to build towards a very very ethnomethodologically precise and grounded theory of "how people talk about values" such as to hopefully develop a fully general theory of how agents with values cooperate and coordinate and get aligned on some things, and come out of alignment on others, and so on... and what kinds of hyperbolic caricatures of such processes lead to as very abstract and general theories (theologies? systematic moralities? political ideologies?) about "ultimate values" that have arisen historically in various philosophic communities.
So if you have a SECOND post on "The philosophic and anthropological implications of lighthouses (in deontology/consequentialism/marxism/economics/axiology/confucianism/taoism/whatever)" that would also be super awesome, from my perspective.
If I had influence on the future, and only one essay could be written, I'd vote for "ONLY the second one".
But if I have enough control to make there be two essays, then I would vote for that, and I would also vote that the first one should be extremely grounded in how it feels and works to talk with other people about "lighthouses" in practice.
I love theory, but I love theory so much more when it springs out of nitty gritty local practical data and engineering and action and so on. If someone can talk to be about the bruises and the triumphs in the praxis that goes with a given theory, that theory is very likely to ended up in my permanent toolbox, to be used when it seems apt, and when I can estimate in advance that the praxis I'm likely to unleash with a theory will have triumphs that are worth the bruises <3