I think that's a worthy ideal to strive for, and the bottleneck is simply bringing together enough different people doing intellectual work on the same topic. Then the niceties of academic freedom will happen mostly by themselves. But the premise is much harder than it seems.
LW approached that ideal for a short while, when Eliezer's writings created a diverse flow of people and the mention of Newcomb's problem channelled some of them into decision theory. It was a fun time and I'm happy to have been part of it. Then Eliezer stopped posting fun stuff for a wide audience, the flow of people started drying up, new ideas became scarce due to lack of outsiders, and the work became more intensely mathematical and shrank to a small core group (MIRI workshops and agentfoundations.org). Now it's mostly met with crickets, and the opportunity for outsiders to make LWish philosophical progress and be rewarded with attention is pretty much gone, even though there's plenty of low hanging fruit. I'm sorry to say I also contributed to this "professionalization", which might have been a mistake in retrospect.
A couple days ago, after two years of silence, I wrote a short LW post about probabilities to test the waters. It got a very good reception, showing that people are still interested in this stuff. But to jumpstart such an effort properly, we need to sell amateurs on some way to do important yet accessible intellectual work. I don't know how to do that.
Scott Garrabrant and I would be happy to see more engagement with the content on Agent Foundations (IAF). I guess you're right that the math is a barrier. My own recent experiment of linking to Two Major Obstacles for Logical Inductor Decision Theory on IAF was much less successful than your post about betting, but I think that there's something inessential about the inaccessiblity.
In that post, for example, I think the math used is mostly within reach for a technical lay audience, except that an understanding of logical induction is assumed, though I may have missed some complexity in looking it over just now. Even for that, it should be possible to explain enough about logical inductors briefly and accessibly enough to let someone understand a version of that post, though I'm not sure if that has been done. People recommend this talk as the best existing introduction.
Note that I played a part in convincing MIRI to create IAF, and wrote the only comment on the IAF post you linked, so rest assured that I'm watching you folks :-) My thinking has changed over time though, and probably diverged from yours. I'll lay it out here, hopefully it won't sound too harsh.
First of all, if your goal is explaining math using simpler math, I think there's a better way to do it. In a good math explanation, you formulate an interesting problem at level n whose solution requires level n+1. (Ideally n should be as low as possible.) In a bad math explanation, you assume the reader understands level n, then write out the basic definitions of level n+1 and formulate a problem using those. That loses the reader, unless they are already interested in level n+1.
But that's still underestimating the problem by a couple orders of magnitude. To jumpstart engagement, you need something as powerful as this old post by Eliezer. That's a much more complicated beast. The technical content is pretty much readable to schoolchildren, yet somehow readers are convinced that something magical is going on and they can contribute, not just read and learn. Coming back to that post now, I'm still in awe of how the little gears work, from the opening sentence to the "win" mantra to the hint that he knows the solution but ain't telling. It hits a tiny target in manipulation-space that people don't see clearly even now, after living for a decade inside the research program that it created.
Apart from finding the right problem and distilling it in the right manner, I think the next hardest part is plain old writing style. For example, Eliezer uses lots of poetic language and sounds slightly overconfident, staying mostly in control but leaving dozens of openings for readers to react. But you can't reuse his style today, the audience has changed and you'll sound phony. You need to be in tune with readers in your own way. If I knew how to do it, I'd be doing it already. These comments of mine are more like meta-manipulation aimed at people like you, so I can avoid learning to write :-)
Note that I ... wrote the only comment on the IAF post you linked
Yes, I replied to it :)
Unfortunately, I don't expect to have more Eliezer-level explanations of these specific lines of work any time soon. Eliezer has a fairly large amount of content on Arbital that hasn't seen LW levels of engagement either, though I know some people who are reading it and benefiting from it. I'm not sure how LW 2.0 is coming along, but it might be good to have a subreddit for content similar to your recent post on betting. There is an audience for it, as that post demonstrated.
I think Eliezer's Arbital stuff would've been popular in blog form. (Converting it to a blog now won't work, the intrigue is gone.) The sequences had lots of similar quality material, like "Created already in motion". I don't like it much because it's so far out, but it gets readers.
The technical content is pretty much readable to schoolchildren, yet somehow readers are convinced that something magical is going on and they can contribute, not just read and learn.
I don't think that's a matter of writing style. It's a matter of whether the prospective "research area" is simple enough that all of its general prerequisites can be stated in a popular blogpost, and otherwise be assumed to be known to the reader. (For example, many OvercomingBias/LessWrong readers have enough of a background in rational-action theory to know what "precommitment" and "dynamic inconsistency" mean, and these notions are indeed necessary for a proper understanding of EY's point.) At one point, that was true of the general area of timeless/updateless decision theory. It seems to be less true of the logical induction problem.
I think logical induction could've been popularized with just as much effort (that is, a lot). For example, the second problem from the post linked by endoself was discussed by Wei and me in 2012, with >40 comments each. If we'd been better at mass appeal, instead of coasting on the audience attracted by Eliezer, we could've had even more engagement. (Note the comment from thescoundrel in the second link, that's the kind of good idea out of nowhere that mass appeal is all about.)
Does popularization produce the goods? Lots of people have the background and skill to contribute to this problem who aren't currently in our community and don't have day jobs.
Choosing the right problem is certainly important, but I don't think it's the bottleneck. There's plenty of low hanging fruit. Knowing how to play your audience seems like more of a bottleneck, and it takes a lot of effort to learn.
I lack motivation myself. I'm interested in AIrisk but I think exploring abstract decision theories where the costs of doing the computation to make the decision are ignored is like trying to build a vehicle and ignoring drag entirely.
I may well be wrong so I still skim the agent foundations stuff, but I am unconvinced of its practicality. So I'm unlikely to be commenting on it or participating in that.
Maybe you've heard this before, but the usual story is that the goal is to clarify conceptual questions that exist in both the abstract and more practical settings. We are moving towards considering such things though - the point of the post I linked was to reexamine old philosophical questions using logical inductors, which are computable.
Further, my intuition from studying logical induction is that practical systems will be "close enough" to satisfying the logical induction critereon that many things will carry over (much of this is just intuitions one could also get from online learning theory). E.g. in the logical induction decision theory post, I expect the individual points made using logical inductors to mostly or all apply to practical systems, and you can use the fact that logical inductors are well-defined to test further ideas building on these.
When computations have costs I think the nature of the problems change drastically. I've argued that we need to go up to meta-decision theories because of it here.
The idea of solomonov induction is not needed for building Neural networks (or useful for reasoning about them). So my pragmatic heart is cold towards a theory of logical induction as well.
We're neither Athenians nor Spartans. Athens and Sparta were city-states. Greek culture thrived because Greece is a mountainous archipelago that prevented large empires from forming. The Greek city-states were constantly at war with each other and with the outside world, and so they had to develop strong new ideas to survive.
You mentioned the Netherlands, which is quite similar in the sense that it was a small country with strong threatening neighbors, but still became successful because of its good social technology. The story of Europe in general is basically the same as Greece. The complexity of European geography meant that after the fall of Rome no power could dominate the whole continent. So Europe was made up of small independent political entities that were constantly fighting each other. This competitive environment meant that they were forced to innovate good social technology.
But we are a community that faces a choice about what values we want: insularity and strong group membership, or openness and intellectualism. This seems fairly analogous to me, and after all don't we need strong new ideas to stop the AI apocalypse or improve lives all over the world? Perhaps the Amish vs. liberal German judaism would be a better analogy.
Compared to whatever we could likely do, the Athenians were insular. We are connected to the internet and to very much information. We have cheap travel that allows traveling to another continent for a conference and then flying back.
That's correct. But I worry that some projects in the rationalistsphere are about turning our backs on modernity in very strong ways.
Athena's shield, spear, and helmet weren't for show, and her major festival placed athletic contests next to musical and artistic ones. It seems dangerous to set up dichotomies and tilt fully to one side, as opposed to collecting and balancing virtues.
Unfortunately, analogies with Greek city states are wasted on me, because I don't have enough knowledge about them to make deep connections. For example, how specifically did Athens solve the problem of refugees bringing their own culture, sometimes incompatible with the original values of Athens? I have no idea. Therefore I have no idea what, according to this analogy, is LW supposed to do in a similar situation.
Unless it is supposed to be just a very superficial analogy, something like " yay individualism, sex and art! boo community, duty and survival!" in which case you have my... uhm... half-support, and I will seek the other half somewhere else.
I am not even sure whether this is supposed to be some kind of political metaphor, e.g. Spartans represent the SJWs who in their imagination take the heroic last stand against fascism, but in fact live in constant fear and keep purging their political opponents and heretics, while Athenians embrace the intellectual diversity, keep their culture alive while appropriating cool parts from other cultures, and most importantly do not forget to reproduce. (Or perhaps it is the other way round? Nope, that would be obviously silly.) Here we have a paradox, whether a part of having an "open mind" is to also include the views of those whose minds are comparatively less "open".
I agree that it would be cool to talk more about what other people are doing. For example, I would like to see relatively short reviews of what is the state of art in other approaches to artificial intelligence. Sadly, I am unqualified to write any of that; but if someone else writes it, and I will be able to understand it and learn something new from it, they will certainly get my upvote.
Unfortunately, analogies with Greek city states are wasted on me, because I don't have enough knowledge about them to make deep connections. For example, how specifically did Athens solve the problem of refugees bringing their own culture, sometimes incompatible with the original values of Athens?
Citizenship, and hence the right to vote, was restricted to people both whose parents were citizens.
Unless it is supposed to be just a very superficial analogy
I think it is a very superficial analogy -- there's a popular high-school-level trope that, basically, Athens made art, Sparta made war, and Athens won. Reality, as you might imagine, was considerably more complicated.
Temporarily: their allies turned on them and Athens soon rebuilt its navy. Ultimately Alexander and Rome ended the whole struggle permanently.
Temporarily
In which sense? Athens never triumphed over Sparta, they just survived, and you know why? Wikipedia tells you:
Corinth and Thebes demanded that Athens should be destroyed and all its citizens should be enslaved. However, the Spartans announced their refusal to destroy a city that had done a good service at a time of greatest danger to Greece
Doesn't quite match your narrative, does it?
It feels a bit strange to talk about needing to be more object, less meta, in a post about cities gone for a thousand years. Do you have any object level suggestions?
I remember a while ago Eliezer wrote this article, titled Bayesians vs. Barbarians. In it, he describes how in a conflict between rationalists and barbarians, or to your analogy Athenians and Spartans, the barbarians/Spartans will likely win. In the world today, low IQ individuals are reproducing at far higher rates than high IQ individuals, so are "winning" in an evolutionary sense. Having universalist, open, trusting values is not necessarily a bad thing in itself, but should not be done to such an extent that this altruism becomes pathological, and leads to the protracted suicide of the rationalist community.
Dysgenesis is worrying, but we have the means to fight it: subsidized egg freezing and childcare, changes to employment culture, and it is a very slow prospect. I don't think that is a correct summary of the essay at all, which is really pointing to a problem with how we think about coordination.
subsidized egg freezing and childcare
Fertility is inversely correlated with income, the problem isn't that people don't have enough money, the problem is that in some sense they don't want children. I think a better approach would be cultural changes that make it high status to have lot's of children.
I don't think that is a correct summary of the essay at all, which is really pointing to a problem with how we think about coordination.
True, his point that Bayesians should be able to overcome these coordination problems by doing X, Y, and Z. Except neither him nor anyone else has should any interest in actually making an effort to do X, Y, and Z.
Seems to me the costs of having children are mostly (1) time, i.e. opportunity costs, and in USA (2) education.
Education is probably expensive because although formally presented as a source of knowledge, it is actually a costly signal of... something (social class? willingness to sacrifice a lot in the name of "education"?)... and costly signals, by definition, are costly. If you would make a cheaper and more available option, some people would signal their superiority by not taking this option, and people who take this option would be perceived as not good enough.
Opportunity costs of time are obviously higher for people with more and better options, such as smart people.
While the US does have higher education costs than Germany it also has a higher birthrate. I don't think those costs are the major factor.
Unfortunately, I don't find data for how birth rate correlates with income in Germany. Only for the US: https://www.statista.com/statistics/241530/birth-rate-by-family-income-in-the-us/
But if education costs are very important than I would expect that at the top percentile the people with more money, who can afford to send more children to college get more children. That doesn't seem to be the case.
I don't think the question if what the main research lines within those fields are is a good one. To use EA phrasing, the major research lines in academia aren't neglected causes. Competing with academics in major academic research lines is a bad idea for anyone who isn't an academic himself and gets credit for solving the problems that academia cares about.
FAI risk wasn't a main research line when this community started being interested in it.
(This is, of course, not very relevant and, moreover, doesn't mean much, but I myself always felt more of a Spartan than an Athenian. Like, forget Athenians, we have a more immediate problem of Xerxes...and not much leeway in solving it.)
So does "holiday modifications" means "our modified versions of tradtiional holidays" or "modifications t ones value systems arising from attending rationalist holidays"?
I'm not exactly sure about the intended meaning with the specific wording but as far as the general content goes:
Petrov day isn't exactly a modification of a specific traditional holiday but it's about reminding us about our values. Solstice celebrations also are about reminding us of the values for which we strive.
I don't have a strong enough model of cultural identity to know how far to take the analogy, but overall I like the attempt. Keep in mind, though:Athens wasn't actually peaceful, just better than most in use of allies, conquests, and trade to expand their empire. Athens did, in fact, surrender to Sparta.
Often, Wikipedia is insufficiently detailed to cite as a reference, but https://en.wikipedia.org/wiki/Peloponnesian_War is pretty darned good. I'd forgotten that Athenian trade sanctions were a major factor in the war.
The Peloponnesian War was a war between two empires: the seadwelling Athenians, and the landlubber Spartans. Spartans were devoted to duty and country, living in barracks and drinking the black broth. From birth they trained to be the caste dictators of a slaveowning society, which would annually slay slaves to forestall a rebellion. The most famous Spartan is Leonidas, who died in a heroic last stand delaying the invasion of the Persians. To be a Spartan was to live a life devoted to toughness and duty.
Famous Athenians are Herodotus, inventor of history, Thucydides, Socrates, Plato, Hippocrates of the oath medical students still take, all the Greek playwrights, etc. Attic Greek is the Greek we learn in our Classics courses. Athens was a city where the students of the entire known Greek world would come to learn from the masters, a maritime empire with hundreds of resident aliens, where slavery was comparable to that of the Romans. Luxury apartments, planned subdivisions, sexual hedonism, and free trade made up the life of the Athenian elite.
These two cities had deeply incompatible values. Spartans lived in fear that the Helots would rebel and kill them. Deeply suspicious of strangers, they imposed oligarchies upon the cities they conquered. They were described by themselves and others cautious and slow to act. Athenians by contrast prized speed and risk in their enterprises. Foreigners could live freely in Athens and even established their own temples. Master and slave comedies of Athens inspired PG Woodhouse.
All intellectual communities are Athenian in outlook. We remember Sparta for its killing and Athens for its art. If we want the rationalist community to tackle the hard problems, if we support a world that is supportive of human values and beauty, if we yearn to end the plagues of humanity, our values should be Athenian: individualistic, open, trusting, enamoured of beauty. When we build social technology, it should not aim to cultivate values that stand against these.
High trust, open, societies are the societies where human lives are most improved. Beyond merely being refugees for the persecuted they become havens for intellectual discussion and the improvement of human knowledge and practice. It is not a coincidence that one city produced Spinoza, Rubens, Rembrandt, van Dyke, Huygens, van Leeuwenhoek, and Grotius in a few short decades, while dominating the seas and being open to refugees.
Sadly we seem to have lost sight of this in the rationality community. Increasingly we are losing touch as a community with the outside intellectual world, without the impetus to study what has been done before and what the research lines are in statistics, ML, AI, epistemology, biology, etc. While we express that these things are important, the conversation doesn't seem to center around the actual content of these developments. In some cases (statistics) we're actively hostile to understanding some of the developments and limitations of our approach as a matter of tribal marker.
Some projects seem to me to be likely to worsen this, either because they express Spartan values or because they further physical isolation in ways that will act to create more small-group identification.
What can we do about this? Holiday modifications might help with reminding us of our values, but I don't know how we can change the community's outlook more directly. We should strive to stop merely acting on the meta-level and try to act on the object level more as a community. And lastly, we should notice that our values are real and not universal, and that they need defending.