LW as rationality hub
One of the goals of LW is to be a forum for the development of rationality, or the application of a particular mindset to oneself and the world. I think this involves a combination of philosophical commitments and practical skills; looking at how that was balanced in a similar situation in the past helps clarify what's happening in the present and what some of the counterfactuals are.
In many ways, psychological phenomena runs into the problems of early astronomy. Different people in different parts of the world see slightly different skies and have different weather; you might only have one observation of where things were on any particular night. Knowledge that a particular tool or technique works for one person might have little bearing on whether or not it works for others, which suggests a focus on local observations.
That said, LW is about a very computational and mathematical view of the world. It reminds me of LaPlace's Demon, the sense that something could have the universe's true dynamics and run all the numbers and predict things exactly, and there's something spiritually solid about yearning for that standard. (It isn't even shaken by quantum indeterminacy, which is neatly resolved by many-world's choice of "branch both ways", such that in the broader multiverse both exist, and you're just uncertain over which branch you end up in. Solomonoff induction and Tegmark's multiverse extend it even further, such that even uncertainty over the universe's true dynamics doesn't upset this view!)
This feels importantly different from, idk, a working mathematician's view of math? Hypothesizing that somewhere out there are all the true propositions and all the false propositions is pretty different from the set of known-true-propositions, and the methodology for determining what goes in that set. Or a working software engineer's understanding of computation, in which runtime considerations are important instead of ignorable.
And so the particularist's observation of a specific interval of spacetime and the universalist's theory of everything clash.
LW as futurist hub
Beyond that, I think LW is a forum with lots of people with a particular outlook (which I'll call 'futurist'). We're living in the hinge of history, AI is important, the humanist neo-Enlightenment is good, many other assumptions that I won't bother to list out.
This, in some ways, reminds me of the relationship between gentlemen-as-a-class and England. (Maybe in a future post I'll elaborate on how I think about classes and their relationship to society; here I'll just mention the association.) When I think about LW as an institution or futurists-as-a-class, many of the same practical social considerations come into play: how do we keep the conversation going? How do we engage in the practical business of living?
And, moreover, what do our roles and templates look like? What is our bricolage from the materials at hand in our culture?
To talk about the universalism vs particularism issue, one of the issues with universalism is that it's trying to solve either provenly hard problems, in the sense that it requires exponential or worse efficiency for an algorithm to do it, or we suspect that it's really hard to do, and we just haven't proved it.
One of the best examples here is learning efficiently from data, and there's a line in a paper that talks about one of the issues for universalism in practice:
Any polynomial-time algorithm for finding a hypothesis consistent with the data would imply a polynomial-time algorithm for breaking widely-used cryptosystems such as RSA!
And this is considered unlikely by complexity theorists and cryptographers.
This seems similar to the issue mentioned below:
This feels importantly different from, idk, a working mathematician's view of math? Hypothesizing that somewhere out there are all the true propositions and all the false propositions is pretty different from the set of known-true-propositions, and the methodology for determining what goes in that set. Or a working software engineer's understanding of computation, in which runtime considerations are important instead of ignorable.
The problem for mathematics in general is at least in the RE complexity class, which is wildly intractable, and even propositional logic is NP-complete in satisfiability. These are really hard problems in the general case, so hard that without radical assumptions about physics will likely remain intractable in the general case.
Universalism is a currently impractical doctrine, but one that is sometimes useful.
This is a chapter-by-chapter summary of A Social History of Truth by Steven Shapin. Focused on Robert Boyle, a founder of the Royal Society considered the first modern chemist, it is interested primarily in his social context and how he (and others) changed it. He was widely considered a role model at the time, and likely saw himself as creating the role of experimental scientist that many would follow. What did he create it from, and why that particular way?
[You may also want to read thru Novum Organum, also available on Less Wrong; published seven years before Boyle was born. While Boyle claims it had little direct influence on him, it undoubtedly had significant indirect influence.]
The Great Civility: Trust, Truth, and Moral Order
"Truth" is often used to refer to correspondence between beliefs and reality. What is there to write a 'social history' about? Shapin isn't interested in the inaccessible truth of philosophers--correspondence between map and territory--but the practical truth of societies--correspondence between a statement and a map. Given that I don't have unmediated access to reality, I can only judge statements as "true according to me" instead of "absolutely true"; and "true according to me" has a bunch of interesting detail behind it.
In particular, Shapin is interested in trust. You probably believe that Caesar was a real person who actually lived on Earth, and you probably never met him, instead following a chain of trust (you believe an author who themselves believed another author, and so on to antiquity). This trust is morally textured; if someone lies about something like the existence of Caesar, it's not a neutral action, and actions are coordinated (or not) based on what beliefs people have trust in, which depends on which people are trusted. Shapin points to thinkers from Cicero to Giddens identifying trust as one of the foundational elements of social order.
Of course, our eventual subject will be Robert Boyle, the Royal Society, and the birth of science, which claim to be opposed to historical systems of trust. The Royal Society's motto is Nullius in verbia, or "take nobody's word for it", and the promotional literature for science foregrounds experiments and direct experience. But radical skepticism or absolute distrust are both impractical and impolite:
Science as it stands today is built almost entirely out of received knowledge instead of experienced knowledge, and this is how it manages to accumulate at all. Society's system of shared knowledge is a communal good, produced like any other. He introduces the phenomenologist's concept of the 'natural attitude', a common-sense realism that views everyone as having access to different perceptions of the same underlying reality; accounts are supposed to not be too discrepant (as that calls into question there being one underlying reality) but some discrepancy is be expected (as observers have different locations, perspectives, perceptual tools, and so on).
Shapin also brings up the idea of 'free action', i.e. being uncoerced by one's situation, which was highly relevant to early modern England. A promise made under duress is not considered a promise (and contracts signed under duress are not enforceable); a person under duress is not trustworthy, as the things they say may be a result of their situation rather than their true beliefs (and society is not going to hold them accountable for things they say).
[This chapter seems a combination of necessary and boring; it deals with a long list of models of basic machinery of believing, choosing, and communicating. As Shapin is trying to carefully analyze another society's version of that machinery, pointing out how 'we' perceive 'our' society's machinery is useful context for what analysis he's hoping to do.]
"Who Was Then A Gentleman?" Integrity and Gentle Identity In Early Modern England
Shapin presents a detailed picture of English society at the time, which I will simplify and compress:
A Social History of Truth-Telling: Knowledge, Social Practice, and the Credibility of Gentlemen
Like the previous chapter, which tried to cover both the story gentlemen told about themselves and the actual practice, this chapter looks at honor and truth.
Concepts like "honor" have both an internal and an external sense; whether you think your actions correspond to some code of conduct, and whether your society respects you. There was controversy over which was more important or real; but the two are deeply linked in a way that makes it a deliberate choice to lump them together. (Similarly, using "noble" to mean both 'powerful' and 'virtuous' can be seen as a psyop on nobles to try to shift their behavior thru an intentional bucket error.)
The chapter gives a brief description of why they cared so much about honor and reputation, which I found somewhat uncompelling. Basically, you can think of gentlemen as existing primarily in the social sphere, coordinating efforts of other people; given that their only real levers on the world are speech acts, the reliability of those speech acts is their only real job, and once shown to be unreliable they lose their privileged access. But... they don't lose their rental income, and the English gentlemen of the time seemed to think that their Continental counterparts were notably less honest and trustworthy. It seems somewhat likely to me that this was a quirk of English gentlemanly society specifically.
Truth and reliability are, in some sense, defined by falsehood and unreliability:
Accusing someone else of lying was grounds for a potentially lethal duel, which led to significant second order effects: a careful system of expressing disagreement without direct accusation, an expectation that gentlemen would express their opinions narrowly and carefully. Together, these ended up forming the basis for the Royal Society's style of considering scientific testimony, in a marked contrast to the divisive pedantry of scholars.
Who Was Robert Boyle? The Creation and Presentation of an Experimental Identity
This has four parts:
That is, Boyle was offered "philosopher, Christian, gentleman" as roles by his society, and cocreated something new out of them.
So, who was Boyle?
Robert Boyle was second to last of 16 children of the first Earl of Cork, widely viewed as a robber baron, who made a fortune colonizing Ireland and managed to rise from obscurity to being one of the highest income Crown subjects over the course of a few decades. Even as a later son who didn't stand to inherit the main titles, Robert would have been known to be rich by basically all of his social contacts.[1] And as a later son, he needed to find something to do to fill up his time.
Somewhat sickly and introverted, he found contemporary polite society "debased and corrupt", while not choosing the life of a hermit. Well aware of the conflicts between the secular gentleman role and religion, he actively promoted the modified role of 'Christian gentleman.' He was basically raised by his siblings and tutors, only meeting his father a handful of times. He was a devout Christian, interested in serious self-examination to become more virtuous, presumably in part because of the influence of some early friends and believing his father's story that their good fortune was the result of divine providence. His arranged marriage fell thru after his father died; he never sought out another and probably died a virgin.
He viewed natural philosophy as being 'a priest of nature'; he apparently considered taking on holy orders but decided against it. Later, when offered a position in the Anglican church, he rejected on the grounds that he hadn't been called to it, and thus it would be 'lying to the Holy Ghost'. He viewed his writings in favor of Christianity more credible given that he was an independent source, rather than someone affiliated with the church.
He gradually turned to philosophy as a vocation, but this conflict needed to be resolved.
The end result was someone who was widely perceived to have little to gain and much to lose by lying.
Epistemological Decorum: The Practical Management of Factual Testimony
This chapter details the epistemological problem they faced, and methods they used to deal with it. It begins by pointing out that Horatio's statement to Hamlet, that there are "more things in heaven and earth than are dreamt of in your philosophy", was at the time not a mere dismissal of formal knowledge but a recognition of its very real limitations. Shapin sees Bacon's contribution here as putting observation first and abstraction second; traditional systems of knowledge dismissed cases they could not easily conform to their categories on first appearance, whereas the empiricist should search for those anomalies, collect them carefully, and deliberate on their implications. This sort of ontological openness rhymed with gentlemanly freedom (in contrast to being constrained by one's ideology).
Of course, you can't believe everything you read. And this was in an era when many reported marvels would not be replicable: suppose investigators in London want to verify the report an explorer to the arctic who reports back to London on the properties of extreme cold. While they had the ability to generate significant heat, they didn't have the reverse, and so couldn't chill anything local to the same degree. (And what about verifying Aurora Borealis?)
Shapin's treatment here differs from the standard story, which emphasizes distrust and individual verification. Taken seriously, you shouldn't believe in Aurora Borealis unless you've seen it yourself, but that isn't standard scientific practice. Boyle and others knew that all government and justice rested on credibility of testimony, and were visibly functional; not thru blind trust or distrust, but prudent consideration of the source and claims.
The broader system for managing testimony, of course, also propped up Christianity, which was viewed as based on historical events like any other bit of history, and a crucial foundation on which social order rested.
I won't go into the maxims for evaluating testimonial plausibility, and just quote Shapin's list:
Of course, each maxim is a rough guide, and there are countermaxims to point at the edge cases (except, importantly, the last); evaluation of testimony is done in a 'commonsense' manner that combines all considerations.
Knowing about People and Knowing about Things: A Moral History of Scientific Credibility
John Locke tells the story of a Dutch ambassador talking with the King of Siam, who hears many stories of Holland but, when the ambassador mentions that rivers can freeze over and an elephant could walk across them, loses his credulity, deciding that the ambassador is a liar.
Of course, we know whether or not rivers freeze, but the problem is general: travelers that tell wondrous tales strain the credulity of their audiences (and perhaps ruin their own reputations), and listeners will end up uninformed about the world if they refuse to acknowledge anything outside of their experience. In order for the London-based Royal Society to make sense of the broader world, it has to carefully consider testimony from it.
The chapter goes thru many individual cases, looking at how Boyle reasoned thru them; I will not recount all of it. The main thing that seems worth extracting out is the gentlemanly style of steering around conflict.
For example, Boyle did some laboratory experiments to establish the buoyancy of ice, and also collected reports of icebergs; the two did not agree with each other. Rather than accepting one account and rejecting the other, Boyle accepted both accounts as true observations--and hypothesized many different potential realities that count generate both, and requested future observations.
Certainty and Civility: Mathematics and Boyle's Experimental Conversation
Here, as everywhere else, Boyle is pragmatic, presenting tables of figures and describing them as having 'reasonable agreement' with the hypothesized regularity.
Shapin notes an oddity: today we think of the scientific revolution as mathematical natural philosophy, but Boyle argues against mathematics, instead presenting a 'mechanical' and 'experimental' view. Some historians view this as the result of innumeracy on his part, but Shapin assembles evidence that Boyle was good at math for his time, and deliberately chose to de-emphasize mathematics in his empirical science. (It's sort of like the opposite of Beautiful Probability.)
The main reason seems to be that math was uncivil. Fewer people were capable of understanding it, and the standards of rigor used in mathematics were entirely too stringent for the empirical sciences; it was good for belief, but not Pareto-optimal for truth.
In the 1660s, Boyle critiqued thought experiments, favoring observations of reality. Thought experiments could have bearings on expectations, but could not stand for actually trying it out--with the knowledge that open things do not go as anticipated. In part this was due to a practical understanding of the variability of materials; metals were known to be of variable purity, air of heterogeneous composition, and so on, well before they had our current understanding of elements.
In cases where observations disagree, the natural inclination is to think that one is wrong and someone has made an error. Boyle's solution was that both observations were probably locally right, and the assumption that they should be identical was probably wrong. This focus on variability was key to preserving civil conversation, which was the mechanism by which science continued.
Invisible Technicians: Masters, Servants, and the Making of Experimental Knowledge
As one might expect from a social history, Shapin doesn't ignore the humans just outside of the spotlight. Boyle's lab was staffed by servants and technicians, and we only know a handful of their names and a bit about their lives. While of human interest, it's mostly not relevant to the parts I'm most interested in here.
Epilogue: The Way We Live Now
Reliability is fundamentally different for us, largely because of our relative size and specialization. Boyle accepts as testimony evidence from a sea captain that he personally knows, and might reasonably expect his readers to know whether or not he is a good judge of character; what testimony we accept relies heavily on mediation by institutions.
But when you look carefully at those institutions, at the lowest levels they are composed of small groups that mostly know each other and are tracking reputations. Familiarity still has a role to play.
Commentary
Published in 1994, A Social History of Truth predates the replication crisis[2] and the revolt of the public. Our culture's solution to the problem of whom to trust does not seem adequate, at the moment.
One of the core questions seems to be how seriously the producers of testimony take honesty. The members of the Royal Society seemed to share idealistic views of honesty much more than modern scientists do today (in part because of secular honor culture, and in part because of religious culture, in part because the gentlemen-scientists all had "fuck you" money and were doing science out of curiosity, rather than as their livelihood). It is not obvious how to prevent fiefdoms from emerging inside of the modern scientific apparatus, and how to detect whether or not someone is the sort of person that would rather lose their job than fabricate data.
That said, I do find the belief of early modern English society that gentlemen were "unconstrained" suspicious; my worldview that foregrounds incentives and views their reliance on rent-seeking as negative. (The Royal Society is the best foot forward from that culture, and I find it telling that Boyle, at least, found them debased and corrupt.)
Boyle's early views that people should share scientific knowledge with the world for free didn't last long; by the end of his life, he was arguing in favor of intellectual property (at least, when it came to prestige markets). Our society has leaned heavily towards the commercial virtues of the merchant class, which thinks it's better for society to incentivize people for results and does not do much in the way of investments in virtue (Boyle would likely find us debased and corrupt as well).[3]
I'm interested in how to apply this to Less Wrong, tho I'll talk about that in the comments.
The closest modern analog might be one of Bill Gates's children--with both the "Micro$oft" backlash and generous spending afterwards. (The elder Boyle was much less humanitarian than Gates.)
Shapin includes Scientific Knowledge and its Social Problems, which predicted the degeneration of modern science in 1971, in his references but in my physical copy I can't easily find where he cites it.
I am reminded of No, it's not the incentives, it's you; it seems more 'earnest' than the mainstream culture, but of course, Boyle was also more earnest than his culture. Almost definitely we have more scientific-truth-seeking in absolute terms than existed back then, but for trust in science as a whole, the relative amounts are perhaps more relevant, and there it's not obvious we're at a historical peak.