Intro

This post compiles my personal comments on the Epistemology book from the Lesswrong 2018 bookset. I put almost no effort into making it legible or relevant to anyone else.

A sketch of good communication

• For research, we want to exchange deep models
• For logistics team, okay to err on side of comparing shallower nodes just to come to agreement.
• Has implications for #sbub . Centralized and federated systems look an awful lot like the bad case of communication. The only way to converse is to straight up adopt the output. If I want #sbub to empower people trying to deeply understand the world, there has to be a lot of space just for interacting with other people's models and drawing intricate connections to your own.

Babble and Prune Sequence

Babble and Prune

• Currently I need to tune down my prune step and increase my babble. A lot. #habits
• #sbub in my current imagination is extremely babble and prune. Some concretizations include:
• Computers are capable of producing babble. That's an idea for a feature right there: integrated gpt-2.
  • Run another Lesswrong poll. Seed statements from babble and prune article. Seed claims of which the reader uses and how well this serves them, what others use and need. Seed claims of good sources of babble, good algorithms of prune, good combinations.
    
    • ask LW team about make a post with the convo embedded
    • make a link post, send delayed invitations to answer and comment
    • advise "if you notice a hint of confusion or not-rightness, sit with it for a moment. Say a short sentence out loud expressing some aspect of the intuition, write it down and submit it if you will."
    • provide a submission form for seed statement ideas, things you don't automatically want to vote in favor of.
       

More Babble

• Akjash is a graph theorist, good to know
• Scrabble babble as an example of the kind of play I want to do with my language generation project.
• Advice in this article seems geared from an assumption of the brain as an implicit graph, namely that individual edges and neighborhoods can be computed in reasonable time. This assumption may not hold for me. I really struggle to generate and re-compact local planning. Expansion may be related? Author clarifies this is not the same thing as connectivity, and doesn't clarify which actions in their example address which metric. I find myself trying to capture any babble I have 2 steps ahead
• Better if this essay had offered its own explanations of expander graph and aji. Wikipedia threw a lot of math symbols at me and I can't tell off-hand how it relates to this context. Go metaphors gave me very aesthetic feels but not a model with moving parts. What are the implications?
  • building an argument from scraps lying on the side of the road captures my experience
• I have insisted for a couple of years now that everyone should randomize their decisions more. If you're stuck in a bad rut, randomize all the details and then use prune-discernment to tweak it until you have something that looks workable. If your pruning can't keep up with your babble, then prune by weighted randomization or pick your favorite out of the first couple draws. (see the secretary problem)
• Mysticism is so great for babble. Absolute best. Form tight connections between everything and anything. Encode important ideas in inscrutable, babbling poetry. Find the essence of hope in a dropped kernel of corn.
• To save the world, I will start by doing the humble and proper things I know how to do within the confines of my own life - one for the Tumblr Likes compilation. #habits
  • Sounds Confucian
• You can increase the number of interconnections in a collaborative babble by hyperlinking. (Wikipedia, RoamResearch and alternatives like Foam or Logseq). You can increase the quality of discussion by generating unique, non-obvious placeholders that expand into nuanced concepts. (such as #sbub, or “The sparkly pink purple ball thing”. see also "correct horse battery staple"). I am in absolute love with the affordance rationalist contexts have for building complex ideas on top of nuanced concepts, with a bunch of mental hyperlinks to specific articles that define/contextualize/elaborate on how I am using my terms. It's like the human version of linked data.
  • I use compound neologisms in my writing A LOT. I think of something, and I want to encode it with multiple examples. or definitions. or synomyms. or checksums. It's hard to locate a point in concept-space. I default to giving as many hints as I can. I really, really want my language to capture the messy, associative nature of thought.

Prune

• Prune has multiple layers, yes. Subconscious, Conscious thought, Spoken word, Written word, Published word. (At times, Written and Spoken are the other way around for me.)
• River of Babble: that's powerful imagery right there.
• This essay has awesome style, it moves the reader in its currents like deep mysticism. We need some of that. We need it in addition to prized technical, specific writing.
• Contains one of the better woo-free explications for meditation practice.

Local Validity as a Key to Sanity and Civilization

• Third example at the beginning is shaky. I get stuck on figuring out whether I agree with the implicit judgment of ape-mammalian social instincts as default, naive, and destructive.
• Your faith in your priors gates your understanding of the rules of allowable argument steps. Being convincable of unintuitive truths by logical validity could get REALLY REALLY BAD OUTCOMES IN PRACTICE if not coupled with the willingness and ability to rethink your foundational assumptions. That skill does not come easily. "Fair to one side but not the other" speaks to this problem; their machinery to determine local validity works but they won't use it to evaluate an argument from premises that don't fit snugly into their understanding of the world.
• I have the weaker version of the low-level attribute, even as I've explicitly championed local validity.
• #habits - When I notice myself steel-manning my own side I will state the original claim's bogus-ness out loud (or write it down) as a simple statement with no equivocation. Only after this will I attend to adjacent, more substantive claims.

In a Comment by bryjnar -

this throws up an important constraint for people designing systems that humans are supposed to interact with: you must make it possible to reason simply and locally about them.

Broken alarms

• My alarms #habits
  • there is too much noise.
  • I need to not bother people.
  • make sense.
  • make myself busy.
  • defensiveness
• Compare to Umeshisms.

Varieties of Argumentative Experience

• good faith surveys of evidence take effort. I don't want to prune my efforts to dredge up faded recall or verbalize intuitions about the high-level generator of disagreement.

• Going over a public political debate with this framework might train a good habit of thought. Combing over a thread in a rationalist space with this framework may generate useful feedback.
• surprised to see double-crux described as a pruning technique. i would think of it as directing seeking: it has a function for determining if you're on track but it also keeps introducing a whole bunch of new avenues to explore.

Naming the Nameless

• Sarah was not exaggerating about chinese opera.
• Consider how to detect style blindness in others. At its best, style is a high level generator of disagreement. So, "Varieties of Argumentative Experience" may help in describing specific error cascades to style-blind individuals.
• Reaction is where the atheism movement was when I first entered it. I have concerns. I would like to co-opt their style to talk rationality to them. On Gab probably.
• project in the vicinity of "making amends between creators and expanders": disco.coop.
• New distinction: bets of expectation vs. bets of taste. The former requires specificity, the latter requires ????emotional connection????.

Toolbox-thinking and Law-thinking

• I'm itching to rewrite this with Scott's Argument pyramid as a rubric. If you knows anyone would find that useful to read. Eliezer's writing style may turn some people off for a reason.
• How Law vs. Tool is relevant for #sbub . Defining the Ideal of a solution to a problem differs from grabbing the best available tool for it. I define the ideal because I believe the best available tools are not good enough; I want a comprehensible metric to make other people more useful help in coordinating towards better tools.
• When I find myself internally protesting that I don't know what math to trust because reality is nearly always messier than proof premises… It's not wrong… but something is funky about that and I need to reframe.
• Mathematical law is timeless. There is no "running the algorithm", really just twisting and turning your head until you hit the right angle to see what was by necessity true the whole time. Math, if you're doing it right, is a whole lot of cleverly disguised tautology. #habits
• Third, related thinking style: Animus. When you don't know the laws, when the metrics you care about are more nebulous and proxied than "distance", you may still need some way to tie together your strategy and experience and hypotheses about a subject. In comes a form of animism: things and concepts have an essential spirit. It is your spirit, of course, the shadow of your own mind that reflects the subject and how you interact with it. It is the memes and archetypes that suffuse your imagination. It is the norms and incentives that push-pull on your actions. You don't use animus-thinking: animus-thinking uses you.
• Thought Eliezer was wrong in the example of sticking to the left-hand rule in a maze. Surely you can just treat the path as a new entrance and resume the left-hand rule? I was wrong. I had fun thinking through it.

• People who I'd want to introduce rationality to could benefit from having the fallback ability to understand whatever-it-is as a fact instead of a normative ideal. It was really hard to find the words for at the time. I kept gesturing to something like "cultivate the skill of knowing what it is right and choosing to do the wrong thing" or "face god and walk backwards into hell".

Toward a New Technical Explanation of Technical Explanation

• If I recall correctly, (and I did), the theory of Logical induction uses prediction markets to eliminate the assumption of logical omniscience from Bayes. Logical Induction was a big breakthrough. Would implementing a market to that logic in #sbub constitute a (haha) market gamechanger?
• Markets aren't simple. Why would Bayes simplify to this weird thing humans invented to do with other humans?
• The market assumption "tastes" like a tool shoved in between the systems to approximate the conversion between them. Markets are a really good tool: but still they're just a tool.
• "I focus on the X, with only a minimal account on how Y makes it work" what a nice, succinct way to establish a context I frequently operate in, 10/10 would plagiarize
• My brain is screaming. Nothing clicks, this model is not deep enough that I can meaningfully integrate it. Did I skip step number 2 in ? I think this article managed to fall down on both high logical validity and encoding gears for installation in human brains.
• This also related to my secret crusade to push Statistical + Systems theory proofs to the memetic fluency and age-appropriateness of aesop's fables. We live in a very large and detailed world! Small gains in this direction would yield lots of benefit for peoples future and present!
• A first level made out of gears, a second level made by gears. This principle underlies the "right to repair" movement regarding consumer goods with literal gears. A company sell you the literal machine made of gears, but they can also sell you a guide to how it's made, usable tools to deconstruct and reconstruct it. Models with easy repair have many of the same useful properties for prediction markets and as right-to-repair goods do for consumer markets.
• Seeing the derivation tracks with what I wrote above about turning your head around a timeless pattern
• Determinism, Reconstructability, Convergence. Hot damn, is Truth a CRDT?

Points for putting suffering in the right spot in your map

Rather, we suffer, when we try to wrap our heads around what's going on.

Closing thoughts

• A sacrament I've been considering: pick a thing-in-my-life and trace back all the actors who contributed to it in concrete detail, then sit with this information and try to wrap my head around it. I expect to want to contribute back, in the end. Mostly in very small ways, as the modern economy coordinates production over many different individuals and institutions.
• To the big movers on this project, I would happily owe one favor per book. You have enriched my life by compiling these articles in such a lovely manner that I could not resist buying it and then digging in full throttle them. This is a book set I would proudly gift or lend, which is good because I wound up with one extra. If you do not feel fully compensated, shoot me a message. Consider me to have asked, right now, "Do you want me to spend a proper 5 minutes thinking about your problems/values and then an hour taking concrete action on your behalf?".