Useful clarification and thanks for writing this up!

Inspired by and building on this, I decided to clean up some thoughts of my own in a similar direction. Here they are on my short forum: What are the actual use cases of memory systems like Anki?

What are the actual use cases of memory systems like Anki?

Epistemic status: spent 30min cleaning up some notes from my Obsidian I jotted down yesterday. This ontology is rough and a bit illegible but potentially useful for narrowing down the actual use cases of memory systems.

Inspired by @Saul Munn's recent short form: Active Recall and Spaced Repetition are Different Things. The concepts of active recall and spaced repetition apply pretty well here, but I saw Saul's post after writing most of the text below.

Roughly, there are types of knowledge in domains (recalling from Scott Young's Ultralearning, I might be slightly off):

• Proceduralized knowledge
• Conceptual knowledge
• Factual knowledge

I think spaced repetition systems are useful for three types of domains based on the nature of the cues in the domains:

1. Domains where you are already cued on the learnings often (e.g., key business procedures in your full-time job). You naturally get spaced repetition in these domains, so memory systems like Anki are less useful.
2. Domains where you are not cued on the learnings often or at properly spaced intervals. This may include domains where you were an expert at one point and then stopped being an expert.
3. Domains where you need to understand cues before being effective. (e.g. understanding a new language or technical domain like quantum mechanics; see Quantum Country.)

Combining the two frameworks above, Anki is useful for:

• Proceduralized knowledge where you can appropriately cue yourself and you're not cued often on.
  • Cueing procuduralized knowledge is hard, though. For example, if I wanted to review my knowledge of 'how-to-play-beach-volleyball' on a regular basis, I have to spend a time coordinating or attending an event.
  • Though, some proceduralized knowledge is possible to cue: for example, I have been using Anki to review LeetCode problems.
• Conceptual and factual knowledge that you're (2 above) not cued often on or (3 above) need to meet a criteria of knowing.
  • Conceptual and factual knowledge is much easier than procedural knowledge because these types of knowledge neatly fit in flashcards. I'm still not sure about the strict boundary between conceptual and factual knowledge, though.
  • And I think the general heuristic that conceptual knowledge is harder to cue than factual knowledge is true: medical students are obsessed with Anki while students in other vocational schools (e.g. law) don't seem to be

Accordingly, memory systems may not be useful for:

• Domains where you are cued on information often enough to get natural spaced repetition.
• Are highly procedural in a way that can't be Ankified and therefore active recall is hard.

Another caveat:

• I am believable and have expertise in very few major life skills, and possibly don't have expertise in the thing you're asking advice for.

Related note: I think developing the skill of identifying believability and expertise is very powerful (though I have only been applying said skill for a couple of years explicitly; caveat emptor, lol.)

Here's Cedric Chin outlining believability defined by Ray Dalio: 

Technique summary:
Believable people are people who have 1) a record of at least three relevant successes and 2) have great explanations of their approach when probed.

You may evaluate a person's believability on the subject matter at hand by applying this heuristic. When interacting with them:

1. If you’re talking to a more believable person, suppress your instinct to debate and instead ask questions to understand their approach. This is far more effective in getting to the truth than wasting time debating.
2. You’re only allowed to debate someone who has roughly equal believability compared to you.
3. If you’re dealing with someone with lower believability, spend the minimum amount of time to see if they have objections that you’d not considered before. Otherwise, don’t spend that much time on them.

Here's Gary Klein, founder of Naturalistic Decision Making, outlining seven dimensions of expertise:

We want pragmatic guidelines for deciding which if any purported experts to listen to when making a difficult and important decision. How can we know who is really credible?

Bottom line: We cannot know for sure. There are no iron-clad criteria.

However, there are soft criteria, indicators we can pay attention to. I have identified seven so far, drawing on papers such as Crispen & Hoffman, 2016, and Shanteau, 2015, and on suggestions by Danny Kahneman and Robert Hoffman. Even though none of these criteria are fool-proof, all of them seem useful and relevant:

(a) Successful performance—measurable track record of making good decisions in the past. (But with a large sample, some we do very well just by luck, such as stock-pickers who have called the market direction accurately in the past 10 years.)

(b) Peer respect. (But peer ratings can be contaminated by a person’s confident bearing or fluent articulation of reasons for choices.)

(c) Career—number of years performing the task. (But some 10-year veterans have one year of experience repeated 10 times and, even worse, some vocations do not provide any opportunity for meaningful feedback.)

(d) Quality of tacit knowledge such as mental models. (But some experts may be less articulate because tacit knowledge is by definition hard to articulate.)

(e) Reliability. (Reliability is necessary but not sufficient. A watch that is consistently one hour slow will be highly reliable but completely inaccurate).

(f) Credentials—licensing or certification of achieving professional standards. (But credentials just signify a minimal level of competence, not the achievement of expertise.)

(g) Reflection. When I ask "What was the last mistake you made?" most credible experts immediately describe a recent blunder that has been eating at them. In contrast, journeymen posing as experts typically say they can't think of any; they seem sincere but, of course, they may be faking. And some actual experts, upon being asked about recent mistakes, may for all kinds of reasons choose not to share any of these, even ones they have been ruminating about. So this criterion of reflection and candor is not any more foolproof than the others.

[I only skimmed the post so you might have addressed this, but…]

I once met a sometime who made super intense eye contact all the time, and it gave me weird vibes. Sort of uncanny valley vibes, if I try to put words to it? It was like they were staring at me in normal conversation. One hypothesis I had as to why is that maybe he did one of these type of activities that you outline here (iirc, I was introduced to the idea being a thing through an old LW post on scientology doing it?).

Epistemic status: I don’t know if an event like this was the cause of this person’s intense eye contact. I don’t know if anyone at your events has this result. But just thought I would mention it in the off chance that “getting in the habit of giving super intense eye contact” is a possible failure mode of doing activities like this.

Tacit Knowledge Videos added to the list from May–August, 2024. Enjoy!

• Kenneth Folk, pranayama breathing.
  • “Kenneth Folk is an instructor of meditation who has received worldwide acknowledgement for his innovative approach to secular Buddhist meditation. After twenty years of training in the Burmese Theravada Buddhist tradition of Mahasi Sayadaw, including three years of intensive silent retreat in monasteries in Asia and the U.S., he began to spread his own findings, successfully stripping away religious dogma to render meditation accessible to modern practitioners" (Website).
• Keith Johnstone, teaching improv.
  • Author of Impro.
  • "A pioneer of improvisational theatre, he was best known for inventing the Impro System, part of which are the Theatresports" (Wikipedia).
• Gwern, "Internet Search Case Studies"
  • Gwern blogs at his well-known gwern.net.
  • Per his about, Gwern has also "worked for, published in, or consulted for: Wired, MIRI/SIAI, CFAR, GiveWell, the FBI, Cool Tools, Quantimodo, New Work Encyclopedia, Bitcoin Weekly, Mobify, Bellroy, Dominic Frisby, and private clients" (Website).

I’ve had a busy past few months (if a three-month meditation retreat counts as busy). There have been more videos submitted than videos added to the post in this batch. I will add these in the coming months.

Thanks!

If there were a bunch of videos of people Google-fu-ing like this, I wouldn't add this article to the post.

However, since this is one of the few good resources on Google-fu that I know of, I'm adding it to the post. Despite it not being a video.

I've added this as a note. Thanks!

Thanks for the kind words! I'd be curious to hear more about what makes you think that!

Edited - thanks!

How would you go about answering this question post-said insight? What would the mental moves look like?

I'm never good at giving an answer to my favorite book/movie/etc.