This sounds like fun!
I think the daily journal should be published to a can-be-private/locked blog that at least n people of the journaler's choosing can access, in addition to maybe needing to be visible to the cohort.
I think this would help me have a midway point between the "no inferential distance" audience of myself (which doesn't provide an affordance for fully spelling out and checking my thinking), and the "too large an inferential distance to be bridgeable in one step" audience of the eventual effortpost.
1Yeah something like that feels good to me.
I actually did end up mostly publishing into private venues during my weeklong self-betatest for idiosyncratic reasons.
One problem I ran into was there was a relatively small number of people that actually made sense to share it with, but like, those people hadn't particularly opted into engaging with me that week and I felt like by posting to a narrower group I was making more of a demand on people's attention on my Thinkslop than publishing publicly would have. (I think this is totally solvable but requires a bit of attention)
I've been doing something similar on my own for the past few weeks. The main difference is that an LLM can answer my questions while your questions are wilder.
Mine have looked like:
- How did Lévi-Strauss respond to Sartre on existentialism?
- What were post-WWII existentialists in France actually like?
- Why is being mild about religion so common?
- How does Blackstone's private equity fund operate? What are the most important financial markers for this type of company?
- Why are Chinese AI models so weakly secured against extraction of bio knowledge and capabilities?
- Is Trump going to strike Iran?
- What is understanding in mathematics? Can you become good at math without ever feeling like you truly understand things?
- More of a goal design nagel-style candle holder that you can print on a
These come from news and books I'm reading at the moment.
Which points at what I expect will be one of the most common failure modes here: asking boring questions that even an LLM can answer.
These sort of questions come up pretty naturally, just, yeah, if an LLM can answer it, it's no longer the interesting part of Thinkhaven. If you came in just planning to ask LLM-answerable questions the mentor/coach staff would be like "okay dude this is not the spirit of the thing, you can do better."
But, synthesizing all the different answers to LLM questions into a coherent bigger picture that matters is still an important part.
(Also, the 500 words and 2500 words definitely need to be human written. The journals/essays can have arbitrary amounts of LLM-content if that's useful, but, for meeting the Goodharty goal, you need to write human-words)
I didn't get it put together in a way I felt ready to ship, but, a mix of LLM-answerable and not-very-LLM-answerable questions I actually asked during my week of Thinkhavening, were:
Initiating questions I was asking:
- What's up with Tsvi/JohnW/ThaneRuthenis thinking that LLMs are missing major ingredients necessary for true AGI?
- Why are LLMs still sometimes ludicrously bad at thinking, despite being apparently good at it?
- Does ASI require at least one major conceptual breakthrough?
- What pieces along the way to modern LLMs required major conceptual breakthroughs (as opposed to just straightforwardly combining the existing ideas)
This resulted in LLM-answered questions along the way like:
- What were the major innovations throughout the entire chain of ML-to-LLMs?
- What prerequisites did each of those have?
- Why didn't the innovations happen sooner?
- What were the details of how the Perceptron was invented?
- (one answer was "it was building off the artificial neuron")
- What were the details of how the Artificial Neuron was invented?
- (half-remembered-answer is "one psychologist/brain-surgeon guy (McCulloch) was obsessed with the question 'how do human brains implement logic?' for 20 years, and eventually met a young logician (Pitts), and then the two of them hashed out the details of how to implement logic in pen-and-paper neurons)
- If McCulloch and Pitts hadn't invented the Artificial Neuron, who would most likely have invented it instead.
- (I think there were a couple answers here, but one was Alan Turing).
I didn't really trust LLM judgment about the previous question, and a lot of the week was trying to think of questions that were pretty grounded/reasonable that seemed useful for synthesizing the answer. i.e. "was anyone working on literally this at literally the same time?")
(An overall takeaway I had was that many innovations are mostly combining prerequisites in straightforward ways but you need one guy who really deeply understands the prequesites)
i'm really not the demographic for things like this, so take this with a grain of salt, but this sort of beeminder-flavored "do the thing or a negative consequence will happen" really seems to me like just about the worst way to cultivate skilled and insightful work. like, could i force myself to write at least 500 words a day if someone put a gun to my head? sure, as you say, it's extremely goodhart-able. however i would expect the writing i produced under those circumstances to be...kinda shit? i often spend up to a month on a single essay, and if i was trying to do some sort of formal research paper it would take even longer. creating a situation where people MUST WRITE OR ELSE, seems like a good way to get a large quantity of slop. i guess it could be helpful if someone needs to get their foot in the door as a writer at all, but i've already got my foot well in the door so this sort of event seems more likely to hinder my writing quality than improve it. i felt this way about inkhaven but never said anything because meh, it's just not my thing. however, your idea here seems to be that you want to help people produce outputs that are more valuable and not slop, and i do think the format is kinda directly at odds with that.
Much of the benefit probably comes from being somewhere with people doing the same and consciously shooting for a specific goal. On the consequences side, for many the social consequence of mildly dissapointing their friends, along with being able to show off to them or mak them happy, has a high motivation to negative valence ratio.
I think it's pretty fine/normal to produce slop alongside your "good" thinking.
A thing I dislike about Inkhaven is, it's sorta necessary to output some amount of "Inkslop", but, there's not a super clear distinction between "posts you shat out because you had to" and "posts that you really wanna promote as interesting."
I think there is totally a muscle to "keep it up" that I found useful even though I think I know how to think and write already. I think Inkhaven and Thinkhaven are both meant to work alongside a spirit-of-the-law intention to be trying to push yourself in some way.
For some people, just getting the words out is the bottleneck. If that's easy for you, focus on whatever the next skill in the chain you want to work.
I think I am a prime example of one that has a serious bottleneck in terms of getting things out of my head and on paper when I try. As you say, developing that ability requires some type of "muscles", just as being able to lift weights or do physically demanding activities. There is also the habit forming aspect. Like noted in one of the comments we forget to do things (often the new things we're trying to do) because they just don't fall into our normal behavior patterns[1].
I would also pose a question about slop concerns. Sure, not bad to avoid if one has a specific target they are aiming for I think but then if we look back as history, I wonder how many of the serendipitous discoveries or starting points wouldn't fall into a slop bucket when views from what the discover was actually trying to do.
- ^
Which in the Thinkhaven vein of at least one new thought a day, seem to lead towards "What don't I know?" What are the gaps in areas one does have some, even a lot, of knowledge about. Seems like a very positive behavioral pattern to cultivate.
All of this is only one possible structure for the underlying goal of "learn to relentlessly find new, useful thoughts every day."
Is that a learnable skill? Like I'm sure that if you spend a month deliberately blogging about interesting questions, your interesting thoughts per day metric will go up for that month, but you might worry about it reverting to baseline at the end of the exercise. Is there a specific reason to expect lasting skill gain or is this a handwavy "Maybe the brain is like a muscle and thinking about stuff makes you better at thinking in general"?
Yeah. Novelty is underrated. Spending even five minutes a day trying to think thoughts I haven't thought before is very fun, I realized it years ago, and I keep forgetting to do it.
Consolidating into a longer post isn't just pushing for effort or original thought or communicating it - it also pushes for 'related' thoughts. I don't think this should be a requirement of the program.
Suppose someone thinks up an original "base level" question every few days, and in intervening time after a base level question they think up some sub questions and make progress until by the end they 'answer' it.
But let's say it's not a big answer per question, and that there is not much connection between them. I then think a single long effort post per every two week would not be a right fit, while a few effort posts every two weeks would.
An example that comes to mind: Recently after hearing a related mathematically simpler (but conceptually lamer) question, I came up with an analogous question ("If a supersonic plane goes fast enough by you, you should hear sounds going both forwards and backwards, right?"), and spent perhaps a couple hours hitting it with math hammers until I was satisfied.
This has extremely little to do with other questions I think about. There are some follow up questions I can think of, but they aren't things I'm excited enough about to pursue instead of other stuff. The most promising place to go with it is to make a good animation and better plot, clean up the explanation, and then put it on Wikipedia. Notably, more effort would ideally lead to a shorter post, and hopefully it wouldn't take as many as 2.5k words.
The first half of this comment feels like it's making some kind of assumption that I was not holding. There are no requirements that any questions particularly relate to each other or that the effortposts should relate to what came before.
The second bit of "more effort might actually be shorter" / "the natural output might not be 'an essay'" does seem significant. I think my current answer is "yep, but, I challenge you to do that and also output a 2500 word essay."
I actually did something sorta-like this in my trial week. The output of the line of thinking I outlined in response to Kaj included both a ~2000 word post and also some attempts a good collection of diagrams and history-snippets with various nice UI features, where the writing was almost entirely AI generated.
I didn't actually end up with a thing I was satisfied with (I was trying to do this during my week of obligatory Lightcone Team Inkhaven-ing, and the structure didn't quite fit in a way that made it easy to finish). But, two attempts along the way were:
My concern about related things wasn't about relating two effort posts, but rather different things in a single effort post. That is, perhaps I have 5 cool things, best described in 500 words - should I make a "Grabbag of stuff" post?
This sounds like something that I'd find valuable and challenging. In particular, that 5-way breakdown of different kinds of styles of thinking was clarifying, especially in crystallizing ways of thinking I wasn't really treating as things in themselves. ("Try to fall in love with this research topic" was an explicit part of my plan for my work last summer. It... sort of worked.) Thanks for the post! If you ran this, I'd apply to participate, depending on scheduling and the price.
Inkhaven has people writing a blogpost a day for 30 days. I think this is a pretty great, straightforward exercise, that I'd definitely want in a hypothetical Rationality Undergraduate Program. But, it's not the only such exercise I'd want to include. It's gotten me excited for a different (superficially similar) program, which I might call "Thinkhaven."
In Thinkhaven, the goal is to learn the skill of "relentlessly think new, useful thoughts you haven't thought before."
Inkhaven had a basic "Goodhart-able" goal of "publish 500+ words every day." For Thinkhaven, I would imagine the Goodhart-y goal being something like:
And, somewhat more opinionatedly:
The spirit of the Goodhart is "are you finding new questions, and making some kind of progress on them?". Along the way, each day, you're thinking at least one new thought you haven't thought before.
One way of "thinking new thoughts" and "asking new questions" is to research stuff that already exists out there (i.e. learn some cool new facts about math/history/science/etc, and then write up good explainers for it, or connect it to other domains).
Another way is by thinking original thoughts, plumbing somehow through your confusions about the world and developing new concepts to help deal with them.
Presumably there are other approaches too, but I list those two to convey that that there's more than one way to go about this. The main thing we'd be trying to not do at Thinkhaven, is to "explain ideas that you've already thought about and just wish everyone else understood." That's also important, it's just not what Thinkhaven is for.
The daily journal is for accountability, to make sure you're making any kind of progress at all. The daily "new question", is to help ensure that progress has some kind of forward momentum, and is exploring new ideas.
The fortnightly 2500 word published writing is to close the loop on "get a new idea all the way from a vague musing, to something written up in a way that other people can critique." (Ideally, this explains some new ideas to the internet. If you didn't really get any new good ideas, you can write "well, I didn't come up with anything, but, here's a cleaned up version of my daily research notes.")
My primary inspiration for this is not actually Inkhaven, it's a period in 2022 where the Lightcone team focused on thinking/research/etc to try to get in touch with what it's like to be an original thinker on LessWrong. We set a goal of writing up a blogpost-worth-of-content per day, which the team would then read over each morning. Even without publishing externally, it was a useful forcing function to keep generating new thoughts and forcing them into a clearer shape.
I personally found it helpful for transitioning from "a guy who mostly defers to other people" to "a guy thinking his own strategic and intellectual thoughts."
Mentors, and Different Styles of Thinking
This is intended to be a fairly openended container. I'd expect to get value out of the pure container listed above, but, I'd ideally want a few different styles of mentors and coaches around, who embody different ways of thinking.
There are a few ways to operationalize that. You could model the thing more similar to MATS where everyone has a mentor they meet with on some cadence. If I were modeling it more on Inkhaven, I think some mentors would give classes, others might be more like mysterious old wizards you just go talk to.
All participants need to have at least one mentor who is enthusiastic about them (as part of the admissions process), but, they could sample from different styles of mentorship over the course of the month.
Possible examples of mentors:
Note: these are examples, not people who agreed to participate or even looked at this post. But they are some archetypes that I'm imagining. I'd be hoping for Thinkhaven to include a mix of mentors or "resident thinkers" with similar range.
John Wentworth-style, focused on tackling some confusing problems we don't understand, asking "what's the hard part?" / "what's the bottleneck?", and systematically making progress, while keeping an eye on Principles Which Will Carry Over To The Next Paradigm.
Logan Strohl-style, focused on openended, patient observation (with a kind of "open curiosity" as opposed to "active curiosity"). Trying to keep as close-to-the-metal on your observations. (See Intro to Naturalism: Orientation for a deep meditation on the sentence "Knowing the territory takes patient and direct observation.")
Elizabeth Van Nostrand-style, with some focus on open-ended "lit review"-ish research. Pick a new field you are curious about, read over lots of existing papers and books. See if you can synthesize some new takeaways that weren't obvious. Be ready to follow threads of information wherever they lead.
Scott Garrabrant-style, go live where the important math problems are, but then marry-for-love. Mull over interesting problems and then get nerdsniped on whatever feels alive.
Chris Olah-style, where... okay honestly I'm not actually sure how Chris Olah does his thinking and he seems particularly unlikely to come. But, reading over his older blogposts I get a sense of both a guy who likes studying lots of little fiddly patterns in the world and making sense of them, in a way that (vaguely) reminds me of an old timey biologist. And, a guy who likes experimenting with new ways of explaining things.
Thinking Assistants / Research Managers
The mentors above are selected for "I respect their thinking and writing."
They're not particularly selected for it being the right-use-of-their-time to help people through daily stumbling blocks, executive dysfunction, etc.
I would want some staff that are more like the research coaches at MATS, who meet with the people on some cadence to check on how things are going and help them resolve obstacles. And, I'd like to try out having dedicated Thinking Assistants available, who can sit with you for a chunk of time as you write or talk out loud through your problem, and notice little microhabits that might be worth paying more attention to.
"FAQ"
Everything above is the core idea. I'm not that confident in that particular format, and expect I'd change my mind about stuff after one iteration. But, here's some explanations of why I picked this structure instead of others, structured as an FAQ.[1]
Why require "a new question each day?"
I'm not sure this will work as well as I hope. But, my reasons are:
Sometimes, when you're exploring and stewing on a set of ideas, you're not really making progress, you're sort of going in circles, or building up some superficial understandings that don't really translate into a clear takeaway. Asking yourself new questions forces you to take your vague musings and confusions and turn them into questions with a meaningful return type.
It also pumps against "explaining ideas you've already thought about." (which again, is totally a useful way to write. It's just not what this program is for). By forcing yourself not to do something, you create space to practice new skills.
And, while it's opinionated on format, I think the "question" framing is still pretty openended as structures go.
What would asking new questions look like, in practice?
One person read the above and was like "okay I kinda get it, but I think I need to see an example of what this looks like to have a clearer sense of what this'd mean."
Here's an example.
(Note: this is just one example. As I just said, the program should be pretty unopinionated. Hopefully, if my line of questioning feels weird to you, it helps you imagine a version that would fit your thought process better).
I might start with a vague frustration/confusion:
I find it fruitful to ask more explicitly:
Which in turn prompt more specific questions like:
As well as meta/tactical questions like:
And then I might learn about domains where progress clearly accumulates, but a lot of it is driven by "taste." I might then spend a day digging into historical example of how people acquired or transmitted taste.
What should a "Daily Journal" look like?
The first answer is "whatever you want."
But, I did find, while beta testing this for myself this month, that it worked better when I gave myself a set of daily prompts to fill out, which looked like:
The "what did I learn?" section is the bit that ends up most shaped like a 500 word blogpost.
Rather than think of this as "the thing I scramble to write before the end of the day", it's more like a thing I write when I first get started in the morning. (I don't really like the "publish by midnight" thing that Inkhaven does, and I think I might want to actually set the deadline at lunchtime).
Another friend who beta-tested the format experimented with changing up the prompts, so that it worked better as an orienting process for them. (By default it felt a bit like a tacked-on-assignment they were doing out of obligation, but, slightly tweaked, it felt more naturally like a useful thing for them to do each day)
Are the daily journals public? Why?
I think so, but, not 100% sure.
(But, my default recommendation would be to put them on an out-of-the-way secondary blog, so you feel more free to think dumb thoughts along the way).
The reason to make them public is to help them function more as an accountability mechanism. You don't need to make a nice polished essay with a conclusion. But, you do need to get your thoughts to a point where they're structured enough someone else can make sense of them.
I considered just requiring them to be published internally to the Thinkhaven cohort. Habryka argued with me that this'd make people feel more like they were writing for the cohort-in-particular, having to care what those people thought, instead of getting to follow their own thought process.
The most important thing is you expect someone to be reading them.
Do we even need the 2500 word effortpost? Why can't it just be research journals all the way down?
Because the point of intellectual progress is to actually contribute to the sum of human knowledge. It's an important part of the process to package it up in a way that other people can understand and build on.
And, it's an important forcing-function that eventually your meandering question needs to turn into something that someone else would want to read.
Why "2500 words every 2 weeks" in particular?
Both of these are numbers I can imagine fine-tuning.
Why not "once a week?"
I thought "once a week" might be a better cadence, but, when I tried it out I found it too short.
During Inkhaven, where I was mostly focused on writing up existing ideas, I was able to write ~2000+ words a day and usually write one full post and make partial progress on an effortpost.
Thinking new meaningful/useful thoughts takes awhile, and sometimes it's important to get lost in the woods for awhile without knowing quite how everything will tie together. Or, just go off and gather a lot of information and digest it.
Why not longer?
I think "real work in the field" often does take more than 2 weeks at a time to output a blogpost worth of content. But, I think that's too slow a feedback loop for people learning. This is still supposed to be a class. I think it'd be hard for people to stay for longer than a month, and seems like people should get at least two reps in of "go from ideation -> publishing."
If this ended up being like a 3-month fellowship, I can imagine once-a-month being a reasonable cadence. But, I think it's just not that hard to turn 2 weeks of thinking into one substantial writeup.
If this were a 3-month fellowship, my current guess is I'd keep the 2-week effortpost but add in a Final Project that's aiming for the level of "significant contribution to whatever field you're exploring."
All of this is only one possible structure for the underlying goal of "learn to relentlessly find new, useful thoughts every day." But, it's a pretty simple structure I'd expect to do pretty well even in its minimal form.
Anyways, happy thinking.
These questions have all been asked at most "once" and sometimes "zero", so "frequently asked questions" is not exactly correct.