LessWrong team member / moderator. I've been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I've been interested in improving my own epistemic standards and helping others to do so as well.
when you attempt to switch from thinking to doing, what happens instead?
Sort of inspired by Erik Jenner's post:
If you've been vaguely following me and thought "man, I wish Ray hurried up and finished making the update that <X>", what are some values of X?
Great writeup.
- Alice's team develops a major product without first checking to see if it's something people actually want -- after a year and a half of development, the product works great, but it turns out there isn't much of any demand. (I would consider this an observation failure -- failure to observe critical information leads to lots of wasted time.)
FYI I'd classify this more as a decision failure. They really would have had to take different actions in order to get this data, so this was more at the point when they were like "do I start building this product, or do I find some random representative users and see what they think of the idea?."
Also, since decision can flow pretty directly from orientation, you may find these two similar enough that you want to group them as one; I'm undecided on whether to make that change to this technique "more formally" and probably need to test it with more participants to see!
I actually normally combine/conflate Observe and Orient.
I think the actual takeaway here is: any two adjacent steps can kind of blend into each other.
You might be in a microloop where you're observing and orienting (and then maybe looking for more observations and then orienting on the new ones).
Then, when you're eventually like "okay I have enough observations", you may be in a loop where you're evaluating decisions, and then looking at your confused model and trying to wrangle the information into a form that's useful for decisionmaking, then look at your decision options again, be dissatisfied with your current ability to make-sense-of-things, and do more orienting.
Then eventually you're in a state where you know how to think about the situation, and you pretty much know what the options are, but as you start thinking about "Acting", your brain starts to see the consequences of each decision in near mode, which changes your guesses about which actions are best.
Then, as you start acting in earnest, each action comes with some immediate observations.
But, you can't really move from "Observe" to "Decide" without having gone through at least a little bit of an orient step on how to classify your observations.
I did a session yesterday with @moonlight, which went pretty well. I ended up consolidating some notes that seemed good to share with new assistants, and then he wrote the introduction he'd personally have preferred.
I generally work out of google docs that serve as shared-external-working-memory, with multiple tabs.
[Written by the first thinking assistant working with Ray, writing here what I’d have liked to read first]
Important things:
For the structure of this document:
Goal: End the acute risk period, and ensure a flourishing human future.
I’ve recently finished a bunch of grieving necessary to say “all right I’m ready to just level up into an Elon-Musk-but-with-empathy-and-cyborg-tools type”, as well as the minimum necessary pieces of a cognitive engine that (I think) is capable of doing so).
I want to be growing in capacity at an exponential rate, both in terms of my personal resources, and the resources available to the x-risk ecosystem that are accomplishing things I think need accomplishing.
This means having a number of resources that are compounding, that are synergistic, which include:
Things I actually do most days:
Things I would like you to do:
I would like to end up with a series of if-then habits you can help me execute. I will mostly write these myself, but as you get to know me well enough to say useful things, you can make suggested-edits
From “Hire or become a Thinking Assistant”
There are also important outside-the-container skillsets, such as:
Even the minimum bar (i.e. "attentive body double") here is a surprisingly skilled position. It requires gentleness/unobtrusiveness, attentiveness, a good vibe.
The skill ceiling, meanwhile, seems quite high. The most skilled versions of this are the sort of therapist or executive coach who would charge hundreds of dollars an hour. The sort of person who is really good at this tends to quickly find their ambitions outgrowing the role (same with good executive assistants, unfortunately).
Common problems I've run into:
Skill I’m working on that hasn’t paid off yet but I believe in:
This is the sort of thing I find appealing to believe, but I feel at least somewhat skeptical of. I notice a strong emotional pull to want this to be true (as well as an interesting counterbalancing emotional pull for it to not be true).
I don't think I've seen output from the people aspiring in this direction without being visibly quite smart to make me think "okay yeah it seems like it's on track in some sense."
I'd be interested in hearing more explicit cruxes from you about it.
I do think it's plausible than the "smart enough, creative enough, strong epistemics, independent, willing to spend years without legible output, exceptionally driven, and so on" are sufficient (if you're at least moderately-but-not-exceptionally-smart). Those are rare enough qualities that it doesn't necessarily feel like I'm getting a free lunch, if they turn out to be sufficient for groundbreaking pre-paradigmatic research. I agree the x-risk pipeline hasn't tried very hard to filter for and/or generate people with these qualities.
(well, okay, "smart enough" is doing a lot of work there, I assume from context you mean "pretty smart but not like genius smart")
But, I've only really seen you note positive examples, and this seems like the sort of thing that'd have a lot of survivorship bias. There can be tons of people obsessed, but not necessarily on the right things, and if you're not naturally the right cluster of obsessed + smart-in-the-right-way, I don't know whether trying to cultivate the obsession on purpose will really work.
I do nonetheless overall probably prefer people who have all your listed qualities, and who also either can:
a) self-fund to pursue the research without having to make it legible to others
b) somehow figure out a way to make it legible along the way
I probably prefer those people to tackle "the hard parts of alignment" over many other things they could be doing, but not overwhelmingly obviously (and I think it should come with a background awareness that they are making a gamble, and if they aren't the sort of person who must make that gamble due to their personality makeup, they should be prepared for the (mainline) outcome that it just doesn't work out)
I'd sort of naively guess doing it with a stranger (esp. one not even in your circles) would be easier on the "feeling private/anxious about your productivity" – does that feel like it wouldn't work?
Okay a few people have DMd me, and I'm feeling some kind of vague friction that feels currently on track to be a dealbreaker so let's think that through here.
Problems:
I have a vision of a whole-ass website dedicated to facilitating this but right now want a quick hacky solution.
A group DM would work, but that feels like it'll produce weird competitive dynamics with who replies first but maybe isn't as good as the person who replies second.
DMing a bunch of people individually I guess is fine but but then I need to go find them.
A requirement for everyone participating as an assistant is that they have a way of being contacted that they'll respond to quickly.
I've added lyrics to this post for now (if you expand each section)
Not sure I quite parsed, but things that makes me think of: