LessWrong team member / moderator. I've been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I've been interested in improving my own epistemic standards and helping others to do so as well.
Neat.
I think I have basically been trying to be meta-honest since the meta-honesty post came out... but, like, not trying super scrupulous committed about it, just, like, keeping it as the obviously-correct-thing to be aspiring to, and being more intentional about when/why to lie.
I didn't find that degree of effort super costly. (I think "specifically tracking meta-honesty" basically just didn't come up as a thing I had to do, because indeed people mostly don't ask questions about when I'm honest)
(You said there was a lot of mental overhead, and I'm not sure if this was more about being "meta-honest" or just "honest"?)
I do think some benefits accrue, not just to you, but, to the rationalists-and-associates as a whole for taking honesty seriously. I don't think the rest of the world cares about "meta-honesty" as an intellectual concept, but, it's a true fact that if you're the sort of person who takes honesty seriously you need to somehow handle the sorts of problems meta-honesty is designed to handle, so it's sort of part-and-parcel to accurately gaining a reputation for serious honesty.
Seems reasonable split, although I try to gesture at / share compressed versions of the background knowledge.
Yeah, I asked for this split precisely because, usually with a LessWrong post I already have at least the gist of the background knowledge, and what I really want to know is "what is the new stuff here?".
But yeah I like the dream of "keep track of the stuff you know, and explain the diff between what you know." But I think for the immediate future, being able to see at a glance "okay, what background context might I not have that, if I'm lost, I might want to read up on separately?"
I feel like I want here is a campaign to make sure history remembers the specific people who let this happen – names of board members, Attorney General Kathy Jennings, etc.
It feels achievable and correct to me for this to be a thing where, if you're going to do this, a lot of people associate your name with enabling the theft.
Do you have own off-the-cuff guesses about how you'd tackle the short feedbackloops problem?
Also, is it more like we don't know how to do short feedbackloops, or more like we don't even know how to do long/expensive loops?
(Putting the previous Wei Dai answer to What are the open problems in Human Rationality? for easy reference, which seemed like it might contain relevant stuff)
I think I meant a more practical / next-steps-generating answer.
I don't think "academia is corrupted" is a bottleneck for a rationalist Get Gud At Philosophy project. We can just route around academia.
The sorts of things I was imagining might be things like "figure out how to teach a particular skill" (or "identify particular skills that need teaching", or "figure out how test whether someone has a particular skill), or "solve some particular unsolved conceptual problem(s) that you expect to unlock much easier progress."
In your mind what are the biggest bottlenecks/issues in "making fast, philosophically competent alignment researchers?"
In this case I spot checked a few random strings from it.
For my personal browsing AI prompt-library-tool I use, it has the ability to click on a highlight, and scroll to that corresponding paragraph. It fails to work if there are any errors (although usually the errors are just "slightly different punctuation"), so it's actually pretty easy to click through and verify.
But if I were building this into an reliably tool I wanted to use at scale, I'd follow it up with a dumb script that checks if the entire paragraphs match, and if not, if there are random subparts that match a given paragraph from the original content, and then reconcile them. (the sort of thing I'm imagining here is a thing that generates interesting highlights from LessWrong posts and some scaffolding for figuring out if you need to read prerequisites)
Yeah that all makes sense.
I'm curious what you say about "which are the specific problems (if any) where you specifically think 'we really need to have solved philosophy / improved-a-lot-at-metaphilosophy' to have a decent shot at solving this?'"
(as opposed to, well, generally it sounds good to be good at solving confusing problems, and we do expect to have some confusing problems to solve, but, like, we might pretty quickly figure out 'oh, the problem is actually shaped like <some paradigmatic system>' and then deal with it?)
Some specific things that have come up for me, some of which are about "metahonesty-qua-metahonesty" and some is just... idk, getting more intentional about honesty.
Honesty oaths
Sometimes, I think someone is maybe lying to me (usually in a fun prankstery way, sometimes more importantly). Sometimes, when it's actually pretty important to me to know if it's a lie, I ask "do you swear that's true upon your honor as a guy who cares about being able to credibly coordinate about things truthfully sometimes?"
and sometimes they say "yes" and sometimes they say "no, because I have a general policy of not being pressured by that sort of question" and sometimes they say "hmm, I'm not sure whether I should glommarize here."
sometimes, I instead say "do you swear on your honor as a guy who cares about credibly coordinating but also cares about fun pranks and gets, a few free passes on lying in answer to this sorta question?"
Mostly this has just come up for fun, but I like it as social tech.
"Honest relationships", vs "Other random kinds of relationships"
Generally, I want to have honest relationships with my close friends. But, there's a cluster of people (usually but not always non-rationalists, old friends, family, etc) who seem like they just don't actually want a relationship with rationalist-levels-of-honesty. If I were fully honest with them they'd be annoyed or sad, and it doesn't seem like this even bothers them. Mostly I still don't lie to them, I just am not as open and don't correct all inaccurate assumptions, but I lie sometimes.