I got the same piece of advice - to think about things in terms of "wants" rather than "shoulds" or "have tos" - from someone outside the LW bubble, but in the context of things like doing my tax returns.
She teaches social skills to nerds at CFAR workshops. She has an incredibly positive view of humanity and of what people are capable of, and meeting her massively increased the size of my reference class for what a rational person is like.
LW Women Submissions
a call for anonymous submissions by the women on LW
Seven women submitted
uh... could this be rephrased?
I'm a very satisfied customer from the March workshop. The biggest win for me has been with social skills - it turns out that anxiety had been making me stupid, and that if I de-stress then whole new parts of my brain spring into action. And that was just one of a large number of practical insights. I was amazed at both how much CFAR know about how we can use our brains effectively, and at how much they were able to teach over 4 days. Really impressive, well-run effort with a buzz of "this is where it's happening".
I promised I'd write about this in more detail, so stay tuned!
The best thing about this was that there was very little status dynamic within the CFAR house - we were all learning together as equals.
Agree with purchasing non-sketchiness signalling and utilons separately. This is especially important if like jkaufman a lot of your value comes as an effective altruist role model
Agree that if diversification is the only way to get the elephant to part with its money then it might make sense.
Similarly, if you give all your donations to a single risky organization and they turn out to be incompetent then it might demotivate your future self. So you should hedge against this, which again can be done separately from purchasing the highest-expected-value thing.
Confused about what to do if we know we're in a situation where we're behaving far from rational agents but aren't sure exactly how. I think this is the case with purchasing xrisk reduction, and with failure to reach Aumann agreement between aspiring effective altruists. To what extent do the rules still apply?
Lots of valid reasons for diversification can also serve as handy rationalizations. Diversification feels like the right thing to do - and hey, here are the reasons why! I feel like diversification should feel like the wrong thing to do, and then possibly we should do it anyway but sort of grudgingly.
I can imagine having a preference for saving at least X lives
I feel like you've got a point here but I'm not quite getting it. Our preferences are defined over outcomes, and I struggle to see how "saving X lives" can be seen as an outcome - I see outcomes more along the lines of "X number of people are born and then die at age 5, Y number of people are born and then die at age 70". You can't necessarily point to any individual and say whether or not they were "saved".
I generally think of "the utility of saving 6 lives" as a shorthand for something like "the difference in utility between (X people die at age 5, Y people die at age 70) and (X-6 people die at age 5, Y+6 people die at age 70)".
We'd have to use more precise language if that utility varies a lot for different choices of X and Y, of course.
I felt like I gained one insight, which I attempted to summarize in my own words in this comment.
It also slightly brought into focus for me the distinction between "theoretical decision processes I can fantasize about implementing" and "decision processes I can implement in practice by making minor tweaks to my brain's software". The first set can include self-less models such as paperclip maximization or optimizing those branches where I win the lottery and ignoring the rest. It's possible that in the second set a notion of self just keeps bubbling up whatever you do.
One and a half insights is pretty good going, especially on a tough topic like this one. Because of inferential distance, what feels like 10 insights to you will feel like 1 insight to me - it's like you're supplying some of the missing pieces to your own jigsaw puzzle, but in my puzzle the pieces are a different shape.
So yeah, keep hacking away at the edges!
I can imagine that if you design an agent by starting off with a reinforcement learner, and then bolting some model-based planning stuff on the side, then the model will necessarily need to tag one of its objects as "self". Otherwise the reinforcement part would have trouble telling the model-based part what it's supposed to be optimizing for.
I've got a few people interested in an effective altruism version of this, plus a small database of cards. Suggestions on how to proceed?