A rationalist interjects: “You should make public predictions about this stuff!” Idk, should I? What should I make predictions about? About whether individual cases succeed, or some broader trends? I'm not sure if it’s worth my time. I really like $ as a metric, not sure what the predictions add. Very open to being convinced here!
Predictions about individual cases would be great. Whenever you take a deposit write down the condition for the bounty being paid out, the amount of the bounty, and your self-assessed likelihood of the person paying the bounty in the following twelve months to you into a public Google Sheet. Maybe, add another row for "time-spent with the person".
The exercise about thinking beforehand about how likely you will solve the issue for the person is useful for you to understand your method better. It also help informing potential customers well about what they can expect from your service.
Finally, it would be great to have a one-year follow-up after a bounty is paid and that information also added to the Google Sheet.
What are the failure rates? So, I would love to share data on the cases I haven’t (yet) been able to help… but I don’t know how?
1) How many cumulative hours have you spent on things where there has been no success and you guys aren't working together anymore on the issue? How does this compare to the number of hours which have resulted in success, and the number where the result is tbd? How many hours have resulted in partial or incremental success, without meeting agreed upon win criteria?
2) Of those where someone bailed how many times did they bail and how many times did you bail? There's some ambiguity here, but probably manageable. If you don't expect to hear back (e.g. because it's been two months), then count it as a bail. If you suggest that their problem isn't in your wheel house and they say "okay" rather than asking to try anyway, I'd count that one as on you.
3) To what extent have you "failed" because the initial goal turned out to be meaningfully mis-specified? E.g. someone wants to be more socially active in a certain group, only to realize their aversion to socializing in that group is actually well grounded, and they no longer want to achieve their initial goal?
4) To what extent have you caused problems by being too successful for the specified goal? E.g. The person actually ends up active in that social circle before realizing that they've been wasting their time doing so. Or maybe you help someone be more secure and they're happy for it, but it did lead to them losing a relationship when they spoke a little too freely.
Please, tell me what metric I should use here!
Is it feasible to just generate a bunch of such metrics, with details about what was included or not included in a particular number, and share all of them?
could you give a few examples?
also seems time-intensive hmmmm
also, i thought about it more and i really like the metric of "results generated per hour"
I think you've already given several examples:
Should I count the people I spoke to for 15 minutes for free at the imbue potlucks? That was year-changing for at least one. But if I count them I have to count all of the free people ever, even those who were uninvested. Then people will respond “Okok, how many bounties have you taken on?” Ok sure, but should I include the people who I told “Your case is not my specialty, idk if i’ll be able to help, but I'm interested in trying for a few hours if you’re into it”? Should I include the people who had an amazing session or two but haven’t communicated in two months? Should I include the people who are being really unagentic and slow?
It would already be informative if you put numbers on each of these questions (i.e. "how often does talking for 15 minutes accomplish something", "how many bounties have you taken on in/outside of your specialty", "what percent of your clients are 'unagentic and slow' (and what does this actually mean)"). Probably one could do much better by generating several metrics that one would expect to be most useful (or top N%tile useful) and share each of them.
oh ok hm. i also don't want to be incentivized to not give easy-for-me help to people with low odds of success though
Disclaimer : I would not pay and want to pay that much money anyway - so I am not your intended audience
I'd trust you more (and I would think members of the rationalist community would too) if you gave several metrics, even if some of them are not so good, with explanations. Right now, it seems you chose a metric so that it looks good.
More metrics would take more time but not much if you have the data easily available. This would be my suggestion :
You can provide three percentages ( like when one provides three quantiles instead of just the mean of data values) :
- the percentage of success in people you discussed for at least an hour
- the percentage among the people with reasonable chances of success (motivated + didn't bail + your expertise + spent at least X hours)
- the percentage among people with great chances of success.
These percentages, with precise information on what determines in which category clients fall in and the percentage of people treated who fall into each category, would give a first sound idea of the success rate.
Taking on low success rate people would not be a problem because their data is treated separately. It's only a problem if 90% of your clients are unlikely to be helped but that would not be a good thing anyway.
Previously: Pay-on-results personal growth: first success
To validate my research, I began offering pay-on-results coaching in July. Clients have paid $40,300 so far upon achieving their goals:
(I spend just ~8 hours per week coaching.)
An innovation here is a way to measure results: Results are measured by bounties paid by clients one or more months after, in my bank account, since I started. Clients set a bounty in advance (e.g.: “If I learn to feel secure in approximately all social situations by working with you, I will pay $25k.”), then choose whether to pay, when to pay, and how much to pay.
This way, when someone pays, it’s good evidence that real growth occurred.
The rest of this post is rough takes written quickly:
Why did I start doing bounties?
Reason 1) Wanted to know if my research actually works
If I were to charge on a per-hour basis (like ~every other coach), it would be harder to know whether the models of growth I've developed actually work. Basically: “Is the client’s life legitimately changing for the better, or do they just like talking to me?”
I also knew that people would generally be more hesitant to pay 4- (now 5-) figure sums than 3- figure sums, especially if they had to wait multiple months to pay and they’re the one who decides success.
Reason 2) Wanted success cases to produce legible evidence to onlookers
When you read the list of results at the start of this post, then (as long as you believe me) you probably thought “Wow.” They’re specific; you can tell stuff actually happened.
Meanwhile, most other coaching pages have extremely vague reviews: “They were nice, they are cool, I had fun, it was ‘transformative’. Big wow! 10/10.” And maybe growth did happen, but there isn’t actually much signal!
(That said, maybe we’re just optimizing for different things. I’m optimizing for what I mean when I say “results”. But I wouldn’t be surprised if lots of coaching is more like “having a smart friend to talk to”. And that’s ok!)
A partial way to address this is to get clients to be more specific, and I do that too… But also I'm not sure good reviews can ever rule out “Maybe the client is just trying to be nice and please the coach?”
This is where bounties come in: If you heard that someone paid a surprisingly large sum of money — an amount that is likely a 1–10% of their net worth — and you learned that they did so months-later retroactively and essentially voluntarily… that gets your attention!
(I'd guess my median client has a net worth on the order of $200k?)
Money is real. Money is skin in the game! Generally, people try to not give up a significant percentage of their power over the world to anything unless they believe it’s worthwhile.
Reason 3) Wanted to make more money
I thought I had something that could, at least in some cases, resolve lifelong anxiety in a single conversation (yes, see below) … and, I wanted more than 3 figures for that!
Reason 4) Wanted better incentive-alignment
Even if I didn't have the ability to do one-session magic yet, I wanted to be incentivized to develop it.
Coaches who charge by the hour don’t actually have that much financial incentive to resolve issues in a session when it’s possible. Frankly it’s the opposite: they get less money if they do that.
(You may think “But if they were just that good, then they could just raise their rates a ton and still get tons of clients!” But in practice I don’t think so: the marketing would be hard. Also no one would believe them. See Reason #2 again. Would you believe the results I put at the beginning of this post without the $$ citations?)
Reason 5) Who was going to pay me by the hour, anyway?
I had no experience! I didn't want to spend months or years building a reputation for skills I thought I already had.
(I am not a licensed psychologist, psychiatrist, therapist, or medical professional. Not therapy or medical treatment.)
Reasons 6+) Probably others I’ve forgotten.
EDIT: a new version of this post will be released on Jan 2.
https://x.com/ChrisChipMonk/status/1873101046199038301
Thanks to Stag Lynn, Anna Salmon, CFAR, Ethan Kuntz, Alex Zhu, Brian Toomey, Kaj Sotala, Damon Sasi, and lots more for mentorship and help.
@ChrisChipmonk, chrislakin.com/now
Place a bounty: chrislakin.com/bounty