Alfred Harwood and I were working through this as part of a Dovetail project and unfortunately I think we’ve found a mistake. The Taylor expansion in Step 2 has the 3rd order term o(δ3)=16[2(√P[X])3](−δ[X])3. This term should disappear as δ[X] goes to zero, but this is only true if √P[X] stays constant. The Γ transformation in Part 1 reduces (most terms of) P[X] and Q[X] at the same rate, so √P[X] decreases at the same rate as δ[X]. So the 2nd order approximation isn’t valid. For example, we could consider two binary random variables with probability distributions P(x=0)=zp and P(X=1)=1−zp and Q(X=0)=zq and Q(X=1)=1−zq. If δ[X]=√P(X)−√Q(X), then δ[X]→0 as z→0. But consider the third order term for X=0 which is 13(√Q(0)−√P(0)√P(0))3=13(√zq−√zp√zp)3=13(√q−√p√p)3 This is a constant term which does not vanish as z→0. We found a counterexample to the whole theorem (which is what led to us finding this mistake), which has KL(X2→X1→Λ′)max[KL(X1→X2→Λ),KL(X2→X1→Λ)]>10, and it can be found in this colab. There are some stronger counterexamples at the bottom as well. We used sympy because we were getting occasional floating point errors with numpy. Sorry to bring bad news! We’re going to keep working on this over the next 7 weeks, so hopefully we’ll find a way to prove a looser bound. Please let us know if you find one before us!

Your process description sounds right (like, the thing I would aspire to, although I don't consistently do it – in particular, I've identified it'd be good if I did more automated testing, but haven't built that into my flow yet).  But, you don't really spell out the "and, here's why I'm pretty confident this is a 5x improvement."  A few months ago I'd have been more open to just buying the "well, I seem to be shipping a lot of complex stuff", but, after the METR "turns out a lot of devs in our study were wrong and were actually slowed down, not sped up", it seems worth being more skeptical about it.  What are the observations that lead you to think you're 5x? (also, 5x is a somewhat specific number, do you mean more like 'observations suggest it's specifically around 5x' or more like 'it seems like a significant speedup, but I can tell I'm still worse than the '10x' programmers around me, and, idk, eyeballing it as in the middle?) (I don't mean this to be like, super critical or judgmental, just want to get a sense of the state of your evidence)

> My mom didn’t buy it. “This is all sounding a bit crazy, Taylor,” she said to me. And she’s usually primed to believe whatever I say, because she knows I’m smart. > > The problem is that these stories are not believable. True, maybe, but not easy to believe. They fail the “mom test”. Only hyper-logical nerds can believe arguments that sound like sci-fi. Maybe only hyper-logical nerds can believe arguments that sound like sci-fi, but your mom only has to believe you. The question is whether you are believable, or whether you're "starting to sound a bit crazy, Taylor". That's her sign to you that you need to show that you can appreciate how crazy it sounds and maintain your belief. Because it does sound a bit crazy. It's quite a leap from demonstrated reality, and most of the time people are making such leaps they're doing fiction/delusion and not actually calling things right in advance. The track record of people saying crazy shit and then insisting "It's not crazy I swear!" isn't good. If instead, you meet her where she's at and admit "Yeah. I know. I wish it was", it hits differently. I can't remember if I've talked to my mom about it, but if I had to talk to her about it, I'd probably say something like "You hear of the idea that AGI is going to be completely transformative, and will have the power to kill us all? Yeah, that's likely real", and she'd probably say something like "Oh.". That's basically how it went when I told her the world was about to change due to the upcoming pandemic. I didn't "try to persuade her" by giving her arguments that she's supposed to buy, let alone spinning stories about how a bat had a virus and then these researchers genetically modified it to better attack humans. I just told her "Here's what I believe to be true", so that she could prepare. I was open to why it was that I believed it, but the heavy lifting was done by the fact that I genuinely believed it and I came off more like I was trying to share information so that she could prepare than like I was trying to convince her of anything. In your shoes, besides making sure to acknowledge her point that it sounds crazy, I'd do a lot of genuine curiosity about her perspective. Has she ever experienced something that sounded crazy as fuck, and then turned out to be real? Not as a rhetorical question, just trying to understand where she's coming from. Is she aware of the massive impact drones are having in the war in Ukraine? Has she thought about what it felt like to be warned of the power of nuclear weapons before anyone had seen them demonstrated? These aren't "rhetorical questions", asked as ways of disguising a push for "Then you should stop being so confident!" but as a genuine inquiry. Maybe she has experienced something "crazy" turning out to be real, and noticing will change her mind. Or maybe she hasn't. Or maybe it seems different to her, and learning in what way it seems different will be relevant for continuing towards resolving the disagreement. Giving people the space to share and examine their perspective without pressure is what allows people to have the experiences that shift views. Maybe she hasn't had the experience of running from a terminator drone, or being outsmarted at every turn, but you could give her that experience -- by pointing out the shared starting point and asking her to imagine where that goes. She'd still have to take you up on that invitation, of course. If I'm wrong about being able to convince my own mom in a single line, it'd be for this reason. Maybe the idea would freak her out so much that she would be motivated to not understand. I don't think she would, but maybe. And if so, that's a very different kind of problem that you deal with by making arguments which are "more believable".