What actual choice is informed by considering this decision problem, what motivates posing it? (I don't know, which is why I originally ruled that out as a possible interpretation, and suggested a less straightforward interpretation that involved an actual choice.)
Consider economic development, population growth, technological development, basic science progress. My intuition is that all of these things are good, but that seems at odds with this analysis. If the new analysis is right, that would cause me to reconsider those valuations, which are relevant to things like "how useful is the science I do" or "how useful is improving the pace of scientific progress in general" or etc.
I think your interpretation at the end is correct.
I think your interpretation at the end is correct.
OK.
Your conclusion still leaves me confused. In any given choice considered by itself, the scale of value also doesn't make sense, you need to compare with something outside of that choice to say that the difference between the available options is insignificant. In a decision problem, you zoom in, not give up. So what is the difference between today and tomorrow insignificant in comparison with? Any personal-level change is much smaller, likely also much smaller than predictable differences in the world...
Over the last few months I've started blogging about effective altruism more broadly, rather than focusing on AI risk. I'm still focusing on abstract considerations and methodological issues, but I hope it is of interest to others here. Going forward I intend to cross-post more often to LW, but I thought I would post the backlog here anyway. With luck, I'll also have the opportunity to post more than bi-weekly.
I welcome thoughts, criticisms, etc.