I'm not sure what you mean by most of this. What is the "last part" and why does it seem wrong? Why do you hope that the idea presented in this story is correct? There seem to be too many factors to determine whether it would be better than some unknown alternative. What does this have to do with many worlds/mangled worlds? The story would still work in a classical universe.
The story makes two claims about decision theory. One of them explains the ending, hence "the last part". This claim leads to odd questions which cast doubt on it. (Note that the linked post itself links to a better trap in the comments.)
If the argument does work, it would apply to hypothetical entities with certain (highly desirable) powers in some forms of Many-Worlds. By "something like Mangled-Worlds" I meant a theory that restricts the set of Everett branches containing intelligent observers. Such a theory might assign P=0 to a par...
Suppose we could look into the future of our Everett branch and pick out those sub-branches in which humanity and/or human/moral values have survived past the Singularity in some form. What would we see if we then went backwards in time and look at how that happened? Here's an attempt to answer that question, or in other words to enumerate the not completely disastrous Singularity scenarios that seem to have non-negligible probability. Note that the question I'm asking here is distinct from "In what direction should we try to nudge the future?" (which I think logically ought to come second).
Sorry if this is too cryptic or compressed. I'm writing this mostly for my own future reference, but perhaps it could be expanded more if there is interest. And of course I'd welcome any scenarios that may be missing from this list.