Kaj_Sotala comments on Evaluating the feasibility of SI's plan - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (186)
As for my own work for SI, I've been trying to avoid the assumption of there necessarily being a hard takeoff right away, and to somewhat push towards a direction that also considers the possibility of a safe singularity through an initial soft takeoff and more heuristic AGIs. (I do think that there will be a hard takeoff eventually, but an extended softer takeoff before it doesn't seem impossible.) E.g. this is from the most recent draft of the Responses to Catastrophic AGI Risk paper:
Here, "human-like architectures" also covers approaches such as OpenCog. To me, a two-pronged approach, both developing a formal theory of Friendliness, and trying to work with the folks who design heuristic AGIs to make them more safe, would seem like the best bet. Not only would it help to make the heuristic designs safer, it could also give SI folks the kinds of skills that would be useful in actually implementing their formally specified FAI later on.