Devising a procedure to figure out what to do in arbitrary situations is obviously even harder than creating a human-equivalent AI

Yes, blowing up the universe with an intelligence explosion is much easier than preserving human values.