I was able to answer part of my earlier question regarding O(3^^^3) by following the link to Torture vs Dust Specks. The '^' in '3^^^3' is the Knuth up-arrow notation and so '3^^^3' is standing in for "astronomically large number". Apologies for not reading that post before asking my question.

That said, if the O is meant to be the Big O notation then O(3^^^3) = O(1) which I'm sure isn't what was intended. An unadorned '3^^^3' works fine by itself to make the point.

Pedantry aside, I'm puzzled by other parts of this discussion too (perhaps because I'm late to the party). For instance, why is the (Friendly) AI always discussed in the singular? Might there not be value in several AIs, none of which are capable of complete self-reflection, keeping others in check. It seems that many of the ethical dilemmas discussed here are usually resolved by some appeal to a social element.