'Unfortunately, Lob's Theorem demonstrates that if we could prove the above within PA, then PA would prove 1 + 2 = 5'.

I believe that that's just a statement of Löb's theorem, and the rest of the Cartoon Guide is a proof.

It says earlier that Godel developed a system for this, but the theorem doesn't seem to explain that system

The exact details aren't important, but Gödel came up with a way of using a system that talks about natural numbers to talk about things like proofs. As Wikipedia puts it:

Thus, in a formal theory such as Peano arithmetic in which one can make statements about numbers and their arithmetical relationships to each other, one can use a Gödel numbering to indirectly make statements about the theory itself.

Actually, going through a proof (it doesn't need to be formal) of Gödel's incompleteness theorem(s) would probably be good background to have for the Cartoon Guide. The one I read long ago was the one in Gödel, Escher, Bach; someone else might be able to recommend a good one that's available online not embedded in a book (although you should read GEB at some point anyway).

arrows and little squares

The rightward-pointing arrows mean "If [thing to the left of the arrow] then [thing to the right of the arrow]". E.g. if A stands for "Socrates is drinking hemlock" and B stands for "Socrates will die" then "A -> B" means "If Socrates is drinking hemlock then Socrates will die".

I suspect the squares were originally some other symbol when this was first posted on Overcoming Bias, and they got messed up when it was moved here [Edit: nope, they're supposed to be squares], but in any case, here they mean "[thing to the right of the square] is provable". And the parentheses are just used for grouping, like in algebra.