Sometimes I'll even apparently remember that I have a dream in which an alarm goes off in my dream, which wakes me up, and then five seconds later the alarm goes off.

When I set my alarm, I routinely (almost infallibly unless I set it for before 5:00 am) wake up before the alarm and turn it off before it has a chance to ring. I simply hate being woken by an alarm so much that I keep track of the time, even while asleep, in order to prevent such a traumatic experience from occurring.

It might be that you really were dreaming about your alarm going off before it actually went off, due to anticipation.