Could be a real problem, but this didn't happen to me. My brain recognizes only the specific alarm sound as important. Just make sure you don't use a sound that resembles anything else in the environment.
Yes, when that happened to me I was using a record of birds singing or something like that as the alarm.
This is the public group instrumental rationality diary for December 1-15.
Thanks to cata for starting the Group Rationality Diary posts, and to commenters for participating.
Immediate past diary: November 16-30