It depends. Writing a paper is not a realtime activity. Answering a free-response question can be. Proving a complex theorem is not a realtime activity, solving a basic math problem can be. It's a matter of calibrating the question difficulty so that is can be answered within the (soft) time-limits of an interview. Part of that calibration is letting the applicant "choose their weapon". Another part of it is letting them use the internet to look up anything they need to.

Our lead dev has passed this test, as has my summer grad student. There are two applicants being called back for second interviews (but the position is still open and it is not too late) who passed during their first interviews. Just to make sure, I first gave it to my 14 year old son and he nailed it in under half an hour.