Seth Roberts used a balance test to measure cognitive functioning to help determine whether various cognition-enhancing supplements were working. He would stand on a small platform and measure the time before he fell off, repeated like 5 times. One of his readers came up with another measure--arithmetic drills. A sheet full of single-digit addition or multiplication problems. The measure, of course, is time to completion.
I have a version of Tetris I think might be useful towards this end. In my version of the game, the speed increases when you clear lines and decreases when you stack them up, (and the lines shift down when you fill up the well so there's no game over,) so your average speed can indicate the level of attention you can sustain. If anyone's interested I can publish it. (Developed on Windows, might possibly work on Linux, or even OSX.) OTOH, I'm not completely sure how well this measure works, because there seems to be a significant degree of variation within a single, short playing session. A good test would have little random variation in the results of immediately repeated tests. You'd probably want to do some experiments to see how long the session would have to be to reach a suitably low variation.
The problem with all of the suggestions in the thread so far is that even if they correlate well with one's mental performance in general, they correlate much less well with individual tasks. Particular tests might be more correlated to particular tasks than others, but unless you know how correlated your test and task are, you won't be able to draw any good conclusions. Ideally one would have some sort of factor analysis that one could categorize tests and tasks in, so you could pick the best matching test for your intended task.
I'd wager the correlation between performance of different tasks in various mental states for a person over a few weeks or months (excluding skill improvements) is probably around .3 - .6.
We all have our good days and our bad days. Due to insufficient sleep, illness, stress, distractions, and many other causes we often find ourselves far below our usual levels of mental performance. When we find ourselves in such a state, it's not really worth putting effort in doing many tasks, like programming or long term planning - as quality will suffer a lot.
The problem is - other than observing deterioration of results, I have no idea if I'm in such a state or not. I cannot be sure if it's also true for others, but I had to find out a few tests of what's my mental performance at the moment. Tests that are deeply flawed, so I'd request better if there are any. I also cannot predict my mental state in advance, as my life isn't terribly regular.
The most reliable test I found, and by accident, was fighting bots on a certain Quake 3 map - me vs 10 or so highest difficulty bots. The challenge was to get 50 frags without dying. As the map was huge and full of power ups, it wasn't really that difficult as long as I could maintain full alertness for 10-15 minutes - but if I was tired or distracted, I would invariably fail. This test was unfortunately extremely slow.
Another test would be to go to goproblems, and do a few random problems at proper difficulty level. If I could think right, I would do most of them, if I was tired, I would fail almost 100%. This didn't test alertness, I guess it would be best described as short term memory test, as that's what used for game tree exploration. Unfortunately what's the proper difficulty varies a lot with how much go I played recently, so it needs to be recalibrated.
One more test would be to go to some decent online IQ test like this one. My results on such test would suffer a lot if I was sleepy or tired. The main problem is that such tests cannot repeated too often, or I'd just remember the answers.
So these are three ways to test how well my mind functions at the moment, all testing something different, and all flawed in one way or another.
How do you test yourself?