There has been previous discussion on LW on the topic of how to quickly determine if someone might be good at programming. This is relevant because this is currently a good career field that can be relatively easy to enter, and because programming-style thinking is often relevant to LW topics (eg decision theory). In light of this I've created the following test, which is based on my memory of a test from an interview process for a programming job. It attempts to test common low-level concepts from programming such as sequence, assignment, indirection, and recursion, in a way that doesn't require any previous programming experience (although previous experience will likely make it easier).
This test is aimed at getting a quick clear positive, so the fact that someone does poorly on it doesn't mean they can't become a programmer (ie I'd guess it's likely to generate false negatives rather than false positives). This test is obviously lacking scientific validation, and is probably too short, but I'd like to start somewhere.
I'd like to invite both programmers and non-programmers to take the test for comparison. It should only take about 5 minutes. If you do the test, please also take the short poll in the comments for feedback and calibration purposes, regardless of what result you got.
----- Test begins below -----
This is a 1-question algorithmic thinking exercise that should take less than 5 minutes.
Pen and paper is required. There should be no prerequisites beyond basic arithmetic.
First, write down the following sequence of numbered boxes. You will be writing numbers in some of the boxes more than once, so either use a pencil or make the boxes big enough to cross out and replace numbers.
1 2 3 4 5 6 7 8
[ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]
Following is a sequence of numbered steps. Do the steps in the order they are numbered (unless instructed otherwise). Note that "write a number in a box" means "cross out the previous number and write the new number".
1. Write 1 in box 3, 2 in box 6, 9 in box 4, 1 in box 5, 5 in box 8, and 0 in the remaining boxes.
2. In box 4, write the sum of the number in box 3 and the number in box 5.
3. In both boxes 2 and 5, write the the number in box 8 minus the number in box 6
4. Write 1 in the box whose number is in box 3
5. In box 3, write the sum of the number in box 3 and the number in box 4
6. In the box whose number is in box 6, write the sum of the number that's in the box whose number is in box 4, and the number that's in box 5.
7. Do step 2 again, then continue directly on to step 8.
8. Do step 4 again, but this time with box 4 instead of box 3, then continue directly to step 9.
9. The final result is the number that is in the box whose number is the number that is in the box whose number is equal to 2 plus the number that is in box 4. End of test.
--------------
Expected Results: http://pastebin.com/wA6xDxVb
Thanks for taking the test! Don't forget to answer the poll in the comments too.
I'd also appreciate any feedback on the test, both if you think its going in the right direction or not and if you think there are specific improvements that could be made.
edit: As some commenters have pointed out, there was a previous attempt at such a test that you may have heard of: http://www.eis.mdx.ac.uk/research/PhDArea/saeed/
However, it seems that further investigation found that their test, while better than nothing, wasn't very accurate. The test given in this post takes a different approach.
Concretely, I have seen this style of test (for want of better terms, natural language code emulation) used as a screening test by firms looking to find non-CS undergraduates who would be well suited to develop code.
In as much as this test targets indirection, it is comparatively easy to write tests which target data driven flow control or understanding state machines. In such a case you read from a fixed sequence and emit a string of outputs. For a plausible improvement, get the user to log the full sequence of writes, so that you can see on which instruction things go wrong.
There also seem to be aspects of coding which are not simply being technically careful about the formal function of code. The most salient to me would be taking an informally specified natural language problem and reducing it to operations one can actually do. Algorithmic / architectural thinking seems at least as rare as fastidiousness about code.