[LINK] Computer program that aces 'guess next' in IQ test

Dmytry

From

http://astrobio.net/pressrelease/4569/computers-that-think-like-humans

The group is therefore using a psychological model of human patterns in their computer programmes. They have integrated a mathematical model that models human-like problem solving. The programme that solves progressive matrices scores IQ 100 and has the unique ability of being able to solve the problems without having access to any response alternatives. The group has improved the programme that specialises in number sequences to the point where it is now able to ace the tests, implying an IQ of at least 150.

'Our programmes are beating the conventional math programmes because we are combining mathematics and psychology. Our method can potentially be used to identify patterns in any data with a psychological component, such as financial data. But it is not as good at finding patterns in more science-type data, such as weather data, since then the human psyche is not involved,' says Strannegård.

That's an awesome study.

I always thought the variations of continue series test (progressive matrices, number sequences, word A is to word B as word C is to ?? etc) are very culturally biased. You solve those best and easiest by sharing with the test maker the learning environment (and for visual ones, sharing visual environment), as well as sharing neural architecture. That lets you pick same choice as the test maker [edit: and do so easily and naturally]. And this research provides very good demonstration.

Of course there will be a correlation of ability to guess the same or secondguess the test maker with intelligence, but so does e.g. height correlate with intelligence (via effect of nutrition on both); perhaps we should add 'what is your height' question to IQ test and then let some giant robot score a genius.

Note: one might think of sequence guessing as task of minimizing Kolmogorov complexity. That's not quite so, sequences are too short, shorter than the generators. Consider sequence 2,3,5,7,11,? . Obviously the answer on IQ test would be 13 (primes). Good luck writing primes generating program that is simpler than this sequence itself, though [edit: i mean, simpler than a program which just prints those numbers followed by whatever garbage. Unless you have a language where 'print primes' is a basic command]. (and of course the length of program will be very dependent on the machine being used)

From

http://astrobio.net/pressrelease/4569/computers-that-think-like-humans

The group is therefore using a psychological model of human patterns in their computer programmes. They have integrated a mathematical model that models human-like problem solving. The programme that solves progressive matrices scores IQ 100 and has the unique ability of being able to solve the problems without having access to any response alternatives. The group has improved the programme that specialises in number sequences to the point where it is now able to ace the tests, implying an IQ of at least 150.

'Our programmes are beating the conventional math programmes because we are combining mathematics and psychology. Our method can potentially be used to identify patterns in any data with a psychological component, such as financial data. But it is not as good at finding patterns in more science-type data, such as weather data, since then the human psyche is not involved,' says Strannegård.

That's an awesome study.

If we're told that the data is a rule-generated sequence, then programs that are just lists can be excluded. That's a conclusion from the data, not from the prior.

Plus, if you come across, out of nowhere, the sequence of prime numbers up to 11, what probability ought you assign that it is the sequence of primes instead of arbitrary numbers? I think the overhead of a computer program in comparison to a statement of mathematical requirement may be too great to give a realistic estimate. In particular, generating a nonexistence test in code carries a lot of overhead, while in math it's quite compact.

Just listing them, we have "2,3,5,7,11"

That's 10 symbols, if we let 11 be considered 1 symbol (its short enough that a reasonable representation would leave it at its present length), but leave the commas as delimiters.

Via mathematical criterion, "a:!b,c>1|b*c=a"

That's 14 characters, of which 2 can be considered overhead (could leave 'a:' implicit). Note, we dropped order information here. I suppose in the event of non-uniformly-increasing sequences, we'll need the spot used here for 'a:' for something else.

Now, code. Count each line as 2 characters, and I'm completely neglecting any runtime optimization.

Assign register 1 the value 2
(label NEXT) Assign register 2 the value 2.
(label CANDIDATE) test for equality between register 2 and register 1
(on success) jump to KEEP
Assign register 3 the value in register 1
apply modulo base register 2 to register 3
test for register 3 being 0.
(on success) jump to NEXT while incrementing register 1
jump to CANDIDATE while incrementing register 2
(label KEEP) output register 1
jump to NEXT while incrementing register 1

22 'characters'

It may be possible to trim a few more away, if the instruction set is optimized for efficient short-range jumps. If the test commands can include longer jumps than 1 step in them (e.g. a skip-2 test and a go-back-4 test) then you get to save 2 lines. If the modulo command takes 3 register arguments, you can save one more line.

With all of these, we're down to 16 characters, still 4 more than the briefer variant of the mathematical criterion. There's no way 'ascending order' counts for 4 characters, or even 2 for the longer variant (since those 2 characters could be taken to specify that), to make up that difference.

Yet... I really suspect that if you run across the sequence "2, 3, 5, 7, 11" with no context other than that 2 was the beginning of the sequence, you ought to assign more than a 1/(sizeof char)^2 probability that it's the sequence of primes - that would be somewhere around 0.0001 probability.

(edited: mathematical statement of primality did not exclude factors of 1)

Well, I dunno.

If I had random C program print me out 2,3,5,7,11 , I would still assume VERY low probability that it is going to print out primes correctly up to reasonably big number. Even more so for Turing machines or anything of this kind. Ditto for any natural processes. Ditto for seeing those numbers in idk child doodling when child didn't learn the primes yet. Child might have invented primes but that list is not remotely enough evidence.

If a human writer of a test tells me this sequence, I would guess that he knows of primes, and is telling primes t... (read more)

1kpreid14y

I think if you want to write symbol-short programs for small problems, you should probably look at models which are more composition-based, so as to remove the identifiers/register names which are not really part of the complexity of the problem. To name some actual existant programming languages, (APL or J) or (Forth or Factor). APL in particular uses single-character symbols in a character set designed for the language. Then there's compression. If you want to keep it reasonably human-understandable and compositional, how about Huffman-coding the set of symbols?