I just did some reading about "Sociosexual Orientation Inventory", a simple 7-item test designed to measure one's openness to sex without love and long term commitment.

Here are the questions. How long will it take you to spot the huge problem ahead...

With how many different partners have you had sex (sexual intercourse) within the last year.
How many different partners do you foresee yourself having sex with during the next five years? (Please give a specific, realistic estimate)
With how many different partners have you had sex on one and only one occasion?
How often do (did) you fantasize about having sex with someone other than your current (most recent) dating partner? (1 never ... 8 at least once a day)
"Sex without love is OK" (1 strongly disagree ... 9 stronly agree)
"I can imagine myself being comfortable and enjoying `casual' sex with different partners (1 strongly disagree ... 9 stronly agree)
"I would have to be closely attached to someone (both emotionally and psychologially) before I could feel comfortable and fully enjoy having sex with him or her" (1 strongly disagree ... 9 stronly agree)

Score is: 5 x item1 + 1 x item2 (capped at 30) + 5 x item3 + 4 x item4 + 2 x (mean of item5, item6, and reversed item7)

Do you see the problem already?

Quite predictably, researchers report that men have much higher SOI scores than women in all countries. But the first three questions (ignoring non-1:1 gender ratios, differently biased sampling for different genders, different rates of homosexuality between genders, different behaviour of homosexuals of different genders, 30 partners cap on the second item, differently biased forecasts of the second item and other small details that won't affect the score much) - simply have to be identical for men and women, so the entire difference would have to be explained by items 4 to 7, which have relatively low weights!

The differences between men and women can be really extreme for some countries, Ukraine has 50.79±28.92 (mean±sd) for men, and 17.36±8.65 for women, which means that either Ukrainian men, or Ukrainian women, or both, are notoriously lying when asked about past and future sex partners. In most countries the differences are more moderate, with total 48-country sample's scores being 46.67±29.68 for men, and 27.34±19.55 for women. Latvia leads the way with smallest difference, and so most likely greatest honesty, with 49.42±23.61 for men, and 41.68±26.68 for women, what can be plausibly explained by just differences in attitude. (fake lie detector experiments have shown it's almost exclusively women who are lying when answering questions like that)

SOI seems to be considered quite useful by psychologists, it correlates with many nice things, not only other questionnaires, but country SOI averages correlate with various demographic, economic, and health scores in quite systematic way. Still, I cannot read papers about it without asking myself - why didn't they bother to perform this basic sanity check - which would detect huge number of outright lies in answers. And more importantly - what proportion of "serious science" suffers from problems like that?

References: The 48 country SOI study, fake lie detectors shows which gender lies more.