Eric Schwitzgebel has done studies on whether moral philosophers behave more ethically (e.g., here). Some of the measures from that research seem to match reasonably well with law-abidingness (e.g., returning library books, paying conference registration fees, survey response honesty) and could be used in studies of mathematicians.

A subquestion here might be "how do you get reasonably unbiased data on criminality that you're able to cross-check with mathematical ability and IQ?"

I'm guessing that somewhere out there are data-dumps that include criminality, IQ and occupation, but don't have much of a sense of how to look for it.

Carefully define mathematician. Working definition: one who has obtained a degree in mathematics at some level (is an undergrad a mathematician, or do you need a phd? Do physics degree holders count as mathematicians? What about accounting and finance degrees?).

List number of non degree holders in the usa, list number of degree holders in the usa, list number of mathematics degree holders at the threshold, calculate ratio.

List number of incarcerated people, list number with degrees. Calculate ratio. List number of incarcerated degree holders with math degrees, calculate ratio to degree holders.

From these ratios, you should be able to see if mathematicians are proportionate, under, or overrepresented in incarcerated populations relative to both similarly educated and the general population.

Second approach: submit surveys to known math degree holders and known holders of similar levels of education. Ask 'do you do things you feel to be unethical on a regular basis?' and 'do you do things that a typical person would feel to be unethical, if they understood it, on a regular basis?' along with some lie scales (to determine whether the person is lying to the test to improve their image; these scales are commonly used on psychological tests).

Check power of your statistics.

Between those two methods, you should get a reasonable answer. I haven't googled and won't do it myself, but I think this project, at least approach 1, is doable. Without approach two, in the case mathematicians are better at not going to prison than the general population, the results of approach 1 will incorrectly make it look like Eliezer is right.

I do not know Eliezer, but have read a decent amount of his work, though not this. I offer the following counterpoint:

A mathematician who has chosen to use his math talents to sell used cars has probably calculated what he views to be prices that maximize his profits, taking into account anything you the consumer could do to impose costs for selling an overpriced lemon.

With the mathematician, 'market for lemons' economics are in play, and probably well executed, and therefore, I should avoid negotiating with him, as it is likely to go badly for me. A non mathematician may have made errors or been lazy in his pricing, creating an opportunity for deals.

If Eliezer considers himself to be a mathematician, this assertion is inherently even more suspect, as it is a member of a group ascribing positive characteristics to himself on the basis of his group membership. (I'm a Mathematician you can trust me, because Mathematicians don't lie, because they're Mathematicians....and I can prove it using the language of Mathematicians, which is known as Mathematics, something I've studied more than you... you're still skeptical? What do you have against Mathematicians you lunatic?!)

On the other hand, a pure mathematician who is dumping his car on craigslist is a mathematician who may not be happy about having to be a used car salesman, and is in addition to being as honest as anyone else, likely to find the 'applied' process of figuring out an asking price for the car distatesful. If the buyer is lucky, the mathematician will not need to be talked out of an elaborate payment scheme, calculated the value of the car lazily (find the book value, round up to the nearest $100), and actually has the paperwork so the buyer can hand over cash and the whole business can be concluded quickly.