Vladimir_Nesov comments on Go Forth and Create the Art! - Less Wrong

38 Post author: Eliezer_Yudkowsky 23 April 2009 01:37AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (104)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 23 April 2009 04:21:10PM *  2 points [-]

That is a good question, and one that I don't have an answer to. Is there one? (you can't?)

You look at the sanity of the theory. How do you tell a sane academic paper from one written by a crackpot (say, you don't look at where it's published and you don't know the author)? Certainly you don't need to go check experimental results or proofs in overwhelming majority of cases. Doing this reliably where absurdity heuristic breaks is claimed as a large part of the Art of rationality. It's necessarily self-referential.

Comment author: thomblake 23 April 2009 04:43:59PM 0 points [-]

How do you tell a sane academic paper from one written by a crackpot (say, you don't look at where it's published and you don't know the author)?

Simple - I don't read it either way. There are plenty of papers written by folks I've heard of or in trusted publications to fill up all of the free time I don't have.

if the ideas seem novel and useful, I expect empirical evidence. If there is none, I withhold judgement until there is.

Comment author: Vladimir_Nesov 23 April 2009 04:48:14PM -1 points [-]

This doesn't sound realistic. Do you deny your ability to tell the difference?

Comment author: thomblake 23 April 2009 05:16:58PM *  1 point [-]

Do you deny your ability to tell the difference?

Between crackpots and valid academic papers, out of context? Yes, I do deny that. Take Louis Savain (please!). I actually have a lot of the same intuitions about the nature of spacetime, and the problems involving even talking about time travel outside of a fictional context. Savain is clearly a crackpot - it's practically painted all over his site. But if I had written a paper discussing some of these intuitions, and nobody noticed that it wasn't published in a peer-reviewed journal nor that I'm not a physicist, then I don't think there would be a clear way to notice that I was wrong.

Of course, a physicist could happily come along and say, "No, our theory is experimentally verified, and yours doesn't even make sense. Here are references to the relevant evidence, and some engineering applications you might not have been aware of".

ETA: I admit, I had "speak of the devil" in mind when I posted this. But it had to be done!

Comment author: Vladimir_Nesov 23 April 2009 05:24:39PM 0 points [-]

You are not doing the categorization as a civil service, you are doing it to efficiently build on gained understanding. If you are not ready to understand, you shouldn't try to do that. But when you are, you play the role of that very physicist, you don't need to see the crackpot's experimental results.

Comment author: thomblake 23 April 2009 05:30:01PM *  1 point [-]

you don't need to see the crackpot's experimental results.

That's because the physicist has her own experimental results already, and the crackpot has none. If the crackpot has experimental results to back up his theory, then the physicist had bloody well better look at them! (if the paper is even being taken seriously enough to be read in the first place)

Comment author: Luke_A_Somers 10 May 2013 02:27:47PM *  5 points [-]

The one crackpot I interacted most strongly did have experimental results, and trumpeted them loudly. The experiment turned out to be a notoriously finnicky one (not quite down to Millikan experiment territory) done in slipshod fashion. This was utterly predictable, given purely theoretical considerations and examination of his style, even before it came to the observations - his theory contradicted, say, the existence of comets.

Experiments can be wrong. Maybe even most attempts at experiments are wrong. What makes a scientist a scientist instead of a crackpot is the debugging and validation. Trying to exclude every way the results might not mean what it seems like they mean - not just doing control-experiment comparison and saying you've done your duty.

Crackpot experiments, lacking these extra checks, are worthless.

Comment author: satt 10 May 2013 09:49:47PM 3 points [-]

Experiments can be wrong. Maybe even most attempts at experiments are wrong.

This wouldn't surprise me much, at least in physics. There are probably more physics students than professional physicists, and those students do lots of tabletop experiments, badly. (My own old lab books document a refractive index measurement of -19.6, a disproof of the equivalence principle, and a laser beam that travelled at (1.05±0.01)c.) Nonetheless...

What makes a scientist a scientist instead of a crackpot is the debugging and validation. Trying to exclude every way the results might not mean what it seems like they mean - not just doing control-experiment comparison and saying you've done your duty.

...this is a bit too strong a distinction between crackpots & non-crackpots, though your basic point is right. The way I'd put it: a non-crackpot confronted with a bizarre result immediately wonders, "what did I do wrong?", but a crackpot confronted with the same result immediately gasps, "I knew it!".

I guess I'm just paraphrasing Dear Leader, really: one's strength as a non-crackpot is one's ability to be more confused by bizarre, inexplicable results than predictable results.

Comment author: Eliezer_Yudkowsky 10 May 2013 08:40:15PM 3 points [-]

What makes a scientist a scientist instead of a crackpot is the debugging and validation. Trying to exclude every way the results might not mean what it seems like they mean - not just doing control-experiment comparison and saying you've done your duty.

This would make most modern professional scientists crackpots which sounds a bit noncentral - they may be no true scientists, but they seem very different from the crackpots I've met.

Comment author: Luke_A_Somers 10 May 2013 09:48:47PM 4 points [-]

There's a bit of a gap between what ordinary not-very-good scientists do to make sure the experiment is right and what they should be doing.

There is a colossal gulf between what crackpots do and ordinary not-very-good scientists do.

Comment author: [deleted] 10 May 2013 09:13:30PM 2 points [-]

On a completely unrelated note, screw the Millikan experiment. That one lab where we had to replicate it in undergrad with the world's shittiest equipment is probably the only reason I'm a mathematician and not a physicist.

Comment author: Kindly 11 May 2013 02:01:23AM 2 points [-]

My high school physics class took the Millikan experiment to a new level: we installed a calculator program in which oil drops were simulated by pixels moving down the screen, and you could press buttons to vary the simulated electric charge.

I wonder if I can blame becoming a mathematician on that, too.

Comment author: Luke_A_Somers 10 May 2013 09:50:40PM 1 point [-]

/me stares in horror.

Undergrad. Millikan. Oil. Drop. Experiment.

shiver