Johnicholas comments on Some Heuristics for Evaluating the Soundness of the Academic Mainstream in Unfamiliar Fields - Less Wrong

73 Post author: Vladimir_M 15 February 2011 09:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (272)
Tags: epistemology academia contrarianism

Comments (272)

You are viewing a single comment's thread.

Comment author: Johnicholas 15 February 2011 08:00:25PM 13 points [-]

I've been surprised by how bad the majority of scholarship is around the "inspired-by" or "metaphorical" genre of algorithms - neural networks, genetic algorithms, Baum's Hayek machine and so on. My guess is that the colorful metaphors allow you to disguise any success as due to the technique rather than a grad student poking and prodding at it until a demo seems to work.

Within the metaphorical algorithms, I've been surprised at reinforcement learning in particular. It may have started with a metaphor of operant conditioning, but it has a useful mathematical foundation related to dynamic programming.

Comment author: Daniel_Burfoot 16 February 2011 03:14:05PM *  0 points [-]

I've been surprised by how bad the majority of scholarship is around the "inspired-by" or "metaphorical" genre of algorithms

Heh. Ultimately, all of AI is in this genre. A particularly bad aspect of this is that it usually means people choose their research problems based on what they think their approach can solve.

I'll briefly plug my proposal for AI-related inquiries, that explicitly rejects the "metaphorical" approach by actually defining the question before finding the solution. At the same time it doesn't rule out NNs, GAs, etc, but requires hard proof, in the form of an ungameable compression score, of their quality.

Comment author: IlyaShpitser 18 February 2011 07:58:27PM 3 points [-]

Modern AI is an odd combination of statistics, applied math, discrete math/combinatorics and logic. My theory is the only reason AI is a subfield of Computer Science at all is founder bias (Turing).

Comment author: Daniel_Burfoot 18 February 2011 09:24:07PM 1 point [-]

Totally agree. My slogan is that for AI to succeed it has to become an empirical science: it should use math, but only to the extent that the math is useful to describe reality. And it should be curiousity-driven and not application-driven like almost all modern research in computer vision and natural language processing.

Comment author: wedrifid 19 February 2011 02:49:04AM 1 point [-]

And it should be curiousity-driven and not application-driven like almost all modern research in computer vision and natural language processing.

I'm curious about how to not die (as an individual and a species). Would that count as curiosity or application drive? :P

Comment author: wedrifid 15 February 2011 08:23:23PM 6 points [-]

I've been surprised by how bad the majority of scholarship is around the "inspired-by" or "metaphorical" genre of algorithms - neural networks, genetic algorithms, Baum's Hayek machine and so on. My guess is that the colorful metaphors allow you to disguise any success as due to the technique rather than a grad student poking and prodding at it until a demo seems to work.

This was my area of research in my postgrad years. Specifically ants and birds. I couldn't agree more. The techniques do work but the scholarship was absolutely abysmal.

Comment author: Lightwave 16 February 2011 08:12:41AM 0 points [-]

but the scholarship was absolutely abysmal

Hey, can you expand a bit on this? Is this lecture an example of this?

Comment author: wedrifid 16 February 2011 08:59:17AM 6 points [-]

Hey, can you expand a bit on this?

Blast, I wrote a couple of paragraphs but accidental bumped the 'cancel' button. So you get dot points this time.

  • Some of the papers by field leaders misused statistical tool tools.
  • Insufficient comparison to the relevant mainstream techniques for the same problem.
  • Excessive amount of ad hoc algorithm selection and cherry picking of results.
  • I shouldn't be able to see blatant problems in the leading research any any field when I am a hack who has had to self teach and pick things up as I go along.

My disillusionment, unfortunately, wasn't just limited to that one field. I had higher expectations of academic research than what seems to be available in quite a few fields. There are some high quality fields but you still have to be careful when you take things at face value.

Is this lecture an example of this?

Not exactly. It is a bit more of a biology course than just a 'metaphor' course. It seems quite good. I'm looking at some of the other lectures now.

Comment author: DSimon 17 February 2011 07:45:59PM 5 points [-]

Blast, I wrote a couple of paragraphs but accidental bumped the 'cancel' button.

I hate it when that happens! There's a good technique to prevent it from happening again, though: form recovery plugins, like Lazarus.

(Heck, it helped me just now; I accidentally pressed cancel on this very comment a moment ago.)

Comment author: wedrifid 18 February 2011 12:13:40AM 0 points [-]

form recovery plugins, like Lazarus.

Thanks, installed. :)

Powered by Reddit Powered by Reddit