See also Modeling human function learning with Gaussian processes, by Tom Griffiths, Chris Lucas, Joseph Jay Williams, and Michael Kalish, in NIPS 21:

[. . .] we look at two quantitative tests of Gaussian processes as an account of human function learning: reproducing the order of difficulty of learning functions of different types, and extrapolation performance. [. . .]

Predicting and explaining people’s capacity for generalization – from stimulus-response pairs to judgments about a functional relationship between variables – is the second key component of our account. This capacity is assessed in the way in which people extrapolate, making judgments about stimuli they have not encountered before. [. . .] Both people and the model extrapolate near optimally on the linear function, and reasonably accurate extrapolation also occurs for the exponential and quadratic function. However, there is a bias towards a linear slope in the extrapolation of the exponential and quadratic functions[. . .]

The first author, Tom Griffiths, is the director of the Computational Cognitive Science Lab at UC Berkeley, and Lucas and Williams are graduate students there. The work of the Computational Cognitive Science Lab is very close to the mission of Less Wrong:

The basic goal of our research is understanding the computational and statistical foundations of human inductive inference, and using this understanding to develop both better accounts of human behavior and better automated systems [. . .]

For inductive problems, this usually means developing models based on the principles of probability theory, and exploring how ideas from artificial intelligence, machine learning, and statistics (particularly Bayesian statistics) connect to human cognition. We test these models through experiments with human subjects[. . .]

Probabilistic models provide a way to explore many of the questions that are at the heart of cognitive science. [. . .]

Griffiths's page recommends the foundations section of the lab publication list.

If the data is actually linear or anything remotely resembling linear, then on distant points a linear model will do much better than a nearest-neighbor estimator. Whereas on nearby points, a nearest-neighbor estimator will do as well as a linear model given enough data. So on distant points nearest-neighbor only works if the curve is a particular shape (constant), while on near points it works so long as the curve has anything resembling local neighborhoods.

Your use of the terms parametric vs. nonparametric doesn't seem to be that used by people working in nonparametric Bayesian statistics, where the distinction is more like whether your statistical model has a fixed finite number of parameters or has no such bound. Methods such as Dirichlet processes, and its many variants (Hierarchical DP, HDP-HMM, etc), go beyond simple modeling of surface similarities using similarity of neighbours.

See, for example, this list of publications coauthored by Michael Jordan:

• Bayesian Nonparametrics http://www.cs.berkeley.edu/~jordan/bnp.html

Parametric extrapolation actually works quite well in some cases. I'll cite a few examples that I'm familiar with:

• prediction of black holes by the general theory of relativity
• Moore's Law
• predicting the number of operations needed to factor an integer of given size

I don't see any examples of nonparametric extrapolation that have similar success.

A major problem in Friendly AI is how to extrapolate human morality into transhuman realms. I don't know of any parametric approach to this problem that isn't without serious difficulties, but "nonparametric" doesn't really seem to help either. What does your advice "don't extrapolate if you can possibly avoid it" imply in this case? Pursue a non-AI path instead?