Paper: Testing ecological models

0 brian_jaress 27 August 2009 10:12PM

You may be interested in a paper of medium age I just read. Testing ecological models: the meaning of validation (PDF) tackles a problem many of you are familiar with in a slightly different context.

To entice you to read it, here are some quotes from its descriptions of other papers:

Holling (1978) pronounced it a fable that the purpose of validation is to establish the truth of the model…

Overton (1977) viewed validation as an integral part of the modelling process…

Botkin (1993) expressed concern that the usage of the terms verification and validation was not consistent with their logical meanings…

Mankin et al. (1977) suggested that the objectives of model-building may be achieved without validating the model…

I have another reason for posting this; I’m looking for more papers on model validation, especially how-to papers. Which ones do you consider most helpful?

The Journal of (Failed) Replication Studies

11 Vladimir_Gritsenko 23 August 2009 09:15AM

One of Seed Magazine's "Revolutionary Minds" is Moshe Pritsker, who created the Journal of Visualized Experiments, which to me looks like a very cool idea. I imagine that early on it may have looked somewhat silly ("he can't implant engineered tissue in a rat heart and he calls himself a scientist?!"), so it's nice to know JoVE is picking up pace.

Many folks keep pointing out how published research is itself biased towards positive results, and how replication (and failed replication!) trumps mere "first!!!11" publication. If regular journals don't have good incentives to publish "mere" (failed) replication studies, why not create a journal that would be dedicated entirely to them? I can't speak about the logistics, but I imagine it can be anything from a start-up (a la JoVE) to an open depository (a la arxiv.org).

I am not part of academia, but I understand that there are a few folks here who are. What do you say?

[EDIT: Andrew Kemendo notes two such journals in the comments: http://www.jnrbm.com/ and http://www.jnr-eeb.org/index.php/jnr.]

Informers and Persuaders

12 Eliezer_Yudkowsky 10 February 2009 08:22PM

Suppose we lived in this completely alternate universe where nothing in academia was about status, and no one had any concept of style.  A universe where people wrote journal articles, and editors approved them, without the tiniest shred of concern for what "impression" it gave - without trying to look serious or solemn or sophisticated, and without being afraid of looking silly or even stupid.  We shall even suppose that readers, correspondingly, have no such impressions.

In this simpler world, academics write papers from only two possible motives:

First, they may have some theory of which they desire to persuade others; this theory may or may not be true, and may or may not be believed for virtuous reasons or with very strong confidence, but the writer of the paper desires to gain adherents for it.

Second, there will be those who write with an utterly pure and virtuous love of the truthfinding process; they desire solely to give people more unfiltered evidence and to see evidence correctly added up, without a shred of attachment to their or anyone else's theory.

People in the first group may want to signal membership in the second group, but people in the second group only want their readers to be well-informed.  In any case, to first order we must suppose that none of this is about signaling - that all such motives are just blanked out.

What do journal articles in this world look like, and how do the Persuaders' articles differ from the Informers'?

continue reading »

The Comedy of Behaviorism

12 Eliezer_Yudkowsky 02 August 2008 08:42PM

Followup toHumans in Funny Suits

"Let me see if I understand your thesis.  You think we shouldn't anthropomorphize people?"
        -- Sidney Morgenbesser to B. F. Skinner

Behaviorism was the doctrine that it was unscientific for a psychologist to ascribe emotions, beliefs, thoughts, to a human being.  After all, you can't directly observe anger or an intention to hit someone.  You can only observe the punch.  You may hear someone say "I'm angry!" but that's hearing a verbal behavior, not seeing anger.  Thoughts are not observable, therefore they are unscientific, therefore they do not exist.  Oh, you think you're thinking, but that's just a delusion - or it would be, if there were such things as delusions.

continue reading »

Do Scientists Already Know This Stuff?

21 Eliezer_Yudkowsky 17 May 2008 02:25AM

Followup toScience Isn't Strict Enough

poke alleges:

"Being able to create relevant hypotheses is an important skill and one a scientist spends a great deal of his or her time developing. It may not be part of the traditional description of science but that doesn't mean it's not included in the actual social institution of science that produces actual real science here in the real world; it's your description and not science that is faulty."

I know I've been calling my younger self "stupid" but that is a figure of speech; "unskillfully wielding high intelligence" would be more precise.  Eliezer18 was not in the habit of making obvious mistakes—it's just that his "obvious" wasn't my "obvious".

No, I did not go through the traditional apprenticeship.  But when I look back, and see what Eliezer18 did wrong, I see plenty of modern scientists making the same mistakes.  I cannot detect any sign that they were better warned than myself.

Sir Roger Penrose—a world-class physicist—still thinks that consciousness is caused by quantum gravity.  I expect that no one ever warned him against mysterious answers to mysterious questions—only told him his hypotheses needed to be falsifiable and have empirical consequences.  Just like Eliezer18.

continue reading »

Science Doesn't Trust Your Rationality

19 Eliezer_Yudkowsky 14 May 2008 02:13AM

Followup toThe Dilemma: Science or Bayes?

Scott Aaronson suggests that Many-Worlds and libertarianism are similar in that they are both cases of bullet-swallowing, rather than bullet-dodging:

Libertarianism and MWI are both are grand philosophical theories that start from premises that almost all educated people accept (quantum mechanics in the one case, Econ 101 in the other), and claim to reach conclusions that most educated people reject, or are at least puzzled by (the existence of parallel universes / the desirability of eliminating fire departments).

Now there's an analogy that would never have occurred to me.

I've previously argued that Science rejects Many-Worlds but Bayes accepts it.  (Here, "Science" is capitalized because we are talking about the idealized form of Science, not just the actual social process of science.)

It furthermore seems to me that there is a deep analogy between (small-'l') libertarianism and Science:

  1. Both are based on a pragmatic distrust of reasonable-sounding arguments.
  2. Both try to build systems that are more trustworthy than the people in them.
  3. Both accept that people are flawed, and try to harness their flaws to power the system.

continue reading »

The Failures of Eld Science

33 Eliezer_Yudkowsky 12 May 2008 10:32AM

Followup toInitiation Ceremony, If Many-Worlds Had Come First

This time there were no robes, no hoods, no masks.  Students were expected to become friends, and allies.  And everyone knew why you were in the classroom.  It would have been pointless to pretend you weren't in the Conspiracy.

Their sensei was Jeffreyssai, who might have been the best of his era, in his era.  His students were either the most promising learners, or those whom the beisutsukai saw political advantage in molding.

Brennan fell into the latter category, and knew it.  Nor had he hesitated to use his Mistress's name to open doors.  You used every avenue available to you, in seeking knowledge; that was respected here.

"—for over thirty years," Jeffreyssai said.  "Not one of them saw it; not Einstein, not Schrödinger, not even von Neumann."  He turned away from his sketcher, and toward the classroom.  "I pose to you to the question:  How did they fail?"

The students exchanged quick glances, a calculus of mutual risk between the wary and the merely baffled.  Jeffreyssai was known to play games.

continue reading »

Many Worlds, One Best Guess

12 Eliezer_Yudkowsky 11 May 2008 08:32AM

Previously in series: Collapse Postulates
Followup toBell's Theorem, Spooky Action at a Distance, Quantum Non-Realism, Decoherence is Simple, Falsifiable and Testable

If you look at many microscopic physical phenomena—a photon, an electron, a hydrogen atom, a laser—and a million other known experimental setups—it is possible to come up with simple laws that seem to govern all small things (so long as you don't ask about gravity).  These laws govern the evolution of a highly abstract and mathematical object that I've been calling the "amplitude distribution", but which is more widely referred to as the "wavefunction".

Now there are gruesome questions about the proper generalization that covers all these tiny cases.  Call an object 'grue' if it appears green before January 1, 2020 and appears blue thereafter.  If all emeralds examined so far have appeared green, is the proper generalization, "Emeralds are green" or "Emeralds are grue"?

The answer is that the proper generalization is "Emeralds are green".  I'm not going to go into the arguments at the moment.  It is not the subject of this post, and the obvious answer in this case happens to be correctThe true Way is not stupid: however clever you may be with your logic, it should finally arrive at the right answer rather than a wrong one.

In a similar sense, the simplest generalizations that would cover observed microscopic phenomena alone, take the form of "All electrons have spin 1/2" and not "All electrons have spin 1/2 before January 1, 2020" or "All electrons have spin 1/2 unless they are part of an entangled system that weighs more than 1 gram."

continue reading »

Beware of Stephen J. Gould

27 Eliezer_Yudkowsky 06 November 2007 05:22AM

Followup to:  Natural Selection's Speed Limit and Complexity Bound

If you've read anything Stephen J. Gould has ever said about evolutionary biology, I have some bad news for you.  In the field of evolutionary biology at large, Gould's reputation is mud.  Not because he was wrong.  Many honest scientists have made honest mistakes.  What Gould did was much worse, involving deliberate misrepresentation of science.

In his 1996 book Full House: The Spread of Excellence from Plato to Darwin, Stephen J. Gould explains how modern evolutionary biology is very naive about evolutionary progress.  Foolish evolutionary biologists, says Gould, believe that evolution has a preferred tendency toward progress and the accumulation of complexity.  But of course - Gould kindly explains - this is simply a statistical illusion, bolstered by the tendency to cite hand-picked sequences like bacteria, fern, dinosaurs, dog, man.  You could equally well explain this apparent progress by supposing that evolution is undergoing a random walk, sometimes losing complexity and sometimes gaining it.  If so, Gould says, there will be a left bound, a minimum at zero complexity, but no right bound, and the most complex organisms will seem to grow more complex over time.  Even though it's really just a random walk with no preference in either direction, the distribution widens and the tail gets longer.

What romantics, ha ha, those silly evolutionary biologists, believing in progress!  It's a good thing we had a statistically sophisticated thinker like Stephen J. Gould to keep their misconceptions from infecting the general public.  Indeed, Stephen J. Gould was a hero - a martyr - because evolutionary biologists don't like it when you challenge their romantic preconceptions, and they persecuted him.  Or so Gould represented himself to the public.

There's just one problem:  It's extremely unlikely that any modern evolutionary theorist, however much a romantic, would believe that evolution was accumulating complexity.

continue reading »

Double Illusion of Transparency

51 Eliezer_Yudkowsky 24 October 2007 11:06PM

Followup to:  Explainers Shoot High, Illusion of Transparency

My first true foray into Bayes For Everyone was writing An Intuitive Explanation of Bayesian Reasoning, still one of my most popular works.  This is the Intuitive Explanation's origin story.

In December of 2002, I'd been sermonizing in a habitual IRC channels about what seemed to me like a very straightforward idea:  How words, like all other useful forms of thought, are secretly a disguised form of Bayesian inference.  I thought I was explaining clearly, and yet there was one fellow, it seemed, who didn't get it.  This worried me, because this was someone who'd been very enthusiastic about my Bayesian sermons up to that point.  He'd gone around telling people that Bayes was "the secret of the universe", a phrase I'd been known to use.

So I went into a private IRC conversation to clear up the sticking point.

continue reading »

View more: Prev | Next