That’s the peril of a historically successful, productive research program. We get locked in to a model; there is the appeal of being able to use solid, established protocols to gather lots of publishable data, and to keep on doing it over and over. It’s real information, and useful, but it also propagates the illusion of comprehension. We are not motivated to step away from the busy, churning machine of data gathering and rethink our theories.
I read the above quote in this blog post by PZ Myers. That post is about the concept of the gene in biology, but it made me think of the use of corpora like the Penn Treebank to measure performance in part-of-speech tagging and parsing.