PEOPLE RESPOND TO INCENTIVES. The Superintendent commends two articles on the nature of scholarly publishing. One, at The Valve, continues the reaction to Chicago's thumbs-down to Professor Drezner. There's a lot to recommend in this article. Read it. I'll be here. There is an excerpt I wish to expand upon.
A less obvious issue is the way the existing tenure system may negatively affect diversity and liveliness of academic thought. To some degree, the charge that tenure is mainly a system of credentialism that rewards the expression of acceptable opinion is an old one. What’s new in the humanities and many of the social sciences is the vast oversupply of academic labor. Departments hiring from overstocked pools find it easy to replicate themselves, making it more likely that they’ll grow more intellectually and politically homogeneous over time. The tenure system potentially worsens this situation. Tenure battles are notoriously unpleasant and time consuming. (Indeed, the whole system of review and evaluation diverts considerable resources away from teaching and scholarship.)
Focus on that topic sentence. Now consider the boiler-plate in any department's formal statement of its tenure policy. Such things exist. Sometimes they're easily gotten to; other times they're safeguarded by laity who will pull them from the relevant file should the archbishop be called before a legislative inquisition. The statement will likely include language about "establishing a reputation" or "established a reputation" as a qualification for tenure. The criterion for promotion to professor will further define the nature of that reputation.

Pull out the catechism.

How does one establish a reputation?

Publish in reputable journals?

What makes a journal reputable?

Other people read and cite the articles in them.

Any other people?

People in leading departments.

Can you quantify that?


Here comes the problem. Newmark's Door recommends an article in the Chronicle of Higher Education (this may be embargoed to the paid side shortly) on the consequences of researching to the quantification. It's easy enough to note who is citing whom (compare and contrast with the Blog Ecosystem) and count which articles and which journals are more frequently cited. In the world of scholarly journals, the Higher Mammals have higher impact factors.

Impact-factor fever is spreading, threatening to skew the course of scientific research, say critics. Investigators are now more likely to chase after fashionable topics — the kind that get into high-impact journals — than to follow important avenues that may not be the flavor of the year, says Yu-Li Wang, a professor of physiology at the University of Massachusetts Medical School. "It influences a lot of people's research direction."

That influence has also led to a creeping sense of cynicism about the business of science publications. Journal editors have learned how to manipulate the system, sometimes through legitimate editorial choices and other times through deceptive practices that artificially inflate their own rankings. Several ecology journals, for example, routinely ask authors to add citations to previous articles from that same journal, a policy that pushes up its impact factor. Authors who have received such requests say that the practice veers toward extortion and represents a violation of scientific ethics.

But landing a paper in a high-impact journal is not per se evidence of a high-impact paper.
The measurement is just an average of all the papers in a journal over a year; it doesn't apply to any single paper, let alone to any author. For example, a quarter of the articles in Nature last year drew 89 percent of the citations to that journal, so a vast majority of the articles received far fewer than the average of 32 citations reflected in the most recent impact factor.
So what is an academic, particularly one starting out, to do? Consider this:
"You're a better scientist if you're a happy scientist."
Interpret it this way. If we really knew what the answers were, we wouldn't have to do research. Therefore, we can choose to research something that appears to be fashionable even if it makes us miserable. The top journals might not take it anyway, and a lesser journal that takes it with the hope of raising its impact factor might be disappointed. Or we can research what interests us and enjoy the work. The top journals might not take it anyway, and there are sufficient lesser journals that a few iterations of shopping it will publish it. There is no guarantee that your work will become a hot topic in the future. But do you really want to be in the position of the woodpusher who is attempting to stay one move ahead of Kasparov in the King's Indian rather than enjoy the game? Or would you rather be cheerful, and quite likely more productive than you would be chasing the current high-impact topics? (That quest, by the way, is likely to be futile if you're not at one of the forty departments claiming to be in the top 25. By the time the high-impact stuff is released as a .pdf or published out of such a place, the folks there have had two years worth of seminars and coffee chats to push it beyond what you're starting on.)

No comments: