Recently I was asked why academics (or at least tenure committees) respect peer-reviewed publications over other high-quality articles published in non-peer reviewed spaces (journals, newsletters, blogs, wikis, etc.). I have been thinking about this quite a bit over the last couple years, considering that I have a career ahead of me (hopefully) involving tenure review. However, I neither believe that traditional “high-quality” journals should have the power that they do over evaluation of academic prowess nor do I believe that they are the best place way to disseminate information. So, which direction do I go?
As journal subscribers, we count on editors to censor to an extent. We want articles that are on topic (as per the focus of the journal) as well as articles that meet generally agreed upon standards as far as method and presentation. These are generally accepted based upon the readership (they vote with their subscriptions). Publication/submission requirements change over time as the readership changes. This filtering for content and format results in publications that are easier to read in that readers know what to expect in both content and form. I like this. It allows me to quickly process many articles in a short amount of time
However, I don’t agree that we should judge the value of an article by the journal in which it is published. While, I see this as a natural tendency for humans to identify with the familiar, I think that we need to be broken from this habit (or safe zone). For example, if I see that an article was published in TESOL Quarterly, I know that, in my experience, articles in this journal are generally of high quality. So, I’ll assume that this one will meet my standards for quality. Whether it does or not will have to wait until I actually read it. This is where the real problem occurs. Most people will assume that the article is of high quality because it appears in the journal. Most often, they simply don’t know how to judge high vs low quality (a somewhat subjective judgment) therefore they leave the decision up to the editors/reviewers. I would include many faculty in this criticism. They point to articles in specific journals when trying to strengthen their arguments that aren’t much more than editorials. They take these statements of opinion as fact and pass it on to their students without disclaimer (or training) for them to make up their own minds.
So, now we are stuck in a vicious circle. Academics are required to publish in these journals and they are judged by the “quality” of the journal in which they publish. Those who don’t want to go in this direction are driven out of the field, thus only those who tow the line remain.
I’d love to see an academic version of Digg (http://digg.com) in place of refereed journals. Let those in the field (everyone in the field) determine the quality of a piece of writing and take in out of the hands of a few. This would get research out to the public faster and with less editorial interference than the current system.
Add to this the ability to create collaborative documents that can be edited beyond publication and you get a system of ranking and improving on publications that goes far beyond the current , slow, closed system.
What do you think?