On the Quantity vs. Quality of Research Output
Recently one of my Facebook friends has posted the picture above with the comment: “Must be some weird area.” Of course, this picture is a spoof produced for an article in the Scientist magazine about the shortage of open positions that require a Ph.D. Because my friend is an accomplished academic, his post prompted the usual wittiness contest, with fellow academics and graduate students commenting on this photo. I commented as well saying: “4 years to get a Ph.D. plus 3*2 years of postdoc = 10 years of scholarly productivity to publish 6 papers. This is 3/5th of a paper a year! I would not hire this person even for a postdoc position.” My statement must have touched the nerve, as several people started arguing with me vehemently that one cannot judge the quality of a researcher based on the number of his or her papers.
A similar argument has come up in the Research Methods in CS class I am teaching this semester. To help students plan their Ph.D. studies, I asked them how many papers and in which venues they plan to publish before they complete their Ph.D. studies. Some students objected to my asking this question, stating that one cannot correlate publication numbers with the quality of one’s research. One student, in particular, was adamantly defending his position that as long as he did something great for his Ph.D., he could easily report the results in a single publication. Although I appreciated the student’s candor, I do not believe that his strategy is feasible.
German philosopher Georg Wilhelm Friedrich Hegel was the first to point out the phenomenon of quantity transforming into quality. This is a powerful idea that applies to multiple fields. At the physical level, one example of this transformation is that as soon as enough heat is applied to water, it turns into steam. Recently, I encountered a fascinating example of this transformation described eloquently by Maxwell Gladwell in his book “Outliers: The Story Of Success.” In particular, Gladwell is describing the so-called 10,000 hours principle. It takes on average 10,000 hours of deliberate practice in a cognitively demanding field for an individual to achieve remarkable results. In other words, 10,000 hours of practice transform a capable person into an outlier in pursuits as diverse as chess, music, computer programming, business, and math. For me, this is the ultimate application of the Hegelian transition from quantity into quality.
How does this principle apply to the quantity and quality of research output? Is it possible for a Ph.D. student to write just one paper that is going to become highly influential? I am sure there are some rare examples of that, but in general I don’t think this is possible. A highly influential paper must report on a real breakthrough. Publishing truly innovative ideas is hard, as one must overcome the resistance of the established understanding and views in the area. To be able to publish such a research paper, one must develop the ability to present research results effectively. The only way to learn this skill is by practicing it incessantly for many years. To develop this ability, a researcher should have published multiple papers having learned how to deal with rejection in the process. Therefore, I believe that researchers also transition from quantity into quality.
Once someone asked me why I was publishing papers in a whole variety of venues. Wouldn’t it make sense just to focus on publishing in top tier venues? I quickly replied that I had a wide range of ideas, and only some of which deserved to be published in top tier venues. Nevertheless, I strongly believe in evaluating and publishing all promising ideas. This practice is necessary to build my research evaluation and presentation skills, so that when I have a truly great idea I’d have the required skills to evaluate and publish it.
Going back to the original example, six publications for someone with a Ph.D. and three postdocs seems unreasonably few, which to me indicates low research productivity or poor work ethics. I also do not believe that someone can publish only six papers, all of which would be in top tier venues. To get accepted to these venues, one needs to build up his or her research skills by publishing other papers, albeit in lower-ranked venues. So I stand by my original Facebook comment on that fake picture.