Archive

Archive for the ‘scholarly publications’ Category

A Commonly Overlooked Aspect of Communication Skills

July 23, 2013 1 comment

Any compendium of advice for budding researchers is never complete without admonitions about the importance of developing good communication skills. However, communication skills are defined almost exclusively as the ability to write technical manuscripts and give formal oral presentations. Although these skills are critically important (don’t even contemplate applying for an academic job having not gained some mastery in them), there is another set of communication skills that are not any less important but are commonly overlooked. I am talking about the ability to manage the expectations of your research peers and superiors as well as keeping them properly informed about your progress or lack thereof.

The modern research enterprise is a fundamentally collaborative process of discovering new knowledge and sharing this knowledge with your research community.  Few researchers work by themselves. Even a researcher who publishes all of her papers as single author has organizational superiors (e.g., dept. head, manager), whose jobs are affected by her progress. A graduate student working on his own research under the guidance of an advisor has the responsibility to inform the advisor about his successes, setbacks, and changes of directions in a timely fashion. Doing so properly requires strong communication skills that graduate students must take time and effort to properly develop. What I am going to say next may sound blasphemous, but an otherwise competent researcher working in a team (e.g., in an industrial research lab) may be forgiven poor formal written or oral presentation skills. Other team members can take the lead on writing research papers or giving presentations. However, not properly communicating with your research team about your personal research progress can be truly disastrous.

Let us consider a hypothetical example of a graduate student working on a project that involves applying an AI algorithm to a software engineering problem. The statement “The AI algorithm we picked does not quite work” from a graduate student to her advisor would have vastly different implications depending on how long in advance of the planned publication deadline it is uttered: (a) two months—normal research process; “let’s find and try another one”, (b) one month—worrisome; “can we make this deadline?” (c) two weeks—frustrating; “forget about making a strong submission now!” (d) one week—exasperating; “why are you telling me this now, when I have wasted all this valuable time working on this manuscript?!” (e) days—catastrophic and maddening; “…!!!”

Nevertheless, I often witness graduate students struggling with developing this important aspect of communication skills. I believe that some cultural norms and common misunderstandings may prevent the effective development of such communication skills. Let us examine them in turn.

  1. Lack of Appreciation for Negative Results
    Some graduate students take an engineering approach to research tasks: something to be successfully executed to completion. Especially for someone who had a career in industry, not performing an assigned task on time successfully often signals incompetence or lack of work ethics. By contrast, doing research entails continuously trying out new ideas to test them out for their promise. In that light, knowing that an idea does not work is often as valuable, if not even more so, than knowing that it works. Hence, negative results should be reported and discussed in a timely fashion.
  2. Not Feeling Comfortable to Ask for Help
    Some students have a false sense of pride in handling all the research challenges by themselves. They see asking for help as a sign of weakness and incompetence. As a result, they often postpone getting the required help until they get dangerously close to the publication deadline. This creates unnecessary stress for everyone and often leads to failure.Nobody was born knowing how to conduct great research. So it is perfectly OK not to excel at each aspect of this demanding cognitive activity. Asking for help in a timely manner makes planning easier. The planning is required for the advisor to be able to plan how best to help the student. This help may entail allocating more of the advisor’s time to the project, a brainstorming session, an inclusion of additional students, or targeting a publication venue with a later deadline. With enough time, all these options are possible.
  3. Poor Articulation in Describing One’s Progress
    Often graduate students fail to properly articulate the progress they are making in their research. Instead of precisely identifying which aspects of the project are going well and which ones are problematic, they report their progress using generalities. Here is a list exemplifying inappropriate answers to the question “How is your project going?”: “It is going fine.” ; “No specific results yet, but I am working on it.” ;“I am writing the paper.”A competent researcher should be able to articulate her progress using concrete terms and specify not only the successes but also the hurdles to be addressed. For example: “I am having a really hard time finding a convincing motivating example—the ones in the literature do not seem very applicable.” “I have finished the implementation of the system. For some reason, I am not seeing the expected performance advantages. I am checking my implementation for bugs, and if my implementation is correct, we may need to tone down our performance efficiency claims.” “My paper draft is almost completed with the exception for the experiments section. For some reason, I find it hard to clearly explain our experimental setup.” “When summarizing the system design, I find it hard morphing the text from my prior papers to avoid self-plagiarism. I may need your help with this part.”

I know that some research teams hold regular meetings, in which each team member reports on their progress and difficulties encountered. Some academic research teams even have successfully adopted Scrum, an approach used for managing Agile software development teams. However, I find the practice of regularly reporting progress in a rigid format contradictory for my vision of the research enterprise, even though these practices may work exceptionally well for other research groups. I see research as a fundamentally creative enterprise, with researchers being more akin to artists than engineers, with individual researchers having a strong ownership of their work. In my view, properly developing the neglected aspect of communication skills discussed in this post can help ensure timely research progress without jeopardizing the spirit of free discovery and exploration that I value so much.

Advertisements

Curbing the Unnecessary Scholarship (an annual April 1 e-mail)

April 1, 2013 Leave a comment

Dear Colleagues:

The Taulbee Survey Report ranks CS departments based on the amount of their external funding: “The U.S. CS data indicate that the higher the ranking, the more external funding is received by the department (both in total and per capita).”
(Source: http://cra.org/uploads/documents/resources/taulbee/CRA_Taulbee_2009-2010_Results.pdf). And yet, as the recently submitted Faculty Activity Reports (FARs) indicate, some of you are spending only between 60% and 90% of your time on “research activities,” defined as bringing in external funding to the department. If we are truly serious about increasing the rank and prestige of our department, we should be spending a 100% of our time on such research activities. Two obstacles stand in the way of realizing this vision: teaching and scholarship. Fortunately, recent advancements in massive open online courses (MOOCs) have great potential to significantly alleviate the first obstacle if not to remove it altogether. Effective handling of the scholarship obstacle, however, will require some creative problem solving. To that end, a new committee has been established: Curbing the Unnecessary Scholarship or The CUrSe Committee for short. How do we know what scholarship is unnecessary? We will use the following intuitive definition: a scholarship activity is necessary if it prepares preliminary results for a successful funding proposal; it is unnecessary otherwise.

Conference publishing is particularly harmful. Publishing at conferences wastes faculty and graduate student time, while traveling to conferences incurs high administrative overhead, draining staff time and resources (and is bad for the environment). Besides, allocating budget for conference travel makes your funding proposals less competitive. All other factors being equal, which proposal has a better chance of being funded: the one that asks for $5K in domestic and international travel or the one that asks for $500 to travel to DC to discuss your next proposal with an NSF program director? It is simply unjustifiable to be wasting your and your graduate students’ time to go through several rejection cycles to have your papers accepted to conferences with ridiculous acceptance rates (e.g., CHI, ICSE, KDD, IPDPS, SC, etc.). Conference publishing is a dangerous addiction and should be treated accordingly. Therefore, the first recommendation of the CUrSE committee is that we go cold turkey on conference publishing for a period of one year. Then, the committee will assess the expected positive impact of this initiative and may recommend occasional recreational conference publishing on a case-by-case basis.

Journal publishing should be curbed as well. Too much time and effort is spent on preparing journal manuscripts and addressing comments from the reviewers. To address this inefficiency, the CUrSE Committee is tasked with compiling a list of journals that do not impose the unnecessary burden of the review process on the authors. The committee needs to identify those journals that fully embrace the value of inclusiveness and welcome all submissions irrespective of their topic, content, or fit.

The journal writing process itself can be streamlined as well. A couple of years ago, SCIGen, a promising technology emerged from MIT that makes it possible to automatically synthesize scientific manuscripts (Source: http://pdos.csail.mit.edu/scigen/). At the time, SCIGen received some negative press due to its technical imperfections stemming naturally from the breakthrough nature of this technology. Only because an innovative technology does not pan out as intended right away, it does not mean it would not mature over time to become ready for practical application. Therefore, the CUrSe Committee will explore whether it is the right time for us to invest time and efforts in mastering SCIGen.

Once all the unnecessary scholarship is properly curbed, we will be able to dedicate all our time and efforts to increasing the amount of research funding in the department. What about graduate students? One point needs to be made perfectly clear: the value of a graduate degree is positively correlated with the rankings of the department awarding the degree. Therefore, our graduate students should be even more interested in and dedicated to increasing our rankings than we are. To help our graduate students really increase the value of their future degrees, we must immediately stop counting publications as a criteria for graduation. Instead of saying “OK, this student deserves a Ph.D., as he has published N papers,” we should be saying “OK, this student is ready to graduate, as he has contributed preliminary results for N successful funding proposals.” When a graduate student asks you “When can I expect to graduate?”, you should have profound answers prepared; for example: “When your thinking reaches the right depth.” If students ask you what devices will be used to gauge the depth of their thinking, just look thoughtfully off into the distance and utter Yoda-like: “You won’t miss it when you get there.”

Please, let me know by EOB today, if you feel prepared and motivated to join the CUrSe Committee. We need a lot of talent, energy, and passion to eliminate the scourge of unnecessary scholarship, thereby fulfilling our raison d’etre of increasing our department’s rankings!

On the Quantity vs. Quality of Research Output

October 16, 2011 Leave a comment

ᄅ JASON VARNEY|VARNEYPHOTO.COM

Recently one of my Facebook friends has posted the picture above with the comment: “Must be some weird area.” Of course, this picture is a spoof produced for an article in the Scientist magazine about the shortage of open positions that require a Ph.D. Because my friend is an accomplished academic, his post prompted the usual wittiness contest, with fellow academics and graduate students commenting on this photo. I commented as well saying: “4 years to get a Ph.D. plus 3*2 years of postdoc = 10 years of scholarly productivity to publish 6 papers. This is 3/5th of a paper a year! I would not hire this person even for a postdoc position.” My statement must have touched the nerve, as several people started arguing with me vehemently that one cannot judge the quality of a researcher based on the number of his or her papers.

A similar argument has come up in the Research Methods in CS class I am teaching this semester. To help students plan their Ph.D. studies, I asked them how many papers and in which venues they plan to publish before they complete their Ph.D. studies. Some students objected to my asking this question, stating that one cannot correlate publication numbers with the quality of one’s research. One student, in particular, was adamantly defending his position that as long as he did something great for his Ph.D., he could easily report the results in a single publication. Although I appreciated the student’s candor, I do not believe that his strategy is feasible.

German philosopher Georg Wilhelm Friedrich Hegel was the first to point out the phenomenon of quantity transforming into quality. This is a powerful idea that applies to multiple fields. At the physical level, one example of this transformation is that as soon as enough heat is applied to water, it turns into steam. Recently, I encountered a fascinating example of this transformation described eloquently by Maxwell Gladwell in his book “Outliers: The Story Of Success.” In particular, Gladwell is describing the so-called 10,000 hours principle. It takes on average 10,000 hours of deliberate practice in a cognitively demanding field for an individual to achieve remarkable results. In other words, 10,000 hours of practice transform a capable person into an outlier in pursuits as diverse as chess, music, computer programming, business, and math. For me, this is the ultimate application of the Hegelian transition from quantity into quality.

How does this principle apply to the quantity and quality of research output? Is it possible for a Ph.D. student to write just one paper that is going to become highly influential? I am sure there are some rare examples of that, but in general I don’t think this is possible. A highly influential paper must report on a real breakthrough. Publishing truly innovative ideas is hard, as one must overcome the resistance of the established understanding and views in the area. To be able to publish such a research paper, one must develop the ability to present research results effectively. The only way to learn this skill is by practicing it incessantly for many years. To develop this ability, a researcher should have published multiple papers having learned how to deal with rejection in the process. Therefore, I believe that researchers also transition from quantity into quality.

Once someone asked me why I was publishing papers in a whole variety of venues. Wouldn’t it make sense just to focus on publishing in top tier venues? I quickly replied that I had a wide range of ideas, and only some of which deserved to be published in top tier venues. Nevertheless, I strongly believe in evaluating and publishing all promising ideas. This practice is necessary to build my research evaluation and presentation skills, so that when I have a truly great idea I’d have the required skills to evaluate and publish it.

Going back to the original example, six publications for someone with a Ph.D. and three postdocs seems unreasonably few, which to me indicates low research productivity or poor work ethics. I also do not believe that someone can publish only six papers, all of which would be in top tier venues. To get accepted to these venues, one needs to build up his or her research skills by publishing other papers, albeit in lower-ranked venues. So I stand by my original Facebook comment on that fake picture.

On the Prudence of Pessimism

October 9, 2011 Leave a comment

I am in beautiful Detroit, MI, sitting at the gate in the airport, having missed my flight by about 10 minutes. The next flight is in 5 hours. Nevertheless, I feel fortunate. No, I don’t need to have my head examined. I am fortunate indeed to have a confirmed seat on the next flight, which is not just full but oversold. My fellow travelers who missed the same connection were not so fortunate–they may have to spend the night here or fly to a “nearby” airport a 5-6 hours drive away from their final destination.

How did I luck out like that? I am a pessimist by nature, and always prepare for the worst possible outcome. When I was waiting to board my first flight this morning, a cheerful announcement notified me that the flight would be slightly delayed. “We have a minor maintenance issue, but should be out of here in 30 minutes. Please ask the agent to rebook your connecting flight only if your layover is less than 30 minutes.” Well, my layover was longer than one hour and thirty minutes, but nevertheless I immediately proceeded to rebook my connecting flight. “But sir, you should be able to make your flight comfortably–we are expected to be only 30 minutes late.” “Lady, just please rebook me for a later flight just in case. If I make my original flight, I just won’t use that ticket.” Lo and behold, the 30 minutes delay turned into an hour and forty minutes delay. If the next flight were delayed by as little as 10 minutes, I would have probably made it. Now I have a pleasant 5 hour wait in the airport, but as it turned out, I am the fortunate holder of the last seat on the next and last flight of the day.

This realization makes the wait sweeter. It also makes me ponder about the value of pessimism. Not getting back home today would have been disastrous for me. I not only have to teach tomorrow early afternoon, but a senior colleague is scheduled to come by my class to conduct a peer teaching evaluation, crucial for my tenure application. Yes, I will be very tired when I get home, but nothing that a good night sleep would not cure. Well, I would have to break my promise to grade that assignment by tomorrow; Sorry, folks!–I hope you understand.

Is preparing for the worst possible outcome always prudent? Harvard economist, John Kenneth Galbraith, claims that “We all agree that pessimism is a mark of superior intellect.” I don’t know about superior intellect, but this strategy has worked for me well in different settings. For example, whenever I submit a manuscript for review, I immediately start making back up plans for the case if the paper gets rejected. I don’t just make abstract plans, but I usually ask a student co-author to reformat the paper for the next submission target. I don’t just want to be prepared for the possibility of rejection psychologically, I want to know what kind of editing I would have to do to resubmit the paper. If the next publication venue has a smaller page limit, I want to know exactly how many pages we should be prepared to cut. If the opposite is the case, I want to plan what new material we can include to strengthen the paper.  What if your paper is accepted for publication right away? Well, I’d be pleasantly surprised. But what about all the work you have done to shorten or lengthen the paper? This work will never go to waste. Whenever reporting on a research project, I have always found myself needing to write about it in different forms: short, long, and everything in between. These shortened or lengthened manuscripts will serve as building blocks for future publications on the same topic.

Well, it looks like I am going to be able to squeeze in some grading after all. Nothing shortens a wait so productively as grading assignments. Thanks to cloud-base course management systems and 3G tethering, nowadays one can be productive almost everywhere.

What makes a good workshop paper

January 26, 2011 Leave a comment

I have been actively involved as a reviewer for both workshops and conferences. Although the end goal of any reviewing exercise is to recommend the submissions for acceptance or rejection, I notice that I tend to apply vastly different acceptance criteria to workshops and conferences. For a workshop paper, my primary acceptance criteria is whether the paper presents new ideas that can spark discussions at the workshop. I tend to consider the issues of how complete the implementation and validation are as secondary for a workshop paper.

In deciding whether to recommend a workshop submission for acceptance, I ask myself the following question: how likely would pursuing the ideas outlined in this paper lead to solid conference publications in the future, once all the required implementation and validation have been properly carried out? In my book, meeting this criteria is what makes a successful workshop submission.