Archive for February, 2011

Don’t you dare compare surgeons and programmers!

February 27, 2011 3 comments

My recent blog post has recently generated a lot of controversy. After it was featured on HackerNews, my blog received more than 5,000 hits in the course of a couple of days. I never expected that my reflections on the nature of programming interviews and introductory Computer Science education could prove of interest to so many computing professionals.

Quite a few readers of the blog chose to comment, both on the blog itself as well as on various other sites (e.g., YCombinator). Most of the comments were quite positive, complementing my approach to teaching job interviewing skills from early on. Nevertheless, a consistent theme that runs through the comments is that my analogy of comparing experienced surgeons and Computer Scientists is ill-conceived.

As a professor, I was a little disappointed that my correspondents chose to focus on the obvious “the way things are: a professional’s experience and background matter a whole lot in medicine, but are completely discounted in computing–so your analogy is wrong.” In essence, the comments focused mostly on how things are rather than on how they can or should be. It would be more interesting to explore deeper questions such as (1) why is this analogy inaccurate? (2) are we happy with the status quo or do we want to change the situation? (3) what can we do as IT professionals for our profession to enjoy as much prestige and admiration as medicine and law?

Let us explore some of these questions by examining some of my correspondents’ arguments.

“Not all CS Ph.D.s are good programmers.” Of course not! Not all cars can handle off-road well. There are different makes and kinds of cars, and some of them cannot be taken off-road at all. The same applies to CS Ph.D.s. Although getting a Ph.D. in CS is a grueling affair, not all CS careers involve extensive programming. Also Ph.D. programs differ in their quality and reputation. One’s Ph.D. work can be concerned with designing and running users studies, and not involve a significant implementation effort. However, in my analogy, I was talking about CS Ph.D. holders whose background and experience is similar to mine: a Ph.D. in computer systems from a top 10 institution. All of my grad school colleagues who pursued systems research were top notch software developers. Furthermore, all my current Ph.D. students have exceptional Software Engineering skills. If they did not posses these skills, they would not choose to work with me.

Thus, the question is “why a systems-oriented Ph.D. degree from a top 10 institution is not considered a valid proof of the holder’s programming proficiency?” I think one issue is the relative ignorance on the part of the general populace of what it entails to earn such a degree. In my Ph.D. institution in the systems area, it took 6.5 years on average to get a Ph.D. Furthermore, each candidate had to implement state-of-the-art systems that contributed new knowledge about computer system design and implementation. My own Ph.D. dissertation is backed up by close to 100K lines of code, which on average can be on the low side. (I also worked as a commercial software developer for four years prior to grad school, but this is besides the point.) And yet, for people with my background and experience the ability to write code is routinely questioned.

“To practice medicine, one must be certified.” Should we certify computing professionals? Major technology companies offer their own certifications, but those are not universally recognized. If software is of vital importance to the modern society, do we want to require a certification for people producing software for a living? If we are to require a certification, should it be another government function? Or an institution similar to Certified Public Accountants would be more appropriate? Should we as a profession have both oral and written portions of board exams? Would such certifications be trusted by employers? Would have a certification process improve the quality of the IT workforce?

“A medical doctor’s job recommendation is rock solid.” The argument goes that when evaluating a job candidate, one can fully trust the recommendations of the candidate’s former co-workers in case if they are medical doctors; the recommendations of fellow programmers are not worth much, as they can be easily fudged. Why is a recommendation from a colleague is treated so differently? Does our profession suffer from a serious integrity problem?

“Medicine is learned through apprenticeship” Isn’t, ultimately,  programming also learned through apprenticeship? How many fresh college graduates are immediately ready for the challenges of industrial software development? I am sure there are some, and I had them among my own students. But in reality, new professionals learn a whole lot during their first couple of years on the job from more experienced programmers. How can we claim that programming is not learned through apprenticeship? Would it be beneficial to make the apprenticeship component more formal? If we do so, what procedure should we follow?

“When practicing medicine, human lives are at stake” Software systems are at the very foundation of modern society. While the domains in which human lives are directly dependent on software are relatively few, software affects vital facets of human existence. Shouldn’t the job of a Software Engineer be afforded an appropriate level of respect by the society? What can we do as computing professionals to elevate the status of computing? Is it the issue that computing is too young as a discipline? Will the problem take care of itself once computing matures?

If I missed any important arguments, please let me know, and I’ll append this discussion.

Categories: careers

On the essentials of a research deliverable

February 19, 2011 Leave a comment

One of the most influential people in my life was my clarinet teacher, with whom I studied when I was in my teenage years. He was a man of high wisdom and originality who could convey important ideas through humor and amusing stories from his vast past experiences as an orchestral musician of great acclaim. Even though I shifted my professional career away from music almost 20 years ago, I still find the lessons of my old clarinet teacher timely and applicable in domains vastly different from music.

My teacher used to say: “A good musician is a combination of three things: a warm heart, a cold head, and a hard behind.” Indeed, every good musician possesses these three essentials. “A warm heart” refers to the ability to feel the music and interpret it in an expressive and interesting way that can touch the listener’s emotions. Without a warm heart, a performance would be uninteresting and boring. “A cold head” refers to the ability to precisely strategize how to practice a musical piece to the point when you can be sufficiently expressive but without losing control. A musician who cannot plan an effective practicing strategy and who gets completely carried away by one’s emotions during the performance is unlikely to achieve much. Finally, “a hard behind” refers to the ability to practice systematically and overcome all the technical difficulties through perseverance and consistency.

It occurred to me that every research deliverable, such as a research publication accepted to a top tier conference, requires the same three essentials. However, in research they are defined slightly differently. Every research deliverable requires an inspiring idea–“a warm heart.” However, an idea alone is never enough, as one must devise an effective strategy to develop the idea to a publication–“a cold head.” Finally, someone has to do the leg work (implement the system, run the experiments, etc.)–” a hard behind.”

The fascinating thing about research is that unlike in music these essentials can be contributed by separate researchers collaborating on a research project. It is totally possible for one person to come up with a research idea, for another person to find a strategy how this idea can be developed, thinking through all the technical hurdles,  and finally yet for another person to simply follow the instructions of the first two collaborators doing all the required leg work. In fact, that what makes the world of research so versatile.

When it comes to academic research, it is appropriate to play different roles at various stages of your research training. It is totally acceptable and even beneficial to start as “a behind” in some project that was initiated and thought through by some other people. This experience can give a student an immediate exposure to a real research problem. To graduate someone with an M.S. thesis, I expect them to at least to play the role of a brain (i.e., “stop being an ass and work your way up to a brain.”). To graduate someone with a Ph.D., I fully expect them to become the heart of a research project, as the ability to generate original ideas that can be publishable is the main purpose of doctoral training.

In my experience, the transition from “a behind” to “a brain” is fairly straightforward, and I have not witnessed many students who had difficulties with this transition. In fact, I was impressed with several of my students who immediately started playing the role of a brain in non-trivial projects. However, becoming a heart is something with which many students tend to struggle very hard. Finding a good research problem requires that a researcher has good answers to the following three questions: (1) which problems remain unsolved? (2) which problems are worth solving? (3) which problems are solvable? A good research problem must lie on the intersection of the answers to these questions. Being the heart of a research problem is a big honor and even bigger responsibility if other people depend on you to identify new ideas that are worthy of their time and efforts. And that’s why research is so hard.

Categories: academic enterprise

Preparing CS students for programming interviews from day one

February 19, 2011 20 comments

Computer Science is an odd discipline when it comes to job interviews for software developers. Specifically, all interviewees are expected to write code on the whiteboard, irrespective of their background and experience. Let’s assume that I get to interview for a job at Google (which will be a very real possibility if I fail to get tenure). Like a fresh college grad, I will be asked to write code on the whiteboard. The fact that I have close to 15 years of combined industrial and academic experience as well as more than 40 peer reviewed, technical publications will not make any difference. I will have to ask questions like: “How would you remove a node from a doubly linked list?” “How can you find all the neighboring links using a heap?” The fact that I have been teaching these skills to undergraduates for more than five years will be ignored completely. Purportedly, some academics who have not written much code in a while claim to hesitate even applying for a job with Google for that reason.

If we are to use a medical analogy, imagine interviewing an experienced (10-15 years on the job) surgeon for a position in a hospital.  Can you imagine asking the surgeon: “What is a scalpel? What kind of scalpels are there?”
“Now, cut this cadaver as if you were to perform an appendectomy. Good. Now imagine that there were some complications–show how you’d cut in that case.” I hope you are having a good laugh, because such a situation is unimaginable.

Nevertheless, this is the way all candidates for software development jobs are interviewed. As an academic, I cannot really change how the computing industry conducts interviews. However, what I can do as a professor is to prepare my students for the realities of a typical programming interview that consists primarily of writing code on the whiteboard. To that end, I am trying something different in my introductory CS class, in which we are talking about software design and data structures.

I have decided to start introducing more and more of the fundamental CS concepts from the position of a programming job interview. To that end, I stopped using PowerPoint slides. Instead, I quickly explain a new concept on the whiteboard and then immediately pose a programming puzzle using this concept to the class. I ask students to work on the problem for 5-10 minutes and then I ask for volunteers to present their solution by writing it on the whiteboard. Then I discuss and correct the presented solutions from the perspective of a potential interviewer.

Last class, we talked about lists implemented as arrays. After I explained the basic operations supported by java.util.ArrayList, I posed the following challenge to the class:

Assume that you have raw data containing the results of a Web-based survey. This survey asked programmers to specify their primary programming language and the years of experience they have in that language. The data is organized into records as follows: {“Java”, 2}, {“C”, 5}, {“C++”, 10}, {“Java”, 4}, {“Ruby”, 1}, {“PHP”, 7}, {“Java”, 4}, {“Cobol”, 20}, {“Python”, 1}. Write a Java class to represent such a record and implement its equals method. Assume that no getter methods are provided. Write a method that prints all the indexes of the records that have “Java” as the language.

As it turns out, implementing the equals method correctly has several caveats. The ability to implement this methods correctly immediately identifies a certain level of proficiency in Java. Then I went over many of the caveats by correcting a proposed solution. Printing the indexes of Java records using the equals method did not prove very difficult.

Then I imposed an additional constraint on the problem: you are not allowed to use method equals directly; you can use only indexOf and subList methods. These constraints complicate the problem significantly, making it very hard to determine how indexes should be incremented to move through the array. I had a hard time trying to find volunteers to present their solutions. And then I explained why this problem was so hard and how my students should go about solving such problems on the whiteboard.

Will I teach all the remaining technical material in the course using this method? Of course not–it would be counterproductive for the educational goals I set out for my course. Nevertheless, I intend to make this “writing code on the whiteboard” component an integral part of my teaching, as I want my students to get used to what will be a reality of their entire professional careers, as long as they want to continue developing software for a living.

Categories: careers, teaching

Why get a Ph.D.?

February 13, 2011 2 comments

Based on the popular demand, I will give my opinion on why one may consider getting a Ph.D. Every Ph.D. degree holder who has gone through the process, has his or her justification of why they had engaged on and persevered throughout the arduous journey of earning this educational degree.

When I just started my Ph.D. training, I posed this question to my Ph.D. advisor. He pointed out that in his view the only worthy reason for getting a Ph.D. was to become a fully trained Jedi Knight, thus becoming a member of an elite order of technical professionals who have gone through a rigorous training process and passed all the tests satisfactorily. I can certainly confirm that my Ph.D. training taught me how to think about hard problems systematically. Because of my prior training as a professional musician, I already had known how to work toward a non-trivial goal for prolong periods of time. However, I posit that certain facets of my personality are certainly due to the Ph.D. training process I have successfully completed, and this has nothing to do with Computer Science or even research in general.

Let me make this point by telling a story that took place soon after I successfully defended my dissertation. I was in this blissful state of mind that I like to call “a prolonged Friday.” On Friday, the work week is ending up, while the new week is still two days away to be seriously contemplated (if you are not on a tenure-track, of course). Indeed, at that time I completed all my Ph.D. milestones, while my faculty job was not going to start for another couple of months. I was told that people who have just defended their Ph.D. dissertations should try to avoid driving a car for at least a couple of days–as these days have been observed as being particularly accidents prone.

No, I did not get into an accident. I simply made an illegal left turn, a transgression that was observed by a campus cop, who promptly wrote me a ticket. I was clearly at fault there, and I could have just paid the ticket. However, I was about to move to another state and would need to get a car insurance quote. Having a ticket on my record could spoil my chances of getting a good insurance rate. Fortunately, I knew that every three years, the state law allowed a driver to plead Nolo Contendere (no-contest) to a traffic offense. By entering this plea but paying the fine in full, the driver would avoid having the offense posted to his driving record. Since I had not have any traffic offenses in the previous three years, I decided to go to the traffic court and enter my no-contest plea in person.

As it turned out, I arrived to the traffic court way too early. Besides me, there was another person waiting for the court to start. When I asked my waiting companion what he was there for, it turned out that he was a long-haul driver who took some wrong highway while driving through the city. As it turned out, the truckers who drive through big cities are forbidden from taking certain highways. I was fascinated by this unique opportunity to learn about the area of occupation about which I knew little.

Naturally, I took full advantage of this opportunity. For the next hour, I systematically asked the truck driver about the realities of his job, which included the training he had received, his biggest frustrations, and future aspirations. The driver gladly shared his substantial expertise and experience with me. During that hour, I got a fairly detailed idea about not only how truckers make their living, but also how the general economy was affecting their profession. I found out the purpose of the weight stations we see on every major interstates. It was fascinating to find out that the scales in different weight stations do not work identically and how this can significantly complicate a long haul. I also learned that high diesel prices were particularly harmful for independent truckers. Overall, I eagerly engaged in a fascinating learning experience, and I was thoroughly enjoying the process. As I was listening to the trucker talking about his trade, I got a couple of research ideas that, if pursued, could develop new types of distributed information systems that could better inform the driver.

The net effect of this experience was that by the time I left the traffic court, I learnt something about a completely new domain and got a couple of research ideas. Overall, getting this traffic ticket exposed me to new knowledge, an experience that I appreciated greatly. And then I realized that I have really become a Ph.D., a Doctor of Philosophy. In Greek, “philosophy” means “love of knowledge.” Having gone through my Ph.D. training, I acquired a new outlook on life, where I treat all my life experiences as a learning opportunity. Later I also started to evaluate unpleasant events in my life from the perspective of whether they’d make a good story to tell to my students.

In my view, the ability to perceive one’s life as a series of learning experiences is one of the most important outcomes of getting a Ph.D. It is certainly possible that some have a similar outlook on life, acquired innately. Nevertheless, in my case, I credit my Ph.D. training for this ability, for which I am extremely grateful. I do realize that this is not a satisfactory answer to the question of why one may want to get a Ph.D., but this is the best answer I can provide.

The “astonish us” advice to have proposals funded

February 6, 2011 Leave a comment

I am not sure what to make of the opinion piece by Douglas Green that first appeared on the F1000 website and was recently published in the Tomorrow’s Professor newsletter. Prof. Green’s contends that to get research proposals funded one must astonish the reviewers. I am not sure how things work in biomedical research, Prof. Green’s research field. I do believe though that if the astonishment factor were indeed required to have Computer Science research proposals funded, the majority of grantees would have to significantly exaggerate the significance of their ideas.

I may be reading too much into Prof. Green’s advice, but I find its ramifications disturbing. The majority of important research is incremental. What is dismissively characterized as incremental research builds the foundation for astonishing new developments. These developments do not arise out of a vacuum–from time to time someone is incredibly lucky to connect the dots in unexpected ways to discover new knowledge that will have a lasting impact. In Computer Science, such impactful developments often result from combining known techniques in a new way. Nevertheless, someone must have developed these known techniques, and most likely they have done so through incremental research!

The implication that only astonishing ideas are funded is disturbing because it treats research funding as an award for having already accomplished something of value. Calling research funding an award is somewhat misleading. In US academia, research funding is a fundamental commodity required to keep one’s academic enterprise running. We need funding to hire our graduate students as research assistants, buy research equipment, as well as to attend conferences and program committee meetings. Research funding is what pays the bills incurred through the very process of pursuing academic research. Research funding should not be an award, but rather a mandate for a productive researcher to carry out the proposed tasks.

Unfortunately, the federal research dollar is very tight, and getting funded is rapidly becoming an exceptional event rather than a standard part of an academic job. Computer Science is a fundamentally problem-based discipline. We do research to solve important problems that practically impact society at large. While many fundamental CS problems may not have an immediate practical impact, they are still quite significant intellectually. Nevertheless, it is almost impossible to get funding to pursue these problems. Perhaps one way to become more successful in having one’s grants funded is to “astonish” reviewers with exaggerated claims hoping that they will buy them.

What would it mean to astonish proposal reviewers in a mostly problem-based engineering discipline such as Computer Science? For example, one can paint the problems to be solved as having apocalyptic consequences if left unsolved. The two problem areas that are hard to beat are catastrophic financial loses and lose of life. For maximum effectiveness, these two areas can be combined. I.e., “If you do not fund me to solve this problem, we will lose all our financial assets and then perish penniless!!!” Voilà–we have the winner! Will the reviewers be sufficiently astonished to fund such a proposal if it does not present a well thought out research plan and promising preliminary results? I am not sure, but for the sake of the integrity of the scientific process I would like to believe that the answer is no.

Categories: academic enterprise

A pointless NSF report

February 4, 2011 Leave a comment

According to a recent National Science Foundation report, in 2008 while the overall US unemployment rate was 6.6%, for Ph.D. degree holders in science, engineering and health fields it was only 1.7%. I find these results totally predictable and lacking any useful insight.

To obtain a Ph.D. degree holder in a technical field, among other things one must have:
1.) gone through 5-9 additional years of post-baccalaureate formal education,
2.) taken advanced classes in his or her major for at least a couple of years,
3.) passed a rigorous qualifying exam,
4.) shown initiative and ingenuity to identify a non-trivial thesis topic,
5.) demonstrated enormous perseverance to develop this topic into new knowledge,
6.) written and passed through a rigorous peer-review process several original technical publications,
7.) passed a rigorous oral defense,
8.) produced a substantial Ph.D. dissertation manuscript.

All in all, a Ph.D. degree holder is a highly educated individual, who possesses well-developed written and oral communication skills as well as wanton perseverance. Ph.D. holders had the audacity and determination to advance knowledge in their area of pursuit to be granted the highest educational degree possible. In essence, a Ph.D. degree holder has all the qualifications of the Bachelor’s and Master’s degree holders. The only issue is that someone with a Ph.D. may be deemed overqualified for certain jobs, but I just do not this as a serious issue in this economy. Why would anyone be surprised that these people would be unemployed in much smaller numbers than the general population?!

Unfortunately, I suspect that this NSF report will be used in the future to support a specious argument that goes as follows: “No, we are not overproducing Ph.D. level scientists! Haven’t you seen the report? Unemployment among Ph.D. degree holders is lower than in the general population.”

The problem is that the statistics presented in this report does not support this argument! The survey that was used to produce this report asked the question: “Are you gainfully employed?” The question that would have provided much more revealing information would be: “Are you gainfully employed at a job that does require you to hold a Ph.D.?” I believe that as a society we do need to start asking this question if we are to make wise decisions about our scientific policy.

I remember meeting several Computer Science Ph.D. holders who worked as regular programmers, a job that certainly does not require a Ph.D. Some of them were quite bitter about the time they spent getting a Ph.D. saying that it was a waste of time that prevented them from starting their programming careers earlier. If a Ph.D. degree holder works at a job that does not require a Ph.D. it is a waste of this person’s potential as well as the time and efforts expended on his or her training. Some Ph.D. holders may be doing this by choice, but I suspect that the majority choose that kinds of jobs because they could not find employment opportunities that were commensurate with their level of educational attainment.

I do not mind that much that this report is stating the obvious. What I am afraid of is that some irresponsible policy makers will use this report to support their self-serving agenda to the detriment of all of us.

“Writing” a CS Ph.D. dissertation?

February 3, 2011 Leave a comment

I often tell my students that producing a Ph.D. dissertation manuscript should be the easiest part of the entire Ph.D. process. The truly difficult part of Ph.D. training in a technical field is funding a promising topic, getting interesting results, and publishing them. Putting everything together in the final dissertation manuscript should be straightforward.

When it was my time to prepare my dissertation manuscript, my advisor told me that: “As far as I am concerned, a stapler is a perfectly good device for putting dissertations together. If you want to put some effort to get a nicely written manuscript, it is up to you.” What I ended up doing was putting some effort into smoothing out the transitions between my published papers, which made the major chapters of my dissertation. The only section that I wrote from scratch was future work. The entire process from start to finish took about a month.

In my view, the only part of a Ph.D. dissertation that survives the test of time is the acknowledgements. In 5-10 years, the technical content of any engineering dissertation will become either obsolete or irrelevant. If your dissertation research is truly cutting edge and has produced new knowledge, other researchers will build on it. Furthermore, to learn about your research, they’ll read your research papers, not your dissertation. If your dissertation research does not pique other researchers’ interest, it will become irrelevant as the time goes. C’est la vie.

Your human experience of getting a Ph.D., however, does not have an expiration date. That’s why I suggest that you spend time and effort to write a thoughtful acknowledgments section that authentically reflects your experiences of getting a Ph.D. Consider that modern electronic publishing makes your dissertation instantly accessible until the end of times. Your great great great * children will be able to read your dissertation. For them, the technical content of your dissertation would most likely be completely irrelevant. But they’d surely be genuinely interested in what it was like for you to go through your Ph.D. level training.

Do not write your acknowledgements section that looks like a laundry list (i.e., I’d like to thank A, B, .. and Z). Try to think and find answers to the following questions. Who were the people who influenced your thinking? Who helped you out? What did you learn in the process? Why are you a different person now than when you started your Ph.D. training? By answering questions like that, you should be able to get through the most difficult section of your dissertation manuscript with flying colors.

Categories: academic enterprise