Evaluating impact of a publication

Question

How do you evaluate publications? Im currently searching for a CS research topic and reading various papers. My dilemma on reading a paper usually is - is it really worthwhile continuing research in this topic?

what are the indicators of impact of research?

btw, im currently interested in - Liveness analysis. what do you think of it?

Answer 1

The highest impact papers are those that are cited most. Citeseer and the ACM will show you how often a paper is cited. Really influential papers are cited long after they cease to actually be useful. Everyone cites papers they haven't read because they are certain that the paper is the definitive reference.

The definitive way to know the good papers is to have looked at everything in the area, but the question really is where to start.

A good strategy I've found is to start in textbooks, as they will sometimes cite the most important work at the time they are written. Obviously, use a recent text. Liveness comes under compilers, so try Cooper/Torczon, Muchnick, or Appel. Look at the end of the chapters, where there are often mini-literature surveys. (I don't usually recommend the Dragon Book. I just checked it though, and there's nothing useful.)

Finally, look for others to do the work for you. Look at the comments on the top of source files in gcc or LLVM. Look for survey papers. Look for papers that you already know the content of who touched on the topic, and follow the citation trail.

Example: Lets take a quick example. I remember a few papers that use liveness. One is Sam Guyer's 2006 PLDI paper, "Free Me". And I did a bit of work on SSA recently, and people use liveness a lot with SSA. I don't remember a specific recent paper, but I expect that Briggs' semi-pruned SSA probably talks about liveness, so that's somewhere to go second.

So looking at Guyer's paper, I went to the bibliography, and there were maybe two papers that mentioned liveness:

• M. Hirzel, A. Diwan, and J. Henkel. On the usefulness of type and liveness accuracy for garbage collection and leak detection. ACM TOPLAS
• H. Inoue, D. Stefanovi´c, and S. Forrest. Object lifetime prediction in Java. Technical Report TR-CS-2003-28, University of New Mexico, May 2003.

TOPLAS is a quality journal, so I'd look there first. And so on...

Answer 2

You should only target publications in very well established journals that are peer-reviewed such as IEEE. Some publications will include a section about possible future work in the Summary and Conclusion section. You could start by seeing if any of their suggestions have been taken in future publications.