One approach for measuring the impact and diffusion of academic research is by studying the quantity and pattern of citations to published research findings. The literature on the diffusion of technological innovations frequently uses patent citation data to study the spread of technological knowledge (for example, the volume by Jaffe and Trajtenberg (2002)). Citations are a data source for investigating linkages from results to subsequent research more generally in the study of the dissemination and creation of ideas and knowledge.The study of citation patterns may be useful for addressing a variety of policy concerns about organisation, funding and incentives in academic research.
Citations to scholarly publications are often used to evaluate the impact of research findings and the contributions of individual researchers. In the natural and social sciences, citation counts have been used to evaluate the productivity of individuals for appointments, salaries, and research awards for years. Empirical studies by Hammermesh, Johnson and Weisbrod (1982), Diamond (1986), and others find that the compensation of economists rises significantly with cumulative citations. Within economics, ranking university departments, journals and economists using raw citation counts or recursively weighted citation counts is increasingly popular.
The origins of citations
The use of citation data to allocate research resources and reward individual effort has generated a substantial literature on the validity of using citations to measure the value of research. Much of the analysis of the use and usefulness of citations is associated with the sociology of science following Robert K. Merton. Merton (1973) observes that citations to the publications of researchers by other researchers may be influenced by considerations beyond the strict linkage between new findings and old findings. Citation practices depend on academic culture and traditions, and the incentives to cite may not accord with the notion that citations provide an approximate record of the origin of ideas used.
Richard Posner (2000) summarises the insights of the literature on the incentives to cite. He distinguishes between informational and strategic citations. Informational citations serve an important expositional role by identifying the context of the article, allowing the incorporation of supporting evidence without repetition, establishing linkages to authoritative works, and acknowledging the priority of others. Each corresponds to the notion that citations measure the importance of a cited work. Strategic citations are made simply to benefit the author and are due to the rewards to being cited and the role of others in the peer reviewing process. The possibilities of strategic citation behaviour and editorial bias have received attention in the economics literature (Posner (2000) and Laband and Piette (1994), respectively).
Information costs are also important for understanding citations. Because scholarship is costly, the rates of citation to specific articles can also depend on the promotion by the authors or more general personal networking by authors. Networking increases the familiarity with a researcher’s work lowering the information cost to others of citing that work. In the economics profession, networking has been studied in the context of citation circles based on graduate education (Stigler and Friedland (1975), gender differences in citation frequency (Ferber (1988)) and the research gains associated with co-authorship (Sauer (1988) and Laband and Tollison (2000)).
Death and citations
In a recent working paper, “The Life Cycle of Scholars and Papers in Economics - the ‘Citation Death Tax’,” we consider whether citations depend only on the intrinsic contribution of a publication or if they are influenced by the author’s professional presence by estimating the impact of premature death of productive economists on their citations. Premature death terminates activities that help enhance the prominence of scholar’s publications, such as presenting papers, pursuing follow-up research, encouraging related research by others, and supervising Ph.D. students. Some of these costs may be mitigated in circumstances where the research was done jointly with active and productive scholars. The loss of some citations may result from the termination of the incentives for strategic citation. A researcher’s death can also have the direct effect of reducing research activity on a particular topic.
We consider the potential importance of professional presence and networking for citations by first constructing a sample of publications and citations to highly-cited economists who died well before retirement age during the period from 1975 to 1997. We identified sixteen such economists and used the Web of Science to assemble a panel of the annual citations of 428 papers written by these 16 economists over the years 1957 to 2006. The members of the sample vary in terms of prominence. Five have average citations per year between 2 and 10, five receive average citations/year between 10 and 20, 4 have average citations/year between 20 and 100, and two, with average citations/year exceeding 100, died before co-authors received the Nobel prize (Fischer Black and Amos Tversky). We removed citations by the authors to their own work.
The dynamics of the citations generally correspond to the expected life cycle pattern of an article in which its citation rate first rises sharply as awareness of the article spreads. Then citations gradually decline over time. This pattern is violated for publications by the two otherwise Nobel laureates in the sample. We constructed the intertemporal path of missing citations of a paper in the sample relative to the hypothetical citations had the author been alive, and calculated for each paper the cumulative missing citations. The break in citations following the economist’s death yields an estimate of the missing citations.
Our results are mixed. For half of the economists in the sample, we identify a large and significant “citation death tax” for the average paper written by these scholars. For these authors, the estimated average missing citations per paper attributed to premature death ranges from 40% to 140% (the overall average is about 90%), and the annual costs of lost citations per paper are in the range 3% and 14%. Hence, a paper written ten years before the author’s death avoids a loss of citations that ranges between 30% and 140%. For the other half of the sample, there is no citation death tax; and for two extraordinary scholars in this second group, Black and Tversky, citations took off over time, reflecting the growing recognition of their contributions.
The value of presence
Our sample of prominent economists who died early suggests that being there is important for generating citations. It is natural to expect that an author’s premature death leads to a sudden drop in strategic citations to his publications. Informational citations may also fall following a researcher’s death because he can no longer promote his own research or continue training students. Researchers play a role in promoting their own research simply by being visible to the research community and continuing to press a current research agenda through presentations and follow-up papers. Identifying the precise importance of all these factors requires much more detailed information about various dimensions of networking than available in the data on publications and citations. Our results do suggest two directions of implication. The first is to confirm the conclusion of Robert Merton and successive sociologists of science that citations are not an especially pure measure of scientific impact and may be affected by strategic considerations or information costs. The other is that networking may have a significant impact on the dissemination of research findings and diffusion of valuable knowledge. This second bears on the organisation of research and can have implications for public policies towards the production of basic research.
Aizenman, Joshua and Kenneth Kletzer (2008), “The Life Cycle of Scholars and Papers in Economics – the ‘Citation Death Tax’,” National Bureau for Economic
Research, Working Paper 13891, March.
Diamond, Arthur M. (1986), “What is a Citation Worth?” Journal of Human Resources, Vol. 21, pp. 200-215.
Ferber, Marianne A. (1988), “Citations and Networking,” Gender and Society, Vol. 2, No. 1, March, pp. 82-89.
Hamermesh, Daniel S., George E. Johnson and Burton A. Weisbrod (1982), “Scholarship, Citations and Salaries: Economic Rewards in Economics,” Southern Economic Journal, Vol. 49, No. 2, pp. 472-481.
Jaffe, Adam and Manuel Trajtenberg (2002), Patents, Citations, and Innovations: A Window on the Knowledge Economy, Cambridge, MA: MIT Press.
Laband David N. and Michael J. Piette (1994) “Favoritism versus Search for Good Papers: Empirical Evidence Regarding the Behavior of Journal Editors” The Journal of Political Economy, Vol. 102, No. 1., pp. 194-203.
Laband David N. and Robert D. Tollison (2000) “Intellectual Collaboration” The Journal of Political Economy, Vol. 108, No. 3., pp. 632-662.
Merton, Robert K. (1973), The Sociology of Science, Chicago: University of Chicago Press.
Posner, Richard A. (2000), “An Economic Analysis of the Use of Citations in the Law,” American Law and Economics Review, Vol. 2, No. 2, pp. 381-406.
Sauer Raymond D. (1988) “Estimates of the Returns to Quality and Coauthorship in Economic Academia,” The Journal of Political Economy, Vol. 96, No. 4., pp. 855-866.
Stigler, George and Claire Friedland (1975), “The Citation Patterns of Doctorates in Economics, Journal of Political Economy, Vol. 83, No. 3, pp. 477-507.