Extended Abstract – EXTENDING TECHMINING METHODS session at “1st Global TechMining Conference” 2011
Author(s):Jan Youtie, Julia Melkers, Luciano Kay (Georgia Institute of Technology)
This paper explores the use of bibliometrics in R&D assessment. In particular, we put forth a bibliographic-coupling based indicator as a measure of “centerness” and present its characteristics relative to the experience of an NSF science center.
Although much of federally-funded science is given to single investigators or small teams, science is increasingly organized in large scale federally-funded science centers.(1) The notion behind this organization is that, in addition enabling research to occur on a greater scale, science centers can yield creative findings that reflect conceptual collisions across the disciplinary or geographic boundaries between the center’s scientific members. One of the criteria for reviewing science centers is based on this notion of encouraging these kinds of conceptual collisions – the criterion of centerness.
Centerness represents the extent to which diverse scientific parties that are part of the center, but likely did not previously conduct collaborative research, come together to produce original and more creative research outputs.(2) Such outputs are believed to be based on multi-dimensional conceptual collisions within the center and occur to a greater extent than they would have if the researchers were working alone or in pre-center research groups. Traditional characterizations of collaborative research using bibliometric analysis rely on scientific publication co-authorships.(3) Co-authorship measures can be conveniently and unobtrusively operationalized, yet they often yield indicators (especially for a small center) with low variance. This low variance is usually because authors tend to be relatively stable in who they like to publish with.(4) As a consequence, co-authorship measures may not uncover, for example, whether an influx of diverse disciplines made possible by a center nourishes different patterns of research outputs. The co-authorship measure thus can understate the extent of the centerness phenomenon in comparison with richer findings from questionnaires orinterviews. These more obtrusive measures can reveal considerable collaborative, multi- and inter-disciplinary activity in the center. The problem is that questionnaires and interviews can be rather intrusive into the already busy schedules of research center members and cannot be well done in a retrospective manner to produce pre-center measures.
This paper characterizes the use of bibliographic coupling as a measure of centerness in an NSF-funded science center.(5) We define bibliographic coupling as the appearance of a reference work in the cited references of articles from two or more center researchers. We compare this measure of centerness in two type periods: before the creation of the center and since the establishment of the center. We also examine changes in bibliographic coupling indicators alongside changes in co-authorship networks and centerness evidence in a social network survey of center participants. The social network survey shows substantial increase in centerness, especially along the dimensions of collaboration on methodological and conceptual tasks.
The two bibliometric measures – co-authorship based and bibliographic coupling based –exhibit divergent results. The bibliographic coupling measure, defined as the sum of all shared references divided by the total number of references, increases dramatically between the pre-center and since-center time periods. In contrast, the co-authorship network remains relatively unchanging, even though the number of publications increases. This increase in centerness as represented in bibliographic coupling does not dramatically change when self-citations are removed.
REFERENCES
(1) Geiger, R. L. (1990). Organized Research Units–Their Role in the Development of University Research. The Journal of Higher Education, 61(1), 1-19. Corley, E., Boardman, P. C. & Bozeman, B. (2006). Design and the management of multi-institutional research collaborations: Theoretical implications from two case studies. Research Policy, 35(7), 975-993. Gray, D., Johnson, E. C. & Gidley, T. R. (1986). Industry-University Projects and Centers: An Empirical Comparison of Two Federally Funded Models of Cooperative Science. Evaluation Review, 10(6), 776-793.
(2) Youtie, J. Corley, E. 2010. Federally-Sponsored Multidisciplinary Research Centers: Learning, Evaluation, and Vicious Circles. Evaluation and Program Planning, Vol 34 No. 1, pp. 13-20.
(3) Melin, G., (2000) “ Pragmatism and self organization: Research collaboration at the individual level” Research Policy 29: 31-40. De Solla Price, D., and Beaver, D. (1966) “ Collaboration in an Invisible College” American Psychologist 21(1): 1011-1018
(4) Rafols I, Meyer M (2007). How cross-disciplinary is bionanotechnology? Explorations in the specialty of molecular motors. Scientometrics 70(3), 633-650.
(5) Kessler, M.M. (1963). Bibliographic coupling between scientific papers. American Documentation, 14, 10–25. Garfield, Eugene (2001). From Bibliographic Coupling to Co-Citation Analysis via Algorithmic Historio-Bibliography, A Citationist’s Tribute to Belver C. Griffith, Philadelphia: Drexel, November 27, 2001.