Measuring the Quality of Journals and Journal Articles: The Impact Factor Tells but a Portion of the Story

ABSTRACT

Much emphasis has been placed on the impact factor, a measure of journal article citation rates used as a surrogate marker of both journal and article quality. There is little doubt that the impact factor is an important audit of journal article usage, as it in essence provides a measure of the level of peer attention being given to articles within journals and (by extrapolation) of the level of attention being given to the journal containing those articles. However, the impact factor has its limitations and only tells a very small fraction of the overall story regarding the utility of journals and the articles within them. In addition, the impact factor can be easily manipulated. The current article includes a brief review of the current and past uses and abuses of the impact factor and describes some of its strengths and limitations. In addition, a review of past publications, primarily from this journal, has been undertaken to help show the potential use of alternative measures of journal utility, such as Internet-based journal sessions and article downloads. The evaluation of previously published articles also helps serve to illustrate, by example, some of the limitations to the use of the impact factor as the sole determinant of a journal's “quality.”

KEYWORDS

Impact factor - journal quality - review - commentary

REFERENCES

• 1 Garfield E. The history and meaning of the journal impact factor.  JAMA. 2006;  295 90-93
• 2 Bergemann D. The impact factor game: counting goals or measuring fair play?.  Thromb Haemost. 2006;  96 105-106
• 3 Lippi G, Franchini M. The growing impact of publications in haemostasis and thrombosis journals.  Thromb Haemost. 2007;  97 1049-1050
• 4 Favaloro E J. More on the impact factor and thrombosis and haemostasis journals: benefits and limitations.  Thromb Haemost. 2007;  98 475-476
• 5 Lee C, Kessler C, Barback J. Haemophilia: January 2008.  Haemophilia. 2008;  14 1-4
• 6 Favaloro E J. Medical research in New South Wales 1993-96 assessed by Medline publication capture.  Med J Aust. 1998;  169 617-622
• 7 Dunlop L C, Skinner M P, Bendall L J et al.. Characterisation of gmp-140 (P-selectin) as a circulating plasma protein.  J Exp Med. 1992;  175 1147-1150
• 8 Favaloro E J. Clinical application of the PFA-100^® .  Curr Opin Hematol. 2002;  9 407-415
• 9 Favaloro E J. Collagen binding assay for von Willebrand Factor (VWF:CBA): detection of von Willebrands disease (VWD), and discrimination of VWD subtypes, depends on collagen source.  Thromb Haemost. 2000;  83 127-135
• 10 Favaloro E J. Utility of the PFA-100^® for assessing bleeding disorders and monitoring therapy: a review of analytical variables, benefits and limitations.  Haemophilia. 2001;  7 170-179
• 11 Favaloro E J, Grispo L, Exner T, Koutts J. Development of a simple collagen binding assay aids in the diagnosis of, and permits sensitive discrimination between, type I and type II von Willebrand's disease.  Blood Coagul Fibrinolysis. 1991;  2 285-291
• 12 Favaloro E J, Facey D, Henniker A. Use of a novel Platelet Function Analyser (PFA-100™) with high sensitivity to disturbances in von Willebrand factor to screen for von Willebrand's disease and other disorders.  Am J Hematol. 1999;  62 165-174
• 13 Favaloro E J. Laboratory assessment as a critical component of the appropriate diagnosis and sub-classification of von Willebrand's disease.  Blood Rev. 1999;  13 185-204
• 14 Favaloro E J. Differential expression of surface antigens on activated endothelium.  Immunol Cell Biol. 1993;  71 571-581
• 15 Favaloro E J, Grispo L, Dinale A, Berndt M, von Koutts J. Willebrand's disease: laboratory investigation using an improved functional assay for von Willebrand factor.  Pathology. 1993;  25 152-158
• 16 Favaloro E J, Facey D, Grispo L. Laboratory assessment of von Willebrand factor: use of different assays can influence the diagnosis of von Willebrand's disease, depending on differing sensitivity to sample preparation and differential recognition of high molecular weight VWF forms.  Am J Clin Pathol. 1995;  104 264-271
• 17 Favaloro E J, Moraitis N, Bradstock K, Koutts J. Co-expression of haemopoietic antigens on vascular endothelial cells: a detailed phenotypic analysis.  Br J Haematol. 1990;  74 385-394
• 18 Favaloro E J, Smith J, Petinos P, Hertzberg M, Koutts J. on behalf of the RCPA Quality Assurance Program (QAP) in Hematology Scientific Hemostasis Advisory Panel . Laboratory testing for von Willebrand's disease: an assessment of current diagnostic practice and efficacy by means of a multi-laboratory survey.  Thromb Haemost. 1999;  82 1276-1282
• 19 Mammen E F, Comp P C, Gosselin R et al.. PFA-100 system: a new method for assessment of platelet dysfunction.  Semin Thromb Hemost. 1998;  24 195-202
• 20 Mammen E F, Koets M H, Washington B C et al.. Hemostasis changes during cardiopulmonary bypass surgery.  Semin Thromb Hemost. 1985;  11 281-292
• 21 Mammen E F, Alshameeri R S, Comp P C. Preliminary data from a field trial of the PFA-100 system.  Semin Thromb Hemost. 1995;  21(Suppl 2) 113-121
• 22 Mammen E F. The haematological manifestations of sepsis.  J Antimicrob Chemother. 1998;  41(Suppl. A) 17-24
• 23 Mammen E F. Antithrombin: its physiological importance and role in DIC.  Semin Thromb Hemost. 1998;  24 19-25
• 24 Mammen E F. Disseminated intravascular coagulation (DIC).  Clin Lab Sci. 2000;  13 239-245
• 25 Wojtukiewicz M Z, Tang D G, Ciarelli J J et al.. Thrombin increases the metastatic potential of tumor cells.  Int J Cancer. 1993;  54 793-806
• 26 Mammen E F. Pathogenesis of venous thrombosis.  Chest. 1992;  102 640S-644S
• 27 Mammen E F. Coagulation defects in liver disease.  Med Clin North Am. 1994;  78 545-554
• 28 Mammen E F. Coagulation abnormalities in liver disease.  Hematol Oncol Clin North Am. 1992;  6 1247-1257
• 29 Jafri S M, Vanrollins M, Ozawa T, Mammen E F, Goldberg A D, Goldstein S. Circadian variation in platelet-function in healthy-volunteers.  Am J Cardiol. 1992;  69 951-954
• 30 Niitsu Y, Jakubowski J A, Sugidachi A et al.. Pharmacology of CS-747 (Prasugrel, LY640315), a novel, potent antiplatelet agent with in vivo P2Y(12) receptor antagonist activity.  Semin Thromb Hemost. 2005;  31 184-194
• 31 Savi P, Herbert J M. Clopidogrel and ticlopidine: P2Y(12) adenosine diphosphate-receptor antagonists for the prevention of atherothrombosis.  Semin Thromb Hemost. 2005;  31 174-183
• 32 van Giezen JJJ, Humphries R G. Preclinical and clinical studies with selective reversible direct P2Y(12) antagonists.  Semin Thromb Hemost. 2005;  31 195-204
• 33 Esmon C T. Inflammation and the activated protein C anticoagulant pathway.  Semin Thromb Hemost. 2006;  32(Suppl 1) 49-60
• 34 Hechler B, Cattaneo M, Gachet C. The P2 receptors in platelet function.  Semin Thromb Hemost. 2005;  31 150-161
• 35 Boeynaems J M, Communi D, Gonzalez N S et al.. Overview of the P2 receptors.  Semin Thromb Hemost. 2005;  31 139-149
• 36 Rak J, Milsom C, May L et al.. Tissue factor in cancer and angiogenesis: the molecular link between genetic tumor progression, tumor neovascularization, and cancer coagulopathy.  Semin Thromb Hemost. 2006;  32 54-70
• 37 Robson S C, Wu Y, Sun X F et al.. Ectonucleotidases of CD39 family modulate vascular inflammation and thrombosis in transplantation.  Semin Thromb Hemost. 2005;  31 217-233
• 38 Jurk K, Kehrel B E. Platelets: physiology and biochemistry.  Semin Thromb Hemost. 2005;  31 381-392
• 39 Favaloro E J. Laboratory identification of von Willebrand disease: technical and scientific perspectives.  Semin Thromb Hemost. 2006;  32 456-471
• 40 Falanga A. Thrombophilia in cancer.  Semin Thromb Hemost. 2005;  31 104-110
• 41 Osterud B, Bjorklid E. Sources of tissue factor.  Semin Thromb Hemost. 2006;  32 11-23
• 42 Levi M, van der Poll T, ten Cate H. Tissue factor in infection and severe inflammation.  Semin Thromb Hemost. 2006;  32 33-39
• 43 Cattaneo M. The P2 receptors and congenital platelet function defects.  Semin Thromb Hemost. 2005;  31 168-173
• 44 Gachet C, Hechler B. The platelet P2 receptors in thrombosis.  Semin Thromb Hemost. 2005;  31 162-167
• 45 Versteeg H H, Ruf W. Emerging insights in tissue factor-dependent signaling events.  Semin Thromb Hemost. 2006;  32 24-32
• 46 Favaloro E J, Wong RCW, Silvestrini R et al.. A multilaboratory peer assessment quality assurance program-based evaluation of anticardiolipin antibody, and beta(2)-glycoprotein I antibody testing.  Semin Thromb Hemost. 2005;  31 73-84
• 47 Mannucci P M. Laboratory detection of inherited thrombophilia: a historical perspective.  Semin Thromb Hemost. 2005;  31 5-10
• 48 Michiels J J, Berneman Z, Van Bockstaele D et al.. Clinical and laboratory features, pathobiology of platelet-mediated thrombosis and bleeding complications, and the molecular etiology of essential thrombocythemia and polycythemia vera: therapeutic implications.  Semin Thromb Hemost. 2006;  32 174-207
• 49 Roberts H R, Hoffman M, Monroe D M. A cell-based model of thrombin generation.  Semin Thromb Hemost. 2006;  32(Suppl 1) 32-38
• 50 Kher N, Marsh J D. Pathobiology of atherosclerosis-a brief review.  Semin Thromb Hemost. 2004;  30 665-672
• 51 Galli M, Barbui T. Antiphospholipid syndrome: clinical and diagnostic utility of laboratory tests.  Semin Thromb Hemost. 2005;  31 17-24
• 52 Gallus A S. Management options for thrombophilias.  Semin Thromb Hemost. 2005;  31 118-126
• 53 Jafri S M, Metha T P. Periprocedural management of anticoagulation in patients on extended warfarin therapy.  Semin Thromb Hemost. 2004;  30 657-664
• 54 Favaloro E J. Diagnostic issues in thrombophilia: a laboratory scientist's view.  Semin Thromb Hemost. 2005;  31 11-16
• 55 Wong RCW, Adelstein S, Gillis D, Favaloro E J. Development of consensus guidelines for anticardiolipin and lupus anticoagulant testing.  Semin Thromb Hemost. 2005;  31 39-48
• 56 Rajagopal V, Bhatt D L. Controversies of oral antiplatelet therapy in acute coronary syndromes and percutaneous coronary intervention.  Semin Thromb Hemost. 2004;  30 649-656
• 57 Warkentin T E. An overview of the heparin-induced thrombocytopenia syndrome.  Semin Thromb Hemost. 2004;  30 273-284
• 58 Fareed J, Leong W L, Hoppensteadt D A et al.. Generic low-molecular-weight heparins: some practical considerations.  Semin Thromb Hemost. 2004;  30 703-713
• 59 Andersen J C. Advances in anticoagulation therapy: the role of selective inhibitors of factor Xa and thrombin in thromboprophylaxis after major orthopedic surgery.  Semin Thromb Hemost. 2004;  30 609-618
• 60 Walenga J M, Hoppensteadt D A. Monitoring the new antithrombotic drugs.  Semin Thromb Hemost. 2004;  30 683-696
• 61 Hertzberg M S. Genetic testing for thrombophilia mutations.  Semin Thromb Hemost. 2005;  31 33-38
• 62 Tang D G, Conti C J. Endothelial cell development, vasculogenesis, angiogenesis, and tumor neovascularization: an update.  Semin Thromb Hemost. 2004;  30 109-118
• 63 Tilley R, Mackman N. Tissue factor in hemostasis and thrombosis.  Semin Thromb Hemost. 2006;  32 5-10
• 64 Bunnett N W. Protease-activated receptors: how proteases signal to cells to cause inflammation and pain.  Semin Thromb Hemost. 2006;  32(Suppl 1) 39-48
• 65 Rock G. The management of thrombotic thrombocytopenic purpura in 2005.  Semin Thromb Hemost. 2005;  31 709-716
• 66 Michiels J J, Gadisseur A, Budde U et al.. Characterization, classification, and treatment of von Willebrand diseases: a critical appraisal of the literature and personal experiences.  Semin Thromb Hemost. 2005;  31 577-601
• 67 Galbusera M, Noris M, Remuzzi G. Thrombotic thrombocytopenic purpura-then and now.  Semin Thromb Hemost. 2006;  32 81-89
• 68 Davie E W, Kulman J D. An overview of the structure and function of thrombin.  Semin Thromb Hemost. 2006;  32(Suppl 1) 3-15
• 69 Zakarija A, Bennett C. Drug-induced thrombotic microangiopathy.  Semin Thromb Hemost. 2005;  31 681-690
• 70 Michiels J J, Raeve H D, Berneman Z et al.. The 2001 World Health Organization and Updated European Clinical and Pathological Criteria for the Diagnosis, Classification, and Staging of the Philadelphia Chromosome-Negative Chronic Myeloproliferative Disorders.  Semin Thromb Hemost. 2006;  32 307-340
• 71 Lowe E J, Werner E J. Thrombotic thrombocytopenic purpura and hemolytic uremic syndrome in children and adolescents.  Semin Thromb Hemost. 2005;  31 717-730
• 72 Ruf W, Mueller B M. Thrombin generation and the pathogenesis of cancer.  Semin Thromb Hemost. 2006;  32(Suppl 1) 61-68
• 73 Qu L, Kiss J E. Thrombotic microangiopathy in transplantation and malignancy.  Semin Thromb Hemost. 2005;  31 691-699
• 74 Girolami B, Girolami A. Heparin-induced thrombocytopenia: a review.  Semin Thromb Hemost. 2006;  32 803-809
• 75 Pini M, Spyropoulos A C. Prevention of venous thromboembolism.  Semin Thromb Hemost. 2006;  32 755-766
• 76 Kucher N, Goldhaber S Z. Risk stratification of acute pulmonary embolism.  Semin Thromb Hemost. 2006;  32 838-847
• 77 Tiu R V, Mountantonakis S E, Dunbar A J, Schreiber M J. Tumor lysis syndrome.  Semin Thromb Hemost. 2007;  33 397-407
• 78 Ageno W, Squizzato A, Garcia D, Imberti D. Epidemiology and risk factors of venous thromboembolism.  Semin Thromb Hemost. 2006;  32 651-658
• 79 Bernardi E, Pesavento R, Prandoni P. Upper extremity deep venous thrombosis.  Semin Thromb Hemost. 2006;  32 729-736
• 80 Stellos K, Gawaz M. Platelets and stromal cell-derived factor-1 in progenitor cell recruitment.  Semin Thromb Hemost. 2007;  33 159-164
• 81 Tefferi A, Elliott M. Thrombosis in myeloproliferative disorders: prevalence, prognostic factors, and the role of leukocytes and JAK2V617F.  Semin Thromb Hemost. 2007;  33 313-320
• 82 Siegel-Axel D I, Gawaz M. Platelets and endothelial cells.  Semin Thromb Hemost. 2007;  33 128-135
• 83 Ho V T, Linden E, Revta C, Richardson P G. Hepatic veno-occlusive disease after hematopoietic stem cell transplantation: review and update on the use of defibrotide.  Semin Thromb Hemost. 2007;  33 373-388
• 84 De Stefano V, Rossi E, Za T, Leone G. Prophylaxis and treatment of venous thromboembolism in individuals with inherited thrombophilia.  Semin Thromb Hemost. 2006;  32 767-781
• 85 Daub K, Lindemann S, Langer H et al.. The evil in atherosclerosis: adherent platelets induce foam cell formation.  Semin Thromb Hemost. 2007;  33 173-178
• 86 Laux V, Perzborn E, Kubitza D, Misselwitz F. Preclinical and clinical characteristics of rivaroxaban: a novel, oral, direct factor Xa inhibitor.  Semin Thromb Hemost. 2007;  33 515-523
• 87 Cohn D M, Roshani S, Middeldorp S. Thrombophilia and venous thromboembolism: implications for testing.  Semin Thromb Hemost. 2007;  33 573-581
• 88 Prandoni P, Lensing AWA, Pesavento R. New strategies for the treatment of acute venous thromboembolism.  Semin Thromb Hemost. 2006;  32 787-792
• 89 Blum W, Porcu P. Therapeutic apheresis in hyperleukocytosis and hyperviscosity syndrome.  Semin Thromb Hemost. 2007;  33 350-354
• 90 Middeldorp S, Levi M. Thrombophilia: an update.  Semin Thromb Hemost. 2007;  33 563-572