Follow-up bibliometric analysis of neurosurgical publications from Pakistan and institutional comparison with other countries using h-index and i-10 index

Aim: The aim of the study was to analyze the neurosurgical research output of Pakistan and compare it with that of developed countries. Methodology: We conducted a bibliometric analysis of publically available databases for all neurosurgical publications from Pakistan. All indexed peer-reviewed publications from January 2009 to December 2014 where at least one author was affiliated with any neurosurgical departments in Pakistan and research was conducted in Pakistan were selected. Manual and electronic search was done using MeSH terms to search for articles from Pakistan. Articles were then categorized according to design, subspecialty, region, and year. Results: Our search identified 121 articles during the defined study period (mean = 20.16 ± 5.2 papers/year). A relatively constant increase was noticed for the last 6 years, i.e., 2009–2014. From the total 121 references, 100 (82.4%) publications were from one city, and on subanalysis, 80 (66.1%) were from a single institution. Three primary authors cumulatively contributed to 76 (62.8%) of these publications. Almost two-thirds (n = 76, 62.8%) of these publications were published in either regional or international journals while only 37.2% (n = 45) were published in local nonneurosurgery-specific journals. Only one study in the 6-year study period was with Level I evidence (meta-analysis). Conclusion: Neurosurgery research in Pakistan has shown modest improvement in terms of quality and quantity. Collaboration between various centers and channelizing different resources to create national data registries along with basic science laboratories is much needed.

Key-words:

Bibliometric analysis - neurosurgery research - Pakistan

Introduction

The global distribution of healthcare-related research efforts is unequal between high- and low-income countries.[[1]] This has been highlighted on multiple global forums since 1990. The World Health Organization Consultative Expert Working Group's report on financing and coordination of research and development related to diseases that mainly affect the world's poorest people demonstrate ongoing disparity.[[1]] Global burden of disease data and publications from developing countries both suggest that spectrum of diseases affecting developing countries is different from that of developed countries.[[2]] Yet, clinical trials on diseases of relevance to high-income countries are seven-to-eight times more often than those related to low- and middle-income countries.[[3]] As funding opportunities for developing countries decline over the years, it has become imperative for developing countries to make special efforts.[[4]] Each health specialty in developing countries needs to go through a process of rigorous self-assessment to identify and address areas of deficiency.

In 2010, the senior author of this study analyzed the overall research output of neurosurgical community of Pakistan.[[5]] The results were presented at multiple forums and published in an article. Pakistani neurosurgeons published an average of only ten PubMed indexed papers per year. There were neither clinical trials nor systematic reviews or basic science research. This generated much debate within the neurosurgical community, and a number of suggestions were floated to encourage and promote nationwide neurosurgical research.

Over the past few years, the country has seen an increase in the number of neurosurgeons with an apparently greater emphasis on clinical research. To determine if there has been an increase in the number of peer-reviewed articles in neurosurgery from Pakistan, we decided to do a repeat bibliometric analysis of the past 6 years. We also chose one of the Pakistani neurosurgical departments on the basis of its relatively outstanding research contribution to draw a comparison with neurosurgical departments around the world, using h and i-10 indices.

Methodology

We conducted a bibliometric analysis of publicly available databases for all neurosurgery publications from Pakistan. This study was exempted from Institutional Review Board by the Ethics Review Committee of the Aga Khan University Hospital (AKUH). All indexed peer-reviewed publications from January 2009 to December 2014 where at least one author was affiliated with any of the neurosurgical departments in Pakistan and research was conducted in Pakistan were selected. Both international and local scientific search engines (including PubMed/Medline, Cochrane database, Embase, Google Scholar, PakMedinet) were used in various combinations of keywords, including both text words and medical subject headings (MeSH). Search for focused topics was performed by grouping MeSH terms according to the associated neurosurgical subspecialty. Furthermore, names of various neurosurgical departmental chairs along with their respective center and major Pakistani cities were used separately as keywords. Senior neurosurgeons and unit heads of major neurosurgical institutes were also contacted individually in order not to miss publication. Locally published indexed journals, Journal of Pakistan Medical Association, Journal of College of Physicians and Surgeons Pakistan, and Journal of Ayub Medical College were manually searched for neurosurgery publications.

All publications that were not directly related to neurosurgery or in which data were acquired from outside of Pakistan were excluded. Nonscientific correspondence such as bibliographies, news items, and roll calls of reviewers was also excluded. The resulting references were screened on the basis of abstracts and full texts where necessary. This method has been described in detail previously.[[5]] Selected publications were evaluated for basic characteristics. Frequencies and proportions were calculated for publication with respect to study design, specialty, city, institute, and authors. All data were recorded using Statistical Package for Social Sciences version 20 (SPSS IBM, Chicago, IL, USA).

The neurosurgery department with the most publications was then selected, and several variables including i-10 index, h-index, number of citations, and publications per faculty per year were calculated using Google Scholar. These numbers were then compared with published reports of international neurosurgical centers.

Results

Follow-up bibliometric analysis

Neurosurgeons from Pakistan published 121 articles during the defined study period (mean = 20.16 ± 5.2 papers/year). A relatively constant increase was noticed for the last 6 years, i.e., 2009–2014. [[Figure 1]] identifies the number of publications during the study period along with publications from previous years. Majority, i.e., 100 (82.4%) of these publications came from a single city of the country.

Figure 1: Trend of neurosurgery publications per year

Neurosurgeons primarily authored 111 (91.7%) studies while the rest were collaborations with other specialties such as neurology, internal medicine, and basic sciences. Departmental affiliation of authors could not be identified in two publications. The mean number of authors per publication was 4.2 ± 1.4. Sixty-two (62.8%) of the articles were published in international journals, and 45 (37.2%) were published in national PubMed indexed journals. These figures showed that Pakistani neurosurgeons were now more likely to choose an international journal than the last audit.

We stratified articles according to the study design. Case series, case reports, letters to the editor, and forum articles constituted the largest segment of these publications (n = 58, 47.9%). Original articles (retrospective and prospective cohort studies) constituted 47.1% (n = 52) of all publications. There were 5 review articles, 1 meta-analysis, and one quasi-experimental study [[Table 1]]. This was similar to the trends of global neurosurgical publications.[[6]] In terms of specialty, neuro-oncology (n = 28, 23.1%), spine (n = 23, 19.0%), and neurosurgical trauma (n = 19, 15.7%) had most number of publications [[Table 2]]. Eight articles addressed hydrocephalus; one of the studies was related to basic sciences and only two studies were funded. When compared with global neurosurgical publication trends, we found fewer publications on vascular and functional neurosurgery and a comparatively larger proportion of publications on central nervous system infections and hydrocephalus.[[6]]

Table 1: Number of articles according to the study design

Table 2: Publications with respect to subspecialties

Comparison of Pakistani neurosurgical center with international centers

When we further analyze the Pakistani neurosurgical department with the highest publication rate, the AKUH in Karachi, we found that during the study period, the department contributed 80 of 121 (66.1%) publications coming from the country (rate of 16 per year). AKUH is a private university hospital with ISO certification and JCIA accreditation, and the neurosurgery department has six full-time faculty members. For the study period, the department showed a publication/faculty/year rate of 2.2. Three primary authors of the department contributed to 76 (62.8%) of all publications coming from the country. For the study period, the mean h-index of the department is 10, mean i-10 index is 13, and the cumulative i-10 index is 78.

Here, we must mention that these comparisons in no way compare the value of the department or the value of the research conducted there and are subject to the inherent limitations of the various tools for measuring research outputs.

India

On comparing these statistics with neurosurgical centers in India, we found a similar pattern of publication where majority of research is conducted in just a few centers.[[7]] The publication rate of AKUH (16 per year) was much higher than the publication/year/institute rate of India – 3.28 ± 8.22. The Indian data have been analyzed for private and government centers, separately. If AKUH is to be ranked on their tables, it would rank higher than any other private center and would be one of the top 7 centers with respect to total publications in 5 years, total number of citations, number of original papers in the past 5 years, 5-year i-10 index, and 10-year h-index.

Saudi Arabia

The mean h-index of neurosurgical centers in Saudi Arabia is 5.04 (range 0–33),[[8]] and although an individual analysis of various center was not available for comparison, AKU with a mean h-index of 10 (sum of h-indices 60) would certainly rank among the top centers in Saudi Arabia.

Great Britain and Ireland

h-index is a function of time. The older institutes would have a higher h-index. Few of the oldest neurosurgical centers are in Great Britain and Ireland. On comparison of the mean h-index and sum of h-indices, AKU will be ranked easily in the top 10 neurosurgery centers in Great Britain and Ireland.[[9]]

United States of America

A significant proportion of global neurosurgical publications come from the USA.[[6]] However, the relative rates of publication of the USA have remained constant where other countries have shown several hundredfold of improvement in their contributions.[[6]] Similarly, several countries, despite having fewer publications, tend to have more impactful publications.[[6]] Nevertheless, the USA continues to be the global leaders in neurosurgical research.[[10]],[[11]] If AKUH was to be ranked on the tables of North American neurosurgical centers, it would lie someplace between 60 and 70, on the basis of i-10[[5]] index, summed h index, total publications, and total citations. However, most North American centers on the list cannot be compared to AKUH by virtue of their size, and when AKUH is compared to centers of its own size (8 or less faculty members), it would lie within the top 10 neurosurgical centers in the USA.

Discussion

Pakistan has a population of 190 million and is the sixth biggest country in the world.[[12]] The presence of less than 150 neurosurgeons in the country, gives it one of the lowest surgeon to patient ratios in the world, higher only to a few African countries.[[13]],[[14]] This points to an extremely high patient load per surgeon. Not surprisingly, the quality and quantity of neurosurgical and neuroscience research in Pakistan, considering its size, has been below par.[[15]]

The study in 2010 which analyzed the number of neurosurgical publications over a 6-year period identified only about 10 papers from Pakistan per year.[[5]] Results of the current study indicate a steady increase in the number of neurosurgery publications from Pakistan. Compared to the previous study, the number of neurosurgery publications has doubled in the last 6 years. Majority of the publications however were still contributed by a few institutes only. Several of the institutes did not have a single publication during the study period.

Our follow-up bibliometric analysis reveals a rising trend in research publication with an annual growth rate of almost 17%. The proportion of publications in international journals has increased by almost thrice the amount (from 22.6% to 62.8%) with a consequent drop in local journal articles (77.4% to 37.2%). Such a finding indicates improvement in literature coming from the country with publication in greater impact factor journals. This improvement in quality is also implicated by the publication of higher evidence studies such as meta-analysis and case–control studies that were seen during our study period.[[16]],[[17]] Trauma, infections, and hydrocephalus are leading neurosurgical problems in developing countries. These are not reflected as priority in the publication profile of the country. Only 15.7% papers concerned trauma while 8 out of 121 articles addressed hydrocephalus.

According to King's analysis, 31 nations produce 98% of the volume of scientific citations in the world and the rest of the world contribute to the remaining 2% citations.[[18]] In another study on biomedical research profile of nations, Pakistan was among the countries with lowest research profiles (0.86 population per million population).[[19]] Neighboring country India had a much better research profile with 2.82 publications per million population. Developed countries like Japan produced over 240 publications per million population but several low-income African countries such as Kenya, Libya, Gambia, and Gabon had higher number of publications per million population, than both Pakistan and India.[[19]] This shows that although a better economic status of countries helps, it is possible to generate research despite limited resources. Through this paper, we have shown that although Pakistan may lag behind in overall research outputs, and neurosurgical research outputs, one center in the same country is still able to produce reasonable research, despite the limitations, proving that these limitations can be overcome. Indeed, several developing countries have been shown to have several hundredfold increases in neurosurgical research outputs over very short period of time.[[6]] Another prime example is the University of Toronto Neurosurgery Program, which in a span of just 5 years, increased its research outputs from a time that it was comparable to AKU, to now where it ranks even higher than the top program in the USA.[[11]] Salgar outlined three key reasons for the low research output in developing countries, lack of scientific writing training, lack of budget for specialized editorial staff, and lack of expert assistance, which we believe are very valid although our comparison also suggests a fourth reason, which is the smaller number of faculty members within the department.[[20]]

There are several important limitations of our study. Since we have restricted our research to single specialty, it is not a true reflection of overall state of research in Pakistan. A comparison with other specialties would have been useful. For our bibliometric analysis, we only included studies that were indexed in Medline, just as it was done for the last review, which excludes all studies published in nonindexed journals. Second, spine surgery is also performed by spine orthopedic surgeons, and their papers coming from Pakistan were excluded. For the institutional comparison part of the study, we must admit that it is limited by the well-discussed, inherent limitations of the tools available for comparison, i.e., h-index, i-10 index, etc.[[21]],[[22]]

Conclusion

Neurosurgery research in Pakistan has shown modest improvement in terms of quality and quantity. There exists a large disparity between institutes in scientific research output within the country. A homogeneous distribution of resources, strong will on the part of clinicians, and collaboration between institutes can help improve the scientific output from the country.

Conflict of Interest

There are no conflicts of interest.

Financial support and sponsorship

Nil.

• References

• 1 World Health Organization. Research and Development to Meet Health Needs in Developing Countries: Strengthening Global Financing and Coordination. Report of the Consultative Expert Working Group on Research and Development: Financing and Coordination. Geneva: World Health Organization; 2012.
• 2 Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: A systematic analysis for the global burden of disease study 2010. Lancet 2012;380:2095-128.
• 3 Røttingen JA, Regmi S, Eide M, Young AJ, Viergever RF, Ardal C, et al. Mapping of available health research and development data: What's there, what's missing, and what role is there for a global observatory? Lancet 2013;382:1286-307.
• 4 Leach-Kemon K, Chou DP, Schneider MT, Tardif A, Dieleman JL, Brooks BP, et al. The global financial crisis has led to a slowdown in growth of funding to improve health in many developing countries. Health Aff (Millwood) 2012;31:228-35.
• 5 Shamim MS, Enam SA, Kazim SF. Neurosurgical research in pakistan: Trends of publication and quality of evidence. Clin Neurol Neurosurg 2011;113:107-10.
• 6 Hauptman JS, Chow DS, Martin NA, Itagaki MW. Research productivity in neurosurgery: Trends in globalization, scientific focus, and funding. J Neurosurg 2011;115:1262-72.
• 7 Madhugiri VS. Publication performance and research output of neurology and neurosurgery training institutes in India: A 5-year analysis. Neurol India 2015;63:338-46.
• 8 Jamjoom AB. Survey of h-index for neurosurgeons in Saudi Arabia. Neurosciences (Riyadh) 2015;20:392-5.
• 9 Knight J. Academic impact rankings of neurosurgical units in the U.K. And Ireland, as assessed with the h-index. Br J Neurosurg 2015;29:637-43.
• 10 Burnett MG, Stein SC, Grady MS. What we research: Survey of american association of neurological surgeons and congress of neurological surgeons member publications. J Neurosurg 2004;100:73-8.
• 11 Ponce FA, Lozano AM. Academic impact and rankings of American and Canadian neurosurgical departments as assessed using the h index. J Neurosurg 2010;113:447-57.
• 12 Population Welfare Department (Punjab); 2015. Available from: http://www.pwd.punjab.gov.pk/. [Last accessed on 2015 May 19].
• 13 El Khamlichi A. African neurosurgery: Current situation, priorities, and needs. Neurosurgery 2001;48:1344-7.
• 14 Shamim MS, Tahir MZ, Godil SS, Kumar R, Siddiqui AA. A critical analysis of the current state of neurosurgery training in Pakistan. Surg Neurol Int 2011;2:183.
• 15 Shamim MS, Shamim MS. Research and publications: Where do we stand? J Pak Med Assoc 2009;59:62-4.
• 16 Zafar SN, Khan AA, Ghauri AA, Shamim MS. Phenytoin versus leviteracetam for seizure prophylaxis after brain injury – A meta analysis. BMC Neurol 2012;12:30.
• 17 Shamim MS, Enam SA, Tahir MZ, Khan M. Prospective case control evaluation of epidural midazolam for improving pain and ambulation after microdiscectomy. J Pak Med Assoc 2012;62:561-5.
• 18 King DA. The scientific impact of nations. Nature 2004;430:311-6.
• 19 Rahman M, Fukui T. Biomedical publication – Global profile and trend. Public Health 2003;117:274-80.
• 20 Salager-Meyer F. Scientific publishing in developing countries: Challenges for the future. J Engl Acad Purposes 2008;7:121-32.
• 21 Spearman CM, Quigley MJ, Quigley MR, Wilberger JE. Survey of the h index for all of academic neurosurgery: Another power-law phenomenon? J Neurosurg 2010;113:929-33.
• 22 Aoun SG, Bendok BR, Rahme RJ, Dacey RG Jr. Batjer HH. Standardizing the evaluation of scientific and academic performance in neurosurgery – Critical review of the “h” index and its variants. World Neurosurg 2013;80:e85-90.

Publication History

Article published online:
09 September 2022