When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings

@article{Adler2009WhenKW,
  title={When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings},
  author={Nancy J. Adler and Anne-Wil Harzing},
  journal={Academy of Management Learning and Education},
  year={2009},
  volume={8},
  pages={72-95}
}
“Not everything that can be counted counts, and not everything that counts can be counted.”—Albert Einstein Has university scholarship gone astray? Do our academic assessment systems reward scholarship that addresses the questions that matter most to society? Using international business as an example, we highlight the problematic nature of academic ranking systems and question if such assessments are drawing scholarship away from its fundamental purpose. We call for an immediate examination of… 

Figures from this paper

Deep impact: unintended consequences of journal rank
TLDR
It is suggested that abandoning journals altogether, in favor of a library-based scholarly communication system, will ultimately be necessary because this new system will use modern information technology to vastly improve the filter, sort and discovery functions of the current journal system.
CARVing Quality vs. Characterizing Capital: The Scholarly Capital Model, a Portfolio Approach
TLDR
This rejoinder to the initial article advocates a disciplined metatheoretical pluralism in which work is evaluated for admission to the discourse not by conformance to “normal science” but rather by its conforming to its stated met Atheoretical commitments.
On Academic Rankings, Unacceptable Methods, and the Social Obligations of Business Schools
TLDR
This study argues that the fundamental methodological shortcomings of existing rankings, and ultimately any ordinal ranking system, limit the value of current rankings and calls on academic institutions and organizers to take a collective stand against existing rankings and to embrace the strategic use of multidimensional alternatives that faithfully serve prospective students, parents, and other key stakeholders.
Quantitative and Qualitative Rankings of Scholars
In a former article we started to argue that publication and citation rankings of individual scholars do not effectively measure research quality, which should in fact be the essence of evaluation
Leading questions: Journal rankings, academic freedom and performativity: What is, or should be, the future of Leadership?
Pressure on academics to publish articles in ‘top’ journals continues to grow. In tandem, we have seen a proliferation of journal rankings, claiming to provide a guide to the quality of journals. As
Examining Scholarly Influence: A Study in Hirsch Metrics and Social Network Analysis
This study is part of a program aimed at creating measures to enable a fairer and more complete assessment of a scholar’s contribution to a field thus bringing greater rationality and transparency to
Scholarly Impact Revisited
Executive Overview Scholarly impact is one of the strongest currencies in the Academy and has traditionally been equated with number of citations—be it for individuals, articles, departments,
Ideational Influence, Connectedness, and Venue Representation: Making an Assessment of Scholarly Capital
TLDR
Three dimensions of scholarly capital (ideational influence, connectedness and venue representation) are identified in this paper as part of a scholarly capital model (SCM) and it is shown how one might use the measures to evaluate scholarly research activity.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 128 REFERENCES
From the Editors: what makes a study sufficiently international?
Journal of International Business Studies (2008) 39, 180–183. doi:10.1057/palgrave.jibs.8400351 From the flow of submissions that arrive on the desk of the JIBS Editors, it appears that the issue of
Aardvark et al.: quality journals and gamesmanship in management studies
TLDR
It is concluded that laughter is both the appropriate reaction to such farce, and also the stimulus to reform, in the situation in management studies.
Lost in publication: how measurement harms science
Measurement of scientific productivity is difficult. The measures used (impact factor of the journal, citations to the paper being measured) are crude. But these measures are now so univer- sally
How business schools lost their way.
TLDR
To regain relevancy, business schools must rediscover the practice of business and find a way to balance the dual mission of educating practitioners and creating knowledge through research.
Identifying the big question in international business research
Buckley (2002) argues that the international business (IB) research agenda may be running out of steam, because no big research question has currently been identified. Buckley also asks whether the
How Much Better are the Most Prestigious Journals? The Statistics of Academic Publication
TLDR
There is much overlap in articles in different prestige strata, and theory implies that about half of the articles published are not among the best ones submitted to those journals, and some of the manuscripts that belong in the highest-value 20% have the misfortune to elicit rejections from as many as five journals.
Are Articles in “Top” Management Journals Necessarily of Higher Quality?
This study investigates the appropriateness of using publication of an article in a top (specifically, top five) management journal as a proxy for its quality. Social Science Citation Index citation
A Theoretical and Empirical Analysis of Journal Rankings: The Case of Formal Lists
This study examines the use of formal rankings of journals by management departments for personnel decision purposes. We posit that the probability of adopting a list of formal rankings is related to
The Rain Man Cometh—Again
For colleges and universities October has traditionally been a tough month — growing darkness, impending rain and cold, the creeping realization that the football team won't win that many games. Not
...
1
2
3
4
5
...