Irreproducibility of the results of the Shanghai academic ranking of world universities

@article{Florian2007IrreproducibilityOT,
  title={Irreproducibility of the results of the Shanghai academic ranking of world universities},
  author={Razvan V. Florian},
  journal={Scientometrics},
  year={2007},
  volume={72},
  pages={25-32}
}
I discuss the difficulties that I encountered in reproducing the results of the Shanghai ranking of world universities. [] Key Result This shows that the results of the Shanghai ranking cannot be reproduced, given raw data and the public methodology of the ranking.
Reproducibility of the Shanghai academic ranking of world universities results
TLDR
The paper presents a set of straightforward procedures to estimate raw results and final relative scores and can safely state that the results of the Shanghai academic ranking of world universities are in fact reproducible.
A fresh approach to evaluating the academic ranking of world universities
TLDR
The statistical I-distance method is proposed to be used and based on a sample containing the top 100 ranked universities, the results show a significant correlation with the official ARWU list.
Should you believe in the Shanghai ranking?
TLDR
The view is that the Shanghai ranking, in spite of the media coverage it receives, does not qualify as a useful and pertinent tool to discuss the “quality” of academic institutions, let alone to guide the choice of students and family or to promote reforms of higher education systems.
Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review
This integrative review reports on methodological questions about the Shanghai Ranking as a tool for the evaluation of universities, questions that are extensible to other rankings. The paper
Should you believe in the Shanghai ranking? An NCDM view
This paper proposes a critical analysis of the “Academic Ranking of World Universities”, published every year by the Institute of Higher Education of the Jiao Tong University in Shanghai and more
Modelo de estimación de los indicadores del Academic Ranking of World Universities (Shanghai Ranking)
The interest in global university rankings has grown significantly in the last 10 years. The use of just a handful of indicators, the ease of interpretation of the information they contain and the
The Shanghai Global Ranking of Academic Subjects: Room for improvement
Global university rankings have achieved public popularity as they are portrayed as an objective measure of the quality of higher education institutions. One of the latest rankings is the Shanghai
An anatomy of the academic ranking of world universities (Shanghai ranking)
International academic rankings of research universities are widely applied and heavily criticised. Amongst the many international rankings, the Shanghai ranking has been particularly influential.
On using the Shanghai ranking to assess the research performance of university systems
TLDR
A new look at the Shanghai Jiao Tong Academic Ranking of World Universities to evaluate the performance of whole university systems is taken, with a clear picture emerges: at the aggregate level ARWU seems to be a very reliable one-dimensional scale.
The effect of university mergers on the Shanghai ranking
TLDR
This paper exhibits the response of the ARWU indicators and rankings to institutional mergers in general, and specifically analyses the universities of France that are engaged in a major amalgamation process motivated in part by a desire for higher international rankings.
...
...

References

SHOWING 1-2 OF 2 REFERENCES
Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods
A system of input, output, and efficiency indicators is sketched out, with each indicator related to basic research, applied research, and experimental development. Mainly, this scheme is inspired by
Universităţile din România şi clasamentul Shanghai
Rezumat. Articolul estimează performanţa ştiinţifică a universităţilor din România comparativ cu cele mai bune universităţi din lume, conform metodologiei folosite îm clasamentul Shanghai din 2005.