Should you believe in the Shanghai ranking?

@article{Billaut2009ShouldYB,
  title={Should you believe in the Shanghai ranking?},
  author={Jean-Charles Billaut and Denis Bouyssou and Philippe Vincke},
  journal={Scientometrics},
  year={2009},
  volume={84},
  pages={237-263}
}
This paper proposes a critical analysis of the “Academic Ranking of World Universities”, published every year by the Institute of Higher Education of the Jiao Tong University in Shanghai and more commonly known as the Shanghai ranking. After having recalled how the ranking is built, we first discuss the relevance of the criteria and then analyze the proposed aggregation method. Our analysis uses tools and concepts from Multiple Criteria Decision Making (MCDM). Our main conclusions are that the… 
Modelo de estimación de los indicadores del Academic Ranking of World Universities (Shanghai Ranking)
The interest in global university rankings has grown significantly in the last 10 years. The use of just a handful of indicators, the ease of interpretation of the information they contain and the
On using the Shanghai ranking to assess the research performance of university systems
TLDR
A new look at the Shanghai Jiao Tong Academic Ranking of World Universities to evaluate the performance of whole university systems is taken, with a clear picture emerges: at the aggregate level ARWU seems to be a very reliable one-dimensional scale.
Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review
This integrative review reports on methodological questions about the Shanghai Ranking as a tool for the evaluation of universities, questions that are extensible to other rankings. The paper
On the internal dynamics of the Shanghai ranking
TLDR
It is proposed that the utility and usability of the ARWU could be greatly improved by replacing the unwanted dynamical effects of the annual re-scaling based on raw scores of the best performers.
The Shanghai Global Ranking of Academic Subjects: Room for improvement
Global university rankings have achieved public popularity as they are portrayed as an objective measure of the quality of higher education institutions. One of the latest rankings is the Shanghai
An anatomy of the academic ranking of world universities (Shanghai ranking)
International academic rankings of research universities are widely applied and heavily criticised. Amongst the many international rankings, the Shanghai ranking has been particularly influential.
What do global university rankings really measure? The search for the X factor and the X entity
TLDR
Results support the existence of an underlying entity profile, characterized by institutions normally from the US that enjoy a high reputation in the ARWU and Times Higher Education rankings, and support the idea that rankings lack the capacity to assess university quality in all its complexity.
A Critical Review on World University Ranking in Terms of Top Four Ranking Systems
TLDR
A number of recommendations are proposed that could address the identified inadequacy and considerably improve the ranking system as well as incorporate more participation of higher education institutes form developing world.
Evaluation of Asian University Rankings: Position and Perspective of Leading Indian Higher Education Institutions
TLDR
The aim of this article is to explore possibilities to enhance ranking methodologies using I-distance method, which could be a foundation for impartial framework of universities' assessment, independent of subjectively formed weighting factors.
On the effects of institutional size in university classifications: the case of the Shanghai ranking
TLDR
Primary components analysis of data from the Shanghai ranking (2013 edition) is used to explore factors that contribute to the variation of the total score and concludes that around 30 % of the variation in the ARWU indicators can be attributed to variation in size.
...
...

References

SHOWING 1-10 OF 184 REFERENCES
Should you believe in the Shanghai ranking? An NCDM view
This paper proposes a critical analysis of the “Academic Ranking of World Universities”, published every year by the Institute of Higher Education of the Jiao Tong University in Shanghai and more
International ranking systems for universities and institutions: a critical appraisal
TLDR
The examination of the existing international ranking systems suggests that generic challenges include adjustment for institutional size, definition of institutions, implications of average measurements of excellence versus measurements of extremes, adjustments for scientific field, time frame of measurement and allocation of credit for excellence.
Challenges in Ranking of Universities
In this paper we discuss recent developments in rankings of universities and the impact of these rankings on academia in the context of international benchmarking and evaluation. We focus on
Irreproducibility of the results of the Shanghai academic ranking of world universities
TLDR
In the Shanghai ranking, the dependence between the score for the SCI indicator and the weighted number of considered articles obeys a power law, instead of the proportional dependence that is suggested by the official methodology of the ranking.
The Academic Ranking of World Universities.
Shanghai Jiao Tong University1 has published on the Internet an Academic Ranking of World Universities that has attracted worldwide attention. Institutions are ranked according to academic or
World-wide university rankings: A Scandinavian approach
TLDR
By resampling and reanalyzing the ARWU data, the paper proposes an input-output analysis for measuring universities’ scientific productivity with special emphasis on those universities which meet the productivity threshold in a certain group of universities.
Academic ranking of world universities using scientometrics  - A comment to the “Fatal Attraction”
SummaryThe Institute of Higher Education, Shanghai Jiao Tong University published on the web the Academic Ranking of World Universities and attracted wide attentions worldwide. 60% of their criteria
Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems
The global expansion of access to higher education has increased demand for information on academic quality and has led to the development of university ranking systems or league tables in many
Higher education rankings : robustness issues and critical assessment : how much confidence can we have in higher education rankings?
University rankings are very appealing, in that they provide one with a single number that allows one, at a glance, to situate a given university in the worldwide context. However, this very
Validating DEA as a ranking tool: An application of DEA to assess performance in higher education
TLDR
It is concluded that the critical aspect in classification and ranking is the model, and suggests that DEA is a suitable tool for these types of studies.
...
...