U of T has been ranked the world’s 18th best university by Times Higher Education (THE) and 29th by Quacquarelli Symonds (QS) — all while remaining the top university in Canada for both publications’ 2020 university rankings. The jump from the 21st to 18th spot on the THE ranking system and the decline from 28th to 29th on the QS system raises questions regarding the methodology behind university rankings. Why has U of T risen in one and declined in the other?
THE and QS look at some of the same factors when deciding the overall ranking of universities worldwide. However, the biggest difference arises in the system used to weigh each category.
The THE ranking places a 30 per cent emphasis on teaching, 30 per cent on research, 30 per cent on citations, 7.5 per cent on international outcome, and 2.5 per cent on industry income.
The QS breakdown follows a different path: 40 per cent is allocated to academic reputation, 10 per cent for employer reputation, 20 per cent for faculty-student ratio, 20 per cent for citations per faculty, 5 per cent for international faculty ratio, and 5 per cent international student ratio.
The most pronounced difference between the two ranking systems is the score that U of T received for the category of citations. THE calculates total citations, while QS calculates citations per faculty. While THE awarded a score of 93.6, U of T received nearly half that score from QS, a 43.9.
This disparity occurred due to the process by which citations per faculty is calculated. QS recognizes that different departments, such as natural sciences and medicine, tend to get more citations than arts and humanities departments. Therefore, after 2015, it adjusted its calculation by weighing five subjects — Arts & Humanities, Engineering & Technology, Life Sciences & Medicine, Natural Sciences, and Social Sciences & Management — equally and attributing a 20 per cent weight to each subject area.
Following this change, from 2015 to 2016, U of T’s rankings dropped significantly in the QS system, from 20th to 34th. During this same time period, U of T remained consistent on the THE rankings, and even jumped up a spot from 20 to 19.
In addition, the ratio of students to faculty members has a 20 per cent weighting in the QS ranking. THE considers this category to fall under the umbrella of “teaching” and only ascribes it a 4.5 per cent weighting.
Criticism of ranking systems
A qualitative factor that is constantly left out of popular ranking systems is student life and student satisfaction. The 2020 Maclean’s student satisfaction survey of Canadian undergraduates saw U of T failing to even crack the top 19.
However, Maclean’s itself faced criticism in 2006 when 11 Canadian universities declined to provide direct information to the magazine — citing their dissatisfaction with Maclean’s methodology as the cause for the boycott.
In a 2013 Varsity article, David Naylor, former U of T president, touched on another failing of the ranking system. He talks about a ‘catch-22’ situation, whereby universities who increase their citations per faculty score see a decrease in their faculty to student ratio score.
These various methodologies highlight concerns with ranking — particularly just how subjective they can be in measuring a university’s value. There is no widely agreed set or system of variables, calling into question how much stock should be put into such lists.