While I was composing my blog You've Got (Misinformation) Mail: on the China Milk Scare and the Bar Code, I remembered reading a mail from UP (via its all-faculty account which I was subscribed to) about its stand on UP's supposed "ranking" among 500 other universities in the world as conducted by Times Higher Education Supplement in collaboration with Quacquarelli Symonds (THES-QS). According to THES-QS as reported by the ABS-CBN News website (http://www.abs-cbnnews.com/nation/10/13/08/ateneo-rise-2008-qs-world-university-rankings), UP's rank rose from 398 to 276, while the Ateneo de Manila University rose from rank 451 to 254 this year.
No, not that I can't accept that the Ateneo ranked higher than UP (which is partly true :p). What I just found out today from the UP System website that UP gave an official comment and stand about the National University's "ranking" among THES-QS's top universities in 2008. Here's the article on the UP website underscoring UP's stand on the issue:
UP did not participate in the THES-QS university rankings
Friday, October 17, 2008
From http://www.up.edu.ph/features.php?i=94
In the 2008 university rankings recently released by the Times Higher Education Supplement (THES) and Quacquarelli Symonds (QS), the University of the Philippines rose from 398 in 2007 to 276 this year. Ateneo de Manila University rose from 401-500 to 254. De La Salle University was ranked 415th and the University of Santo Tomas was ranked 470th.
This is the third year that such a survey has been conducted and its results given prominence by local dailies. But according to UP Vice President for Public Affairs Cristina Pantoja Hidalgo, UP has never agreed to participate in this survey. In fact, this year, President Emerlinda R. Roman did not even receive an invitation to be a part of it. Nor did she receive any questionnaire to answer.
What she did receive was an email message from QS Asia Regional Director (Asia Pacific), Mandy Mok, informing her that UP had “gone up in the rankings” for 2008. The email also contained an invitation to buy “an attractive package” from THES-QS. The “package price,” which includes a banner on topuniversities.com, a full page full color ad in Top Universities Guide 2009, and a booth at Top Universities Fair 2009, amounts to $48,930.
Since UP was not invited to participate and therefore had not provided any data, UP officials do not know where and how the figures were obtained on which the ranking was based, Hidalgo said.
“UP can hardly be expected to spend more than 2 million pesos on publicity for itself involving a survey conducted by an organization that refuses to divulge where it obtains its data,” she added.
In 2007, UP was invited to participate in the survey, but when THES-QS refused to explain where it obtained the data used to determine UP’s rank in the 2006 survey, university officials decided not to accept the invitation to be part of the 2007 survey. In 2006, UP was ranked No. 299, and Ateneo was ranked No. 500.
UP wrote THES-QS in July 2007, informing them of UP’s decision not to be a part of the survey; and again in September 2007, requesting the organization to respect UP’s decision. In response, research assistant Saad Shabir wrote back saying that if it did not receive the information it would be “forced to use last year’s data or some form of average.”
Surveys and rankings obviously have their usefulness. But, as the National University—status officially granted to it with its new Charter on its centennial year—UP feels that before it agrees to participate in such an exercise, it must carefully examine the indices by which it is to be evaluated. It also needs to be convinced about the reliability of the methodology used in the exercise.
The THES-QS ranking is supposedly meant to serve as “the definitive guide to universities around the world which truly excel.” In evaluating institutions it computes half of the index based on its reputation as perceived by academics (peer review 40%) and global employers (recruiter review 10%). Since it does not specify who are surveyed or what questions are asked, the methodology is problematic.
In an earlier statement released in August this year, and carried by several national dailies, UP said: “Even peers require standardized input data to review. But according to the International Ranking Systems for Universities and Institutions: A Critical Appraisal, published by BioMed Central, the Times simply asks 190,000 ‘experts’ to list what they regard as the top 30 universities in their field of expertise without providing input data on any performance indicators (http://www.biomedcentral.com/1741-7015/5/30). Moreover, the survey response rate among selected experts was found to be below 1%. In other words, on the basis of possible selection biases alone, the validity of the measurement is shaky.” (See Panao, “Only Two RP Universities Made It…” UP Newsletter, August 2007, p. 5.)
According to the statement, the other half of the index is based on such indicators as student-to faculty ratio, the number of foreign faculty and foreign students in the university, and the number of citations in internationally accredited publications. “Data for these indicators depend on the information that participating institutions submit. An institution’s index may be easily distorted if it fails to submit data for the pertinent indicators, or if it chooses not to participate.”
Hara, hara, hara, another misinformation for everyone especially the UP community.... And to think that many media outfits - major ones at that - picked up the story. for sure many have either been disillusioned by UP's big jump in the rankings, or has made some *tsk tsk* because despite of the increase, Ateneo managed to "overtake" UP in the rankings (in which I can say, "I can't believe it").
I am also at a loss as to why THES-QS still included UP in the 2008 rankings despite its signification (thank you CRS for the new word :p) to withdraw from the survey. What benefit will THES-QS get from UP's continued inclusion in its surveys?
I don't know if UP sent their comment to those media companies who bannered the story. I have yet to read one or hear one from those media companies. I just hope that the issue has been rectified so that UP students and alumni will not be deceived by questionable information gathered and supplied by a supposedly "respectable" institution like THES-QS.
Notes:
1. Another article reiterated UP's stand on another questionable THES-QS ranking made in 2007. For details, read the article Only two RP universities made it to the world’s 500 best; but must the rankings be taken seriously? on http://www.up.edu.ph/features.php?i=87
2. To see the actual "rankings" mentioned here, visit the QS Top Universities website, http://www.topuniversities.com/.
Interestingly enough, a certain Professor Judith Kinnear, the Vice Chancellor of Massey University, testified that "(The THE - QS Rankings is a) wonderful external acknowledgement of several university attributes, including the quality of its research, research training, teaching and employability." Oh, really? :p
