FAQ on University Ranking Survey 2005Back


Note: Below are some frequently-asked questions, as well as their answers concerning the Opinion Survey on the Public Ranking of Universities. We welcome the public to submit other questions, so that we can keep on increasing and revising our questions and answers. Please email your questions to <[email protected]>. We will reply to them as soon as possible, and have them uploaded to our webiste.

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Q1: HKU Public Opinion Programme is affiliated to the University of Hong Kong, is there a conflict of roles when conducting an opinion survey on the public ranking of universities?
 
 
A1: In order to ensure the interviewers are not affected by the above-mentioned factor, our interviewers would emphasize HKU Public Opinion Programme is a neutral research body, and we would remind the respondents not to make any decisions based on the fact that we are affiliated to the University of Hong Kong. Should we have not been commissioned by any independent organization to carry out this research independently, we would not take the initiative to conduct any survey of related kind. On the other hand, if we did not carry out the surveys because of our affiliation with the University of Hong Kong, which means in a broader sense, all tertiary institutions and academic staff should not conduct similar kind of survey, or to participate in any universities ranking, and all government departments should not carry out their own public opinion survey too. Therefore, when this is inevitable, the best way is to declare our interests in conducting this survey.
Back to TOP  
 
Q2: Most ideally, who should be in charge of the survey on the public ranking of universities?
 
 
A2: The most ideal research, of course, should be carried out by the most credible organization. It would be the best if all tertiary institutions can collaborate and apply scientific method to carry out a joint survey. However, it seems that the majority of similar type of surveys was usually conducted by independent bodies in many countries. If Hong Kong can develop a better system to rank the tertiary institutions, we are very pleased to pass on all of our data and experience to related research organizations.
Back to TOP  
 
Q3: Should the most ideal survey on the public ranking of universities include public opinion components?
 
 
A3: That really depends. We started conducting surveys on public perception of tertiary institutions in 2001, because we think related statistics is valuable as a reference. Academic organizations always used professional knowledge to monitor the society, but they should also be monitored by the society. Whereas for other organizations, including our partner, Education 18.com, it is neither our responsibility nor strength to understand how they apply the survey data in their professional ranking. Before 2001, Education 18.com have conducted ranking of universities for many times, and we trust their experience and talent, we never participate in or comment on their ranking methodology.
Back to TOP  
 
Q4: Can different universities be compared?
 
 
A4: Different universities have different characteristics; it can be compared but not fully compared. No matter it is public opinion poll or professional rating; comparable units include schools, faculties, departments, subjects, students, courses, or even individual objective-oriented efficiency, etc. However, if we only say differences exist but they are not comparable, it is not reasonable at all. Someone says, orange and apples cannot be compared. But they are both fruits, aren't they? If someone says he/she prefers apple to orange, this is a result of comparison, isn't it? If orange and apple can never be compared, that means red apple cannot be compared with green apple, and milky bananas and wild bananas cannot be compared too. In fact, everything can be compared, it depends on the weight of its meanings. If people think comparing universities is meaningful, that's it.
Back to TOP  
 
Q5: Are there any examples to conduct universities ranking by public opinion polls around the world?
 
 
Q5: US Gallup Poll collaborated with different medias to conduct a telephone survey related to US universities ranking in August 1999 and July 2003. They interviewed more than 1000 respondents in each survey. Both survey polls suggested that Harvard University is always the top university in the US, and this is also widely reported by the CNN. The follow websites cover the news about the two polls.
//www.gallup.com/poll/content/login.aspx?ci=3634 (1999 survey)
//www.gallup.com/poll/content/login.aspx?ci=9109 (2003 survey)
Gallup Poll emphasized, results obtained from public opinion survey would be different from professional ratings. But this represents how the US citizens perceive, and this is also part of Gallup Poll's job.

Back to TOP  
 
Q6: In the survey sample, many respondents have never studied in the university before, do you think it is appropriate for these people to rate the universities or the Vice-Chancellor/President?
 
 
A6: People who have never entered the university do not mean that they know nothing about the universities. For example, elders or parents can understand the good and bad things about the universities through their children or relatives. Many parents have spent a lot of time to compare the universities for the sake of their children. Their opinions would not be the same as the experts, but they still have the freedom to express their opinions. This is exactly the same when the general public can express their opinions about government officials or social policies, although these people may have superficial knowledge, or they know nothing at all.
Back to TOP  
 
Q7: The general public are not experts, what is the meaning of their opinions?
 
 
A7: As reflected from the name, "Opinion Survey on the Public Ranking of Universities in Hong Kong" aims to reflect the general public's subjective appraisal of all the universities in Hong Kong. It is not professional assessments. However, the government and those privately or publicly owned organizations should always understand their own standing through evaluation of the general public. Shortcomings should be fixed whereas strengths should be rewarded. In fact, many universities have conducted public perceptions surveys for internal references. If universities can seek improvements on their public image according to these survey results, why shouldn't an independent organization carry out similar surveys and release the data as public information?

Back to TOP  
 
Q8: If the survey results revealed that the ranking of universities had remarkable variations among people from different levels, how should we interpret the results?
 
 
A8: Our survey revealed that, people from different levels did rate the universities differently, showing that they have not used the same criterion in the rating questions. For example, people with higher education level usually adopt a wider spectrum due to their stronger analytical abilities. However, in terms of the order of ranks, those of the eight universities have remained almost the same, reflecting the extremely stable and reliable relative rankings.

Back to TOP  
 
Q9: Why don't we conduct a ranking survey targeting at people working in the education sector?
 
 
A9: We definitely welcome this kind of survey so as to examine this topic in various angles. Yet, according to our data analyses over the past years, there were in fact no apparent differences on the perceived ranking of universities between respondents with tertiary or primary education standards. Thus, we can see that education standard in fact is not an important factor here. If a survey is really conducted in the education sector, we should better be careful with the possible conflicts of interest as well, especially among those who are working in the tertiary institutions.
Back to TOP  
 
Q10: Are there any problems with the survey when staff or students have to assess their own universities?
 
 
A10: One have to first examine the survey's details, like the sampling method, response rate and questionnaire design, in order to comment on the survey, and therefore no simple conclusion could be drawn here. Overall speaking, it is possible that members from different institutions will have different demands on their own institutions, while those variations may be due to the different backgrounds of members. For example, students with excellent results from a certain university may have higher demands on their university, and therefore will tend to give less positive appraisal to it, and vice versa. Besides, if members from a university do not answer honestly as they worry the survey results will affect the university's reputation, there will be another kind of deviation. This is the shortcoming.
Back to TOP  
 
Q11: Some comments suggesting that in our questionnaire, there are some items which are hard to be understood by the public, such as qualification of the universities' teaching staff, academic research performance and leadership abilities of Vice-Chancellors/Presidents. Are there problems with the design of the questionnaire?
 
 
A11: This is another subjective question which should be judged by independent professionals. For key questions in our previous questionnaires, respondents were asked to "use a scale of 0-10, with 0 representing the worst, 10 representing the best and 5 being half-half, to describe your overall assessment towards XXX". This is a frequently-used scale in Hong Kong and the Western countries. Items mentioned in the questions, such as qualification of the universities' teaching staff, academic research performance, leadership abilities and vision of Vice-Chancellors/Presidents, in fact were just hints to assist the respondents to give comprehensive thoughts to the questions, as well as diluting the "labeling effect" of the universities. Since the phrasing of questions was exactly the same for all institutions, it should be fair even though the respondents' answers were influenced.
Back to TOP  
 
Q12: Why don't we simply ask the respondents to name their perceived best university, but have to use a scale of 0-10 marks to rate each university?
 
 
A12: Absolute rating according to individual attributes, rather than relative ranking was used here, as the former is more conducive to profile analysis of relative strength and weaknesses for individual institutions. Although the amount of resources used is 8 times more than that of a single question, polarization of differences between the best and the worst can be avoided and thus, is a more carefully-designed research method. Take for an example, if University A is a bit better than University B and all respondents can only choose one from two, the former will win with a landslide majority. On the other hand, if respondents can rate according to individual attributes, A may obtain 9 marks and B may obtain 8 marks, and both can be considered as good.
Back to TOP  
 
Q13: In the employers' section of this survey, the sample size was less than 300. Are the results representative?
 
 
A13: Whether the figures are representative enough in fact depend on the acceptable standard error margins. It should be fine if both the standard errors and results are shown together, as readers can judge the accuracy of the figures on their own. In this year's survey, the standard error for this part was less than +/-6 percentage points at 95% confidence level. Many differences among answers could be told from here indeed.

Back to TOP  
 
Q14: There are always controversies arising from the ranking of universities, how could we benefit by mutual discussion?
 
 
A14: We welcome rational discussion, but we are unwilling to participate in any emotional arguments like money-related betting. We have never asked either explicitly and implicitly for support from academics and friends, and we are not afraid of pressure beyond the academic sector as well, including intentions to resort to the public discussion or administrative intervention. Our only concern is to do our best in the public opinion survey part that we are responsible for. We are more than willing to provide comments and assistance should anyone can come up with a set of more well-designed ranking mechanism.
Back to TOP