HKU POP releases popularity figures of CE and the GovernmentBack

Press Release on April 1, 2014

Special Announcement

To facilitate academic study and rational discussion, the Public Opinion Programme (POP) of the University of Hong Kong has already released for public examination some time ago via the “HKU POP Site” (http://hkupop.pori.hk) the raw data of all 43 regular rating surveys of CE CY Leung, along with related demographics of respondents. POP today releases the raw data of the latest which is the 44th CE rating survey. Users can process the data using SPSS, which should be freely accessible by all university staff and students from their own university. Ordinary users can choose to use a variety of freeware to run their own analysis. Please follow normal academic standards when using or citing such data. Because the raw data is already open, POP will simply follow its usual practice of releasing the following statistics in it press release: mean scores, sampling errors at 95% confidence level, latest change in value, whether such change is statistically significant, as well as other basic contact information. Readers are welcome to compile other statistics at will, preferably covering all 44 datasets.

Moreover, because the Hong Kong Institute of Asia-Pacific Studies at the Chinese University of Hong Kong (HKIAPS) and POP are releasing almost parallel surveys of CE popularity, and because POP was recently criticized for producing findings different from that of HKIAPS back in May 2012, Robert Chung, the Director of POP today repeats in this commentary some of the points he made in his article published on 2 June 2012 entitled “Differences in Polls on CE-elect”, in order to facilitate today’s discussion on such differences. The article can be downloaded in full from http://hkupop.pori.hk/english/columns/columns125.html.

Abstract

POP interviewed 1,030 Hong Kong people between 24 and 27 March 2014 by means of a random telephone survey conducted by real interviewers. Our latest survey shows that the latest support rating of CE CY Leung stands at 46.6 marks, his approval rate 27%, disapproval rate 56%, giving a net popularity of negative 30 percentage points. The figures did not change much as compared to two weeks ago. For the SAR Government, compared to one month ago, its popularity has significantly improved. Satisfaction rate goes up by 7 percentages points, while dissatisfaction rate goes down by 4 percentage points, giving a net satisfaction rate of negative 10 percentage points. As for the five specific policy areas, in terms of net satisfaction rate, only one item has registered a positive value, which is relations with the Central Government at positive 8 percentage points. The other four items that have registered negative values are maintaining economic prosperity at negative 2 percentage points, improving people’s livelihood at negative 10 percentage points, protection of human rights and freedom at negative 18 percentage points, and developing democracy at negative 27 percentage points. Compared to three months ago, only the net satisfaction rate of improving people’s livelihood has registered significant improvement, while that of the other four remained stable. The maximum sampling error of all percentage figures is +/-4 percentage points at 95% confidence level, while the sampling error of rating figures and net values need another calculation. The response rate of the survey is 69%.

Points to note:
[1] The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists can check out the details of the survey there.

[2] The sample size of this survey is 1,030 successful interviews, not 1,030 x 68.7% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of percentages is +/-4 percentage points at 95% confidence level, while the sampling error of rating figure and net value needs another calculation. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state “sampling error of rating not more than +/-1.6 and sampling error of percentages not more than +/-4%, sampling error of net values not more than +/-7% at 95% confidence level”. Because POP introduced “rim weighting” in 2014, during the transition period, whether changes in various figures are beyond sampling errors are based on tests using the same weighting methods. That is, to test whether the first set of figures collected in 2014 is significantly different from that of the previous survey, both sets of data are rim weighted before testing, instead of using simple computation of the published figures.
[4] Because of sampling errors in conducting the survey(s) and the rounding procedures in processing the data, the figures cannot be too precise, and the totals may not be completely accurate. Therefore, when quoting percentages of the survey(s), journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses “computerized random telephone survey” to camouflage its IVS operation, it should be considered unprofessional.

Latest Figures

POP today releases on schedule via the “POP SITE” the latest popularity figures of CE CY Leung and the HKSAR Government. From 2014, POP enhanced the previous simple weighting method based on age and gender distribution to “rim weighting” based on age, gender and education (highest level attended) distribution. The latest figures released today have been rim-weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in 2013 year-end and the educational attainment (highest level attended) distribution collected in the 2011 Census. Herewith the contact information of various surveys:

Date of survey	Overall sample size	Response rate	Maximum sampling error of percentages^[6]
24-27/3/2014	1,030	68.7%	+/-3%

[6] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Questions using only sub-samples would have bigger sampling error. Sampling errors of ratings are calculated according to the distribution of the scores collected.
[7] The figures shown in the “latest change” column of this press release have been tested after “rim weighting” data collected in this and last surveys. The structural effect of using the new weighting method is small, around -2% to +1% for percentage figures, while statistical significance tests are not affected.

Recent popularity figures of CE CY Leung and people’s satisfaction of the overall performance of the HKSAR Government are summarized as follows:

Date of survey	15/1/14^[8]	18-22/1/14	4-6/2/14	17-20/2/14	3-6/3/14	24-27/3/14	Latest Change
Sample base	1,017	1,014	1,030	1,031	1,017	1,030	--
Overall response rate	66.7%	67.6%	65.5%	67.8%	65.9%	68.7%	--
Finding	Finding	Finding	Finding	Finding	Finding	Finding and error^[9]	--
Rating of CE CY Leung	48.9^[10]	47.0^[10]	47.9	46.4	47.5	46.6+/-1.6	-0.9
Vote of confidence in CE CY Leung	29%	29%	25%^[10]	23%	25%	27+/-3%	+2%
Vote of no confidence in CE CY Leung	53%^[10]	54%	56%	56%	56%	56+/-3%	--
Net approval rate	-24%^[10]	-26%	-32%^[10]	-33%	-31%	-30+/-5%	+1%
Satisfaction rate of SARG performance^[11]	--	26%	--	24%	--	31+/-4%	+7%^[10]
Dissatisfaction rate of SARG performance^[11]	--	43%	--	45%	--	41+/-4%	-4%
Net satisfaction rate	--	-16%	--	-21%	--	-10+/-7%	+11%^[10]
Mean value^[11]	--	2.7+/-0.1 (Base =551)	--	2.6+/-0.1 (Base =567)	--	2.8+/-0.1 (Base=566)	+0.2^[10]

[8] This survey was the instant survey after the Policy Address and only asked rating of CE as well as his vote of confidence.
[9] All error figures in the table are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Media can state “sampling error of rating not more than +/-1.6, sampling error of percentages not more than +/-4%, sampling error of net values not more than +/-7% at 95% confidence level” when quoting the above figures. The error margin of previous survey can be found at the POP Site.
[10] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[11] Collapsed from a 5-point scale. The mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of positive level, where 1 is the lowest and 5 the highest, and then calculate the sample mean. Starting from March 2011, this question only uses sub-samples of the tracking surveys concerned. The sample size for this series is 573.

The latest survey showed that, CE Leung Chun-ying scored 46.6 marks, and 27% supported him as CE, his net approval rate is negative 30 percentage points. Regarding people’s appraisal of the overall performance of the HKSAR Government, the latest figures revealed that 31% were satisfied, whereas 41% were dissatisfied, thus net satisfaction stands at negative 10 percentage points. The mean score is 2.8, meaning close to “half-half”.

Recent figures on people's appraisal of the five specific policy areas of the HKSAR Government are tabulated as follows:

Date of survey	21-27/3/13	13-19/6/13	23-26/9/13	16-19/12/13	24-27/3/14	Latest Change^[12]
Sample base^[12]	605-681	568-688	566-589	545-691	555-600	--
Overall response rate	67.1%	68.0%	63.8%	68.6%	68.7%	--
Finding	Finding	Finding	Finding	Finding	Finding & error^[13]	--
Relation with the Central Government: Satisfaction rate^[14]	44%^[15]	39%^[15]	44%^[15]	41%	41+/-4%	--
Relation with the Central Government: Dissatisfaction rate^[14]	25%	30%^[15]	26%	32%^[15]	33+/-4%	+1%
Net satisfaction rate	19%^[15]	9%^[15]	18%^[15]	9%^[15]	8+/-7%	-1%
Mean value^[14]	3.2+/-0.1^[15] (Base=600)	3.1+/-0.1 (Base=590)	3.2+/-0.1 (Base=507)	3.0+/-0.1^[15] (Base=489)	3.0+/-0.1 (Base=523)	--
Maintaining economic prosperity: Satisfaction rate^[14]	36%	28%^[15]	38%^[15]	31%^[15]	36+/-4%	+5%
Maintaining economic prosperity: Dissatisfaction rate^[14]	33%	41%^[15]	36%^[15]	39%	38+/-4%	-1%
Net satisfaction rate	3%	-13%^[15]	2%^[15]	-8%^[15]	-2+/-7%	+6%
Mean value^[14]	3.0+/-0.1 (Base=603)	2.7+/-0.1^[15] (Base=552)	2.9+/-0.1^[15] (Base=572)	2.8+/-0.1 (Base=668)	2.9+/-0.1 (Base=585)	+0.1
Improving people’s livelihood: Satisfaction rate^[14]	27%^[15]	27%	34%^[15]	26%^[15]	33+/-4%	+7%^[15]
Improving people’s livelihood: Dissatisfaction rate^[14]	42%	51%^[15]	44%^[15]	48%	43+/-4%	-5%^[15]
Net satisfaction rate	-15%^[15]	-24%^[15]	-10%^[15]	-23%^[15]	-10+/-7%	+13%^[15]
Mean value^[14]	2.7+/-0.1 (Base=589)	2.6+/-0.1 (Base=650)	2.8+/-0.1^[15] (Base=571)	2.6+/-0.1^[15] (Base=576)	2.8+/-0.1 (Base=556)	+0.2^[15]
Protecting human rights and freedom: Satisfaction rate^[14]	28%^[15]	31%	37%^[15]	29%^[15]	29+/-4%	--
Protecting human rights and freedom: Dissatisfaction rate^[14]	43%^[15]	43%	35%^[15]	49%^[15]	47+/-4%	-2%
Net satisfaction rate	-15%^[15]	-12%	2%^[15]	-20%^[15]	-18+/-7%	+2%
Mean value^[14]	2.7+/-0.1^[15] (Base=646)	2.7+/-0.1 (Base=663)	2.9+/-0.1^[15] (Base=530)	2.6+/-0.1^[15] (Base=571)	2.6+/-0.1 (Base=528)	--
Pace of democratic development: Satisfaction rate^[14]	22%	25%	28%	25%	24+/-4%	-1%
Pace of democratic development: Dissatisfaction rate^[14]	50%^[15]	50%	48%	52%	50+/-4%	-2%
Net satisfaction rate	-28%	-25%	-20%	-27%	-27+/-7%	--
Mean value^[14]	2.5+/-0.1 (Base=604)	2.5+/-0.1 (Base=634)	2.6+/-0.1 (Base=532)	2.5+/-0.1 (Base=526)	2.5+/-0.1 (Base=549)	--

[12] The frequency of this series of questions is different from that of CE popularity and SARG overall performance. Comparisons, if made, should be synchronized using the same intervals. Starting from 2011, these questions only uses sub-samples of the tracking surveys concerned, the sample size for each question also varies.
[13] All error figures in the table are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Media can state “sampling error of percentages not more than +/-4%, sampling error of net values not more than +/-7% at 95% confidence level” when quoting the above figures. The error margin of previous survey can be found at the POP Site.
[14] Collapsed from a 5-point scale. The mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of positive level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[15] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Of the 5 specific policy areas, people were most satisfied with the government’s handling of its relation with the Central Government, with a net satisfaction rate of positive 8 percentage points. The Government’s performance in maintaining economic prosperity and improving people’s livelihood followed, attaining a net satisfaction rate of negative 2 and negative 10 percentage points respectively. The Government’s performance in protecting human rights and freedom attained a net satisfaction rate of negative 18 percentage points. Finally, the net satisfaction rate of the Government’s performance in developing democracy stands at negative 27 percentage points. The mean scores of these 5 specific areas are 3.0, 2.9, 2.8, 2.6 and 2.5 respectively, which are in between “quite dissatisfied” and “half-half” in general.

Opinion Daily

In January 2007, POP opened a feature page called “Opinion Daily” at the “POP Site”, to record significant events and selected polling figures on a day-to-day basis, in order to let readers judge by themselves the reasons for the ups and downs of different opinion figures. In July 2007, POP collaborated with Wisers Information Limited whereby Wisers supplies to POP each day starting from July 24, a record of significant events of that day, according to the research method designed by POP. These daily entries would be uploaded to “Opinion Daily” as soon as they are verified by POP.

For the polling items covered in this press release, the previous survey of some items was conducted from 16 to 19 December, 2013 while this survey was conducted from 24 to 27 March, 2014. During this period, herewith the significant events selected from counting newspaper headlines and commentaries on a daily basis and covered by at least 25% of the local newspaper articles. Readers can make their own judgment if these significant events have any impacts to different polling figures.

13/3/14	Premier Li Keqiang holds a press conference.
11/3/14	Communications Authority indicates that mobile TV services of the HKTV violates the Broadcasting Ordinance.
6/3/14	Zhang Dejiang mentions issue of Hong Kong universal suffrage at a group meeting with National People’s Congress deputies from Hong Kong.
3/3/14	The opening of the National People’s Congress and the Chinese People’s Political Consultative Conference.
2/3/14	The Press Coalition against Violence initiates march.
26/2/14	Financial Secretary John Tsang Chun-wah releases the 2014-2015 Budget.
25/2/14	The central government changes the venue of the Asia-Pacific Economic Cooperation meeting of finance ministers to Beijing.
17/2/14	Government officials condemn the anti-mainland-tourists demonstration in Tsim Sha Tsui.
13/2/14	Li Wei-ling accuses Commercial Radio at the press conference for unfair dismissal due to renewal of license.
15/1/14	The Chief Executive CY Leung delivers 2014 Policy Address.

Commentary

Robert Ting-Yiu Chung, Director of POP, observed, “Our latest survey shows that the latest support rating of CE CY Leung stands at 46.6 marks, his approval rate 27%, disapproval rate 56%, giving a net popularity of negative 30 percentage points. The figures did not change much as compared to two weeks ago. For the SAR Government, compared to one month ago, its popularity has significantly improved. Satisfaction rate goes up by 7 percentages points, while dissatisfaction rate goes down by 4 percentage points, giving a net satisfaction rate of negative 10 percentage points. As for the five specific policy areas, in terms of net satisfaction rate, only one item has registered a positive value, which is relations with the Central Government at positive 8 percentage points. The other four items that have registered negative values are maintaining economic prosperity at negative 2 percentage points, improving people’s livelihood at negative 10 percentage points, protection of human rights and freedom at negative 18 percentage points, and developing democracy at negative 27 percentage points. Compared to three months ago, only the net satisfaction rate of improving people’s livelihood has registered significant improvement, while that of the other four remained stable. As for the reasons affecting the ups and downs of these figures, we leave it to our readers to form their own judgment using detailed records displayed in our ‘Opinion Daily’. Recently there are criticisms on POP surveys based on different findings of CY Leung popularity surveys conducted by HKIAPS and POP back in May 2012. I therefore abstract below some of the points I made in my article published around that time, entitled “Differences in Polls on CE-elect”, to explain the difference and to facilitate today’s discussion.”

“Differences in Polls on CE-elect” (released on 2/6/2012): “The author’s general practice is try not to comment on the surveys of other survey organizations, in order to avoid any conflict of interest… The question used in the CUHK survey was… ‘We would like to ask you for some evaluations of Chief Executive-Elect Leung Chun-Ying. With 0 as the minimum and 100 as the maximum, and 50 as passing mark, how would you rate the overall performance of Leung Chun-Ying?’, while the question used in the HKU survey was… ‘Please use a scale of 0-100 to rate your extent of support to the Chief Executive-Elect Leung Chun-Ying, with 0 indicating absolutely not supportive, 100 indicating absolutely supportive and 50 indicating half-half. How would you rate the Chief Executive-Elect Leung Chun-Ying?’… The most important differences between the two surveys, other than the questionnaire, were the frequencies and intervals of the surveys. From the HKU survey, if we are to track the change of CY Leung’s popularity from 17-23 April, then 3-10 May, and then 18-24 May, we can see a big jump then recession. If we only look at the first and third points, the change would disappear… This interval issue is just common sense.”

The article can be downloaded in full from http://hkupop.pori.hk/english/columns/columns125.html.

Future Release (Tentative)

April 8, 2014 (Tuesday) 1pm to 2pm: Ratings of top 10 cross-strait political figures