HKU POP releases the results of the Policy Address second follow-up surveyBack
Press Release on February 7, 2017
| Detailed Findings (Policy Address Feature Page) |
Abstract
The Public Opinion Programme (POP) of The University of Hong Kong interviewed 506 Hong Kong people between 1 and 2 February by means of a random telephone survey conducted by real interviewers, in order to measure again people’s reaction to this year’s Policy Address. According to our Policy Address instant survey, among respondents who had some knowledge of the fifth address of CY Leung, net satisfaction was positive 5 percentage points and satisfaction rating at 52.3 marks. In our first follow-up survey, it decreased to negative 3 percentage points, while satisfaction rating dropped to 48.8 marks. Two weeks later, net satisfaction has further decreased by 12 percentage points to negative 15 percentage points, while satisfaction rating also decreases to 46.9 marks. People’s net satisfaction with CY Leung’s policy direction now stands at negative 18 percentage points. Regarding the theme of the Address, those who agree that “Make Best Use of Opportunities, Develop the Economy, Improve People’s Livelihood, Build an Inclusive Society” meets the need of society have not changed much over the past two weeks. People’s appraisal of the housing policies proposed as well as the general measures taken to address Hong Kong’s current problems both turn negative. Besides, 65% oppose to the saying made by CY Leung that “all commitments in my Election Manifesto have basically been implemented” in his conclusion, the net support rate is negative 46 percentage points, which has significantly dropped by 12 percentage points over the past two weeks. All in all, after many rounds of discussion, people’s appraisal of the policy address has shown no improvement but turned more negative. The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while that of rating figure is +/-2.2 and net value needs another calculation. The response rate of the survey is 70%.
Points to note:
[1] The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists
can check out the details of the survey there.
[2] The sample size of this survey is 506 successful interviews, not 506 x
70.0% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is +/-4 percentage points
at 95% confidence level, while the sampling error of rating figures and net
values needs another calculation. "95% confidence level" means that if we
were to repeat a certain survey 100 times, using the same questions each
time but with different random samples, we would expect 95 times getting a
figure within the error margins specified. When quoting these figures,
journalists can state "sampling error of various ratings not more than
+/-2.2, that of percentages not more than +/-4% and net values no more than
+/-7% at 95% confidence level".
[4] Because of sampling errors in conducting the survey(s) and the rounding
procedures in processing the data, the figures cannot be too precise, and
the totals may not be completely accurate. Therefore, when quoting
percentages of the survey(s), journalists should refrain from reporting
decimal places, but when quoting the rating figures, one decimal place can
be used.
[5] The data of this survey is collected by means of random telephone
interviews conducted by real interviewers, not by any interactive voice
system (IVS). If a research organization uses "computerized random
telephone survey" to camouflage its IVS operation, it should be considered
unprofessional.
Background
Since 1992, POP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. In general, such surveys would be repeated some time later to measure people's more matured reactions. In 2008, we further enhanced our survey design by splitting our Policy Address instant survey into two. In our instant survey, we measure people’s overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE’s popularity. One to two days later, we started to conduct our first follow-up survey, which mainly studies people’s reactions towards different government proposals, and any change in their satisfaction of the Policy Address. Our second follow-up survey would be conducted a short period later, to repeat our measurement of people’s reactions towards different government proposals, and any change in their satisfaction of the Policy Address. We believe this is a better way to study public opinion on these issues: measuring people's instant reaction first, and then repeat our measurement some time later to check people's more matured reaction. Our Policy Address’s instant and first follow-up surveys this year were released on January 18, 19 and 24 respectively, while the findings of the second follow-up poll are released today.
Latest Figures
POP today releases the latest findings of the second follow-up survey of Policy Address. From 2014, POP enhanced the previous simple weighting method based on age and gender distribution to “rim weighting” based on age, gender and education (highest level attended) distribution. The latest figures released today have been rim-weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-year 2016 and the educational attainment (highest level attended) distribution collected in the 2011 Census. Herewith the contact information of various surveys:
|
Survey series |
Date of survey |
Sample base |
Overall response rate |
Sampling error of percentages[6] |
|
2017 Second follow-up |
1-2/2/17 |
506 |
70.0% |
+/-4% |
|
2017 First follow-up |
19-20/1/17 |
513 |
68.4% |
+/-4% |
|
2017 Instant |
18/1/17 |
664 |
67.1% |
+/-4% |
|
2016 Second follow-up |
1-2/2/16 |
515 |
66.5% |
+/-4% |
|
2016 First follow-up |
14-15/1/16 |
514 |
65.8% |
+/-4% |
|
2016 Instant |
13/1/16 |
608 |
64.1% |
+/-4% |
|
2015 Second follow-up |
3-4/2/15 |
514 |
67.9% |
+/-4% |
|
2015 First follow-up |
15-16/1/15 |
500 |
65.7% |
+/-4% |
|
2015 Instant |
14/1/15 |
640 |
67.4% |
+/-4% |
|
2014 Second follow-up |
28-29/1/14 |
516 |
65.2% |
+/-4% |
|
2014 First follow-up |
16-17/1/14 |
519 |
68.7% |
+/-4% |
|
2014 Instant |
15/1/14 |
1,017 |
66.7% |
+/-3% |
|
2013 Second follow-up |
22-24/1/13 |
507 |
66.6% |
+/-4% |
|
2013 First follow-up |
17-18/1/13 |
530 |
66.2% |
+/-4% |
|
2013 Instant |
16/1/13 |
1,021 |
68.7% |
+/-3% |
|
2011 Second follow-up |
17-20/10/11 |
518 |
73.9% |
+/-4% |
|
2011 First follow-up |
13-14/10/11 |
520 |
65.5% |
+/-4% |
|
2011 Instant |
12/10/11 |
1,032 |
65.6% |
+/-3% |
|
2010 Second follow-up |
26-27/10/10 |
523 |
64.0% |
+/-4% |
|
2010 First follow-up |
14-16/10/10 |
507 |
64.9% |
+/-4% |
|
2010 Instant |
13/10/10 |
1,020 |
66.9% |
+/-3% |
|
2009 Second follow-up |
20-26/10/09 |
513 |
72.1% |
+/-4% |
|
2009 First follow-up |
15-17/10/09 |
508 |
70.6% |
+/-4% |
|
2009 Instant |
14/10/09 |
1,007 |
71.9% |
+/-3% |
|
2008 Second follow-up |
27-29/10/08 |
1,015 |
70.3% |
+/-3% |
|
2008 First follow-up |
17-19/10/08 |
505 |
70.9% |
+/-4% |
|
2008 Instant |
15/10/08 |
1,011 |
74.9% |
+/-3% |
[6] Calculated at 95% confidence level using full sample size."95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
As different questions involve different sub-samples, the sample errors will vary accordingly. The table below briefly shows the relationship between sampling errors and sample size for the readers to capture the corresponding changes:
|
Sample size (total sample or sub-sample) |
Sampling error of percentages[7 ] (maximum values) |
Sample size (total sample or sub-sample) |
Sampling error of percentages[7 ] (maximum values) |
|
|
1,300 |
+/- 2.8 % |
1,350 |
+/- 2.7 % |
|
|
1,200 |
+/- 2.9 % |
1,250 |
+/- 2.8 % |
|
|
1,100 |
+/- 3.0 % |
1,150 |
+/- 3.0 % |
|
|
1,000 |
+/- 3.2 % |
1,050 |
+/- 3.1 % |
|
|
900 |
+/- 3.3 % |
950 |
+/- 3.2 % |
|
|
800 |
+/- 3.5 % |
850 |
+/- 3.4 % |
|
|
700 |
+/- 3.8 % |
750 |
+/- 3.7 % |
|
|
600 |
+/- 4.1 % |
650 |
+/- 3.9 % |
|
|
500 |
+/- 4.5 % |
550 |
+/- 4.3 % |
|
|
400 |
+/- 5.0 % |
450 |
+/- 4.7 % |
[7] Based on 95% confidence interval.
Results of the second follow-up survey of Policy Address, together with the instant and first follow-up surveys, for 2016 (CY Leung’s fourth Policy Address) and 2017 (CY Leung’s fifth Policy Address) are tabulated below:
|
2016 |
2017 |
|||||||
|
Date of survey Sample base Overall response rate Latest Finding |
Instant survey 13/1/16 608[8] 64.1% Finding |
First follow-up survey 14-15/1/16 514 65.8% Finding |
Second follow-up survey 1-2/2/16 515 66.5% Finding |
Change -- -- -- -- |
Instant survey 18/1/17 664 67.1% Finding |
First follow-up survey 19-20/1/17 513 68.4% Finding |
Second follow-up survey 1-2/2/17 506 70.0% Finding and error[ 9 ] |
Latest change -- -- -- -- |
|
Policy Address: Satisfaction rate[10 ] |
19% |
17% |
18% |
+1% |
34% |
33% |
24+/-4% |
-9%[11] |
|
Policy Address: Dissatisfaction rate[10] |
39% |
53%[11] |
54% |
+1% |
29% |
36%[11] |
39+/-4% |
+3% |
|
Net value |
-20% |
-37%[11] |
-36% |
+1% |
5% |
-3%[11] |
-15+/-7% |
-12%[11] |
|
Mean Value[10] |
2.5 (Base=423) |
2.3[11] (Base=451) |
2.3 (Base=460) |
-- |
3.0 (Base=431) |
2.9 (Base=456) |
2.6+/-0.1 (Base=425) |
-0.3 [11] |
|
Rating of Policy Address (0 to 100 marks) |
41.1 |
37.5[11] |
39.7 |
+2.2 |
52.3 |
48.8[11] |
46.9+/-2.2 |
-1.9 |
|
Satisfaction rate of Leung’s policy direction [10] |
-- |
22% |
25% |
+3% |
-- |
37%[11] |
27+/-4% |
-10% [11] |
|
Dissatisfaction rate of Leung’s policy direction [10] |
-- |
54% |
53% |
-1% |
-- |
40%[11] |
45+/-4% |
+5%[11] |
|
Net value |
-- |
-31% |
-29% |
+2% |
-- |
-3%[11] |
-18+/-7% |
-15%[11] |
|
Mean Value[10] |
-- |
2.4 (Base=487) |
2.4 (Base=487) |
-- |
-- |
2.8[11] (Base=492) |
2.6+/-0.1 (Base=476) |
-0.2 [11] |
[8] Excluding respondents who were not clear about the Policy Address. The sub-sample size in 2016 was 522 and that in 2017 was 512.
[9] Errors are calculated at 95% confidence level using full sample size.
“95% confidence level” means that if we were to repeat a certain survey 100
times, using the same questions each time but with different random
samples, we would expect 95 times getting a figure within the error margins
specified. Sampling errors of ratings are calculated according to the
distribution of the scores collected.
[10] Collapsed from a 5-point scale, the mean value is calculated by
quantifying all individual responses into 1, 2, 3, 4, 5 marks according to
their degree of importance level, where 1 is the lowest and 5 the highest,
and then calculate the sample mean.
[11] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
Latest second follow-up survey revealed that 24% of the respondents were satisfied with the Policy Address and 39% were dissatisfied, giving net satisfaction of negative 15 percentage points. The mean score is 2.6, which is between “half-half” and “quite dissatisfied” in general. The average rating registered for the Policy Address was 46.9 marks. As for people’s satisfaction with CY Leung's policy direction, 27% of the respondents showed satisfaction while 45% were not satisfied, giving net satisfaction of negative 18 percentage points. The mean score is 2.6, which is between “half-half” and “quite dissatisfied” in general.
Results of people’s satisfaction with the Policy Address in previous similar surveys (follow-up survey of Policy Address in 1997, 1998 and 2000 – 2007 and second follow-up surveys of Policy Address in 1999, 2008 – 2017) are tabulated below:
|
Date of survey |
Sub-sample base |
Appraisal of PA: Satisfaction [12 ] |
Appraisal of PA: Half-half |
Appraisal of PA: Dissatisfaction [12 ] |
Net value (Satisfaction minus dissatisfaction) |
|
Finding & error[13] |
Finding & error[13] |
Finding & error[13] |
Finding & error[13] |
||
|
1-2/2/17 |
506 |
24 [14 ] +/-4% |
21+/-4% |
39 [14 ] +/-4% |
-15 [14 ] +/-7% |
|
1-2/2/16 |
515 |
18[14]+/-3% |
17[14]+/-3% |
54[14]+/-4% |
-36[14]+/-7% |
|
3-4/2/15 |
514 |
22[14] +/-4% |
22[14] +/-4% |
49[14] +/-4% |
-26[14] +/-7% |
|
28-29/1/14 |
516 |
27[14] +/-4% |
27+/-4% |
37[14] +/-4% |
-11[14] +/-7% |
|
22-24/1/13 |
506 |
22+/-4% |
29+/-4% |
45+/-4% |
-23+/-7% |
|
17-20/10/11 |
517 |
33+/-4% |
32+/-4% |
32+/-4% |
1+/-7% |
|
26-27/10/10 |
517 |
31[14]+/-4% |
30+/-4% |
33[14]+/-4% |
-2[14]+/-7% |
|
20-26/10/09 |
506 |
20+/-4% |
28[14]+/-4% |
45[14]+/-4% |
-25[14]+/-7% |
|
27-29/10/08 |
556 |
24[14]+/-4% |
36[14] +/-4% |
35[14]+/-4% |
-11[14]+/-6% |
|
22-23/10/07 |
526 |
43[14]+/-4% |
31[14]+/-4% |
18[14]+/-3% |
25+/-6% |
|
23-24/10/06 |
506 |
26[14]+/-4% |
41[14]+/-4% |
23[14]+/-4% |
3[14]+/-6% |
|
25-27/10/05 |
511 |
41+/-4% |
24+/-4% |
5+/-2% |
36+/-5% |
|
27-31/1/05 |
1,012 |
17+/-2% |
37[14]+/-3% |
23[14]+/-3% |
-6[14]+/-4% |
|
14-16/1/04 |
987 |
10[14]+/-2% |
27[14]+/-3% |
29[14]+/-3% |
-19[14]+/-4% |
|
23-28/1/03 |
1,049 |
13+/-2% |
22[14] +/-3% |
37[14]+/-3% |
-24[14]+/-4% |
|
21-23/10/01 |
1,056 |
14+/-2% |
32[14]+/-3% |
31[14]+/-3% |
-16[14]+/-4% |
|
23-25/10/00 |
1,026 |
15[14]+/-2% |
28+/-3% |
25+/-3% |
-10[14]+/-4% |
|
22/10/99 |
553 |
12[14]+/-3% |
28+/-4% |
27[14]+/-4% |
-15+/-5% |
|
20/10/98 |
460 |
22[14]+/-4% |
31+/-4% |
37[14]+/-5% |
-15+/-7% |
|
14-15/10/97 |
515 |
31[14]+/-4% |
27+/-4% |
14+/-3% |
17+/-6% |
[12] Collapsed from a 5-point scale.
[13] Errors are calculated at 95% confidence level using full sample size.
“95% confidence level” means that if we were to repeat a certain survey 100
times, using the same questions each time but with different random
samples, we would expect 95 times getting a figure within the error margins
specified.
[14] Such changes have gone beyond the sampling errors at the 95%
confidence level under the same weighting method, meaning that they are
statistically significant prima facie. However, whether numerical
differences are statistically significant or not is not the same as whether
they are practically useful or meaningful.
Compared to similar surveys conducted after the handover, people’s net satisfaction with CY Leung’s fifth Policy Address is the second highest among his five addresses, but it is still lower than those registered for former CE Tsang’s six addresses and Tung's three addresses. Other results of both first and second follow-up surveys of Policy Address 2017 are tabulated below:
|
Date of survey Sample base Overall response rate Latest Finding |
First follow-up survey 19-20/1/17 513 68.4% Finding |
Second follow-up survey 1-2/2/17506 70.0% Finding and error[15] |
Change -- -- -- -- |
|
The theme of this year’s Policy Address “Make Best Use of Opportunities, Develop the Economy, Improve People’s Livelihood, Build an Inclusive Society” concurs with the current needs of the society. |
69% |
70 +/-4% |
+ 1 % |
|
The theme of this year’s Policy Address “Make Best Use of Opportunities, Develop the Economy, Improve People’s Livelihood, Build an Inclusive Society” does not concur with the current needs of the society. |
15% |
16 +/-3% |
+1% |
|
Regard the effect of the housing and land supply policies proposed by CY Leung on tackling housing problems to be large. |
28% |
23+/- 4 % |
-5%[16] |
|
Regard the effect of the housing and land supply policies proposed by CY Leung on tackling housing problems to be small (including no effect).. |
50% |
54+/-4% |
+4% |
|
Regard the effect of the various policies proposed by CY Leung in the Policy Address on tackling current problems facing Hong Kong to be large. |
17% |
12+/-3% |
-5%[16] |
|
Regard the effect of the various policies proposed by CY Leung in the Policy Address on tackling current problems facing Hong Kong to be small (including no effect). |
52% |
59+/-4% |
+ 7 %[16] |
|
Support the saying that CY Leung mentioned in his conclusion that “all commitments in my Election Manifesto have basically been implemented”. |
20% |
19+/-3% |
-1% |
|
Oppose the saying that CY Leung mentioned in his conclusion that “all commitments in my Election Manifesto have basically been implemented”. |
54% |
65+/-4% |
+11%[16] |
|
Net support |
-34% |
-46+/-7% |
-12%[16] |
[15] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[16] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
Results of the second follow-up poll showed that 70% thought the theme of the Policy Address “Make Best Use of Opportunities, Develop the Economy, Improve People’s Livelihood, Build an Inclusive Society” concurred with the current needs of the society while 16% did not think so. Besides, 23% considered the effect of the housing and land supply policies on tackling housing problems big while 54% considered it small (including no effect). As for the effect of various policies proposed by CY Leung in the Policy Address on tackling current problems facing Hong Kong, 12% said the effect would be big, while 59% said it would be small (including no effect). Regarding the saying that “all commitments in my Election Manifesto have basically been implemented” mentioned by CY Leung in his conclusion, 19% agreed with the saying, while 65% opposed.
Commentary
Note: The following commentary was written by Research Manager of POP, Frank Wai-Kin Lee.
According to our Policy Address instant survey, among respondents who had some knowledge of the fifth address of CY Leung, net satisfaction was positive 5 percentage points and satisfaction rating at 52.3 marks. In our first follow-up survey, it decreased to negative 3 percentage points, while satisfaction rating dropped to 48.8 marks. Two weeks later, net satisfaction has further decreased by 12 percentage points to negative 15 percentage points, while satisfaction rating also decreases to 46.9 marks. People’s net satisfaction with CY Leung’s policy direction now stands at negative 18 percentage points. Regarding the theme of the Address, those who agree that “Make Best Use of Opportunities, Develop the Economy, Improve People’s Livelihood, Build an Inclusive Society” meets the need of society have not changed much over the past two weeks. People’s appraisal of the housing policies proposed as well as the general measures taken to address Hong Kong’s current problems both turn negative. Besides, 65% oppose to the saying made by CY Leung that “all commitments in my Election Manifesto have basically been implemented” in his conclusion, the net support rate is negative 46 percentage points, which has significantly dropped by 12 percentage points over the past two weeks. All in all, after many rounds of discussion, people’s appraisal of the policy address has shown no improvement but turned more negative.
Future Release (Tentative)
