HKU POP releases the results of the Policy Address second follow-up surveyBack

 
Press Release on February 12, 2015

| Special Announcement| Abstract| Background | Latest Figures | Commentary | Future Release (Tentative) |
| Detailed Findings (Second Follow-up Survey on the Third Policy Address of Leung Chun-ying) |


Special Announcement

The Public Opinion Programme (POP) of The University of Hong Kong conducted a headcount exercise on February 1 and released the preliminary results on the same day. POP has now released the details of the headcount via the “HKU POP Site” (http://hkupop.pori.hk), and will upload the video clippings onto the “PopCon Site” (http://popcon.hk) tomorrow for people to verify the figures this week.



Abstract


POP interviewed 514 Hong Kong people between 3 and 4 February by means of a random telephone survey conducted by real interviewers, in order to measure again people’s reaction to this year’s Policy Address. According to our Policy Address instant survey, among respondents who had some knowledge of the third address of CY Leung, net satisfaction was negative 5 percentage points. In our first follow-up survey, it plunged to negative 27 percentage points, while satisfaction rating plunged to 43.4. About three weeks later, net satisfaction slightly rises 1 percentage point to negative 26 percentage points, but satisfaction rating continues to drop to 41.7 marks, which is the poorest policy address rating since record begins in 2005. People’s net satisfaction with CY Leung’s policy direction now stands at negative 25 percentage points. Regarding the theme of the Address, those who agree that “Uphold the Rule of Law, Seize the Opportunities, Make the Right Choices, Pursue Democracy, Boost the Economy, Improve People’s Livelihood” meets the need of society have increased over the past three weeks. In terms of key policy areas, net support for the suspension of the Capital Investment Entrant Scheme has dropped. People’s appraisal of the housing and education policies proposed, as well as the general measures taken to address Hong Kong’s current problems, has not changed much and remains negative. About 60% continue to object CY Leung’s criticizing The Hong Kong University Students’ Union magazine, “Undergrad”, at the beginning of his Address, for advocating “self-reliance” and “self-determination” in Hong Kong. All in all, after many rounds of discussion, people’s appraisal of the policy address has not changed much, still negative. The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while that of rating figure is +/-2.2 and net value needs another calculation. The response rate of the survey is 68%.

 

 


Points to note:

[1] The address of the “HKU POP SITE” is http://hkupop.pori.hk, journalists can check out the details of the survey there.
[2] The sample size of this survey is 514 successful interviews, not 514 x 67.9% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while the sampling error of rating figures and net values needs another calculation. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state “sampling error of various ratings not more than +/-2.2, that of percentages not more than +/-4% and net values no more than +/-7% at 95% confidence level”.  Because POP introduced “rim weighting” in 2014, during the transition period, whether changes in various figures are beyond sampling errors are based on tests using the same weighting methods. That is, to test whether the first set of figures collected in 2014 is significantly different from that of the previous survey, both sets of data are rim weighted before testing, instead of using simple computation of the published figures.
[4] Because of sampling errors in conducting the survey(s) and the rounding procedures in processing the data, the figures cannot be too precise, and the totals may not be completely accurate. Therefore, when quoting percentages of the survey(s), journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses “computerized random telephone survey” to camouflage its IVS operation, it should be considered unprofessional.



Background

Since 1992, POP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. In general, such surveys would be repeated some time later to measure people's more matured reactions. In 2008, we further enhanced our survey design by splitting our Policy Address instant survey into two. In our instant survey, we measure people’s overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE’s popularity. One to two days later, we started to conduct our first follow-up survey, which mainly studies people’s reactions towards different government proposals, and any change in their satisfaction of the Policy Address. Our second follow-up survey would be conducted a short period later, to repeat our measurement of people’s reactions towards different government proposals, and any change in their satisfaction of the Policy Address. We believe this is a better way to study public opinion on these issues: measuring people's instant reaction first, and then repeat our measurement some time later to check people's more matured reaction. Findings of our Policy Address’s instant and first follow-up surveys this year were released on January 14, 15 and 20 respectively, while that of the second follow-up poll are released today.


Latest Figures

POP today releases the latest findings of the second follow-up survey of Policy Address. From 2014, POP enhanced the previous simple weighting method based on age and gender distribution to “rim weighting” based on age, gender and education (highest level attended) distribution. The latest figures released today have been rim-weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-year 2014 and the educational attainment (highest level attended) distribution collected in the 2011 Census. Herewith the contact information of various surveys:


           

Survey series

Date of survey

Sample base

Overall response rate

Sampling error of percentages[6]

2015 Second follow-up

3-4/2/15

514

67.9%

+/-4%

2015 First follow-up

15-16/1/15

500

65.7%

+/-4%

2015 Instant

14/1/15

640

67.4%

+/-4%

2014 Second follow-up

28-29/1/14

516

65.2%

+/-4%

2014 First follow-up

16-17/1/14

519

68.7%

+/-4%

2014 Instant

15/1/14

1,017

66.7%

+/-3%

2013 Second follow-up

22-24/1/13

507

66.6%

+/-4%

2013 First follow-up

17-18/1/13

530

66.2%

+/-4%

2013 Instant

16/1/13

1,021

68.7%

+/-3%

2011 Second follow-up

17-20/10/11

518

73.9%

+/-4%

2011 First follow-up

13-14/10/11

520

65.5%

+/-4%

2011 Instant

12/10/11

1,032

65.6%

+/-3%

2010 Second follow-up

26-27/10/10

523

64.0%

+/-4%

2010 First follow-up

14-16/10/10

507

64.9%

+/-4%

2010 Instant

13/10/10

1,020

66.9%

+/-3%

2009 Second follow-up

20-26/10/09

513

72.1%

+/-4%

2009 First follow-up

15-17/10/09

508

70.6%

+/-4%

2009 Instant

14/10/09

1,007

71.9%

+/-3%

2008 Second follow-up

27-29/10/08

1,015

70.3%

+/-3%

2008 First follow-up

17-19/10/08

505

70.9%

+/-4%

2008 Instant

15/10/08

1,011

74.9%

+/-3%

[6] Calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.


As different questions involve different sub-samples, the sampling errors will vary accordingly. The table below briefly shows the relationship between sampling errors and sample size for the readers to capture the corresponding changes:

                   

Sample size
(total sample or sub-sample)

Sampling error of percentages[7]
(maximum values)

Sample size
(total sample or sub-sample)

Sampling error of percentages[7]
(maximum values)

1,300

 +/- 2.8 %

1,350

 +/- 2.7 %

1,200

 +/- 2.9 %

1,250

 +/- 2.8 %

1,100

 +/- 3.0 %

1,150

 +/- 3.0 %

1,000

 +/- 3.2 %

1,050

 +/- 3.1 %

900

 +/- 3.3 %

950

 +/- 3.2 %

800

 +/- 3.5 %

850

 +/- 3.4 %

700

 +/- 3.8 %

750

 +/- 3.7 %

600

 +/- 4.1 %

650

 +/- 3.9 %

500

 +/- 4.5 %

550

 +/- 4.3 %

400

 +/- 5.0 %

450

 +/- 4.7 %

[7] Based on 95% confidence interval.


Results of the second follow-up survey of Policy Address, together with the instant and first follow-up surveys, for 2014 (CY Leung’s second Policy Address) and 2015 (CY Leung’s third Policy Address) are tabulated below:

 

 

2014

2015

 

Instant survey

First follow-up survey

Second follow-up survey

Change

Instant survey

First follow-up survey

Second follow-up survey

Latest
change

Date of survey

15/1/14

16-17/1/14

28-29/1/14

--

14/1/15

15-16/1/15

3-4/2/15

--

Sample base

1,017[8]

519

516

--

640[8]

500

514

--

Overall response rate

66.7%

68.7%

65.2%

--

67.4%

65.7%

67.9%

--

Latest Finding

Finding

Finding

Finding

--

Finding

Finding

Finding and error[9]

--

Policy Address: Satisfaction rate[10]

36%

23%[11]

27%

+4%

30%

20%[11]

22+/-4%

+2%

Policy Address: Dissatisfaction rate[10]

31%

41%[11]

37%

-4%

35%

47%[11]

49+/-4%

+2%

Net value

5%

-18%[11]

-11%

+7%

-5%

-27%[11]

-26+/-7%

+1%

Mean Value[10]

3.0
(Base=593)

2.6 [11]
(Base=479)

2.8
(Base=466)

+0.2 [11]

2.8
(Base=449)

2.8[11]
(Base=449)

2.5+/-0.1
(Base=476)

-0.3[11]

Rating of Policy Address (0 to 100 marks)

54.1[11]

48.1[11]

48.1

--

49.5

43.4[11]

41.7+/-2.2

-1.7

Satisfaction rate of Leung’s policy direction[10]

--

29%

32%

+3%

--

24%

23+/-4%

-1%

Dissatisfaction rate of Leung’s policy direction[10]

--

42%

39%

-3%

--

52%

48+/-4%

-4%

Net value

--

-12%

-8%

+4%

--

-27%

-25+/-7%

+2%

Mean Value[10]

--

2.7
(Base=489)

2.8
(Base=486)

+0.1

--

2.5
(Base=483)

2.5+/-0.1
(Base=488)

--

[8] Excluding respondents who were not clear about the Policy Address. The sub-sample size in 2014 was 611 and that in 2015 was 503.
[9] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Sampling errors of ratings are calculated according to the distribution of the scores collected.
[10] Collapsed from a 5-point scale, the mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of importance level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[11] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.


The second follow-up survey revealed that 22% of the respondents were satisfied with the Policy Address and 49% were dissatisfied, giving a net satisfaction of negative 26 percentage points. The mean score is 2.5, which is between “half-half” and “quite dissatisfied” in general. The average rating registered for the Policy Address was 41.7 marks. As for people’s satisfaction with CY Leung’s policy direction, 23% of the respondents showed satisfaction while 48% were not satisfied, giving net satisfaction of negative 25 percentage points. The mean score is 2.5, which is between “half-half” and “quite dissatisfied” in general.

 

Results of people’s satisfaction with the Policy Address in previous similar surveys (follow-up survey of Policy Address in 1997, 1998 and 2000 – 2007 and second follow-up surveys of Policy Address in 1999, 2008 – 2015) are tabulated below:

 

Date of survey

Sub-sample base

Appraisal of PA: Satisfaction [12]

Appraisal of PA: Half-half

Appraisal of PA: Dissatisfaction [12]

Net value
(Satisfaction minus dissatisfaction)

Finding & error[13]

Finding & error[13]

Finding & error[13]

Finding & error[13]

3-4/2/15

514

22[14] +/-4%

22[14] +/-4%

49[14] +/-4%

-26[14] +/-7%

28-29/1/14

516

27[14] +/-4%

27+/-4%

37[14] +/-4%

-11[14] +/-7%

22-24/1/13

506

22 +/-4%

29+/-4%

45 +/-4%

-23+/-7%

17-20/10/11

517

33+/-4%

32+/-4%

32+/-4%

1+/-7%

26-27/10/10

517

31[14]+/-4%

30 +/-4%

33[14]+/-4%

-2[14]+/-7%

20-26/10/09

506

20+/-4%

28[14]+/-4%

45[14]+/-4%

-25[14]+/-7%

27-29/10/08

556

24[14]+/-4%

36[14] +/-4%

35[14]+/-4%

-11[14]+/-6%

22-23/10/07

526

43[14]+/-4%

31[14]+/-4%

18[14]+/-3%

25+/-6%

23-24/10/06

506

26[14]+/-4%

41[14]+/-4%

23[14]+/-4%

3[14]+/-6%

25-27/10/05

511

41+/-4%

24+/-4%

5+/-2%

36+/-5%

27-31/1/05

1,012

17+/-2%

37[14]+/-3%

23[14]+/-3%

-6[14]+/-4%

14-16/1/04

987

10[14]+/-2%

27[14]+/-3%

29[14]+/-3%

-19[14]+/-4%

23-28/1/03

1,049

13 +/-2%

22[14] +/-3%

37[14]+/-3%

-24[14]+/-4%

21-23/10/01

1,056

14 +/-2%

32[14]+/-3%

31[14]+/-3%

-16[14]+/-4%

23-25/10/00

1,026

15[14]+/-2%

28 +/-3%

25 +/-3%

-10[14]+/-4%

22/10/99

553

12[14]+/-3%

28 +/-4%

27[14]+/-4%

-15+/-5%

20/10/98

460

22[14]+/-4%

31 +/-4%

37[14]+/-5%

-15+/-7%

14-15/10/97

515

31[14]+/-4%

27 +/-4%

14 +/-3%

17+/-6%

[12] Collapsed from a 5-point scale.
[13] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[14] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

 

Compared to similar surveys conducted after the handover, people’s net satisfaction with CY Leung’s third Policy Address is lower than those registered for all seven addresses of Tsang and all eight addresses of Tung. Other results of both first and second follow-up surveys of Policy Address 2015 are tabulated below:


 

First follow-up survey

Second follow-up survey

Change

Date of survey

15-16/1/15

3-4/2/15

--

Sample base

500

514

--

Overall response rate

65.7%

67.9%

--

Latest Finding

Finding

Finding and error[15]

--

The theme of this year’s Policy Address “Uphold the Rule of Law, Seize the Opportunities, Make the Right Choices, Pursue Democracy, Boost the Economy, Improve People’s Livelihood” concurs the current needs of the society.

54%

60+/-4%

+6%[16]

The theme of this year’s Policy Address “Uphold the Rule of Law, Seize the Opportunities, Make the Right Choices, Pursue Democracy, Boost the Economy, Improve People’s Livelihood” does not concur the current needs of the society.

28%

21+/-4%

-7%[16]

Regard the effect of the housing and land supply policies proposed by CY Leung on tackling housing problems to be large.

19%

23+/-4%

+4%

Regard the effect of the housing and land supply policies proposed by CY Leung on tackling housing problems to be small (including no effect).

57%

55+/-4%

-2%

Regard the effect of the youth education and development policies proposed by CY Leung on tackling youth problems to be large.

14%

13+/-3%

-1%

Regard the effect of the youth education and development policies proposed by CY Leung on tackling youth problems to be small (including no effect).

58%

58+/-4%

--

Regard the effect of the various policies proposed by CY Leung in the Policy Address on tackling current problems facing Hong Kong to be large.

12%

10+/-3%

-2%

Regard the effect of the various policies proposed by CY Leung in the Policy Address on tackling current problems facing Hong Kong to be small (including no effect).

60%

58+/-4%

-2%

Support the suspension of the Capital Investment Entrant Scheme.

53%

51+/-4%

-2%

Oppose to the suspension of the Capital Investment Entrant Scheme.

21%

26+/-4%

+5%[16]

Net support

32%

24+/-7%

-8%[16]

Support CY Leung’s criticism made in the introduction of the Policy Address on the official magazine of The Hong Kong University Student’s Union, “Undergrad”, for advocating Hong Kong should find a way to self-reliance and self-determination, and called on people to stay alert and discourage it.

25%

24+/-4%

-1%

Oppose CY Leung’s criticism made in the introduction of the Policy Address on the official magazine of The Hong Kong University Student’s Union, “Undergrad”, for advocating Hong Kong should find a way to self-reliance and self-determination, and called on people to stay alert and discourage it.

60%

61+/-4%

+1%

Net support

-36%

-37+/-7%

-1%

[15] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[16] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.


Results of the second follow-up poll showed that 60% thought the theme of the Policy Address “Uphold the Rule of Law, Seize the Opportunities, Make the Right Choices, Pursue Democracy, Boost the Economy, Improve People’s Livelihood” concurred with the current needs of the society while 21% did not think so. Besides, 23% considered the effect of the housing and land supply policies on tackling housing problems big while 55% considered it small (including no effect). As for the effect of the youth education and development policies proposed by CY Leung on tackling youth problems, 13% said the effect would be big while 58% said it would be small (including no effect). Besides, 10% believed the effect of various policies proposed by CY Leung in the Policy Address on tackling current problems facing Hong Kong would be big, while 58% said there would be small or even no effect. Meanwhile, 51% supported CY Leung’s the suspension of the Capital Investment Entrant Scheme while 26% opposed. Regarding CY Leung’s criticism made in the introduction of the Policy Address on the official magazine of The Hong Kong University Student’s Union, “Undergrad”, for advocating Hong Kong should find a way to self-reliance and self-determination, and called on people to stay alert and discourage it, 24% supported this approach, while 61% opposed.



Commentary

Note: The following commentary was written by Director of POP Robert Chung.

 

According to our Policy Address instant survey, among respondents who had some knowledge of the third address of CY Leung, net satisfaction was negative 5 percentage points. In our first follow-up survey, it plunged to negative 27 percentage points, while satisfaction rating plunged to 43.4. About three weeks later, net satisfaction slightly rises 1 percentage point to negative 26 percentage points, but satisfaction rating continues to drop to 41.7 marks, which is the poorest policy address rating since record begins in 2005. People’s net satisfaction with CY Leung’s policy direction now stands at negative 25 percentage points. Regarding the theme of the Address, those who agree that “Uphold the Rule of Law, Seize the Opportunities, Make the Right Choices, Pursue Democracy, Boost the Economy, Improve People’s Livelihood” meets the need of society have increased over the past three weeks. In terms of key policy areas, net support for the suspension of the Capital Investment Entrant Scheme has dropped. People’s appraisal of the housing and education policies proposed, as well as the general measures taken to address Hong Kong’s current problems, has not changed much and remains negative. About 60% continue to object CY Leung’s criticizing The Hong Kong University Students’ Union magazine, “Undergrad”, at the beginning of his Address, for advocating “self-reliance” and “self-determination” in Hong Kong. All in all, after many rounds of discussion, people’s appraisal of the policy address has not changed much, still negative.



Future Release (Tentative)

  • February 17, 2015 (Tuesday) 1pm to 2pm: Social, freedom and legal indicators


| Special Announcement| Abstract| Background | Latest Figures | Commentary | Future Release (Tentative) |
| Detailed Findings (Second Follow-up Survey on the Third Policy Address of Leung Chun-ying) |