HKU POP releases the results of the Policy Address second follow-up surveyBack

Press Release on February 6, 2014

Abstract

The Public Opinion Programme (POP) at the University of Hong Kong interviewed 516 Hong Kong people between 28 and 29 January by means of a random telephone survey conducted by real interviewers, in order to measure again people’s reaction to this year’s Policy Address. According to our Policy Address instant survey, among respondents who had some knowledge of the second address of CY Leung, 36% said they were satisfied. However, in our first follow-up survey, the figure significantly dropped to 23%, while dissatisfaction rate increased from 31% to 41%. Two weeks later, people’s satisfaction rate rebounds by 4 percentage points to 27% while dissatisfaction rate drops by 4 percentage points to 37%, narrowing the net satisfaction rate to negative 11 percentage points, which is within sampling errors. People’s net satisfaction with CY Leung’s policy direction now stands at negative 8 percentage points, up 4 percentage points from our first follow-up survey, also within sampling errors. People’s rating of the Policy Address now stands at 48.1 marks, same as that of the first follow-up survey. Besides, people’s appraisal of many policy proposals has also turned positive, but only that of housing target at “470,000 units in the coming ten years” has increased beyond sampling errors. All in all, after many rounds of discussion, people’s appraisal of the address seems to have rebounded, but the change is not too obvious. The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while that of rating figure is +/-2.0 and net value needs another calculation. The response rate of the survey is 65%.

Points to note:
[1] The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists can check out the details of the survey there.
[2] The sample size of this survey is 516 successful interviews, not 516 x 65.2% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while the sampling error of rating figures and net values needs another calculation. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state "sampling error of various ratings not more than +/-2.0, that of percentages not more than +/-4% and net values no more than +/-8% at 95% confidence level". Because POP introduced “rim weighting” in 2014, during the transition period, whether changes in various figures are beyond sampling errors are based on tests using the same weighting methods. That is, to test whether the first set of figures collected in 2014 is significantly different from that of the previous survey, both sets of data are rim weighted before testing, instead of using simple computation of the published figures.
[4] Because of sampling errors in conducting the survey(s) and the rounding procedures in processing the data, the figures cannot be too precise, and the totals may not be completely accurate. Therefore, when quoting percentages of the survey(s), journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Background

Since 1992, POP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. In general, such surveys would be repeated some time later to measure people's more matured reactions. In 2008, we further enhanced our survey design by splitting our Policy Address instant survey into two. In our instant survey, we measure people’s overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE’s popularity. One to two days later, we started to conduct our first follow-up survey, which mainly studies people’s reactions towards different government proposals, and any change in their satisfaction of the Policy Address. Our second follow-up survey would be conducted a short period later, to repeat our measurement of people’s reactions towards different government proposals, and any change in their satisfaction of the Policy Address. We believe this is a better way to study public opinion on these issues: measuring people's instant reaction first, and then repeat our measurement some time later to check people's more matured reaction. Our Policy Address’s instant and first follow-up surveys this year were released on January 16 and 21 respectively, while the findings of the second follow-up poll are released today.

Latest Figures

POP today releases the latest findings of the second follow-up survey of Policy Address. From 2014, POP enhanced the previous simple weighting method based on age and gender distribution to “rim weighting” based on age, gender and education (highest level attended) distribution. The latest figures released today have been rim-weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-year 2013 and the educational attainment (highest level attended) distribution collected in the 2011 Census. Herewith the contact information of various surveys:

Survey series	Date of survey	Sample base	Overall response rate	Sampling error of percentages ^[6]
Survey series	Date of survey	Sample base	Overall response rate	Sampling error of percentages ^[6]
2014 Second follow-up	28-29/1/14	516	65.2%	+/-4%
2014 First follow-up	16-17/1/14	519	68.7%	+/-4%
2014 Instant	15/1/14	1,017	66.7%	+/-3%
2013 Second follow-up	22-24/1/13	507	66.6%	+/-4%
2013 First follow-up	17-18/1/13	530	66.2%	+/-4%
2013 Instant	16/1/13	1,021	68.7%	+/-3%
2011 Second follow-up	17-20/10/11	518	73.9%	+/-4%
2011 First follow-up	13-14/10/11	520	65.5%	+/-4%
2011 Instant	12/10/11	1,032	65.6%	+/-3%
2010 Second follow-up	26-27/10/10	523	64.0%	+/-4%
2010 First follow-up	14-16/10/10	507	64.9%	+/-4%
2010 Instant	13/10/10	1,020	66.9%	+/-3%
2009 Second follow-up	20-26/10/09	513	72.1%	+/-4%
2009 First follow-up	15-17/10/09	508	70.6%	+/-4%
2009 Instant	14/10/09	1,007	71.9%	+/-3%
2008 Second follow-up	27-29/10/08	1,015	70.3%	+/-3%
2008 First follow-up	17-19/10/08	505	70.9%	+/-4%
2008 Instant	15/10/08	1,011	74.9%	+/-3%

[6] Calculated at 95% confidence level using full sample size."95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[7] The figures shown in the “latest change” column of this press release have been tested after “rim weighting” data collected in this and last surveys. The structural effect of using the new weighting method is small, around -1% to +1% for percentage figures, while statistical significance tests are not affected.

As different questions involve different sub-samples, the sample errors will vary accordingly. The table below briefly shows the relationship between sampling errors and sample size for the readers to capture the corresponding changes:

Sample size (total sample or sub-sample)	Sampling error of percentages^[8] (maximum values)	Sample size (total sample or sub-sample)	Sampling error of percentages^[8] (maximum values)
Sample size (total sample or sub-sample)	Sampling error of percentages^[8] (maximum values)	Sample size (total sample or sub-sample)	Sampling error of percentages^[8] (maximum values)
1,300	+/- 2.8 %	1,350	+/- 2.7 %
1,200	+/- 2.9 %	1,250	+/- 2.8 %
1,100	+/- 3.0 %	1,150	+/- 3.0 %
1,000	+/- 3.2 %	1,050	+/- 3.1 %
900	+/- 3.3 %	950	+/- 3.2 %
800	+/- 3.5 %	850	+/- 3.4 %
700	+/- 3.8 %	750	+/- 3.7 %
600	+/- 4.1 %	650	+/- 3.9 %
500	+/- 4.5 %	550	+/- 4.3 %
400	+/- 5.0 %	450	+/- 4.7 %

[8] Based on 95% confidence interval.

Results of the second follow-up survey of Policy Address, together with the instant and first follow-up surveys, for 2013 (CY Leung’s first Policy Address) and 2014 (CY Leung’s second Policy Address) are tabulated below:

	2013				2014
	Instant survey	First follow-up survey	Second follow-up survey	Change	Instant survey	First follow-up survey	Second follow-up survey	Latest change
Date of survey	16/1/13	17-18/1/13	22-24/1/13	--	15/1/14	16-17/1/14	28-29/1/14	--
Sample base	1,021^[9]	530	507	--	1,017^[9]	519	516	--
Overall response rate	68.7%	66.2%	66.6%	--	66.7%	68.7%	65.2%	--
Latest Finding	Finding and error^[10]	Finding and error^[10]	Finding and error^[10]	--	Finding and error^[10]	Finding and error^[10]	Finding and error^[10]	--
Policy Address: Satisfaction rate ^[11]	36+/-3%	27+/-4%^[12]	22+/-4%	-5%^[12]	36+/-4%	23+/-4%^[12]	27+/-4%	+4%
Policy Address: Dissatisfaction rate^[11]	24+/-3%	39+/-4%^[12]	45+/-4%	+6%^[12]	31+/-4%	41+/-4%^[12]	37+/-4%	-4%
Net value	12+/-6%	-12+/-7%	-23+/-7%	-11%^[12]	5+/-7%	-18+/-7%^[12]	-11+/-7%	+7%
Mean Value^[11]	3.1+/-0.1 (Base=717)	2.8+/-0.1^[12] (Base=487)	2.6+/-0.1 (Base=482)	-0.2^[12]	3.0+/-0.1 (Base=593)	2.6+/-0.1^[12] (Base=479)	2.8+/-0.1 (Base=466)	+0.2^[12]
Rating of Policy Address (0 to 100 marks)	56.4+/-1.7^[12]	48.2+/-2.1^[12]	43.8+/-2.1	-4.4^[12]	54.1+/-1.9^[12]	48.1+/-2.0^[12]	48.1+/-2.0	--
Satisfaction rate of Leung’s policy direction ^[11]	--	35+/-4%	29+/-4%	-6%^[12]	--	29+/-4%	32+/-4%	+3%
Dissatisfaction rate of Leung’s policy direction ^[11]	--	32+/-4%	39+/-4%	+7%^[12]	--	42+/-4%	39+/-4%	-3%
Net value	--	3+/-7%	-10+/-7%	-13%^[12]	--	-12+/-7%	-8+/-7%	+4%
Mean Value^[11]	--	3.0+/-0.1 (Base=491)	2.8+/-0.1 (Base=488)	-0.2^[12]	--	2.7+/-0.1 (Base=489)	2.8+/-0.1 (Base=486)	+0.1

[9] Excluding respondents who were not clear about the Policy Address. The sub-sample size in 2013 was 759 and that in 2014 was 611.
[10] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Sampling errors of ratings are calculated according to the distribution of the scores collected.
[11] Collapsed from a 5-point scale, the mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of importance level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[12] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Latest second follow-up survey revealed that 27% of the respondents were satisfied with the Policy Address and 37% were dissatisfied, giving net satisfaction of negative 11 percentage points. The mean score is 2.8, meaning close to “half-half”. The average rating registered for the Policy Address was 48.1 marks. As for people's satisfaction with CY Leung's policy direction, 32% of the respondents showed satisfaction while 39% were not satisfied, giving net satisfaction of negative 8 percentage points. The mean score is 2.8, meaning close to “half-half”.

Results of people’s satisfaction with the Policy Address in previous similar surveys (follow-up survey of Policy Address in 1997, 1998 and 2000 – 2007 and second follow-up surveys of Policy Address in 1999, 2008 – 2014) are tabulated below:

Date of survey	Sub-sample base	Appraisal of PA: Satisfaction ^[13]	Appraisal of PA: Half-half	Appraisal of PA: Dissatisfaction ^[13]	Net value (Satisfaction minus dissatisfaction)
Date of survey	Sub-sample base	Finding & error^[14]	Finding & error^[14]	Finding & error^[14]	Finding & error^[14]
28-29/1/14	516	27^[15] +/-4%	27+/-4%	37^[15] +/-4%	-11^[15] +/-7%
22-24/1/13	506	22 +/-4%	29+/-4%	45 +/-4%	-23+/-7%
17-20/10/11	517	33+/-4%	32+/-4%	32+/-4%	1+/-7%
26-27/10/10	517	31^[15] +/-4%	30 +/-4%	33^[15] +/-4%	-2^[15]+/-7%
20-26/10/09	506	20+/-4%	28^[15] +/-4%	45^[15] +/-4%	-25^[15]+/-7%
27-29/10/08	556	24^[15]+/-4%	36%^[15] +/-4%	35^[15] +/-4%	-11^[15]+/-6%
22-23/10/07	526	43^[15] +/-4%	31^[15] +/-4%	18^[15] +/-3%	25+/-6%
23-24/10/06	506	26^[15] +/-4%	41^[15] +/-4%	23^[15]+/-4%	3^[15]+/-6%
25-27/10/05	511	41+/-4%	24+/-4%	5+/-2%	36+/-5%
27-31/1/05	1,012	17+/-2%	37^[15] +/-3%	23^[15] +/-3%	-6^[15]+/-4%
14-16/1/04	987	10^[15] +/-2%	27^[15] +/-3%	29^[15] +/-3%	-19^[15]+/-4%
23-28/1/03	1,049	13 +/-2%	22^[15] +/-3%	37^[15] +/-3%	-24^[15]+/-4%
21-23/10/01	1,056	14 +/-2%	32^[15] +/-3%	31^[15] +/-3%	-16^[15]+/-4%
23-25/10/00	1,026	15^[15] +/-2%	28 +/-3%	25 +/-3%	-10^[15]+/-4%
22/10/99	553	12^[15] +/-3%	28 +/-4%	27^[15] +/-4%	-15+/-5%
20/10/98	460	22^[15] +/-4%	31 +/-4%	37^[15]+/-5%	-15+/-7%
14-15/10/97	515	31^[15]+/-4%	27 +/-4%	14 +/-3%	17+/-6%

[13] Collapsed from a 5-point scale.
[14] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[15] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Compared to similar surveys conducted after the handover, people’s satisfaction rate with CY Leung’s second Policy Address is lower than those registered for Tsang’s four addresses, while it is still higher than all those registered for Tung's second to eighth addresses.

Other results of both first and second follow-up surveys of Policy Address 2014 are tabulated below:

	First follow-up survey	Second follow-up survey	Change
Date of survey	16-17/1/14	28-29/1/14	--
Sample base	519	516	--
Overall response rate	68.7%	65.2%	--
Latest Finding	Finding and error^[16]	Finding and error^[16]	--
The theme of this year’s Policy Address “Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential” concurs the current needs of the society.	56+/-4%	58+/-4%	+2%
The theme of this year’s Policy Address “Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential” does not concur the current needs of the society.	22+/-4%	22+/-4%	--
Support CY Leung’s the new housing target of providing a total of 470,000 units in the coming ten years.	76+/-4%	81+/-3%	+5%^[17]
Oppose to CY Leung’s the new housing target of providing a total of 470,000 units in the coming ten years.	13+/-3%	10+/-3%	-3%
Net support	62+/-6%	71+/-6%	+9%^[17]
Support that the middle class is ignored in this Policy Address.	70+/-4%	68+/-4%	-2%
Oppose that the middle class is ignored in this Policy Address.	15+/-3%	17+/-3%	+2%
Net support	55+/-7%	51+/-7%	-4%
Support the introduction of Low-income Working Family Allowance, to assist non-CSSA working families living below the poverty line.	62+/-4%	61+/-4%	-1%
Oppose to the introduction of Low-income Working Family Allowance, to assist non-CSSA working families living below the poverty line.	25+/-4%	26+/-4%	+1%
Net support	37+/-8%	36+/-8%	-1%
Support the development of the eastern of Lantau Island and neighbouring areas, including studies on reclamation and artificial islands, with a view to developing an “East Lantau Metropolis” as a new core business district.	48+/-4%	52+/-4%	+4%
Oppose to the development of the eastern of Lantau Island and neighbouring areas, including studies on reclamation and artificial islands, with a view to developing an “East Lantau Metropolis” as a new core business district.	35+/-4%	31+/-4%	-4%
Net support	13+/-8%	21+/-8%	+8%

[16] Errors are calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[17] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Results showed that 58% thought the theme of the Policy Address “Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential” concurred with the current needs of the society while 22% did not think so. Besides, 81% supported the new housing target of providing a total of 470,000 units in the coming ten years while 10% opposed. Besides, for the view that the middle class is ignored in this Policy Address, 68% supported this view while 17% opposed. Regarding the introduction of Low-income Working Family Allowance, to assist non-CSSA working families living below the poverty line, 61% supported this practice while 26% opposed. As for the development of the eastern of Lantau Island and neighbouring areas, 52% supported this practice while 31% opposed.

Commentary

Note: The following commentary was written by Director of POP Robert Chung.

According to our Policy Address instant survey, among respondents who had some knowledge of the second address of CY Leung, 36% said they were satisfied. However, in our first follow-up survey, the figure significantly dropped to 23%, while dissatisfaction rate increased from 31% to 41%. Two weeks later, people’s satisfaction rate rebounds by 4 percentage points to 27% while dissatisfaction rate drops by 4 percentage points to 37%, narrowing the net satisfaction rate to negative 11 percentage points, which is within sampling errors. People’s net satisfaction with CY Leung’s policy direction now stands at negative 8 percentage points, up 4 percentage points from our first follow-up survey, also within sampling errors. People’s rating of the Policy Address now stands at 48.1 marks, same as that of the first follow-up survey. Besides, people’s appraisal of many policy proposals has also turned positive, but only that of housing target at “470,000 units in the coming ten years” has increased beyond sampling errors. All in all, after many rounds of discussion, people’s appraisal of the address seems to have rebounded, but the change is not too obvious.

Future Release (Tentative)

February 11, 2014 (Tuesday) 1pm to 2pm: Popularity of CE and Principal Officials