HKU POP releases the results of Policy Address first follow-up surveyBack

Press Release on January 20, 2015

Abstract

According to the Policy Address instant survey conducted by the Public Opinion Programme (POP) at The University of Hong Kong, among respondents who had some knowledge of CY Leung’s third Policy Address, net satisfaction was negative 5 percentage points. In our follow-up survey, it plunges 22 percentage points to negative 27 percentage points, while satisfaction rating plunges 6.1 marks to 43.4. The former is a record low since the handover in 1997, while the latter is the poorest rating since record begins in 2008. In other words, after some initial discussions, people’s appraisal of this year’s Policy Address has turned significantly more negative. Most of those who did not express an opinion on the day of the Address now hold a negative view. POP will conduct another round of follow-up survey to map people’s further reaction. Regarding the theme of the Address, 54% agree that “Uphold the Rule of Law, Seize the Opportunities, Make the Right Choices, Pursue Democracy, Boost the Economy, Improve People’s Livelihood” meets the need of society, which is quite positive. However, in terms of key policy areas, other than the suspension of the Capital Investment Entrant Scheme which enjoys majority support, about 60% are negative about the housing and education policies proposed, as well as the general measures taken to address Hong Kong’s current problems. Moreover, 60% object to CY Leung’s criticizing The Hong Kong University Students’ Union magazine, “Undergrad”, at the beginning of his Address, for advocating “self-reliance” and “self-determination” in Hong Kong. People’s net satisfaction with CE’s policy direction now stands at negative 27 percentage points, a plunge of 15 percentage points from that of last year. POP will release another round of Policy Address survey findings in a little more than three weeks’ time. Whether public opinion would change after many rounds of discussion remains to be seen. The follow-up survey interviewed 500 Hong Kong people by means of a random telephone survey conducted by real interviewers. The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while that of rating figure is +/-2.5 and net value needs another calculation. The response rate of the survey is 66%.

Points to note:

[1] The address of the “HKU POP SITE” is http://hkupop.pori.hk, journalists can check out the details of the survey there.
[2] The sample size of this survey is 500 successful interviews, not 500 x 65.7% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while the sampling error of rating figures and net values needs another calculation. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state “sampling error of various ratings not more than +/-2.5, that of percentages not more than +/-4% and net values not more than +/-8% at 95% confidence level”. Because POP introduced “rim weighting” in 2014, during the transition period, whether changes in various figures are beyond sampling errors are based on tests using the same weighting methods. That is, to test whether the first set of figures collected in 2014 is significantly different from that of the previous survey, both sets of data are rim weighted before testing, instead of using simple computation of the published figures.
[4] Because of sampling errors in conducting the survey(s) and the rounding procedures in processing the data, the figures cannot be too precise, and the totals may not be completely accurate. Therefore, when quoting percentages of the survey(s), journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Background

Since 1992, POP has been conducting Policy Address instant surveys every year. In 1998, we expanded our instant surveys to cover the Budget Talks. In general, such instant polls which measure people’s instant reactions would be repeated later by a follow-up survey which measure people’s more matured reactions. We believe this is the correct way to study public opinion. In 2008, we further split our instant survey into two. In our first survey, we measure people’s overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong’s future, and CE’s popularity. One to two days later, we would conduct our first follow-up survey to study people’s reactions towards different government proposals, and any change in their satisfaction of the Policy Address. The findings of this year’s instant survey were already released on January 14 and 15. Today, we release the results of our first follow-up survey.

Latest Figures

POP today releases the latest findings of the Policy Address follow-up survey. From 2014, POP enhanced the previous simple weighting method based on age and gender distribution to “rim weighting” based on age, gender and education (highest level attended) distribution. The latest figures released today have been rim-weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-year 2014 and the educational attainment (highest level attended) distribution collected in the 2011 Census. Herewith the contact information of various surveys:

Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
2015 Follow-up	15-16/1/15	500	65.7%	+/-4%
2015 Instant	14/1/15	640	67.4%	+/-4%
2014 Follow-up	16-17/1/14	519	68.7%	+/-4%
2014 Instant	15/1/14	1,017	66.7%	+/-3%
2013 Follow-up	17-18/1/13	530	66.2%	+/-4%
2013 Instant	16/1/13	1,021	68.7%	+/-3%
2011 Follow-up	13-14/10/11	520	65.5%	+/-4%
2011 Instant	12/10/11	1,032	65.6%	+/-3%
2010 Follow-up	14-16/10/10	507	64.9%	+/-4%
2010 Instant	13/10/10	1,020	66.9%	+/-3%
2009 Follow-up	15-17/10/09	508	70.6%	+/-4%
2009 Instant	14/10/09	1,007	71.9%	+/-3%
2008 Follow-up	17-19/10/08	505	70.9%	+/-4%
2008 Instant	15/10/08	1,011	74.9%	+/-3%
2007 Instant	10/10/07	1,023	69.9%	+/-3%
2006 Instant	11/10/06	1,027	60.7%	+/-3%
2005 Instant	12/10/05	914	66.1%	+/-3%
2004 Instant	7/1/04	1,040	67.5%	+/-3%
2003 Instant	8-9/1/03	1,259	68.9%	+/-3%
2001 Instant	10/10/01	1,051	66.0%	+/-3%
2000 Instant	11/10/00	1,059	69.7%	+/-3%
1999 Instant	6/10/99	888	54.5%	+/-3%
1998 Instant	7/10/98	1,494	56.5%	+/-3%
1997 Instant	8/10/97	1,523	61.5%	+/-3%

[6] Calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Questions using only sub-samples would have bigger sample error. Sampling errors of ratings are calculated according to the distribution of the scores collected.

Results of the follow-up survey of Policy Address, together with the instant poll, for 2014 and 2015 are tabulated below:

	2014			2015
	Instant survey ^[7]	Follow-up survey	Change	Instant survey^[8]	Follow-up survey	Latest Change
Date of survey	15/1/14	16-17/1/14	--	14/1/15	15-16/1/15	--
Sample base	1,017	519	--	640	500	--
Overall response rate	66.7%	68.7%	--	67.4%	65.7%	--
Latest finding	Finding	Finding and error^[9]	--	Finding	Finding and error^[9]	--
Appraisal of Policy Address: Satisfaction rate ^[10]	36%	23%	-13%^[11]	30%	20+/-4%	-10%^[11]
Appraisal of Policy Address: Dissatisfaction rate^[10]	31%	41%	+10%^[11]	35%	47+/-4%	+12%^[11]
Net value	5%	-18%	-23%^[11]	-5%	-27+/-7%	-22%^[11]
Mean value^[10]	3.0 (Base=593)	2.6 (Base=479)	-0.4^[11]	2.8 (Base=449)	2.5+/-0.1 (Base=465)	-0.3^[11]
Satisfaction rating of Policy Address (0 to 100 marks)	54.1	48.1	-6.0^[11]	49.5	43.4+/-2.5	-6.1^[11]

[7] Excluding respondents who did not answer this question because they had not heard of / did not have any knowledge of the Policy Address. The sub-sample size was 611.
[8] Excluding respondents who did not answer this question because they had not heard of / did not have any knowledge of the Policy Address. The sub-sample size was 503.
[9] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Sampling errors of ratings are calculated according to the distribution of the scores collected.
[10] Collapsed from a 5-point scale, the mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of importance level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[11] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Our latest survey revealed that 20% of the respondents were satisfied with the Policy Address and 47% were dissatisfied, giving a net satisfaction rate of negative 27 percentage points. The mean score is 2.5, which is in between “half-half” and “quite dissatisfied”. The average rating registered for the Policy Address was 43.4 marks. With respect to people’s specific reactions towards the contents of this year’s Policy Address, relevant findings are summarized below:

	Finding and error^[12]
	Big	Half-half	Small (including no effect)	Don’t know/ hard to say	Total
Do you think the effect of the housing and land supply policies proposed by CY Leung on tackling housing problems would be big or small?	19+/-4%	18+/-3%	57+/-4%	6+/-2%	100%
Do you think the effect of the youth education and development policies proposed by CY Leung on tackling youth problems would be big or small?	14+/-3%	18+/-3%	58+/-4%	9+/-3%	100%
Do you think the effect of the various policies proposed by CY Leung in the Policy Address on tackling current problems facing Hong Kong would be big or small?	12+/-3%	20+/-4%	60+/-4%	7+/-2%	100%

[12] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.

Results showed that, 19% consider the effect of the housing and land supply policies on tackling housing problems would be big while 57% said it would be small (including no effect). As for the effect of the youth education and development policies proposed by CY Leung on tackling youth problems, 14% said the effect would be big while 58% said it would be small (including no effect). Besides, 12% believed the effect of various policies proposed by CY Leung in the Policy Address on tackling current problems facing Hong Kong would be big, while 60% said there would be small or even no effect.

	Finding and error^[13]
	Support	Half-half	Oppose	Don’t know / hard to say	Total	Net Support
CY Leung decided to suspend the Capital Investment Entrant Scheme. Do you support or oppose this decision?	53+/-4%	17+/-3%	21+/-4%	9+/-3%	100%	32+/-7%

[13] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.

Specifically, 53% supported the suspension of the Capital Investment Entrant Scheme whereas 21% opposed.

	Finding and error^[14]
	Support	Half-half	Oppose	Don’t know / hard to say	Total	Net Support
CY Leung criticized in the introduction of the Policy Address that “Undergrad” the official magazine of The Hong Kong University Student’s Union advocated Hong Kong should find a way to self-reliance and self-determination, and called on people to stay alert and discourage it. Do you support or oppose this approach?	25+/-4%	9+/-3%	60+/-4%	7+/-2%	100%	-36+/-8%

^[14] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.

Regarding CY Leung’s criticism made in the introduction of the Policy Address on the official magazine of The Hong Kong University Student’s Union, “Undergrad”, for advocating Hong Kong should find a way to self-reliance and self-determination, and called on people to stay alert and discourage it, 25% supported this approach, while 60% opposed.

Findings on people’s opinion on whether the theme of Policy Address concurred with the current needs of the society from 1997 up to this year are summarized as follows:

People’s opinion on whether the theme of Policy Address delivered by CY Leung concurred with the current needs of the society from 2013 till 2015 ^[15]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Finding and error ^[17]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Yes	Half-half	No	Don’t know / hard to say
15-16/1/15	500	3rd	Uphold the Rule of Law Seize the Opportunities Make the Right Choices Pursue Democracy Boost the Economy Improve People’s Livelihood	54+/-4%	14+/-3%	28^[18]+/-4%	3+/-2%
16-17/1/14	519	2nd	Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential	56%	17%^[18]	22%	5%^[18]
17-18/1/13	530	1st	Seek Change Maintain Stability Serve the People with Pragmatism	53%	12%	27%	8%

People’s opinion on whether the theme of Policy Address delivered by Donald Tsang concurred with the current needs of the society from 2005 till 2011 ^[15]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Finding and error ^[17]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Yes	Half-half	No	Don’t know / hard to say
13-14/10/11	520	7th	From Strength to Strength	43%^[18]	9%^[18]	36%^[18]	12%^[18]
14-16/10/10	507	6th	Sharing Prosperity for a Caring Society	66%^[18]	13%	18%^[18]	4%^[18]
15-17/10/09	506	5th	Breaking New Ground Together	45%^[18]	16%^[18]	30%^[18]	9%
17-19/10/08	503	4th	Embracing New Challenges	57%^[18]	10%	24%^[18]	9%^[18]
10/10/07	512	3rd	A New Direction for Hong Kong	69%	8%	9%^[18]	15%^[18]
11/10/06	582	2nd	Proactive Pragmatic Always People First	71%	10%	14%^[18]	5%^[18]
12/10/05	913	1st	Strong Governance for the People	72%	10%	8%	10%

People’s opinion on whether the theme of Policy Address delivered by Tung Chee-hwa concurred with the current needs of the society from 1997 till 2005 ^[15]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Finding and error^[17]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Yes	Half-half	No	Don’t know / hard to say
12/1/05	1,031	8th	Working Together for Economic Development and Social Harmony	77%^[18]	7%^[18]	10%^[18]	6%^[18]
7/1/04	1,031	7th	Seizing Opportunities for Development: Promoting People-based Governance	49%^[18]	12%^[18]	19%	20% ^[18]
8-9/1/03	1,250	6th	Capitalising on Our Advantages: Revitalizing our Economy	61%^[18]	7%^[18]	18%^[18]	13%^[18]
10/10/01	1,048	5th	Building on our Strengths, Investing in our Future	45%^[18]	12%^[18]	25%^[18]	18%^[18]
11/10/00	1,041	4th	Serving the Community, Sharing Common Goals	63%^[18]	6%	17%	15%^[18]
6/10/99	888	3rd	Quality People, Quality Home	69%	7%	15%	8%
7/10/98^[19]	--	2nd	From Adversity to Opportunity	--	--	--	--
8/10/97^[19]	--	1st	Building Hong Kong for a New Era	--	--	--	--

[15] The question wordings were “The theme of this year’s Policy Address is ‘XXXX’. Do you think this theme concurs with the current needs of the society?”
[16] Excluding those respondents who refused to answer this question. In 2006-2014 surveys, this series of question only use sub-sample.
[17] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[18] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[19] This question was not covered in the instant Policy Address poll in 1997 and 1998.

The latest results showed that 54% thought the theme of the Policy Address “Uphold the Rule of Law, Seize the Opportunities, Make the Right Choices, Pursue Democracy, Boost the Economy, Improve People’s Livelihood” concurred with the current needs of the society while 28% did not think so.

Respondents’ appraisal of CY Leung’s policy direction, together with people’s appraisals of Donald Tsang’s policy direction at the same period in previous years, are tabulated below:

Date of survey	13-14/10/11	17-18/1/13	16-17/1/14	15-16/1/15	Latest change
Sample base	520	530	519	500	--
Overall response rate	65.5%	66.2%	68.7%	65.7%	--
Latest finding and error	Finding	Finding	Finding	Finding and error ^[20]
Satisfaction rate of Tsang’s / Leung’s policy direction ^[21]	33%^[23]	35%	29%^[22]	24+/-4%	-5%^[22]
Dissatisfaction rate of Tsang’s / Leung’s policy direction ^[21]	36%^[22]^[23]	32%	42%^[22]	52+/-4%	+10%^[22]
Net value	-3%^[23]	3%	-12^[22]	-27+/-7%	-15%^[22]
Mean value^[21]	2.9^[23] (Base=495)	3.0 (Base=491)	2.7^[22] (Base=489)	2.5+/-0.1 (Base=483)	-0.2^[22]

[20] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. The error margin of previous survey can be found at the POP Site.
[21] Collapsed from a 5-point scale, the mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of importance level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[22] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[23] The ruling CE was Donald Tsang. For the figures of previous surveys, please refer to the respective tables in POP Site.

Lastly, as for people’s satisfaction with CY Leung’s policy direction, 24% of the respondents showed satisfaction while 52% were not satisfied, giving a net satisfaction rate of negative 27 percentage points. The mean score is 2.5, which is in between “half-half” and “quite dissatisfied”.

Commentary

Note: The following commentary was written by Director of POP Robert Chung.

According to our Policy Address instant survey, among respondents who had some knowledge of CY Leung’s third Policy Address, net satisfaction was negative 5 percentage points. In our follow-up survey, it plunges 22 percentage points to negative 27 percentage points, while satisfaction rating plunges 6.1 marks to 43.4. The former is a record low since the handover in 1997, while the latter is the poorest rating since record begins in 2008. In other words, after some initial discussions, people’s appraisal of this year’s Policy Address has turned significantly more negative. Most of those who did not express an opinion on the day of the Address now hold a negative view. POP will conduct another round of follow-up survey to map people’s further reaction.

Regarding the theme of the Address, 54% agree that “Uphold the Rule of Law, Seize the Opportunities, Make the Right Choices, Pursue Democracy, Boost the Economy, Improve People’s Livelihood” meets the need of society, which is quite positive. However, in terms of key policy areas, other than the suspension of the Capital Investment Entrant Scheme which enjoys majority support, about 60% are negative about the housing and education policies proposed, as well as the general measures taken to address Hong Kong’s current problems. Moreover, 60% object to CY Leung’s criticizing The Hong Kong University Students’ Union magazine, “Undergrad”, at the beginning of his Address, for advocating “self-reliance” and “self-determination” in Hong Kong. People’s net satisfaction with CE’s policy direction now stands at negative 27 percentage points, a plunge of 15 percentage points from that of last year.

POP will release another round of Policy Address survey findings in a little more than three weeks’ time. Whether public opinion would change after many rounds of discussion remains to be seen.

Future Release (Tentative)

January 27, 2015 (Tuesday) 1pm to 2pm: Popularity of CE and HKSAR Government