HKU POP releases the results of Policy Address first follow-up surveyBack

Press Release on January 21, 2014

Abstract

According to the Policy Address instant survey conducted by the Public Opinion Programme (POP) at the University of Hong Kong, among respondents who had some knowledge of CY Leung’s second Policy Address, net satisfaction was positive 5 percentage points. In our follow-up survey, it plunges 23 percentage points to negative 18 percentage points. This is similar to last year’s drop of 23 percentage point to negative 12 percentage points. In terms of rating, our follow-up survey records an average score of 48.1, a drop of 6 marks from 54.1 registered in our instant poll, which is also very similar to last year’s drop of over 8 marks. All in all, after some initial discussions, people’s appraisal of this year’s Policy Address has turned from positive to negative. Most of those who did not express an opinion on the day of the Address now seem to hold a negative view. POP will conduct another round of follow-up survey to map people’s further reaction. Regarding the theme of the Address, 56% of respondents agree that “Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential” meets the need of society. In terms of key policy areas, 76% support the new housing target of providing a total of 470,000 units in the coming ten years, 62% support the introduction of Low-income Working Family Allowance, whereas 48% support the development of the eastern of Lantau Island and neighbouring areas. The net support rates of three proposals are positive 62, 37 and 13 percentage points respectively. However, 70% agree that the middle class is ignored in this Policy Address, with a net support rate of positive 55 percentage points. Finally, according to the results of our tracking question, people's net satisfaction with CY Leung's policy direction now stands at negative 12 percentage points, which is a big drop of 15 percentage points from that of last year. POP will release another round of Policy Address survey findings in a little more than two weeks’ time. Whether public opinion would change after many rounds of discussion remains to be seen. The follow-up survey interviewed 519 Hong Kong people by means of a random telephone survey conducted by real interviewers. The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while that of rating figure is +/-2.0 and net value needs another calculation. The response rate of the survey is 69%.

Points to note:
[1] The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists can check out the details of the survey there.
[2] The sample size of this survey is 519 successful interviews, not 519 x 68.7% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is +/-4 percentage points at 95% confidence level, while the sampling error of rating figures and net values needs another calculation. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state “sampling error of various ratings not more than +/-2.0, that of percentages not more than +/-4% and net values no more than +/-8% at 95% confidence level”. Because POP introduced “rim weighting” in 2014, during the transition period, whether changes in various figures are beyond sampling errors are based on tests using the same weighting methods. That is, to test whether the first set of figures collected in 2014 is significantly different from that of the previous survey, both sets of data are rim weighted before testing, instead of using simple computation of the published figures.
[4] Because of sampling errors in conducting the survey(s) and the rounding procedures in processing the data, the figures cannot be too precise, and the totals may not be completely accurate. Therefore, when quoting percentages of the survey(s), journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Background

Since 1992, POP has been conducting Policy Address instant surveys every year. In 1998, we expanded our instant surveys to cover the Budget Talks. In general, such instant polls which measure people's instant reactions would be repeated later by a follow-up survey which measure people's more matured reactions. We believe this is the correct way to study public opinion. In 2008, we further split our instant survey into two. In our first survey, we measure people's overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE's popularity. One to two days later, we would conduct our first follow-up survey to study people's reactions towards different government proposals, and any change in their satisfaction of the Policy Address. The findings of this year’s instant survey were already released on January 16. Today, we release the results of our first follow-up survey.

Latest Figures

POP today releases the latest findings of the Policy Address follow-up survey. From 2014, POP enhanced the previous simple weighting method based on age and gender distribution to “rim weighting” based on age, gender and education (highest level attended) distribution. The latest figures released today have been rim-weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-year 2013 and the educational attainment (highest level attended) distribution collected in the 2011 Census. Herewith the contact information of various surveys:

Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
2014 Follow-up	16-17/1/14	519	68.7%	+/-4%
2014 Instant	15/1/14	1,017	66.7%	+/-3%
2013 Follow-up	17-18/1/13	530	66.2%	+/-4%
2013 Instant	16/1/13	1,021	68.7%	+/-3%
2011 Follow-up	13-14/10/11	520	65.5%	+/-4%
2011 Instant	12/10/11	1,032	65.6%	+/-3%
2010 Follow-up	14-16/10/10	507	64.9%	+/-4%
2010 Instant	13/10/10	1,020	66.9%	+/-3%
2009 Follow-up	15-17/10/09	508	70.6%	+/-4%
2009 Instant	14/10/09	1,007	71.9%	+/-3%
2008 Follow-up	17-19/10/08	505	70.9%	+/-4%
2008 Instant	15/10/08	1,011	74.9%	+/-3%
2007 Instant	10/10/07	1,023	69.9%	+/-3%
2006 Instant	11/10/06	1,027	60.7%	+/-3%
2005 Instant	12/10/05	914	66.1%	+/-3%
2004 Instant	7/1/04	1,040	67.5%	+/-3%
2003 Instant	8-9/1/03	1,259	68.9%	+/-3%
2001 Instant	10/10/01	1,051	66.0%	+/-3%
2000 Instant	11/10/00	1,059	69.7%	+/-3%
1999 Instant	6/10/99	888	54.5%	+/-3%
1998 Instant	7/10/98	1,494	56.5%	+/-3%
1997 Instant	8/10/97	1,523	61.5%	+/-3%

[6] Calculated at 95% confidence level using full sample size. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Questions using only sub-samples would have bigger sample error. Sampling errors of ratings are calculated according to the distribution of the scores collected.
[7] The figures shown in the “latest change” column of this press release have been tested after “rim weighting” data collected in this and last surveys. The structural effect of using the new weighting method is small, around -2% to 0% for percentage figures, while statistical significance tests are not affected.

Results of the follow-up survey of Policy Address, together with the instant poll, for 2013 and 2014 are tabulated below:

	2013			2014
	Instant survey ^[8]	Follow-up survey	Change	Instant survey^[9]	Follow-up survey	Latest Change
Date of survey	16/1/13	17-18/1/13	--	15/1/14	16-17/1/14	--
Sample base	1,021	530	--	1,017	519	--
Overall response rate	68.7%	66.2%	--	66.7%	68.7%	--
Latest finding	Finding	Finding and error^[10]	--	Finding	Finding and error^[10]	--
Appraisal of Policy Address: Satisfaction rate ^[11]	36%	27%	-9%^[12]	36%	23+/-4%	-13%^[12]
Appraisal of Policy Address: Dissatisfaction rate^[11]	24%	39%	+15%^[12]	31%	41+/-4%	+10%^[12]
Net value	11%	-12%	-23%^[12]	5%	-18+/-7%	-23%^[12]
Mean value^[11]	3.1+/-0.1 (Base=717)	2.8+/-0.1 (Base=487)	-0.3^[12]	3.0+/-0.1 (Base=593)	2.6+/-0.1 (Base=479)	-0.4^[12]
Satisfaction rating of Policy Address (0 to 100 marks)	56.4	48.2	-8.2^[12]	54.1	48.1+/-2.0	-6.0^[12]

[8] Excluding respondents who did not answer this question because they had not heard of / did not know the details of the Policy Address. The sub-sample size was 759.
[9] Excluding respondents who did not answer this question because they had not heard of / did not know the details of the Policy Address. The sub-sample size was 611.
[10] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Sampling errors of ratings are calculated according to the distribution of the scores collected.
[11] Collapsed from a 5-point scale, the mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of importance level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[12] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Our latest survey revealed that 23% of the respondents were satisfied with the Policy Address and 41% were dissatisfied. The mean score is 2.6, which is in between “quite dissatisfied” and “half-half”. The average rating registered for the Policy Address was 48.1 marks. With respect to people's specific reactions towards the contents of this year's Policy Address, relevant findings are summarized below:

Date of survey	16-17/1/14
Sample base	519
Overall response rate	68.7%
Latest finding	Finding and error^[13]
	Yes		Half-half		No		Don’t know/ hard to say			Total
Do you think the theme of this year’s Policy Address “Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential” concurs with the current needs of the society?	56+/-4%		17+/-3%		22+/-4%		5+/-2%			100%
	Support	Half-half		Oppose		Don’t know/ hard to say		Total	Net Support
CY Leung proposed that the new housing target is to provide a total of 470,000 units in the coming ten years, with public housing accounting for 60%. Do you support or oppose this proposal?	76+/-4%	8+/-2%		13+/-3%		3+/-2%		100%	62+/-6%
There are views that the middle class is ignored in the policy address. Do you support or oppose this view?	70+/-4%	6+/-2%		15+/-3%		9+/-3%		100%	55+/-7%
CY Leung proposed to introduce a Low-income Working Family Allowance, to assist non-CSSA working families living below the poverty line. The basic allowance is from $300 to $1,000, and those taking care of children are entitled to an additional allowance. Do you support or oppose this proposal?	62+/-4%	8+/-2%		25+/-4%		5+/-2%		100%	37+/-8%
CY Leung proposed to develop the eastern of Lantau Island and neighbouring areas, including studies on reclamation and artificial islands, with a view to developing an “East Lantau Metropolis” as a new core business district. Do you support or oppose this proposal?	48+/-4%	7+/-2%		35+/-4%		9+/-3%		100%	13+/-8%

[13] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.

Findings on people’s opinion whether the theme of Policy Address concurred with the current needs of the society from 1997 up to this year are summarized as follows:

People’s opinion whether the theme of Policy Address delivered by CY Leung concurred with the current needs of the society from 2013 till 2014 ^[14]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Finding and error ^[15]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Yes	Half-half	No	Don’t know/ hard to say
16-17/1/14	519	2nd	Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential	56 +/-4%	17^[17] +/-3%	22 +/-4%	5^[17] +/-2%
17-18/1/13	530	1st	Seek Change Maintain Stability Serve the People with Pragmatism	53+/-4%	12+/-3%	27+/-4%	8+/-2%

People’s opinion whether the theme of Policy Address delivered by Donald Tsang concurred with the current needs of the society from 2005 till 2011 ^[14]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Finding and error ^[15]
Date of Survey	Sample/ Sub sample base^[16]	Policy Address	Theme	Yes	Half-half	No	Don’t know/ hard to say
13-14/10/11	520	7th	From Strength to Strength	43^[17] +/-4%	9^[17] +/-3%	36^[17] +/-4%	12^[17] +/-3%
14-16/10/10	507	6th	Sharing Prosperity for a Caring Society	66^[17] +/-4%	13 +/-3%	18^[17] +/-3%	4^[17] +/-2%
15-17/10/09	506	5th	Breaking New Ground Together	45^[17] +/-4%	16^[17] +/-3%	30^[17] +/-4%	9 +/-3%
17-19/10/08	503	4th	Embracing New Challenges	57^[17] +/-4%	10 +/-3%	24^[17] +/-4%	9^[17] +/-3%
10/10/07	512	3rd	A New Direction for Hong Kong	69 +/-4%	8 +/-2%	9^[17] +/-3%	15^[17] +/-3%
11/10/06	582	2nd	Proactive Pragmatic Always People First	71 +/-4%	10 +/-2%	14^[17] +/-3%	5^[17] +/-2%
12/10/05	913	1st	Strong Governance for the People	72+/-3%	10+/-2%	8+/-2%	10+/-2%

People’s opinion whether the theme of Policy Address delivered by Tung Chee-hwa concurred with the current needs of the society from 1997 till 2005 ^[14]
Date of Survey	Sample/ Subsample base^[16]	Policy Address	Theme	Finding and error^[15]
Date of Survey	Sample/ Subsample base^[16]	Policy Address	Theme	Yes	Half-half	No	Don’t know/ hard to say
12/1/05	1,031	8th	Working Together for Economic Development and Social Harmony	77^[17] +/-3%	7^[17] +/-2%	10^[17] +/-2%	6^[17] +/-1%
7/1/04	1,031	7th	Seizing Opportunities for Development: Promoting People-based Governance	49^[17] +/-3%	12^[17] +/-2%	19 +/-2%	20^[17] +/-2%
8-9/1/03	1,250	6th	Capitalising on Our Advantages: Revitalizing our Economy	61^[17] +/-3%	7^[17] +/-1%	18^[17] +/-2%	13^[17] +/-2%
10/10/01	1,048	5th	Building on our Strengths, Investing in our Future	45^[17] +/-3%	12^[17] +/-2%	25^[17] +/-3%	18^[17] +/-2%
11/10/00	1,041	4th	Serving the Community, Sharing Common Goals	63^[17] +/-3%	6 +/-1%	17 +/-2%	15^[17] +/-2%
6/10/99	888	3rd	Quality People, Quality Home	69+/-3%	7+/-2%	15+/-2%	8+/-2%
7/10/98^[18]	--	2nd	From Adversity to Opportunity	--	--	--	--
8/10/97^[18]	--	1st	Building Hong Kong for a New Era	--	--	--	--

[14] The question wordings were “The theme of this year’s Policy Address is ‘XXXX’. Do you think this theme concurs with the current needs of the society?”
[15] Errors are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[16] Excluding those respondents who refused to answer this question. Since 2006, this series of question only use sub-sample.
[17] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[18] This question was not covered in the instant Policy Address poll in 1997 and 1998.

Results showed that 56% thought the theme of the Policy Address “Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential” concurred with the current needs of the society while 22% did not think so. Besides, 76% supported the new housing target of providing a total of 470,000 units in the coming ten years while 13% opposed. Besides, for the view that the middle class is ignored in this Policy Address, 70% supported this view while 15% opposed. Regarding the introduction of Low-income Working Family Allowance, to assist non-CSSA working families living below the poverty line, 62% supported this practice while 25% opposed. As for the development of the eastern of Lantau Island and neighbouring areas, including studies on reclamation and artificial islands, with a view to developing an “East Lantau Metropolis” as a new core business district, 48% supported this practice while 35% opposed.

Respondents’ appraisal of CY Leung’s policy direction, together with people’s appraisals of Donald Tsang’s policy direction at the same period in previous years, are tabulated below:

Date of survey	14-16/10/10	13-14/10/11	17-18/1/13	16-17/1/14	Latest change
Sample base	507	520	530	519	--
Overall response rate	64.9%	65.5%	66.2%	68.7%	--
Latest finding and error^[19]	Finding	Finding	Finding	Finding and error ^[19]
Satisfaction rate of Tsang’s / Leung’s policy direction ^[20]	32%^[21] ^[22]	33%^[22]	35%	29+/-4%	-6%^[21]
Dissatisfaction rate of Tsang’s / Leung’s policy direction ^[20]	28%^[21] ^[22]	36%^[21] ^[22]	32%	42+/-4%	+10%^[21]
Net value	4%^[21] ^[22]	-3%^[22]	3%	-12+/-7%	-15%^[21]
Mean value ^[20]	3.0+/-0.1^[21] ^[22] (Base=477)	2.9+/-0.1^[22] (Base=495)	3.0+/-0.1 (Base=491)	2.7+/-0.1 (Base=489)	-0.3^[21]

[19] Errors are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. The error margin of previous survey can be found at the POP Site.
[20] Collapsed from a 5-point scale, the mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of importance level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[21] Such changes have gone beyond the sampling errors at the 95% confidence level under the same weighting method, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[22] The ruling CE was Donald Tsang. For the figures of previous surveys, please refer to the respective tables in POP Site.

As for people's satisfaction with CY Leung's policy direction, 29% of the respondents showed satisfaction while 42% were not satisfied. The mean score is 2.7, which is in between “quite dissatisfied” and “half-half”.

Commentary

Note: The following commentary was written by Director of POP Robert Chung.

According to our Policy Address instant survey, among respondents who had some knowledge of CY Leung’s second Policy Address, net satisfaction was positive 5 percentage points. In our follow-up survey, it plunges 23 percentage points to negative 18 percentage points. This is similar to last year’s drop of 23 percentage point to negative 12 percentage points. In terms of rating, our follow-up survey records an average score of 48.1, a drop of 6 marks from 54.1 registered in our instant poll, which is also very similar to last year’s drop of over 8 marks. All in all, after some initial discussions, people’s appraisal of this year’s Policy Address has turned from positive to negative. Most of those who did not express an opinion on the day of the Address now seem to hold a negative view. POP will conduct another round of follow-up survey to map people’s further reaction.

Regarding the theme of the Address, 56% of respondents agree that “Support the Needy Let Youth Flourish Unleash Hong Kong’s Potential” meets the need of society. In terms of key policy areas, 76% support the new housing target of providing a total of 470,000 units in the coming ten years, 62% support the introduction of Low-income Working Family Allowance, whereas 48% support the development of the eastern of Lantau Island and neighbouring areas. The net support rates of three proposals are positive 62, 37 and 13 percentage points respectively. However, 70% agree that the middle class is ignored in this Policy Address, with a net support rate of positive 55 percentage points.

Finally, according to the results of our tracking question, people's net satisfaction with CY Leung's policy direction now stands at negative 12 percentage points, which is a big drop of 15 percentage points from that of last year.

POP will release another round of Policy Address survey findings in a little more than two weeks’ time. Whether public opinion would change after many rounds of discussion remains to be seen.

Future Releases (Tentative)

January 23, 2014 (Thursday) 1pm to 2pm: Ratings of Top 10 Legislative Councillors
January 28, 2014 (Tuesday) 1pm to 2pm: Popularity of CE and HKSAR Government