HKU POP releases findings of an instant poll on Policy AddressBack

Press Release on January 17, 2013

Abstract

The Public Opinion Programme at the University of Hong Kong interviewed 1,021 Hong Kong people last night (January 16, 2013) by means of a random telephone survey conducted by real interviewers. According to this Policy Address instant survey, among those who had some knowledge of CE CY Leung's first address, 36% said they were satisfied, giving a net satisfaction rate of positive 12 percentage points. This is significantly poorer than the first addresses delivered by CH Tung and Donald Tsang, probably due to the low popularity of Leung himself at the moment. On a scale of 0-100, this year's Address scored 56.4 marks. Looking back, among the 16 policy addresses delivered after the handover, 4 had negative satisfaction, namely, those of 1998, 2000, 2003 and 2004. In this perspective, Leung's first address is already on steady ground, but whether he can work against the tides remains to be seen. As for Leung's own popularity after the address, his support rating has gone up significantly by 3.3 marks, back to the 50-plus level. His net popularity has also improved from negative 20 to negative 11 percentage points, meaning that the Address has a positive effect on his popularity. Looking back at the instant effect of CH Tung and Donald Tsang's Policy Addresses across the years, Tung's addresses usually have a stimulating effect, while Tsang's addresses usually have a dampening effect. Whether the positive effect of Leung's address could be sustained remains to be seen. Furthermore, after excluding those who did not respond for various reasons, 31% said their confidence in the future of Hong Kong had increased after the Policy Address, 23% said their confidence had dropped, while 38% said "no change", thereby giving a net positive effect of 8 percentage points on people's confidence. Further analysis shows that those of age 18 to 29 are significantly less satisfied with the address, they give significantly lower rating to the address and to CE, their opposition against him as CE is the highest, and their confidence in Hong Kong increased the least because of the address. Our instant survey has described people's instant reaction towards the Policy Address. How public opinion would change after CE and his officials explain their policies remains to be seen. The sampling error of all percentages is +/-3 percentage points at 95% confidence level, while the sampling errors of rating figures and net values need another calculation. The response rate of the survey is 69%.

Points to note:
[1] The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists can check out the details of the survey there.
[2] The sample size of this survey is 1,021 successful interviews, not 1,021 x 68.7% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is +/-3 percentage points at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state "sampling error of all percentages not more than +/-3%, that of ratings not more than +/-1.7 and net values not more than +/-6 percentage points, at 95% confidence level".
[4] When quoting percentages of this survey, journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used, in order to match the precision level of the figures.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Background

Since 1992, HKUPOP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. Starting 2008, we split up previous years' instant survey into two surveys. In our first survey, we measure people's overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE's popularity. In our second survey, we focus on people's reactions towards different government proposals, their satisfaction with CE's policy direction, and other relevant issues. The instant survey released today is our fifth release under our new operation.

Latest Figures

The findings of the Policy Address instant poll released by the POP SITE today have been weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in 2012 mid-year. Herewith the contact information of various surveys:

Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
2013	16/1/13	1,021	68.7%	+/-3%
2011	12/10/11	1,032	65.6%	+/-3%
2010	13/10/10	1,020	66.9%	+/-3%
2009	14/10/09	1,007	71.9%	+/-3%
2008	15/10/08	1,011	74.9%	+/-3%
2007	10/10/07	1,023	69.9%	+/-3%
2006	11/10/06	1,027	60.7%	+/-3%
2005	12/10/05	914	66.1%	+/-3%
2004	7/1/04	1,040	67.5%	+/-3%
2003	8-9/1/03	1,259	68.9%	+/-3%
2001	10/10/01	1,051	66.0%	+/-3%
2000	11/10/00	1,059	69.7%	+/-3%
1999	6/10/99	888	54.5%	+/-3%
1998	7/10/98	1,494	56.5%	+/-3%
1997	8/10/97	1,523	61.5%	+/-3%

Recent figures on CY Leung's popularity are summarized as follows:

Date of survey	14-22/11/12	1-4/12/12	18-28/12/12	2-9/1/13	16/1/13	Latest change
Sample base	1,020	1,006	1,013	1,010	1,021	--
Overall response rate	64.2%	66.2%	65.7%	67.7%	68.7%	--
Latest finding	Finding	Finding	Finding	Finding	Finding and Error ^[7]	--
Rating of CE CY Leung	52.3	49.2	49.1	48.9	52.2+/-1.6	+3.3^[8]
Vote of confidence in CE CY Leung	39%	37%	34%	31%	35+/-3%	+4%^[8]
Vote of no confidence in CE CY Leung	45%	49%	47%	51%^[8]	46+/-3%	-5%^[8]
Net approval rate	-6%	-12%	-13%	-20%^[8]	-11+/-6%	+9%^[8]

[7] All error figures in the table are calculated at 95% confidence level."95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[8] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Figures on Chief Executive's popularity before and after the Policy Address Speech from the handover till present are summarized as follows:

CE Tung Chee-hwa's popularity before and after the Policy Address Speech from 1997 to 2005
	1st Policy Address	2nd Policy Address	3rd Policy Address	4th Policy Address	5th Policy Address	6th Policy Address	7th Policy Address	8th Policy Address
Date of PA Speech	8/10/97	7/10/98	6/10/99	11/10/00	10/10/01	8/1/03	7/1/04	12/1/05
CE's popularity rating before the PA & error^[9]	65.8 +/-1.4	55.8 +/-1.6	54.0 +/-1.8	48.2 +/-2.2	48.4 +/-1.4	46.6 +/-1.4	42.9 +/-1.4	47.2 +/-1.2
CE's popularity rating at PA instant survey & error^[9]	66.1 +/-1.0	56.1 +/-1.0	54.3 +/-1.4	50.7 +/-1.4	50.6 +/-1.4	47.3 +/-1.4	44.6 +/-1.4	48.4 +/-1.4
Change in CE's rating	+0.3	+0.3	+0.3	+2.5^[10]	+2.2^[10]	+0.7	+1.7^[10]	+1.2

CE Donald Tsang's popularity before and after the Policy Address Speech from 2005 to 2011
	1st Policy Address	2nd Policy Address	3rd Policy Address	4th Policy Address	5th Policy Address	6th Policy Address	7th Policy Address
Date of Policy Address Speech	12/10/05	11/10/06	10/10/07	15/10/08	14/10/09	13/10/10	12/10/11
CE's popularity rating before the PA & error^[9]	68.0+/-1.0	62.9 +/-1.2	65.8+/-1.2	52.7+/-1.3	55.2+/-1.2	55.4+/-1.2	48.4+/-1.4
CE's popularity rating at PA instant survey & error^[9]	67.4+/-1.1	59.8+/-1.1	64.4+/-1.0	53.9+/-1.6	54.2+/-1.4	56.2+/-1.6	50.6+/-1.6
Change in CE's rating	-0.6	-3.1^[10]	-1.4^[10]	+1.2	-1.0	+0.8	+2.2^[10]
CE's approval rate before the PA & error^[9]	77+/-3%	64+/-3%	63+/-3%	44+/-3%	45+/-3%	42+/-3%	24+/-3%
CE's approval rate at PA instant survey & error^[9]	74+/-3%	56+/-3%	62+/-3%	44+/-4%	45+/-4%	41+/-4%	25+/-3%
Change in CE's approval rate ^[11]	-3%	-8%^[10]	-1%	--	--	-1%	+1%

CE Tung Chee-hwa, Donald Tsang and CY Leung's popularity before and after the first Policy Address Speech
	Tung Chee-hwa's 1st Policy Address	Donald Tsang's 1st Policy Address	CY Leung's 1st Policy Address
Date of Policy Address Speech	8/10/97	12/10/05	16/1/13
CE's popularity rating before the PA & error^[9]	65.8 +/-1.4	68.0+/-1.0	48.9+/-1.6
CE's popularity rating at PA instant survey & error^[9]	66.1 +/-1.0	67.4+/-1.1	*52.2+/-1.6*
Change in CE's rating	+0.3	-0.6	+3.3^[10]
CE's approval rate before the PA & error^[9]	--	77+/-3%	31+/-3%
CE's approval rate at PA instant survey & error^[9]	--	74+/-3%	*35+/-3%*
Change in CE's approval rate ^[11]	--	-3%	+4%^[10]

[9] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[10] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[11] Instant surveys on Policy Address included CE's approval rate since 2004, so it is not listed under Tung's series.

The install poll conducted last night showed that, after CE CY Leung announced his Policy Address, his latest support rating was 52.2 marks, with an approval and disapproval rate of 35% and 46% respectively. As for people's satisfaction of various Policy Addresses after the handover, the figures are summarized below:

Date of Survey	Sub-sample base^[14]	Appraisal of Policy Address: Satisfaction rate^[13]	Appraisal of Policy Address: Half-half	Appraisal of Policy Address: Dissatisfaction rate^[13]	Mean value^[13]	Satisfaction rating of Policy Address^[14]
16/1/13	759	36+/-3%^[16]	35+/-3%	24+/-3%^[16]	3.1+/-0.1 (Base=717)	56.4+/-1.7^[16]
12/10/11	816	47+/-3%^[16]	32+/-3%	18+/-3%	3.3+/-0.1 (Base=791)	59.1+/-1.4
13/10/10	747	41+/-4%^[16]	33+/-3%^[16]	19+/-3%^[16]	3.2+/-0.1 (Base=695)	58.9+/-1.4^[16]
14/10/09	462	30+/-4%	37+/-4%	28+/-4%	3.0+/-0.1 (Base =434)	53.5+/-2.1
15/10/08	515	31+/-4%^[16]	35+/-4%^[16]	26+/-4%^[16]	3.0+/-0.1 (Base =474)	53.8+/-2.0^[16]
10/10/07	602	52+/-4%^[16]	29+/-4%^[16]	10+/-2%^[16]	3.5+/-0.1 (Base =551)	65.2+/-1.6^[16]
11/10/06	445	30+/-4%^[16]	37+/-5%	22+/-4%^[16]	3.0+/-0.1 (Base =397)	55.8+/-2.0^[16]
12/10/05	377	48+/-5%^[16]	33+/-5%	9+/-3%^[16]	3.5+/-0.1 (Base =338)	66.4+/-1.9^[16]
12/1/05	391	38+/-5%^[16]	30+/-5%	20+/-4%^[16]	3.2+/-0.1 (Base =342)	56.3+/-2.4^[16]
7/1/04	381	25+/-4%	26+/-4%	33+/-5%^[16]	2.8+/-0.1 (Base =322)	49.3+/-2.4
8/1/03^[15]	377	22+/-4%^[16]	29+/-5%	27+/-5%	2.8+/-0.1 (Base =561)	51.6+/-2.6^[16]
10/10/01	433	29+/-4%	33+/-5%	27+/-4%	3.0+/-0.1 (Base =386)	56.7 +/-2.2
11/10/00	262	25+/-5%^[16]	28+/-6%	31+/-6%	2.9+/-0.1 (Base =219)	55.2 +/-2.8
6/10/99	236	31+/-6%^[16]	30+/-6%	25+/-6%^[16]	3.0+/-0.1 (Base =202)	57.3 +/-2.8
7/10/98	508	22+/-4%^[16]	35+/-4%^[16]	35+/-4%^[16]	2.8+/-0.1 (Base =469)	--
8/10/97	534	45+/-4%	30+/-4%^[16]	14+/-3%^[16]	3.4+/-0.1 (Base =474)	--

[12] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[13] Collapsed from a 5-point scale. The mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of positive level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[14] Excluding respondents who did not answer this question because they had not heard of / did not know the details of the Policy Address. Because of the smaller sample size, the sampling error has increased accordingly.
[15] The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our "POP SITE".
[16] Result changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

After excluding those respondents who said they did not know the details of the Policy Address, this year's instant survey showed that 36% were satisfied with it, 24% were dissatisfied and 35% said "half-half". Meanwhile, the average rating registered for the Policy Address was 56.4 marks. Because part of the respondents said they were not familiar with the Policy Address during the instant poll, the valid sub-sample of this item was smaller. The sampling error for this question has increased accordingly.

The survey also gauged the change of people's confidence towards Hong Kong's future after CE CY Leung has delivered his Policy Address. Results are as follows:

Date of survey	Sub-sample base^[18]	Overall response rate	Effect of the Policy Address on one's confidence in HK's future^[17]
Date of survey	Sub-sample base^[18]	Overall response rate	Increased	Unchanged	Decreased	Don't know/ Hard to say
16/1/13	913	68.7%	31+/-3%	38+/-3%^[20]	23+/-3%	7+/-2%^[20]
12/10/11	957	65.6%	29+/-3%	45+/-3%	21+/-3%	5+/-1%
13/10/10	914	66.9%	31+/-3%^[20]	45+/-3%	18+/-3%^[20]	6+/-2%
14/10/09	749	71.9%	27+/-3%^[20]	47+/- 4%^[20]	22+/-3%^[20]	5+/-2%
15/10/08	761	74.9%	23+/-3%^[20]	38+/- 4%^[20]	32+/-3%^[20]	7+/-2%
10/10/07	388	69.9%	53+/-5%^[20]	31%/- 5%^[20]	7+/-3%^[20]	9+/-3%
11/10/06	431	60.7%	25+/-4%^[20]	51+/- 5%^[20]	16+/-4%^[20]	8+/-3%
12/10/05	476	66.1%	54+/-5%^[20]	33+/- 4%^[20]	5+/-2%^[20]	8+/-2%^[20]
12/1/05	658	66.5%	34+/-4%	41+/-4%	12+/-3%^[20]	14+/-3%
7/1/04	602	67.5%	32+/-4%^[20]	40+/-4%	16+/-3%^[20]	12+/-3%
8/1/03^[19]	513	67.3%	25+/-4%	40+/- 4%^[20]	22+/-4%	14+/-3%^[20]
10/10/01	591	66.0%	22+/-3%	50+/- 4%^[20]	21+/-3%^[20]	7+/-2%^[20]
11/10/00	292	69.7%	22+/-5%^[20]	40+/-6%	15+/-4%	22+/-5%^[20]
6/10/99	233	54.5%	40+/-6%^[20]	36+/- 6%^[20]	16+/-5%^[20]	8+/-4%
7/10/98	505	56.5%	21+/-4%	52+/-4%	22+/-4%	5+/-2%

[17] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. This survey series began in 1998.
[18] Excluding respondents who did not answer this question for various reasons. Because of the smaller sample size, the sampling error has increased accordingly.
[19] The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our "HKU POP SITE".
[20] Result changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Results showed that, excluding those who did not answer this question for various reasons, 31% said their confidence in the future of Hong Kong had increased, 38% opted for "no change", whilst 23% said their confidence had dropped.

Indepth Analysis

In the survey, we also asked respondents for their age. If they were reluctant to give their exact age, they could give us a range. According to their answers, we grouped them into 18-29, 30-49, and 50 years or older. Herewith further analysis of public's satisfaction rate of the Policy Address, effect of the Policy Address on their confidence in HK's future and CE's popularity by respondents' age:

Date of survey: 16/1/13		18-29	30-49	50 or above	Overall
Satisfaction rating of Policy Address^[22]		49.6+/-3.4 (134)	56.5+/-2.7 (259)	59.0+/-2.6 (325)	56.4+/-1.7 (718)
Satisfaction rate of the Policy Address^[21] ^[22]	Satisfaction	19+/-7% (26)	35+/-6% (94)	44+/-5% (149)	36+/-3% (269)
	Half-half	45+/-8% (63)	36+/-6% (97)	29+/-5% (101)	35+/-3% (261)
	Dissatisfaction	32+/-8% (44)	23+/-5% (63)	22+/-5% (77)	24+/-3% (183)
	Don't know/ hard to say	4+/-3% (6)	6+/-3% (17)	4+/-2% (15)	5+/-2% (38)
	Total	100% (139)	100% (271)	100% (342)	100% (752)
	Mean value^[21]	2.8+/-0.2 (133)	3.1+/-0.1 (254)	3.2+/-0.1 (326)	3.1+/-0.1 (714)

Date of survey: 16/1/13		18-29	30-49	50 or above	Overall
Effect of the Policy Address on their confidence in HK's future ^[22]	Increased	20+/-6% (35)	28+/-5% (83)	39+/-5% (153)	31+/-3% (281)
	Unchanged	52+/-8% (90)	41+/-5% (138)	30+/-5% (119)	38+/-3% (347)
	Decreased	25+/-7% (43)	26+/-5% (87)	21+/-4% (83)	24+/-3% (213)
	Don't know/ hard to say	4+/-3% (6)	6+/-3% (20)	9+/-3% (37)	7+/-2% (63)
	Total	100% (174)	100% (338)	100% (392)	100% (904)

Date of survey: 16/1/13		18-29	30-49	50 or above	Overall
Rating of CE CY Leung^[23]		44.2+/-3.2 (189)	51.3+/-2.5 (388)	56.6+/-2.5 (418)	52.2+/-1.6 (995)
Vote of confidence/ no confidence in CE CY Leung^[22]	Support	23+/-6% (43)	36+/-5% (140)	40+/-5% (171)	35+/-3% (354)
	Oppose	65+/-7% (124)	49+/-5% (191)	37+/-5% (158)	47+/-3% (472)
	Don't know/ hard to say	12+/-5% (23)	15+/-4% (60)	24+/-4% (102)	18+/-2% (185)
	Total	100% (190)	100% (390)	100% (431)	100% (1,011)

[21] Collapsed from a 5-point scale. The mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of positive level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[22] Differences among sub-groups are tested to be statistically significant at 95% confidence level.

Commentary

Note: The following commentary was written by Director of POP Robert Chung.

According to our Policy Address instant survey, among those who had some knowledge of CE CY Leung's first address, 36% said they were satisfied, giving a net satisfaction rate of positive 12 percentage points. This is significantly poorer than the first addresses delivered by CH Tung and Donald Tsang, probably due to the low popularity of Leung himself at the moment. On a scale of 0-100, this year's Address scored 56.4 marks. Looking back, among the 16 policy addresses delivered after the handover, 4 had negative satisfaction, namely, those of 1998, 2000, 2003 and 2004. In this perspective, Leung's first address is already on steady ground, but whether he can work against the tides remains to be seen.

As for Leung's own popularity after the address, his support rating has gone up significantly by 3.3 marks, back to the 50-plus level. His net popularity has also improved from negative 20 to negative 11 percentage points, meaning that the Address has a positive effect on his popularity. Looking back at the instant effect of CH Tung and Donald Tsang's Policy Addresses across the years, Tung's addresses usually have a stimulating effect, while Tsang's addresses usually have a dampening effect. Whether the positive effect of Leung's address could be sustained remains to be seen.

Furthermore, after excluding those who did not respond for various reasons, 31% said their confidence in the future of Hong Kong had increased after the Policy Address, 23% said their confidence had dropped, while 38% said "no change", thereby giving a net positive effect of 8 percentage points on people's confidence.

Further analysis shows that those of age 18 to 29 are significantly less satisfied with the address, they give significantly lower rating to the address and to CE, their opposition against him as CE is the highest, and their confidence in Hong Kong increased the least because of the address.

Our instant survey has described people's instant reaction towards the Policy Address. How public opinion would change after CE and his officials explain their policies remains to be seen.

Future Releases (Tentative)

January 22, 2013 (Tuesday) 1pm to 2pm: Policy Address First Follow-up survey
January 24, 2013 (Thursday) 1pm to 2pm: Ratings of Top 10 Legislative Councillors

About HKUPOP: "Outline of our operation for the Policy Address instant survey of 2013"

After the HKSAR government announced the date of Policy Address, we started our planning for the instant survey.
About one month ago, we began to keep track of news about the Policy Address, in order to lay the ground work of questionnaire design.
About one week ago, we began our manpower deployment and internal preparation.
On the day CE announces the address, we monitored the media and the Internet, including the entire address and CE's subsequent press conferences, and then drafted the questionnaire.
Our random telephone interviews began at 6 p.m. on that day, involving around 120 interviewers and staff. The interviews finished around 10:00 p.m., after collecting 1,021 samples.
Data verification and quantitative analyses followed immediately, together with the drafting of the press release.
On the following day, the survey findings were verified again, while our POP Site was re-designed. Our press release was compiled, proofread, and then released for public consumption.