HKU POP releases findings of an instant poll on Policy AddressBack

Press Release on October 13, 2011

Abstract

The Public Opinion Programme at the University of Hong Kong interviewed 1,032 Hong Kong people last night (October 12, 2011) by means of a random telephone survey conducted by real interviewers. According to this Policy Address instant survey, among those who had some knowledge of CE Donald Tsang's seventh Address, 47% said they were satisfied, up 6 percentage points from that of last year, with a net satisfaction rate of positive 29 percentage points. On a scale of 0-100, this year's Address scored 59.1 marks, which is also higher than that of last year. All in all, people's instant reaction to this year's Policy Address is quite positive, third among Tsang's seven Addresses. As for CE Donald Tsang's popularity, after giving his Address, his support rating has gone up significantly by 2.2 marks, back to 50-plus. His net popularity has also slightly improved from negative 45 to negative 41 percentage points, meaning that the Address has a positive effect on his popularity. Looking back at the instant effect of CH Tung and Donald Tsang's Policy Addresses across the years, Tung's Addresses usually have a stimulating effect, while Tsang's Addresses usually have a dampening effect. The effect this year is the most positive one for Tsang's popularity, but whether it could be sustained remains to be seen. Furthermore, after excluding those who did not respond for various reasons, 29% said their confidence in the future of Hong Kong had increased after the Policy Address, 21% said their confidence had dropped, while 45% said "no change", thereby giving a net positive effect of 8 percentage points on people's confidence, which is fifth among Tsang's seven Addresses. Further analysis shows that those of age 30 to 49 are significantly less satisfied with the CE Donald Tsang's seventh Policy Address, they give significantly lower rating to the Address and to CE, and their confidence in Hong Kong did not increase because of the Address. Our instant survey has described people's instant reaction towards the Policy Address. How public opinion would change after CE and his officials explain their policies remains to be seen. The sampling error of all percentages is between +/-1 and +/-3 percentage points at 95% confidence level, while the sampling error of rating figures needs another calculation. The response rate of the survey is 66%.

Points to note:
[1] The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists can check out the details of the survey there.
[2] The sample size of this survey is 1,032 successful interviews, not 1,032 x 65.6% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is between +/-1 and +/-3 percentage points at 95% confidence level, while the sampling error of rating figures needs another calculation. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state "sampling error of various ratings not more than +/-1.6 and sampling error of percentages not more than +/-3% at 95% confidence level".
[4] When quoting percentages of this survey, journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used, in order to match the precision level of the figures.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Background

Since 1992, HKUPOP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. Starting 2008, we split up previous years' instant survey into two surveys. In our first survey, we measure people's overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE's popularity. In our second survey, we focus on people's reactions towards different government proposals, their satisfaction with CE's policy direction, and other relevant issues. The instant survey released today is our fourth release under our new operation.

Latest Figures

The findings of the Policy Address instant poll released by the POP SITE today have been weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-2011. Herewith the contact information of various surveys:

Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
2011	12/10/11	1,032	65.6%	+/-3%
2010	13/10/10	1,020	66.9%	+/-3%
2009	14/10/09	1,007	71.9%	+/-3%
2008	15/10/08	1,011	74.9%	+/-3%
2007	10/10/07	1,023	69.9%	+/-3%
2006	11/10/06	1,027	60.7%	+/-3%
2005	12/10/05	914	66.1%	+/-3%
2004	7/1/04	1,040	67.5%	+/-3%
2003	8-9/1/03	1,259	68.9%	+/-3%
2001	10/10/01	1,051	66.0%	+/-3%
2000	11/10/00	1,059	69.7%	+/-3%
1999	6/10/99	888	54.5%	+/-3%
1998	7/10/98	1,494	56.5%	+/-3%
1997	8/10/97	1,523	61.5%	+/-3%

[6] Calculated at 95% confidence level using full sample size. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Questions using only sub-samples would have bigger sample error. Sampling errors of ratings are calculated according to the distribution of the scores collected.

Recent figures on Donald Tsang's popularity are summarized as follows:

Date of survey	16-23/8/11	29/8-6/9/11	13-20/9/11	26/9-4/10/11	12/10/11^[8]	Latest change
Sample base	1,009	1,012	1,001	1,025	1032	--
Overall response rate	66.0%	64.6%	65.5%	66.1%	65.6%	--
Latest finding	Finding	Finding	Finding	Finding	Finding and Error^[7]	--
Rating of CE Donald Tsang	47.9	49.0	47.3[9]	48.4	50.6+/-1.6	+2.2^[9]
Vote of confidence in CE Donald Tsang	35%^[9]	47%^[9]	44%	42%	*25+/-3%*	+1%
Vote of no confidence in CE Donald Tsang	49%	37%^[9]	39%	43%^[9]	*66+/-3%*	-3%
Net approval rate	-41%	-41%	-43%	-45%	-41%	+4%

[7] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[8] Starting from 2008, these questions only use sub-samples of the tracking surveys concerned. The sub-sample sizes of questions on CE's support rating and hypothetical voting were 687 and 671 respectively in 2008, 751 and 749 respectively in 2009, 581 and 563 respectively in 2010, and 770 and 843 respectively this year.
[9] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Figures on Chief Executive's popularity before and after the Policy Address Speech from the handover till present are summarized as follows:

CE Tung Chee-hwa's popularity before and after the Policy Address Speech from 1997 to 2005
	1st Policy Address	2nd Policy Address	3rd Policy Address	4th Policy Address	5th Policy Address	6th Policy Address	7th Policy Address	8th Policy Address
Date of PA Speech	8/10/97	7/10/98	6/10/99	11/10/00	10/10/01	8/1/03	7/1/04	12/1/05
CE's popularity rating before the PA & error^[10]	65.8 +/-1.4	55.8 +/-1.6	54.0 +/-1.8	48.2 +/-2.2	48.4 +/-1.4	46.6 +/-1.4	42.9 +/-1.4	47.2 +/-1.2
CE's popularity rating at PA instant survey & error^[10]	66.1 +/-1.0	56.1 +/-1.0	54.3 +/-1.4	50.7 +/-1.4	50.6 +/-1.4	47.3 +/-1.4	44.6 +/-1.4	48.4 +/-1.4
Change in CE's rating	+0.3	+0.3	+0.3	+2.5^[11]	+2.2^[11]	+0.7	+1.7^[11]	+1.2

CE Donald Tsang's popularity before and after the Policy Address Speech from 2005 to 2011
	1st Policy Address	2nd Policy Address	3rd Policy Address	4th Policy Address	5th Policy Address	6th Policy Address	7th Policy Address
Date of Policy Address Speech	12/10/05	11/10/06	10/10/07	15/10/08	14/10/09	13/10/10	12/10/11
CE's popularity rating before the PA & error^[10]	68.0+/-1.0	62.9 +/-1.2	65.8+/-1.2	52.7+/-1.3	55.2+/-1.2	55.4+/-1.2	48.4+/-1.4
CE's popularity rating at PA instant survey & error^[10]	67.4+/-1.1	59.8+/-1.1	64.4+/-1.0	53.9+/-1.6	54.2+/-1.4	56.2+/-1.6	50.6+/-1.6
Change in CE's rating	-0.6	-3.1^[11]	-1.4^[11]	+1.2	-1.0	+0.8	+2.2^[11]
CE's approval rate before the PA & error^[10]	77+/-3%	64+/-3%	63+/-3%	44+/-3%	45+/-3%	42+/-3%	24+/-3%
CE's approval rate at PA instant survey & error^[10]	74+/-3%	56+/-3%	62+/-3%	44+/-4%	45+/-4%	41+/-4%	25+/-3%
Change in CE's approval rate^[12]	-3%	-8%^[11]	-1%	--	--	-1%	+1%

[10] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[11] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[12] Instant surveys on Policy Address included CE's approval rate since 2004, so it is not listed under Tung's series.

The install poll conducted last night showed that, after CE Donald Tsang announced his Policy Address, his latest support rating was 50.6 marks, with an approval and disapproval rate of 25% and 66% respectively. As for people's satisfaction of various Policy Addresses after the handover, the figures are summarized below:

Date of Survey	Sub-sample base^[15]	Appraisal of Policy Address: Satisfaction rate^[14]	Appraisal of Policy Address: Half-half	Appraisal of Policy Address: Dissatisfaction rate^[14]	Mean value^[14]	Satisfaction rating of Policy Address^[15]
12/10/11	816	47+/-3%^[17]	32+/-3%	18+/-3%	3.3+/-0.1 (Base=791)	59.1+/-1.4
13/10/10	747	41+/-4%^[17]	33+/-3%^[17]	19+/-3%^[17]	3.2+/-0.1 (Base=695)	58.9+/-1.4^[17]
14/10/09	462	30+/-4%	37+/-4%	28+/-4%	3.0+/-0.1 (Base =434)	53.5+/-2.1
15/10/08	515	31+/-4%^[17]	35+/-4%^[17]	26+/-4%^[17]	3.0+/-0.1 (Base =474)	53.8+/-2.0^[17]
10/10/07	602	52+/-4%^[17]	29+/-4%^[17]	10+/-2%^[17]	3.5+/-0.1 (Base =551)	65.2+/-1.6^[17]
11/10/06	445	30+/-4%^[17]	37+/-5%	22+/-4%^[17]	3.0+/-0.1 (Base =397)	55.8+/-2.0^[17]
12/10/05	377	48+/-5%^[17]	33+/-5%	9+/-3%^[17]	3.5+/-0.1 (Base =338)	66.4+/-1.9^[17]
12/1/05	391	38+/-5%^[17]	30+/-5%	20+/-4%^[17]	3.2+/-0.1 (Base =342)	56.3+/-2.4^[17]
7/1/04	381	25+/-4%	26+/-4%	33+/-5%^[17]	2.8+/-0.1 (Base =322)	49.3+/-2.4
8/1/03^[16]	377	22+/-4%^[17]	29+/-5%	27+/-5%	2.8+/-0.1 (Base =561)	51.6+/-2.6^[17]
10/10/01	433	29+/-4%	33+/-5%	27+/-4%	3.0+/-0.1 (Base =386)	56.7 +/-2.2
11/10/00	262	25+/-5%^[17]	28+/-6%	31+/-6%	2.9+/-0.1 (Base =219)	55.2 +/-2.8
6/10/99	236	31+/-6%^[17]	30+/-6%	25+/-6%^[17]	3.0+/-0.1 (Base =202)	57.3 +/-2.8
7/10/98	508	22+/-4%^[17]	35+/-4%^[17]	35+/-4%^[17]	2.8+/-0.1 (Base =469)	--
8/10/97	534	45+/-4%	30+/-4%^[17]	14+/-3%^[17]	3.4+/-0.1 (Base =474)	--

[13] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[14] Collapsed from a 5-point scale. The mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of positive level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[15] Excluding respondents who did not answer this question because they had not heard of / did not know the details of the Policy Address. Because of the smaller sample size, the sampling error has increased accordingly.
[16] The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our "POP SITE".
[17] Result changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

After excluding those respondents who said they did not know the details of the Policy Address, this year's instant survey showed that 47% were satisfied with it, 18% were dissatisfied and 32% said "half-half". Meanwhile, the average rating registered for the Policy Address was 59.1 marks. Because part of the respondents said they were not familiar with the Policy Address during the instant poll, the valid sub-sample of this item was smaller. The sampling error for this question has increased accordingly.

The survey also gauged the change of people's confidence towards Hong Kong's future after CE Donald Tsang has delivered his Policy Address. Results are as follows:

Date of survey	Sub-sample base^[19]	Overall response rate	Effect of the Policy Address on one's confidence in HK's future^[18]
Date of survey	Sub-sample base^[19]	Overall response rate	Increased	Unchanged	Decreased	Don't know/ Hard to say
12/10/11	957	65.6%	29+/-3%	45+/-3%	21+/-3%	5+/-1%
13/10/10	914	66.9%	31+/-3%^[21]	45+/-3%	18+/-3%^[21]	6+/-2%
14/10/09	749	71.9%	27+/-3%^[21]	47+/-4%^[21]	22+/-3%^[21]	5+/-2%
15/10/08	761	74.9%	23+/-3%^[21]	38+/-4%^[21]	32+/-3%^[21]	7+/-2%
10/10/07	388	69.9%	53+/-5%^[21]	31%/-5%^[21]	7+/-3%^[21]	9+/-3%
11/10/06	431	60.7%	25+/-4%^[21]	51+/-5%^[21]	16+/-4%^[21]	8+/-3%
12/10/05	476	66.1%	54+/-5%^[21]	33+/-4%^[21]	5+/-2%^[21]	8+/-2%^[21]
12/1/05	658	66.5%	34+/-4%	41+/-4%	12+/-3%^[21]	14+/-3%
7/1/04	602	67.5%	32+/-4%^[21]	40+/-4%	16+/-3%^[21]	12+/-3%
8/1/03^[20]	513	67.3%	25+/-4%	40+/- 4%^[21]	22+/-4%	14+/-3%^[21]
10/10/01	591	66.0%	22+/-3%	50+/- 4%^[21]	21+/-3%^[21]	7+/-2%^[21]
11/10/00	292	69.7%	22+/-5%^[21]	40+/-6%	15+/-4%	22+/-5%^[21]
6/10/99	233	54.5%	40+/-6%^[21]	36+/- 6%^[21]	16+/-5%^[21]	8+/-4%
7/10/98	505	56.5%	21+/-4%	52+/-4%	22+/-4%	5+/-2%

[18] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[19] Excluding respondents who did not answer this question for various reasons. Because of the smaller sample size, the sampling error has increased accordingly.
[20] The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our "HKU POP SITE".
[21] Result changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful. This survey series began in 1998.

Results showed that, excluding those who did not answer this question for various reasons, 29% said their confidence in the future of Hong Kong had increased, 45% opted for "no change", whilst 21% said their confidence had dropped.

Indepth Analysis

In the survey, we also asked respondents for their age. If they were reluctant to give their exact age, they could give us a range. According to their answers, we grouped them into 18-29, 30-49, and 50 years or older. Herewith further analysis of public's satisfaction rate of the Policy Address, effect of the Policy Address on their confidence in HK's future and Donald Tsang's rating and approval rates by respondents' age:

Date of survey: 12/10/11		18-29	30-49	50 or above	Overall
Satisfaction rating of Policy Address^[23]		60.3+/-2.4 (138)	55.2+/-2.5 (308)	62.2+/-2.2 (343)	59.2+/-1.4 (789)
Satisfaction rate of the Policy Address^[22]^[23]	Satisfaction	50+/-8% (70)	39+/-6% (122)	53+/-5% (189)	47+/-4% (380)
	Half-half	45+/-8% (63)	33+/-5% (102)	26+/-5% (94)	32+/-3% (258)
	Dissatisfaction	5+/-4% (7)	26+/-5% (80)	17+/-4% (59)	18+/-3% (147)
	Don't know/ hard to say	0% (0)	3+/-2% (10)	4+/-2% (14)	3+/-1% (24)
	Total	100% (140)	100% (314)	100% (356)	100% (810)
	Mean value^[22]	3.4+/-0.1 (140)	3.1+/-0.1 (304)	3.4+/-0.1 (342)	3.3+/-0.1 (786)

Date of survey: 12/10/11		18-29	30-49	50 or above	Overall
Effect of the Policy Address on their confidence in HK's future^[23]	Increased	31+/-7% (55)	22+/-4% (82)	34+/-5% (137)	29+/-3% (274)
	Unchanged	51+/-8% (89)	49+/-5% (183)	40+/-5% (160)	46+/-3% (432)
	Decreased	15+/-5% (26)	23+/-4% (87)	20+/-4% (80)	20+/-3% (193)
	Don't know/hard to say	3+/-2% (5)	5+/-2% (20)	6+/-2% (23)	5+/-1% (48)
	Total	100% (175)	100% (372)	100% (400)	100% (948)

Date of survey: 12/10/11		18-29	30-49	50 or above	Overall
Rating of CE Donald Tsang^[23]		50.2+/-2.9 (155)	47.8+/-2.7 (295)	54.0+/-2.6 (304)	50.8+/-1.6 (753)
Vote of confidence/ no confidence in CE Donald Tsang^[23]	Support	23+/-7% (38)	22+/-5% (73)	29+/-5% (99)	25+/-3% (210)
	Oppose	74+/-7% (123)	68+/-5% (222)	60+/-5% (205)	66+/-3% (549)
	Don't know/ hard to say	3+/-3% (5)	10+/-3% (33)	11+/-3% (37)	9+/-2% (76)
	Total	100% (166)	100% (328)	100% (341)	100% (836)

[22] Collapsed from a 5-point scale. The mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of positive level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[23] Differences among sub-groups are tested to be statistically significant at 95% confidence level.

Commentary

Note: The following commentary was written by Director of POP Robert Chung.

According to our Policy Address instant survey, among those who had some knowledge of CE Donald Tsang's seventh Address, 47% said they were satisfied, up 6 percentage points from that of last year, with a net satisfaction rate of positive 29 percentage points. On a scale of 0-100, this year's Address scored 59.1 marks, which is also higher than that of last year. All in all, people's instant reaction to this year's Policy Address is quite positive, third among Tsang's seven Addresses.

As for CE Donald Tsang's popularity, after giving his Address, his support rating has gone up significantly by 2.2 marks, back to 50-plus. His net popularity has also slightly improved from negative 45 to negative 41 percentage points, meaning that the Address has a positive effect on his popularity. Looking back at the instant effect of CH Tung and Donald Tsang's Policy Addresses across the years, Tung's Addresses usually have a stimulating effect, while Tsang's Addresses usually have a dampening effect. The effect this year is the most positive one for Tsang's popularity, but whether it could be sustained remains to be seen.

Furthermore, after excluding those who did not respond for various reasons, 29% said their confidence in the future of Hong Kong had increased after the Policy Address, 21% said their confidence had dropped, while 45% said "no change", thereby giving a net positive effect of 8 percentage points on people's confidence, which is fifth among Tsang's seven Addresses.

Further analysis shows that those of age 30 to 49 are significantly less satisfied with the CE Donald Tsang's seventh Policy Address, they give significantly lower rating to the Address and to CE, and their confidence in Hong Kong did not increase because of the Address.

Our instant survey has described people's instant reaction towards the Policy Address. How public opinion would change after CE and his officials explain their policies remains to be seen.

Future Release (Tentative)

October 18, 2011 (Tuesday) 1pm to 2pm: Policy Address First Follow-up survey

About HKUPOP: "Outline of our operation for the Policy Address instant survey of 2011"

After the HKSAR government announced the date of Policy Address, we started our planning for the instant survey.
About one month ago, we began to keep track of news about the Policy Address, in order to lay the ground work of questionnaire design.
About one week ago, we began our manpower deployment and internal preparation.
On the day CE announces the address, we monitored the media and the Internet, including the entire address and CE's subsequent press conferences, and then drafted the questionnaire.
Our random telephone interviews began at 6 p.m. on that day, involving more than 70 interviewers and staff. The interviews finished before 9:30 p.m., after collecting 1,032 samples.
Data verification and quantitative analyses followed immediately, together with the drafting of the press release.
On the following day, the survey findings were verified again, while our POP Site was re-designed. Our press release was compiled, proofread, and then released for public consumption.