HKU POP SITE releases findings of an instant poll on Policy AddressBack

Press Release on October 14, 2010

Abstract

The Public Opinion Programme at the University of Hong Kong interviewed 1,020 Hong Kong people last night (13 October, 2010) by means of a random telephone survey conducted by real interviewers. The survey finds that among our sub-sample of people who had some knowledge of CE Donald Tsang's 6th Address, 41% said they were satisfied, up 11 percentage points from that of last year, with a net satisfaction rate of positive 22 percentage points. On a scale of 0-100, they gave 58.9 marks to the Address, which is an increase of 5.4 points. All in all, people's instant reaction to this year's Policy Address is fairly positive, third among Tsang's six Addresses. As for CE Donald Tsang's popularity, after giving his Address, both his support rating and approval rate have fluctuated within narrow margins, meaning that the Address has no significant effect on his popularity. Looking back at the instant effect of CH Tung and Donald Tsang's Policy Addresses across the years, we found that Tung's Addresses usually have a stimulating effect, while Tsang's Addresses by and large have a dampening effect. However, because Tsang's popularity figures are consistently higher than those of Tung, such an effect seems to have set very little pressure on Tsang. Furthermore, after excluding those who did not respond for various reasons, 31% said their confidence in the future of Hong Kong had increased after the Policy Address, 18% said their confidence had dropped, while 45% said "no change", thereby giving a net positive effect of 13 percentage points, which is also third among Tsang's six Addresses. The sampling error of all percentages is between +/-2 and +/-4 percentage points at 95% confidence level, while the sampling error of rating figures needs another calculation. The response rate of the survey is 67%.

Points to note:
[1] The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists can check out the details of the survey there.
[2] The sample size of this survey is 1,020 successful interviews, not 1,020 x 66.9% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is between +/-2 and +/-4 percentage points at 95% confidence level, while the sampling error of rating figures needs another calculation. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state "sampling error of various ratings not more than +/-1.6 and sampling error of percentages not more than +/-4% at 95% confidence level".
[4] When quoting percentages of this survey, journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used, in order to match the precision level of the figures.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Background

Since 1992, HKUPOP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. Starting 2008, we split up previous years' instant survey into two surveys. In our first survey, we measure people's overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE's popularity. In our second survey, we focus on people's reactions towards different government proposals, their satisfaction with CE's policy direction, and other relevant issues. The instant survey released today is our third release under our new operation.

Latest Figures

The findings of the Policy Address instant poll released by the POP SITE today have been weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-2010. Herewith the contact information of various surveys:

Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %^[6]
2010	13/10/10	1,020	66.9%	+/-3%
2009	14/10/09	1,007	71.9%	+/-3%
2008	15/10/08	1,011	74.9%	+/-3%
2007	10/10/07	1,023	69.9%	+/-3%
2006	11/10/06	1,027	60.7%	+/-3%
2005	12/10/05	914	66.1%	+/-3%
2004	7/1/04	1,040	67.5%	+/-3%
2003	8-9/1/03	1,259	68.9%	+/-3%
2001	10/10/01	1,051	66.0%	+/-3%
2000	11/10/00	1,059	69.7%	+/-3%
1999	6/10/99	888	54.5%	+/-3%
1998	7/10/98	1,494	56.5%	+/-3%
1997	8/10/97	1,523	61.5%	+/-3%

[6] Calculated at 95% confidence level using full sample size. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Questions using only sub-samples would have bigger sample error. Sampling errors of ratings are calculated according to the distribution of the scores collected.

Recent figures on Donald Tsang's popularity are summarized as follows:

Date of survey	17-20/8/10	31/8-3/9/10	18-24/9/10	5-8/10/10	13/10/10^[8]	Latest change
Sample base	1,006	1,010	1,010	1,014	1,020	--
Overall response rate	64.4%	65.2%	66.2%	60.0%	66.9%	--
Latest finding	Finding	Finding	Finding	Finding	Finding and Error^[7]	--
Rating of CE Donald Tsang	51.3	58.5^[9]	57.9	55.4^[9]	56.2+/-1.6	+0.8
Vote of confidence in CE Donald Tsang	35%^[9]	47%^[9]	44%	42%	41+/-4%	-1%
Vote of no confidence in CE Donald Tsang	49%	37%^[9]	39%	43%^[9]	41+/-4%	-2%

[7] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[8] Starting from 2008, these questions only use sub-samples of the tracking surveys concerned. The sub-sample sizes of questions on CE's support rating and hypothetical voting were 687 and 671 respectively in 2008, 751 and 749 respectively in 2009, and 581 and 563 respectively this year.
[9] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Figures on Chief Executive's popularity before and after the Policy Address Speech from the handover till present are summarized as follows:

CE Tung Chee-hwa's popularity before and after the Policy Address Speech from 1997 to 2005
	1st Policy Address	2nd Policy Address	3rd Policy Address	4th Policy Address	5th Policy Address	6th Policy Address	7th Policy Address	8th Policy Address
Date of PA Speech	8/10/97	7/10/98	6/10/99	11/10/00	10/10/01	8/1/03	7/1/04	12/1/05
CE's popularity rating before the PA & error^[10]	65.8 +/-1.4	55.8 +/-1.6	54.0 +/-1.8	48.2 +/-2.2	48.4 +/-1.4	46.6 +/-1.4	42.9 +/-1.4	47.2 +/-1.2
CE's popularity rating at PA instant survey & error^[10]	66.1 +/-1.0	56.1 +/-1.0	54.3 +/-1.4	50.7 +/-1.4	50.6 +/-1.4	47.3 +/-1.4	44.6 +/-1.4	48.4 +/-1.4
Change in CE's rating	+0.3	+0.3	+0.3	+2.5#	+2.2#	+0.7	+1.7#	+1.2

CE Donald Tsang's popularity before and after the Policy Address Speech from 2005 to 2010
	1st Policy Address	2nd Policy Address	3rd Policy Address	4th Policy Address	5th Policy Address	6th Policy Address
Date of Policy Address Speech	12/10/05	11/10/06	10/10/07	15/10/08	14/10/09	13/10/10
CE's popularity rating before the PA & error^[10]	68.0+/-1.0	62.9 +/-1.2	65.8+/-1.2	52.7+/-1.3	55.2+/-1.2	55.4+/-1.2
CE's popularity rating at PA instant survey & error^[10]	67.4+/-1.1	59.8+/-1.1	64.4+/-1.0	53.9+/-1.6	54.2+/-1.4	56.2+/-1.6
Change in CE's rating	-0.6	-3.1^[11]	-1.4^[11]	+1.2	-1.0	+0.8
CE's approval rate before the PA & error^[10]	77+/-3%	64+/-3%	63+/-3%	44+/-3%	45+/-3%	42+/-3%
CE's approval rate at PA instant survey & error^[10]	74+/-3%	56+/-3%	62+/-3%	44+/-4%	45+/-4%	41+/-4%
Change in CE's approval rate^[12]	-3%	-8%^[11]	-1%	--	--	-1%

[10] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[11] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
[12] Instant surveys on Policy Address included CE's approval rate since 2004, so it is not listed under Tung's series.

The install poll conducted last night showed that, after CE Donald Tsang announced his Policy Address, his latest support rating was 56.2 marks, with an approval and disapproval rate of 41% and 41% respectively. As for people's satisfaction of various Policy Addresses after the handover, the figures are summarized below:

Date of Survey	Sub-sample base^[15]	Appraisal of Policy Address: Satisfaction rate^[14]	Appraisal of Policy Address: Half-half	Appraisal of Policy Address: Dissatisfaction rate^[14]	Satisfaction rating of Policy Address
13/10/10	747	41+/-4%^[17]	33+/-3%^[17]	19+/-3%^[17]	58.9+/-1.4^[17]
14/10/09	462	30+/-4%	37+/-4%	28+/-4%	53.5+/-2.1
15/10/08	515	31+/-4%^[17]	35+/-4%^[17]	26+/-4%^[17]	53.8+/-2.0^[17]
10/10/07	602	52+/-4%^[17]	29+/-4%^[17]	10+/-2%^[17]	65.2+/-1.6^[17]
11/10/06	445	30+/-4%^[17]	37+/-5%	22+/-4%^[17]	55.8+/-2.0^[17]
12/10/05	377	48+/-5%^[17]	33+/-5%	9+/-3%^[17]	66.4+/-1.9^[17]
12/1/05	391	38+/-5%^[17]	30+/-5%	20+/-4%^[17]	56.3+/-2.4^[17]
7/1/04	381	25+/-4%	26+/-4%	33+/-5%^[17]	49.3+/-2.4
8/1/03^[16]	377	22+/-4%^[17]	29+/-5%	27+/-5%	51.6+/-2.6^[17]
10/10/01	433	29+/-4%	33+/-5%	27+/-4%	56.7 +/-2.2
11/10/00	262	25+/-5%^[17]	28+/-6%	31+/-6%	55.2 +/-2.8
6/10/99	236	31+/-6%^[17]	30+/-6%	25+/-6%^[17]	57.3 +/-2.8
7/10/98	508	22+/-4%^[17]	35+/-4%^[17]	35+/-4%^[17]	--
8/10/97	534	45+/-4%	30+/-4%^[17]	14+/-3%^[17]	--

[13] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[14] Collapsed from a 5-point scale.
[15] Excluding respondents who did not answer this question because they had not heard of / did not know the details of the Policy Address. Because of the smaller sample size, the sampling error has increased accordingly.
[16] The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our "POP SITE".
[17] Result changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

After excluding those respondents who said they did not know the details of the Policy Address, this year's instant survey showed that 41% were satisfied with it, 19% were dissatisfied and 33% said "half-half". Meanwhile, the average rating registered for the Policy Address was 58.9 marks. Because part of the respondents said they were not familiar with the Policy Address during the instant poll, the valid sub-sample of this item was smaller. The sampling error for this question has increased accordingly.

The survey also gauged the change of people's confidence towards Hong Kong's future after CE Donald Tsang has delivered his Policy Address. Results are as follows:

Date of survey	Sub-sample base^[19]	Overall response rate	Effect of the Policy Address on one's confidence in HK's future^[18]
Date of survey	Sub-sample base^[19]	Overall response rate	Increased	Unchanged	Decreased	Don't know/ Hard to say
13/10/10	914	66.9%	31+/-3%^[21]	45+/-3%	18+/-3%^[21]	6+/-2%
14/10/09	749	71.9%	27+/-3%^[21]	47+/-4%^[21]	22+/-3%^[21]	5+/-2%
15/10/08	761	74.9%	23+/-3%^[21]	38+/-4%^[21]	32+/-3%^[21]	7+/-2%
10/10/07	388	69.9%	53+/-5%^[21]	31%/-5%^[21]	7+/-3%^[21]	9+/-3%
11/10/06	431	60.7%	25+/-4%^[21]	51+/-5%^[21]	16+/-4%^[21]	8+/-3%
12/10/05	476	66.1%	54+/-5%^[21]	33+/-4%^[21]	5+/-2%^[21]	8+/-2%^[21]
12/1/05	658	66.5%	34+/-4%	41+/-4%	12+/-3%^[21]	14+/-3%
7/1/04	602	67.5%	32+/-4%^[21]	40+/-4%	16+/-3%^[21]	12+/-3%
8/1/03^[20]	513	67.3%	25+/-4%	40+/- 4%^[21]	22+/-4%	14+/-3%^[21]
10/10/01	591	66.0%	22+/-3%	50+/- 4%^[21]	21+/-3%^[21]	7+/-2%^[21]
11/10/00	292	69.7%	22+/-5%^[21]	40+/-6%	15+/-4%	22+/-5%^[21]
6/10/99	233	54.5%	40+/-6%^[21]	36+/- 6%^[21]	16+/-5%^[21]	8+/-4%
7/10/98	505	56.5%	21+/-4%	52+/-4%	22+/-4%	5+/-2%

[18] All error figures in the table are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
[19] Excluding respondents who did not answer this question for various reasons. Because of the smaller sample size, the sampling error has increased accordingly.
[20] The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our "HKU POP SITE".
[21] Result changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful. This survey series began in 1998.

Results showed that, excluding those who did not answer this question for various reasons, 31% said their confidence in the future of Hong Kong had increased, 45% opted for "no change", whilst 18% said their confidence had dropped.

Commentary

Note: The following commentary was written by Director of POP Robert Chung.

According to our Policy Address instant survey, among our sub-sample of people who had some knowledge of CE Donald Tsang's 6th Address, 41% said they were satisfied, up 11 percentage points from that of last year, with a net satisfaction rate of positive 22 percentage points. On a scale of 0-100, they gave 58.9 marks to the Address, which is an increase of 5.4 points. All in all, people's instant reaction to this year's Policy Address is fairly positive, third among Tsang's six Addresses.

As for CE Donald Tsang's popularity, after giving his Address, both his support rating and approval rate have fluctuated within narrow margins, meaning that the Address has no significant effect on his popularity. Looking back at the instant effect of CH Tung and Donald Tsang's Policy Addresses across the years, we found that Tung's Addresses usually have a stimulating effect, while Tsang's Addresses by and large have a dampening effect. However, because Tsang's popularity figures are consistently higher than those of Tung, such an effect seems to have set very little pressure on Tsang.

Furthermore, after excluding those who did not respond for various reasons, 31% said their confidence in the future of Hong Kong had increased after the Policy Address, 18% said their confidence had dropped, while 45% said "no change", thereby giving a net positive effect of 13 percentage points, which is also third among Tsang's six Addresses.

Our instant survey has shown people's instant reaction towards the Policy Address. How public opinion would change after CE and his officials explain their policies remains to be seen.

Future Release (Tentative)

October 19, 2010 (Tuesday) 1pm to 2pm: Follow-up survey of Policy Address

About HKUPOP: "Outline of our operation for the Policy Address instant survey of 2010"

After the HKSAR government announced the date of Policy Address, we started our planning for the instant survey.
About one month ago, we began to keep track of news about the Policy Address, in order to lay the ground work of questionnaire design.
About one week ago, we began our manpower deployment and internal preparation.
On the day CE announces the address, we monitored the media and the Internet, including the entire address and CE's subsequent press conferences, and then drafted the questionnaire.
Our random telephone interviews began at 6 p.m. on that day, involving more than 70 interviewers and staff. The interviews finished at 10:30 p.m., after collecting 1,020 samples.
Data verification and quantitative analyses followed immediately, together with the drafting of the press release.
On the following day, the survey findings were verified again, while our POP Site was re-designed. Our press release was compiled, proofread, and then released for public consumption.