HKU POP SITE releases findings of an instant poll on Policy AddressBack

Press Release on October 15, 2009

Abstract

The Public Opinion Programme at the University of Hong Kong interviewed 1,007 Hong Kong people last night (14 October, 2009) by means of a random telephone survey conducted by real interviewers. The survey finds that among the nearly 500 respondents who have some knowledge of the Address, 30% said they were satisfied, 28% were not, and the Address scored 53.5 marks. These figures are very similar to those of last year, meaning that overall appraisal is not too positive, but nevertheless outweighs negative appraisal by just a little. Net satisfaction rate now stands at positive 2 percentage points. As for CE Donald Tsang's popularity, after giving his Address, his support rating has dropped a bit, while his approval rate remains intact. This shows that the Address has no significant effect on his popularity. Looking back at the instant effect of CH Tung and Donald Tsang's Policy Addresses across the years, we found that Tung's Addresses usually have a stimulating effect, while Tsang's Addresses by and large have a dampening effect. Furthermore, after excluding those who did not respond for various reasons, 27% said their confidence in the future of Hong Kong had increased after the Policy Address, 22% said their confidence had dropped, while 47% said 「no change」. Director of POP Robert Chung observed, because CE Tsang's popularity figures are consistently higher than those of CE Tung, the dampening effect of the Address seems to have set very little pressure on CE Tsang. Moreover, although more positive than last year, the Address' little effect in boosting people's confidence probably because the economy is already recovering. The sampling error of all percentages is between +/-2 and 4 percentage points at 95% confidence level, while the sampling error of rating figures needs another calculation. The response rate of the survey is 72%.

Points to note:
* The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists can check out the details of the survey there.
* The sample size of this survey is 1,007 successful interviews, not 1,007 x 71.9% response rate. In the past, many media made this mistake.
* The maximum sampling error of all percentages is between +/-2 and 4 percentage points at 95% confidence level, while the sampling error of rating figures needs another calculation. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state "sampling error of various ratings not more than +/-2.1 and sampling error of percentages not more than +/-4% at 95% confidence level".
* When quoting percentages of this survey, journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used, in order to match the precision level of the figures.
* The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Background

Since 1992, HKUPOP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. Starting 2008, we split up previous years' instant survey into two surveys. In our first survey, we measure people's overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE's popularity. In our second survey, we focus on people's reactions towards different government proposals, their satisfaction with CE's policy direction, and other relevant issues. The instant survey released today is our second release under our new operation.

Latest Figures

The findings of the Policy Address instant poll released by the POP SITE today have been weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-2009. Herewith the contact information of various surveys:

Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %*
Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %*
2009	14/10/09	1,007	71.9%	+/-3%
2008	15/10/08	1,011	74.9%	+/-3%
2007	10/10/07	1,023	69.9%	+/-3%
2006	11/10/06	1,027	60.7%	+/-3%
2005	12/10/05	914	66.1%	+/-3%
2004	7/1/04	1,040	67.5%	+/-3%
2003	8-9/1/03	1,259	68.9%	+/-3%
2001	10/10/01	1,051	66.0%	+/-3%
2000	11/10/00	1,059	69.7%	+/-3%
1999	6/10/99	888	54.5%	+/-3%
1998	7/10/98	1,494	56.5%	+/-3%
1997	8/10/97	1,523	61.5%	+/-3%

* Calculated at 95% confidence level using full sample size. 「95% confidence level」 means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Questions using only sub-samples would have bigger sample error. Sampling errors of ratings are calculated according to the distribution of the scores collected.

Recent figures on Donald Tsang's popularity are summarized as follows:

Date of survey	18-24/8/09	1-4/9/09	14-17/9/09	6-11/10/09^	14/10/09^	Latest change
Sample base	1,020	1,002	1,004	1,010	1,007	--
Overall response rate	66.7%	60.5%	65.4%	67.5%	71.9%	--
Latest finding	Finding	Finding	Finding	Finding	Finding and Error*	--
Rating of CE Donald Tsang	54.5	52.8#	54.8#	55.2	54.2 +/-1.4	-1.0
Vote of confidence in CE Donald Tsang	42%	42%	47%#	45%	45% +/-4%	--
Vote of no confidence in CE Donald Tsang	38%	42%#	40%	38%	37% +/-4%	-1%

*Errors are calculated at 95% confidence level. ?"95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
^ Starting from 2008, these questions only uses sub-samples of the tracking surveys concerned. The sub-sample sizes of questions on CE's support rating and hypothetical voting were 687 and 671 respectively in 2008, while the sub-sample sizes were 735 and 742 this year.
# Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

Figures on Chief Executive's popularity before and after the Policy Address Speech from the handover till present are summarized as follows:

CE Tung Chee-hwa's popularity before and after the Policy Address Speech from 1997 to 2005
	1st Policy Address	2nd Policy Address	3rd Policy Address	4th Policy Address	5th Policy Address	6th Policy Address	7th Policy Address	8th Policy Address
Date of PA Speech	8/10/97	7/10/98	6/10/99	11/10/00	10/10/01	8/1/03	7/1/04	12/1/05
CE's popularity rating before the PA & error*	65.8 +/-1.4	55.8 +/-1.6	54.0 +/-1.8	48.2 +/-2.2	48.4 +/-1.4	46.6 +/-1.4	42.9 +/-1.4	47.2 +/-1.2
CE's popularity rating at PA instant survey & error*	66.1 +/-1.0	56.1 +/-1.0	54.3 +/-1.4	50.7 +/-1.4	50.6 +/-1.4	47.3 +/-1.4	44.6 +/-1.4	48.4 +/-1.4
Change in CE's rating	+0.3	+0.3	+0.3	+2.5#	+2.2#	+0.7	+1.7#	+1.2

CE Donald Tsang's popularity before and after the Policy Address Speech from 2005 to 2009
	1st Policy Address	2nd Policy Address	3rd Policy Address	4th Policy Address	5th Policy Address
Date of Policy Address Speech	12/10/05	11/10/06	10/10/07	15/10/08	14/10/09
CE's popularity rating before the PA & error*	68.0+/-1.0	62.9 +/-1.2	65.8+/-1.2	52.7+/-1.3	55.2+/-1.2
CE's popularity rating at PA instant survey & error*	67.4+/-1.1	59.8+/-1.1	64.4+/-1.0	53.9+/-1.6	54.2 +/-1.4
Change in CE's rating	-0.6	-3.1#	-1.4#	+1.2	-1.0
CE's approval rate before the PA & error*	77% +/-3%	64% +/-3%	63% +/-3%	44% +/-3%	45% +/-3%
CE's approval rate at PA instant survey & error*	74% +/-3%	56% +/-3%	62% +/-3%	44% +/-4%	45% +/-4%
Change in CE's approval rate ^	-3%	-8%#	-1%	--	--

*Errors are calculated at 95% confidence level. 「95% confidence level」 means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
# Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.
^Instant surveys on Policy Address included CE's approval rate since 2004, so it is not listed under Tung's series.

The install poll conducted last night showed that, after CE Donald Tsang announced his Policy Address, his latest support rating was 54.2 marks, with an approval and disapproval rate of 45% and 37% respectively. As for people's satisfaction of various Policy Addresses after the handover, the figures are summarized below:

Date of Survey	Sub-sample base^	Appraisal of Policy Address: Satisfaction rate**	Appraisal of Policy Address: Half-half	Appraisal of Policy Address: Dissatisfaction rate**	Satisfaction rating of Policy Address
14/10/09	462	30% +/- 4%	37% +/- 4%	28% +/- 4%	53.5 +/- 2.1
15/10/08	515	31%# +/- 4%	35%# +/- 4%	26%# +/- 4%	53.8# +/- 2.0
10/10/07	602	52%# +/- 4%	29%# +/- 4%	10%# +/- 2%	65.2# +/- 1.6
11/10/06	445	30%# +/- 4%	37% +/- 5%	22%# +/- 4%	55.8# +/- 2.0
12/10/05	377	48%# +/- 5%	33% +/- 5%	9%# +/- 3%	66.4# +/- 1.9
12/1/05	391	38%# +/- 5%	30% +/- 5%	20%# +/- 4%	56.3# +/- 2.4
7/1/04	381	25% +/- 4%	26% +/- 4%	33%# +/- 5%	49.3 +/- 2.4
8/1/03^^	377	22%# +/- 4%	29% +/- 5%	27% +/- 5%	51.6# +/- 2.6
10/10/01	433	29% +/- 4%	33% +/- 5%	27% +/- 4%	56.7 +/- 2.2
11/10/00	262	25%# +/- 5%	28% +/- 6%	31% +/- 6%	55.2 +/- 2.8
6/10/99	236	31%# +/- 6%	30% +/- 6%	25%# +/- 6%	57.3 +/- 2.8
7/10/98	508	22%# +/- 4%	35%# +/- 4%	35%# +/- 4%	--
8/10/97	534	45% +/- 4%	30%# +/- 4%	14%# +/- 3%	--

* Errors are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
** Collapsed from a 5-point scale.
^ Excluding respondents who did not answer this question because they had not heard of / did not know the details of the Policy Address. Because of the smaller sample size, the sampling error has increased accordingly.
^^ The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our 「HKU POP SITE」.
# Result changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

After excluding those respondents who said they did not know the details of the Policy Address, this year's instant survey showed that 30% were satisfied with it, 28% were dissatisfied and 37% said "half-half" while the average rating registered for the Policy Address was 53.5 marks. Because part of the respondents said they were not familiar with the Policy Address during the instant poll, the valid sub-sample of this item was smaller. The sampling error for this question has increased accordingly.

The survey also gauged the change of people's confidence towards Hong Kong's future after CE Donald Tsang has delivered his Policy Address. Results are as follows:

Date of survey	Sub-sample base^	Overall response rate	Effect of the Policy Address on one's confidence in HK's future*
Date of survey	Sub-sample base^	Overall response rate	Increased	Unchanged	Decreased	Don't know/ Hard to say
14/10/09	749	71.9%	27%# +/- 3%	47%# +/- 4%	22%# +/- 3%	5% +/- 2%
15/10/08	761	74.9%	23%# +/- 3%	38%# +/- 4%	32%# +/- 3%	7% +/- 2%
10/10/07	388	69.9%	53%# +/- 5%	31%# +/- 5%	7%# +/- 3%	9% +/- 3%
11/10/06	431	60.7%	25%# +/- 4%	51%# +/- 5%	16%# +/- 4%	8% +/- 3%
12/10/05	476	66.1%	54%# +/- 5%	33%# +/- 4%	5%# +/- 2%	8%# +/- 2%
12/1/05	658	66.5%	34% +/- 4%	41% +/- 4%	12%# +/- 3%	14% +/- 3%
7/1/04	602	67.5%	32%# +/- 4%	40% +/- 4%	16%# +/- 3%	12% +/- 3%
8/1/03^^	513	67.3%	25% +/- 4%	40%# +/- 4%	22% +/- 4%	14%# +/- 3%
10/10/01	591	66.0%	22% +/- 3%	50%# +/- 4%	21%# +/- 3%	7%# +/- 2%
11/10/00	292	69.7%	22%# +/- 5%	40% +/- 6%	15% +/- 4%	22%# +/- 5%
6/10/99	233	54.5%	40%# +/- 6%	36%# +/- 6%	16%# +/- 5%	8% +/- 4%
7/10/98	505	56.5%	21% +/- 4%	52% +/- 4%	22% +/- 4%	5% +/- 2%

*Errors are calculated at 95% confidence level. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
^ Excluding respondents who did not answer this question for various reasons. Because of the smaller sample size, the sampling error has increased accordingly.
^^ The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our 「HKU POP SITE」.
# Result changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful. This survey series began in 1998.

Results showed that, excluding those who did not answer this question for various reasons, 27% said their confidence in the future of Hong Kong had increased, 47% opted for 「no change」, whilst 22% said their confidence had dropped.

Commentary

Note: The following commentary was written by Director of POP Robert Chung.

According to our Policy Address instant survey, among the nearly 500 respondents who have some knowledge of the CE Donald Tsang's 5th Address, 30% said they were satisfied, 28% were not, and the Address scored 53.5 marks. These figures are very similar to those of last year, meaning that overall appraisal is not too positive, but nevertheless outweighs negative appraisal by just a little. Net satisfaction rate now stands at positive 2 percentage points.

As for CE Donald Tsang's popularity, after giving his Address, his support rating has dropped a bit, while his approval rate remains intact. This shows that the Address has no significant effect on his popularity. Looking back at the instant effect of CH Tung and Donald Tsang's Policy Addresses across the years, we found that Tung's Addresses usually have a stimulating effect, while Tsang's Addresses by and large have a dampening effect. However, because CE Tsang's popularity figures are consistently higher than those of CE Tung, such an effect seems to have set very little pressure on CE Tsang.

Furthermore, after excluding those who did not respond for various reasons, 27% said their confidence in the future of Hong Kong had increased after the Policy Address, 22% said their confidence had dropped, while 47% said 「no change」. This shows that the Address has little effect in boosting people's confidence, but is already more positive than last year, probably because the economy is recovering.

Our instant survey has shown people's instant reaction towards the Policy Address. How public opinion would change after CE and his officials explain their policies remains to be seen.

Next Release (Tentative)

October 20, 2009 (Tuesday) 1pm to 2pm: Follow-up survey of Policy Address
October 22, 2009 (Thursday) 1pm to 2pm: Ratings of Top 10 Legislative Councillors

About HKUPOP: 「Outline of our operation for the Policy Address instant survey of 2009」

After the HKSAR government announced the date of Policy Address, we started our planning for the instant survey.
About one month ago, we began to keep track of news about the Policy Address, in order to lay the ground work of questionnaire design.
About one week ago, we began our manpower deployment and internal preparation.
On the day CE announces the address, we monitored the media and the Internet, including the entire address and CE's subsequent press conferences, and then drafted the questionnaire.
Our random telephone interviews began at 6 p.m. on that day, involving more than 60 interviewers and staff. The interviews finished at 10 p.m., after collecting 1,007 samples.
Data verification and quantitative analyses followed immediately, together with the drafting of the press release.
On the following day, the survey findings were verified again, while our POP Site was re-designed. Our press release was compiled, proofread, and then released for public consumption.