HKU POP SITE releases findings of an instant poll on Policy AddressBack

Press Release on October 16, 2008

Abstract

The Public Opinion Programme at the University of Hong Kong interviewed 1,011 Hong Kong people last night (15 October, 2008) by means of a random telephone survey conducted by real interviewers. The survey finds that people's satisfaction with CE Donald Tsang's fourth Policy Address is 21 percentage points lower than that of last year, but is somewhat similar to that in 2006. On a scale of 0-100, they gave 53.8 marks to the Address, which is also similar to that in 2006. As for CE Donald Tsang's popularity, after giving his Address, his support rating has gone up slightly, his approval rate remains the same, while his disapproval rate has gone down significantly. This means the Address has a slightly positive effect on CE's popularity. However, after excluding those who said they did not know the content of the policy address, 23% said their confidence in the future of Hong Kong had increased, 32% said their confidence had dropped, while 38% said "no change". This shows that the Address has failed to increase people's confidence. Robert Chung, Director of Public Opinion Programme, however, praised CE Donald Tsang for swiftly and actively communicating with people directly after delivering his Address this year, through various forums, phone-in programmes, and question-and-answer sessions. Their effects, however, could only be told by our next survey. The sampling error of all percentages is below +/-2 to 4 percentage points at 95% confidence level, while the sampling error of rating figures needs another calculation. The response rate of the survey is 75%.

Points to note:

* The address of the "HKU POP SITE" is http://hkupop.pori.hk, journalists can check out the details of the survey there.
* The sample size of this survey is 1,011 successful interviews, not 1,011 x 74.9% response rate. In the past, many media made this mistake.
* The maximum sampling error of all percentages is below +/-2 to 4 percentage points at 95% confidence level, while the sampling error of rating figures needs another calculation. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state "sampling error of various ratings not more than +/-2.0 and sampling error of percentages not more than +/-4% at 95% confidence level".
* When quoting percentages of this survey, journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used, in order to match the precision level of the figures.
* The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Background

Since 1992, HKUPOP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. Starting 2008, we split up previous years' instant survey into two surveys. In our first survey, we measure people's overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE's popularity. In our second survey, we focus on people's reactions towards different government proposals, their satisfaction with CE's policy direction, and other relevant issues. The instant survey released today is our first release under our new operation.

Latest Figures

The findings of the Policy Address instant poll released by the POP SITE today have been weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in mid-2008. Herewith the contact information of various surveys:

Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %*
Year of survey	Date of survey	Total sample size	Response rate	Sampling error of %*
2008	15/10/08	1,011	74.9%	+/-3%
2007	10/10/07	1,023	69.9%	+/-3%
2006	11/10/06	1,027	60.7%	+/-3%
2005	12/10/05	914	66.1%	+/-3%
2004	7/1/04	1,040	67.5%	+/-3%
2003	8-9/1/03	1,259	68.9%	+/-3%
2001	10/10/01	1,051	66.0%	+/-3%
2000	11/10/00	1,059	69.7%	+/-3%
1999	6/10/99	888	54.5%	+/-3%
1998	7/10/98	1,494	56.5%	+/-3%
1997	8/10/97	1,523	61.5%	+/-3%

* Calculated at 95% confidence level using full sample size. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Questions using only sub-samples would have bigger sample error. Sampling errors of ratings are calculated according to the distribution of the scores collected.

Recent figures on Donald Tsang's popularity are summarized as follows:

Date of survey	18-20/8/08	1-5/9/08	17-19/9/08	8-10/10/08	15/10/08**		Latest change
Sample base	1,000	1,010	1,003	1,007	1,011		--
Overall response rate	69.6%	68.7%	63.6%	61.9%	74.9%		--
Sampling error of ratings (at 95% confidence level)*	+/-1.3	+/-1.3	+/-1.2	+/-1.3	+/-1.6		--
Sampling error of percentages (at 95% confidence level)*	+/-3%	+/-3%	+/-3%	+/-3%	+/-4%		--
Finding for each question/Sampling error*	Finding	Finding	Finding	Finding	Finding	Sampling error	--
Rating of CE Donald Tsang	54.1	51.8	52.3	52.7	53.9	+/-1.6	+1.2
Vote of confidence in CE Donald Tsang	39%	41%	42%	44%	44%	+/-4%	--
Vote of no confidence in CE Donald Tsang	41%	41%	36%	39%	34%	+/-4%	-5%#

* "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
** Starting from 2008, these questions only uses sub-samples of the tracking surveys concerned. The sub-sample sizes of questions on CE's support rating and hypothetical voting were 687 and 671 respectively.
# Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

The install poll conducted last night showed that, after CE Donald Tsang announced his Policy Address, his latest rating registered was 53.9 marks, with an approval and disapproval rate of 44% and 34% respectively. In terms of people's satisfaction of the Policy Address, the figures are summarized below:

Date of survey	Sub-sample base#	Appraisal of Policy Address: Satisfaction rate**	Appraisal of Policy Address: Half-half	Appraisal of Policy Address: Dissatisfaction rate**	Satisfaction rating of Policy Address*
15/10/08	515	31% +/- 4%	35% +/- 4%	26% +/- 4%	53.8 +/- 2.0
10/10/07	602	52% +/- 4%	29% +/- 4%	10% +/- 2%	65.2 +/- 1.6
11/10/06	445	30% +/- 4%	37% +/- 5%	22% +/- 4%	55.8 +/- 2.0
12/10/05	377	48% +/- 5%	33% +/- 5%	9% +/- 3%	66.4 +/- 1.9
12/1/05	391	38% +/- 5%	30% +/- 5%	20% +/- 4%	56.3 +/- 2.4
7/1/04	381	25% +/- 4%	26% +/- 4%	33% +/- 5%	49.3 +/- 2.4
8/1/03^	377	22% +/- 4%	29% +/- 5%	27% +/- 5%	51.6 +/- 2.6
10/10/01	433	29% +/- 4%	33% +/- 5%	27% +/- 4%	56.7 +/- 2.2
11/10/00	262	25% +/- 5%	28% +/- 6%	31% +/- 6%	55.2 +/- 2.8
6/10/99	236	31% +/- 6%	30% +/- 6%	25% +/- 6%	57.3 +/- 2.8
7/10/98	508	22% +/- 4%	35% +/- 4%	35% +/- 4%	--
8/10/97	534	45% +/- 4%	30% +/- 4%	14% +/- 3%	--

* "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified.
** Collapsed from a 5-point scale.
# Excluding respondents who said they would not answer because they had not heard of / did not know the details of the Policy Address. Because of the smaller sample size, the sampling error has increased accordingly.
^ The 2003 Policy Address instant poll was conducted for two days. Only figures registered in the first day of fieldwork are listed in this table for direct comparison and analysis. Aggregate results are available in our "HKU POP SITE".

After excluding those respondents who said they did not know the details of the Policy Address, this year's instant survey showed that 31% were satisfied with it, 26% were dissatisfied and 35% said "half-half" while the average rating registered for the Policy Address was 53.8 marks. Because part of the respondents said they were not familiar with the Policy Address during the instant poll, the valid sub-sample of this item was smaller. The sampling error for this question has increased accordingly.

The survey also gauged the change of people's confidence towards Hong Kong's future after CE Donald Tsang has delivered his Policy Address. Results are as follows:

Date of survey	Sub-sample base#	Overall response rate	Effect of the Policy Address on one's confidence in HK's future*
Date of survey	Sub-sample base#	Overall response rate	Increased	Unchanged	Decreased	Don't know/ Hard to say
15/10/08	761	74.9%	23% +/- 3%	38% +/- 4%	32% +/- 3%	7% +/- 2%
10/10/07	388	69.9%	53% +/- 5%	31% +/- 5%	7% +/- 3%	9% +/- 3%
11/10/06	431	60.7%	25% +/- 4%	51% +/- 5%	16% +/- 4%	8% +/- 3%
12/10/05	476	66.1%	54% +/- 5%	33% +/- 4%	5% +/-2%	8% +/- 2%
12/1/05	658	66.5%	34% +/- 4%	41% +/- 4%	12% +/- 3%	14% +/- 3%
7/1/04	602	67.5%	32% +/- 4%	40% +/- 4%	16% +/- 3%	12% +/- 3%
8/1/03^	513	67.3%	25% +/- 4%	40% +/- 4%	22% +/- 4%	14% +/- 3%
10/10/01	591	66.0%	22% +/- 3%	50% +/- 4%	21% +/- 3%	7% +/- 2%
11/10/00	292	69.7%	22% +/- 5%	40% +/- 6%	15% +/- 4%	22% +/- 5%
6/10/99	233	54.5%	40% +/- 6%	36% +/- 6%	16% +/- 5%	8% +/- 4%
7/10/98	505	56.5%	21% +/- 4%	52% +/- 4%	22% +/- 4%	5% +/- 2%

Commentary

Note: The following commentary was written by Director of POP Robert Chung.

Since 1992, HKUPOP has been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. In February this year after the Budget Talk, we modified our instant survey operation by splitting it into two. Our first round of survey would concentrate on satisfaction rates and related ratings, while our second round would focus on specific policy proposals. Our purpose is to shorten the time of our operation to collect opinion data for public consumption.

In free and democratic societies, instant surveys are indispensable sources of free information. Combined with appropriate follow-up surveys, and in parallel to expert analyses, they give a multi-dimensional picture of opinion development. They are an important part of a society's interactive development. Today at 09:00 to 10:30 Hong Kong Time, it so happens that the American presidential candidates Barack Obama and John McCain had their final television debate. At 10:27, CBS announced its instant poll result to show that Obama has won the debate. Locally, some critics have repeatedly said that instant polls are useless. Maybe they should also study the situation overseas.

According to our Policy Address instant survey, among the over 500 respondents who have some knowledge of the Address, 31% said they were satisfied, down 21 percentage points from that of last year, but somewhat similar to that in 2006. On a scale of 0-100, they gave 53.8 marks to the Address, which is also similar to that in 2006. As for CE Donald Tsang's popularity, after giving his Address, his support rating has gone up slightly, his approval rate remains the same, while his disapproval rate has gone down significantly. This means the Address has a slightly positive effect on CE's popularity.

However, after excluding those who said they did not know the content of the Policy Address, 23% said their confidence in the future of Hong Kong had increased, 32% said their confidence had dropped, while 38% said "no change". This shows that the Address has failed to increase people's confidence.

What needs to be praised is that CE Donald Tsang, after delivering his Address this year, has swiftly and actively communicated with people directly, through various forums, phone-in programmes, and question-and-answer sessions. Their effects would be told by our next survey.

News about POP

POP's normal practice is to release the results of our regular surveys every Tuesday afternoon via our POP Site, except during public holidays, each time with a forecast of the items to be released in the next 7 days. According to schedule, our next release of regular survey findings will be October 21, Tuesday, between 1pm and 2pm, when the results of second round of Policy Address Survey will be released. Then on October 23, 2008, Thursday, between 1pm and 2pm, POP will release latest results of people's appraisal of local news media.

POP will also follow the rhythm of the WorldPublicOpinion.org (WPO) to globally release the Chinese versions of WPO's press releases regularly, via our "World Public Opinion Platform" accessible through our POP Site and the "Hong Kong People's Opinion Platform" at http://www.hkpop.hk.

Our general practice is to answer all questions on the research design of the surveys published in the POP Site as soon as we receive them, but we will not further comment on the findings. We welcome questions for follow-up purpose, please email them to us at <[email protected]>. We will keep such an arrangement under constant review, suggestions most welcome. Please note that everything carried in the POP Site does not represent the stand of the University of Hong Kong. Dr Robert Ting-Yiu Chung, Director of POP, is responsible for everything posted herewith, except for column articles which represent the stand of their authors.

Starting from January 2006, we have included in our regular press releases a small educational section for the purpose of general civic education, so that we can share our experience with the general public. The subject of our education section today is "About HKUPOP".

About HKUPOP

The development and operation of Policy Address instant surveys

HKUPOP was established in 1991, we began our first Policy Address instant survey in October 1992, and our operation has not changed much since then. By "instant survey", we mean a survey which begins on the same day when the Policy Address is announced, in order to gauge the immediate reaction of the people. We have explained the development of our Policy Address instant surveys in our press releases of October 12, 2006 and October 11, 2007. Today we post it again, so that readers can refresh their memory.

Our instant surveys comprise a set of core questions asked every year for repeated measurement, plus another set of ad hoc questions designed to match the content of each year's Policy Address. Core questions include CE's popularity, people's overall appraisal of the Address, and its effect on people's confidence in the future of Hong Kong. Starting 2008, we split up previous years' instant survey into two surveys. In our first survey, we measure people's overall appraisal of the Policy Address, their rating of the Policy Address, their change in confidence towards Hong Kong's future, and CE's popularity. In our second survey, we focus on people's reactions towards different government proposals, their satisfaction with CE's policy direction, and other relevant issues.

Regarding the sample size, since 1992, it is always set at slightly over 1,000.

If case we have media sponsorship, our results would be released immediately, even on a real time basis. Otherwise, we generally announce the findings online through our website on the following day. The findings from our Policy Address instant surveys conducted on or before November 2000 have been published in our newsletter POP Express, and they are available in our POP Site in various formats.

Herewith an outline of our operation for the Policy Address instant survey of 2008:

After the HKSAR government announced the date of Policy Address, we started our planning for the instant survey.

About one month ago, we began to keep track of news about the Policy Address, in order to lay the ground work of questionnaire design.

About one week ago, we began our manpower deployment and internal preparation.

On the day CE announces the address, we monitored the media and the Internet, including the entire address and CE's subsequent press conferences, and then drafted the questionnaire.

Our random telephone interviews began at 6 p.m. on that day, involving nearly 50 interviewers and staff. The interviews finished at 10 p.m., after collecting 1,011 samples.

Data verification and quantitative analyses followed immediately, together with the drafting of the press release.

On the following day, the survey findings were verified again, while our POP Site was re-designed. Our press release was compiled, proofread, and then released for public consumption.