HKU POP releases the third survey on this year’s BudgetBack

To facilitate academic study and rational discussion, the Public Opinion Programme (POP) of the University of Hong Kong released on Tuesday for public examination via the “HKU POP Site” (http://hkupop.pori.hk) the raw data of all 43 regular rating surveys of CE CY Leung, along with related demographics of respondents. This degree of transparency has surpassed normal academic and professional standards, and should be treasured. Users can process the data using SPSS, which should be freely accessible by all university staff and students from their own university. Ordinary users can choose to use a variety of freeware to run their own analysis. Please follow normal academic standards when using or citing such data.

POP interviewed 1,027 Hong Kong people from 17 to 20 March by means of a random telephone survey conducted by real interviewers. Over the years, we find people’s receptiveness of the government’s Budget drops after it has been debated for some time. The situation this year is that people’s initial appraisal of the Budget was rather negative, its satisfaction rating even registered a record low since 2008. People’s satisfaction did not change much after widespread coverage by the media for a day or two. Three weeks later, dissatisfaction tapers while satisfaction rebounds slightly but remains negative, as reflected by a number of indicators. These include the slight rebound of satisfaction rating from 48.5 to 48.8 marks, and the small improvement in net satisfaction rate from negative 22 to negative 20 percentage points. In terms of specific policies, people remain negative on the relief measures suggested by the FS, but continue to support his suggestion to contain expenditure growth, preserve revenue base, and save for future, in order to prevent the occurrence of structural deficit. Our survey also shows that people continue to agree to the saying that there is an internal contradiction of financial management concept indicated by the benefits given away in the Policy Address and warning of deficit in the Budget. Finally, people continue to consider Hong Kong’s tax system as fair, but consider the distribution of wealth unreasonable. People’s satisfaction with the government’s fiscal policies remains negative, with a net satisfaction of negative 11 percentage points. The maximum sampling error of all percentages is +/-3 percentage points at 95% confidence level, while the sampling error of rating figure is +/-1.3, and that of net values needs another calculation. The response rate of the survey is 68%.

[2] The overall sample size of this survey is 1,027 successful interviews, not 1,027 x 68.1% response rate. In the past, many media made this mistake.
[3] The maximum sampling error of all percentages is +/-3 percentage points at 95% confidence level, while the sampling error needs another calculation. "95% confidence level" means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. When quoting these figures, journalists can state "sampling error of rating not more than +/-1.3, that of percentages not more than +/-3% and net values not more than +/-5% at 95% confidence level". Because POP introduced “rim weighting” in 2014, during the transition period, whether changes in various figures are beyond sampling errors are based on tests using the same weighting methods. That is, to test whether the first set of figures collected in 2014 is significantly different from that of the previous survey, both sets of data are rim weighted before testing, instead of using simple computation of the published figures.
[4] Because of sampling errors in conducting the survey(s) and the rounding procedures in processing the data, the figures cannot be too precise, and the totals may not be completely accurate. Therefore, when quoting percentages of the survey(s), journalists should refrain from reporting decimal places, but when quoting the rating figures, one decimal place can be used.
[5] The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

Since 1992, HKUPOP has already been conducting Policy Address instant surveys every year. From 1998 onwards, we expanded our instant surveys to cover the Budget Talks. In 2008, we revised our research design by splitting up our instant survey into two rounds, while our follow-up survey operation weeks later remains intact, to become the third survey under our new design. Starting 2011, we revised our design to concentrate on people’s appraisal of the Budget and FS’s popularity in our instant survey, and move the remaining questions to our follow-up surveys. There is no change to our operation this year.

For this year’s Budget, in our first survey, we measured people’s overall appraisal of the Budget, their rating of the Budget, and the Financial Secretary’s popularity. Our results were released on February 27. For our second survey, we focused on people’s reactions towards major government proposals, their satisfaction with the government’s fiscal policies, and other relevant issues. Our results were released on March 4 respectively. Today we are releasing the results of our third and last survey.

POP today releases the findings of the Budget follow-up poll. From 2014, POP enhanced the previous simple weighting method based on age and gender distribution to “rim weighting” based on age, gender and education (highest level attended) distribution. The latest figures released today have been rim-weighted according to provisional figures obtained from the Census and Statistics Department regarding the gender-age distribution of the Hong Kong population in 2013 year-end and the educational attainment (highest level attended) distribution collected in the 2011 Census. Herewith the contact information of various surveys:

As different questions involve different sub-samples, the sample errors will vary accordingly. The table below briefly shows the relationship between sampling errors and sample size for the readers to capture the corresponding changes:

[7] Based on 95% confidence interval.

Results of the Budget follow-up poll, together with the previous two surveys, are tabulated below:

[8] All error figures in the table are calculated at 95% confidence level. “95% confidence level” means that if we were to repeat a certain survey 100 times, using the same questions each time but with different random samples, we would expect 95 times getting a figure within the error margins specified. Sampling errors of ratings are calculated according to the distribution of the scores collected. Media can state "sampling error of various ratings not more than +/-1.3, that of percentages not more than +/-3% and net values not more than +/-5% at 95% confidence level" when quoting the above figures. The error margin of previous survey can be found at the POP Site.
[9] Excluding respondents who had not heard of/were not clear about the Budget.
[10] Collapsed from a 5-point scale. The mean value is calculated by quantifying all individual responses into 1, 2, 3, 4, 5 marks according to their degree of positive level, where 1 is the lowest and 5 the highest, and then calculate the sample mean.
[11] Collapsed from a 4-point scale.
[12] Such changes have gone beyond the sampling errors at the 95% confidence level, meaning that they are statistically significant prima facie. However, whether numerical differences are statistically significant or not is not the same as whether they are practically useful or meaningful.

The third survey conducted in mid-March revealed that, 25% of the respondents were satisfied with the Budget and 44% were dissatisfied, thus net satisfaction stands at negative 20 percentage points. The mean score is 2.6 marks, meaning in between “half-half” and “quite dissatisfied”. The average rating registered for the Budget was 48.8 marks. 29% were satisfied with the government's strategy in monetary arrangement, whereas 40% were dissatisfied, thus net satisfaction stands at negative 11 percentage points. The mean score is 2.8 marks, meaning in between “half-half” and “quite dissatisfied”.

On the other hand, 33% of the respondents thought that the relief measures suggested by the Financial Secretary are enough, 61% thought the opposite. Meanwhile, 52% agreed with Financial Secretary’s suggestion to contain expenditure growth, preserve revenue base, and save for future, in order to prevent the occurrence of structural deficit, whereas 29% of the respondents disagreed. Besides, 53% of the respondents agreed to the saying that there was an internal contradiction of financial management concept indicated by the benefits given away in the Policy Address and warning of deficit in the Budget, 24% disagreed to it.

With respect to Hong Kong’s tax system, 54% considered it fair, whilst 35% thought the opposite. Last of all, 22% perceived the distribution of wealth in Hong Kong reasonable, as contrast to 64% who regarded it unreasonable.

Robert Ting-Yiu Chung, Director of Public Opinion Programme, observed, “Over the years, we find people’s receptiveness of the government’s Budget drops after it has been debated for some time. The situation this year is that people’s initial appraisal of the Budget was rather negative, its satisfaction rating even registered a record low since 2008. People’s satisfaction did not change much after widespread coverage by the media for a day or two. Three weeks later, dissatisfaction tapers while satisfaction rebounds slightly but remains negative, as reflected by a number of indicators. These include the slight rebound of satisfaction rating from 48.5 to 48.8 marks, and the small improvement in net satisfaction rate from negative 22 to negative 20 percentage points. In terms of specific policies, people remain negative on the relief measures suggested by the FS, but continue to support his suggestion to contain expenditure growth, preserve revenue base, and save for future, in order to prevent the occurrence of structural deficit. Our survey also shows that people continue to agree to the saying that there is an internal contradiction of financial management concept indicated by the benefits given away in the Policy Address and warning of deficit in the Budget. Finally, people continue to consider Hong Kong’s tax system as fair, but consider the distribution of wealth unreasonable. People’s satisfaction with the government’s fiscal policies remains negative, with a net satisfaction of negative 11 percentage points.”

• April 1, 2014 (Tuesday) 1pm to 2pm: Popularity of CE and HKSAR Government