Sunday, January 4, 2015

Allen Myers: The Asia Foundation, surveys and media blindness

By Allen Myers


In early December, the Asia Foundation released the results of a survey of Cambodian public opinion indicating that “Most see Kingdom on wrong path”, as the Phnom Penh Post headline put it.

Predictably, media coverage of the survey was based on the Asia Foundation’s accompanying press release and/or on the words of AF’s country representative. Few if any journalists appear to have taken a serious look inside the survey report, and so they didn’t notice any of the discrepancies between what it actually says and what the AF says about it, or consider how the survey’s questions might have been framed to produce desired answers. And none of the coverage that I have seen informed readers/viewers/listeners about the AF itself.

What is the Asia Foundation?
The Asia Foundation is not a benevolent and impartial NGO, but has always been an undercover tool of the US government. It was set up in 1951 as the Committee for Free Asia, with secret government funding through the CIA. In 1954 it adopted its current name and began pretending to be a public charity. To the extent that people were fooled by this, it saved money for the CIA as well as giving cover to its real purpose.

In the 1960s, as US citizens and media began asking questions about what their government was up to, the CIA became worried about the Asia Foundation being exposed. In June 1966, it sent a memorandum about this to the 303 Committee, a secret government body overseeing and coordinating undercover operations, including those of the CIA. The subject of the memo was “The Asia Foundation: Proposed Improvements in Funding Procedures”. It begins:

“The Asia Foundation (TAF), a Central Intelligence Agency proprietary, was established in 1954 to undertake cultural and educational activities on behalf of the United States Government in ways not open to official U.S. agencies. Over the past twelve years TAF has accomplished its assigned mission with increasing effectiveness and has, in the process, become a widely-known institution, in Asia and the United States. TAF is now experiencing inquiries regarding its sources of funds and connections with the U.S. Government … It is conceivable that such inquiries will lead to a published revelation of TAF’s CIA connection.”

Therefore, the CIA recommended, the government should find less smelly ways of funding the Asia Foundation: “In the long run, we feel TAF’s vulnerability to press attack can be reduced and its viability as an instrument of U.S. foreign policy in Asia can be assured by relieving it of its total dependence upon covert funding support from this Agency. In the belief that TAF contributes substantially to U.S. national interests in Asia, and can continue to contribute if its viability is sustained, CIA requests the Committee’s study and attention to possible alternative means of supporting it.” [Emphasis added.]

In short, because of fears that the AF’s role as an instrument of the US government would be exposed, the CIA recommended hiding its funding by using other channels. There was no suggestion that the AF might change its role or that the government might stop funding it, although it continues to solicit funds from friends and/or gullible organisations. According to Wikipedia, subsequent funds have come from “the U.S. Agency for International Development, the World Bank, the Asian Development Bank, the United Nations Development Program, official development assistance agencies of Australia, Canada, Netherlands and the United Kingdom, an annual appropriation from the U.S. Congress, and contributions from private corporations and foundations.”

Of course, I don’t begrudge the US government the right to spend its money to pay pretendedly independent organisations that produce pretendedly objective reports that coincide with what the US government would like people to believe. I assume that at least a few other governments do the same thing, although I can’t think of any that have as much money available for that purpose as the US does.

However, I do think there ought somewhere to have been an English-language journalist reporting to Cambodia who, in summarising the Asia Foundation’s press release, would have alluded to what the Asia Foundation was and is. No, not on Voice of America, whose funding is very much like that of the Asia Foundation. But neither the Cambodia Daily’s nor the Phnom Penh Post’s stories on the AF survey even hinted at the information above, all of which has long been freely available on the internet. It’s a bit like reporting a North Korean survey that found great hostility to critics of Kim Jong-un, without mentioning that the survey organiser had connections to the North Korean government.

Careless reporting
The report of the survey is full of assurances that this is really scientific, objective stuff. However, even a moderately attentive reading reveals frequent evidence of carelessness, not to use a stronger term, including numerous typographical errors. On page 40, the report, explaining increased support for women in commune councils, states, “Potential causes of this shift include increased exposure to international norms through the media, the influence of civic education, and direct experience with female commune councilors.” Two pages later, it repeats the same explanation, almost word for word and with one slight addition, in regard to whether women take men’s advice on voting – a situation that had not changed significantly and which had no connection with women councillors. Did the AF’s report writer push the wrong macro key?

Whether deliberately or not, at times the carelessness and sloppy formulations make it almost impossible to figure out what the report is saying. For example, the Introduction (page 7) states: “The study builds on the findings of an earlier phase of the research, involving in-depth interviews with voters and extensive pre-testing of the questionnaire.” An earlier phase of research? In-depth interviews? Extensive pre-testing? When? With how many people? What were the “findings” that the current research builds on?

The report provides no clear answers – or even unclear answers in most cases – to such obvious questions. However, in a later section, a bit more detail is provided on those in-depth interviews. They were about translation rather than the views of voters: “Specific sections of the questionnaire were tested … through in-depth interviews to get a better understanding on how to translate certain concepts and how applicable some of the questions were to the local context.”

And the next sentence gave some specifics about the “extensive pre-testing”: “After this phase, the survey was tested with 13 randomly selected respondents, in Phnom Penh.” Talking to 13 people in Phnom Penh is “extensive” testing! If they had added two people in Battambang or Siem Reap, I suppose they would have called the pre-testing “exhaustive” or maybe “universal”.

In fact, the report’s authors seem to find numbers quite challenging. On page 16, they write: “With 13 interviewers and team leaders, no one interviewer conducted more than 4% of the total number of target completed interviews (4 questionnaires per day).” If there were only 13 people doing interviews, and none of them interviewed more than 4% of the total, that means that a maximum of 13 x 4% = 52% of the targeted interviews were conducted. Is there any sense in this?

Loaded questions
The sloppiness of the report extends to the questionnaire, which is appended to the report. For instance, if respondents reply that they have not previously heard of the NEC, the interviewer is instructed to go to a question that asks whether they think NEC decisions are influenced by political parties; if respondents say that they have never attended a political demonstration, the interviewer is told to ask them how they heard about the demonstration.

But the major problem is that the wording and presentation of many questions appear designed to produce a desired response.

The Asia Foundation commissioned a market research organisation called Indochina Research to conduct the survey. On its web page, Indochina Research says: “We understand that information is useless unless it leads to results. Our approach is committed to outcomes. We help businesses focus on turning market insights into practical actions.”
So the survey was intended, not to gather information for its own sake, but to lead to “outcomes”, “practical actions”. Which outcomes? That is indicated in the title of the questionnaire, which is “Cambodia Election Reform Study”. The planned outcome is to “reform” Cambodia’s elections, and the survey is designed to elicit answers which appear to justify such “reform”.

The AF report is not above misrepresenting the survey results in pursuit of this goal. For example, the report’s Executive Summary says that “the majority of respondents support a constitutional amendment to provide for a ‛balanced’ National Election Committee”. But while respondents were asked their attitude to a “balanced” NEC, they were not asked about a constitutional amendment, on the NEC or any other subject. The main body of the report compounds the distortion: “Survey findings support the July 2014 agreement made between the CPP and CNRP to make a constitutional amendment to provide for a ‛balanced’ National Election Committee”. Not only were respondents not asked about a constitutional amendment; they also could not have expressed an opinion on the 22 July 2014 agreement, because the survey was conducted from 19 May to 9 June.

Still, it is instructive to look at the question on a “balanced” NEC. Question 75 was phrased: “Currently election commissioners should be neutral individuals with no party affiliation, but some people say it’s hard to find neutral people and that it would be better to have a balanced commission with representatives from each party. Which is closer to your view?” The text of the report describes this question and the response: “When asked if they preferred the current system, with ‛neutral’ commissioners, or a ‛balanced’ commission, with equal representation from all parties, a huge majority (82%) preferred a ‛balanced’ commission.”

But it is not true that survey respondents supported equal representation from all parties. They were not asked about equal representation. The survey did not define what was meant by “balanced” except that it included “representatives from each party”. Some people would consider it balanced to have equal representation of all parties regardless of their size, and some people would consider that unbalanced and that big parties should have more representatives than small ones; but both would say they are in favour of a balanced NEC. Similarly, some might think a balanced NEC needs representatives only from parties with seats in the National Assembly, while others would think it should have representatives from all parties competing in an election. Again, people holding these contrary views could both call their favoured NEC “balanced”.

“Balanced” is very nearly as approving and vague a word as “good”: who would say that they are not in favour of a balanced NEC? And just to make doubly sure of getting the desired response, the current situation is presented as an alternative to balance – therefore unbalanced, bad. It “should be neutral”, which implies that it isn’t. The respondent is furthermore reminded that “some people” think that a neutral NEC is basically impossible; he or she is not told that anyone considers “balanced NEC” as too vague a term to serve as a practical goal.

Imagine how much different the response to the question might have been if it had been phrased, as it could have been: “Originally the NEC included representatives from political parties, but foreign experts advised that this made the NEC too political, so in 2002 it was changed to be made up of neutral people who do not belong to any party, a move welcomed by the UNDP as ‛global good practice’. Some people want to continue this arrangement, and others want to return to having the NEC composed of party representatives. Which is closer to your view?”

There are similar distortions in the survey’s question on electoral reform. In addition, two different versions of the question are presented.

According to the report, question 79 was phrased: “Some people say elections should be held as soon as possible, while others think it will be important to enact necessary reforms before elections are held, or to wait until the current government’s term expires. Which is closer to your view?”

However, in the appended questionnaire, the quoted text is preceded by: “The current government won the last election, but some parties dispute the result and demand election reform.”

It is impossible to know for sure, but it seems likely that the questionnaire used the full question, and the presentation in the report abbreviated it. The question then is: why? Why tell the reader only part of what respondents were asked?

A possible answer is this: The full question 79 contains evidence of manipulation. It puts election reform into the context of the CNRP’s claims that it really won the election and that therefore various reforms were required. Reminding respondents who are CNRP supporters of this connection would incline them to support this demand of their party. And perhaps supporters of some of the smaller parties would be taken in by the question’s assertion that not just the CNRP but “some parties” – maybe including theirs – were making this demand. Further, it might incline some to support this alternative simply as a way of ending nearly a year of often disruptive demonstrations and disputes. On the other hand, it would not automatically incline supporters of the CPP or other parties, or people with no particular allegiance, to oppose undefined election “reform”, because “reform” is, like “balanced”, one of those vague words that nearly everyone can say they are in favour of – until the specifics of a “reform” are spelled out.

Look again at the choices offered respondents: new elections quickly; reforms and then elections at some unspecified time; and elections at the end of the current term. The choices do not allow a respondent to express support for both continuing with the existing election schedule and reforming election procedures. And suppose that the National Assembly does not legislate reforms – something that is still possible, though not likely. In that case, option 2 – that elections would not be held until after reforms were made – would mean that the present government continued indefinitely, past 2018. Does anyone think that a majority would favour that scenario if there were no reforms? Neither do I, but that is what the AF’s distorted survey makes 56% of Cambodians say.

I am not a professional, or even amateur, survey creator, but I can explain how this muddle has arisen. If each of two questions can be answered in two ways, then there are four possible combinations of how the two questions can be answered. The questionnaire authors combined two different questions – election timing and election reforms – into a single one. This did not change the number of possible answers, but the survey authors rewrote the four logical answers into only three choices for the respondents, thus guaranteeing distortion.

Was this a deliberate attempt to produce the appearance of a big majority for undefined election reform? Maybe it never occurred to the AF and its hired researchers that public attitude on these topics could easily have been determined by separating the questions: 1. “Do you favour new elections in the near future, or elections at the end of the current government term?” 2. “Do you think electoral reforms are necessary before the next election?”

The 97th question – “Generally speaking, do you think things in Cambodia today are going in the right direction, or do you think they are going in the wrong direction?” – is the one that got the biggest media coverage. According to the report, 59% of respondents said “wrong direction” and 32% said “right direction”.

Respondents were not offered a third option: “… or do you think things are not changing much?” However, when asked about their personal economic situation, respondents were able to answer that things had stayed the same. It should be obvious that countries, like individuals, might be considered to be going neither forward nor backward, but AF’s survey didn’t allow respondents to say that. It seems likely that people who think things aren’t changing much, but would of course like them to improve, would choose the negative of the two options they were offered.

The survey found that Cambodians’ evaluation of their own situation was much more positive, on average, than their view of the national situation. Of course, it is possible for individuals to think “I’m okay [or doing better], but things aren’t so good for the country overall”. But it would be absurd to maintain that people’s individual situation has no effect on how they see the country’s situation. And it is a coincidence, but nevertheless an interesting one, that the total of respondents who said their economic situation was less favourable (22%) or unchanged (37%) is precisely the same as the proportion who said Cambodia is heading in the “wrong direction”.

The question’s placing in the survey also seems significant. It comes after 96 other questions. Most of those that are not seeking simple factual information (“Did you vote in the election?” “Do you own a mobile phone?”) imply that something is not as it should be, or at least that significant numbers of people think this is the case.

Also, questions on personal situation or the state of the commune, which could be expected to contribute to how respondents viewed the country’s situation, were asked after the question on the country’s direction. The view that the respondent’s commune was headed in the right direction was shared by 11% more respondents than thought the same of the country. Maybe, if they had been asked to think first about their local situation, some respondents would have had a more favourable view of the country’s direction.

Conclusion
In addition to those described above, there are numerous other flaws in the Asia Foundation survey and the report of it. These flaws are of varied significance: some reveal political bias, others appear to be the result of carelessness and/or incompetence. Overall, the document is clearly unreliable. It may contain or point to some truths about Cambodian public opinion, but there is no way to distinguish those truths from the repeated instances of unreliable or distorted information.

By Allen Myers
http://letters2pppapers.wordpress.com/