The Independent Uganda: You get the Truth we Pay the Price
What polling strategy means for Afrobarometer survey findings
“Most Ugandans favour proposed reforms to improve elections and Parliament, new Afrobarometer survey shows.”
That was the title of a statement to journalists from Afrobarometer’s agent in Uganda, Hatchile Consult Ltd, on April 28 when the results of the survey were released. It was also the message Afrobarometer hoped would make the headline the next morning.
As has become the norm in Uganda, and world over, Afrobarometer’s report titled: “Outcry versus disdain? Understanding public support for proposals to improve Parliament and elections in Uganda” has sparked debate on the relevance, reliability and objectivity of opinion polls.
The survey had 12 questions; two on presidential terms and age limits, five on quality of elections, two on post-election issues, and three on cars for MPs, their number, and whether they should be appointed ministers.
The latest survey was released after another Afrobarometer report released on April 24 titled “Social services rank tops in Ugandans’ priorities, but not in national budget”. In this report respondents were asked: “In your opinion, what are the most important problems facing this country that government should address?”
The respondents, who could give up to three responses, put social services as their top priority. This comprises health (53% of respondents) and education (36%). This was followed by infrastructure/roads/transport (34%), water supply (28%), poverty/destitution (21%), corruption (19%), unemployment (19%), agriculture/food shortage (19%), electricity (10%), and finally crime and security (8%).
The difference between what Ugandans list as their priorities and the issues that Afrobarometer chose to focus on have become major talking points since the latest report was released.
The questions
Questions are being asked during debates on radio about how the pollsters decide which issues to focus on and how they frame the questions on issues that they want Ugandans to form opinions on. Questions are also asked about how the way Afrobarometer frames the questions could influence the outcome of the surveys.
At the launch of the latest survey, the public ambivalence towards the polls became an issue. Guests; including outspoken former Ethics and Integrity Minister Miria Matembe, said it would have been better if officials who had been invited from the Ministry of Justice and Constitutional Affairs and the opposition political parties had been around to debate the issues.
Without this, she said, there is a tendency for people to “keep talking to themselves”. Generally, however, it could be that the issues polling agencies focus on are not the issues most people are interested in.
But Dr. Frederick Golooba Mutebi, Afrobarometer’s national investigator for Uganda, told The Independent on May 4 that research teams in each of the 36 countries where Afrobarometer does similar survey rounds identify the issues in their country and these are normally issues that are of public interest or are very much in the public domain.
“We have no interest in projecting our views, and we only project the views of the public,” Golooba said.
Dr. Patrick Wakida, the executive director of Research World International, a Kampala-based polling agency, also told The Independent that the themes for polling are normally developed from the public domain.
“At the time of the polling, you should be polling issues for which people are engaged and the whole objective is to redirect the discussion so that it becomes an informed discussion.”
Wakida says “at the moment, discussion in Uganda is about (presidential) succession, age limit for the president, term limits and constitutional amendments”.
These views mirror the opinion of one of the most respected and experienced polling agencies in the world, the Washington-based Pew Research Centre, which says the most important part of the survey process, is the creation of questions that accurately measure the opinions, experiences, and behaviours of the public.
The first step in this is selecting what topics would be covered in the survey. For Pew Research Centre surveys, this involves thinking about what is happening in the country and what would be relevant to the public, policy makers, and the media.
But Paul Nyende, a lecturer of psychology at Makerere University says he would be more concerned about the polling agency’s sampling strategy when looking for potential influences on the outcome of polls.
Nyende is referring to how a pollster determines the subset sample of the population to participate in the survey to ensure it represents the population from which the sample is picked.
In this case, Afrobarometer chooses the random sampling strategy which covers the whole population. Between December 26, 2016 and Jan.08, it interviewed 1,200 adult Ugandans to represent the demographic composition of Uganda basing on the 2014 National Population and Housing Census. It picked 50% women and a similar number of men.
The majority, 75% of respondents were villagers and only 25% were from towns. They were also not educated; 13% had zero formal education and 45% had only been to primary school. It means 58% of those interviewed could possibly not write or read yet they were required to respond to fairly complex questions. One such question was: “The quality of Uganda’s elections have been described by many stakeholders including the opposition, ruling party, civil society, media, and voters, as poor due to allegations of election irregularities and mismanagement. I am going to read from a list of proposals aimed at improving the quality of Uganda’s elections. For each of the following proposals, please tell me whether you disagree or agree.”
This question had five proposals including; improve transparency during tallying, transmission, and declaration of election results, tighten law on campaign financing and accountability for all political parties, and increase the time within which to complete the hearing of and ruling on presidential election petitions from 30 to 60 days as proposed by court?
First, I should convey the Afrobarometer’s appreciation to The Independent magazine for using our data in writing this article. This is very much encouraged, and the network has developed an online-data-analysis facility (available at http://www.afrobarometer.org/online-data-analysis/analyze-online) to enable anyone run analysis from our data without the need for any statistical analysis packages.
This article raises two questions related to opinion polling: First – the choice and framing of questions asked, especially that survey questions asked did not mirror public interest and second – that the survey targeted people who are not qualified to answer these kind of questions. To briefly answer to these questions, (1) the framing of Afrobarometer survey questions meets international guidelines, this survey is conducted in 36 African countries and is wholly peer-reviewed. The Afrobarometer has been a recipient of the “Best Data Set Award 2004” from the American political Science Association, which is no easy feat. The 12 questions reported under this particular release are part of 281 questions directed asked to the respondent or 377 questions carried by the survey.
There are different types of questions, each with an array of possible ways to be framed depending on the survey context or target respondents. For some questions we have asked respondents to mention (off head) which issues the government should address first and foremost (i.e. spontaneous off-head responses), while in other instances we have asked respondents to make a choice of opinion given a public interest issue. The 12 law-reform proposals fall in this latter category. Responses from off-head spontaneous set or itemized choice items are essentially telling of the same opinion, and one is preferred over the other given a range of research factors. The key factor in the framing of questions is to avoid bias, a concept that is central to research design throughout the methodology of the research in question.
With respect to this article, the April 24 release (Afrobarometer Dispatch paper number 141 and available at http://afrobarometer.org/sites/default/files/publications/Dispatches/ab_r6_dispatchno141_uganda_social_services.pdf), refers to analysis of the Public Development Agenda component of our Round6 survey conducted in May 2015, while the latest release refers to polling of public support for some of the many reform issues that are relevant to the quality of our democracy and governance. With respect to this article, it should therefore not be expected that the batch of 12 questions on public support for law reforms should have been part of the spontaneous mentions of public development agenda, as the two sets are from two separate constructs, polled almost 2 years apart.
How then are poll questions selected? The general rule-of-thumb is that poll questions need to be relevant, scientifically testable interrogations of plausible hypotheses. And this could be to the discretion of researchers or study population or (frankly) both. The most critical element however is that questions – or the research setup in general – should avoid bias. All stages in the research process – including reporting – should avoid bias including statistical bias.
Lastly, the assertion in this article that ordinary people cannot form valid opinions on issues of democracy or governance is perhaps over-stated. Afrobarometer Working Paper #124 (available at http://afrobarometer.org/publications/wp124-understanding-citizens-attitudes-democracy-uganda, see page 17 – 18) shows that expert views on these kind of issues follows a similar trajectory as views held by ordinary citizens. Research suggests that both public opinion and expert views appear to agree on a direction (such as democratic governance or cost of living), such noting that such a condition is increasing or decreasing – but may not mention this observation at the same level. And that is critical in evaluating the importance of public opinion.
In sum, it would be wrong to ask ordinary people how to formulate these 12 proposals into new laws – that could be for technocrats – but polling public support for these proposals is technically possible.
And lastly, allow me note that the Afrobarometer is a pan-African network of research organizations now working in 36 African countries. This network conducts and disseminates its own research, and does not accept funding from governments or political parties/groups and is wholly donor funded. The Afrobarometer core objective is to 1) conduct high-quality scientific research of public opinion on the African continent, 2) provide a voice to public opinion in the policy-making arena and 3) provide capacity for research in Africa. Since the year 1999, the Afrobarometer has conducted over 200,000 interviews with ordinary African in over 145 surveys across 36 African countries, and still counting. The network is wholly peer-reviewed, has an international advisory board and takes technical support from the University of Cape Town and Michigan State University. More info at http://www.afrobarometer.org
Attractive part of content. I simply stumbled upon your website and in accession capital to assert that I acquire in fact enjoyed account your blog posts.
Anyway I’ll be subscribing in your augment and even I
success you get admission to consistently rapidly.
Fantastic website. Lots of useful information here.
I’m sending it to several friends ans additionally sharing in delicious.
And certainly, thank you on your sweat!
I hardly leave remarks, however i did some searching and wound up here AFROBAROMETER:
Right poll, wrong people. And I actually do have some questions for
you if it’s allright. Is it only me or does it appear like
some of the remarks appear like coming from brain dead visitors?
😛 And, if you are posting at other social sites, I would like to
follow everything new you have to post. Could you make a
list of every one of all your social sites like your twitter feed, Facebook page or linkedin profile?
I just like the helpful information you supply in your articles.
I’ll bookmark your blog and take a look at again here frequently.
I am somewhat sure I’ll be informed a lot of new stuff proper here!
Best of luck for the next!
I real delighted to find this site on bing, just what I
was searching for 😀 also saved to my bookmarks.
that is really a good tip especially to those fresh to the blogging simple but very accurate info many thanks for sharing this one a must read post