Audience Dialogue

Simple quota survey in Indonesia

Case study 28

A quota sample is designed by dividing the population into groups, and interviewing a fixed number in each group. For example, if there are equal numbers of men and women in the population, the quotas for men and women should be equal. But unlike a random sample of the population, where respondents must be first contacted at home, a quota sample can find respondents anywhere.

Audience Dialogue have had good results with quota sampling in a 2001 series of surveys in Indonesia, with a design like this (slightly improved now, after further experience).

Planned total sample size = 200 people.
Principle: interview approximately equal numbers at home, in public places, and 80 at their workplaces.
Also, interview equal numbers of men and women.
We could also have included a quota for age groups, to ensure the correct representation of old and young people, but this would have been too complex for our trainee interviewers.

The area to be surveyed was divided (roughly) into "rich places" and "poor places". The wealth of the areas was determined informally, using local knowledge - not from official data - which was unobtainable anyway.

A quarter of the interviews were in an area where rich people live, a quarter in a poor areas. and the other half in an area of average wealth.(Bear in mind that not everybody found in a rich area is rich - or vice versa.) The purpose of dividing the sample between rich, poor, and average areas was to reach a wide range of the population.

We had 16 interviewers - who in fact were only trainees, and had never done research interviews before. They went out in pairs: one asked the questions, and the other recorded the answers, and kept track of the quotas. The questionnaire had only 12 questions, and the interviews lasted on average about 5 minutes. Each pair of interviewers had a quota of up to 30 interviews to do, with a minimum of 25. Over 200 interviews were finished in one morning.

A good way to check the results of a quota survey is to repeat some questions that were included in the census. In this case, we asked everybody's age group and educational achievement, and compared the answers with census data. Everything was within a few percent, which helped us feel confident that the other results were accurate. Though it's possible that we missed out on some types of people entirely, they'd have had to be the same age groups, sexes, and educational level as the others.

To keep track of the quota, each pair of interviewers had a log to record their work. It looked like this, with 30 boxes to be filled in at the top, and another 30 at the bottom.

Home         [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]
Work         [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]
Public place [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]

Man          [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]
Woman        [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]

Every time they got an interview, they numbered that questionnaire, and wrote the number twice on their log, once in the top section (showing where the interview was done) and once in the bottom (to show the sex of the respondent). When the log was filled, the day's work was finished. In practice,that worked out at 8 interviews per hour, with about two thirds of the time spent interviewing, and one third finding respondents.

The number of each interview was entered on the log twice: once in its location (home, work, public place) and again for sex. Why? Wouldn't it be easier to have a grid, and interview (say) 5 men and 5 women in each type of place? The problem is that sexes are not necessarily evenly spread between locations. Generally, most people interviewed at home are women, and most interviewed at work are men. In Islamic countries, few women are found in public places. So to set an equal quota for sexes in each location could produce a less representative sample.

Rich/poor/average areas didn't appear on the logs, because each pair of interviewers worked in only one area: rich, poor, or average. They were also asked to make no more than 3 interviews in any one street or workplace. Public places included streets, shopping centres, markets, bus stations, etc. People working in public places (e.g. driving buses) were counted in the work quota, not the public place quota.

Mixing home, work, and public-place interviews works well because of the mix of age groups. People at home tend to be older than average, and more women. People at work included more men, and more people in the middle age groups. People in public places tended to be the youngest group, including students and unemployed.

Did you wonder: why write in the questionnaire number? Why not just tick each box? There¹s a good reason: writing in the number ensures that a box isn't ticked twice by mistake. That¹s a very easy mistake to make, when the interviewer is interrupted.

If you wanted to do a more elaborate quota survey, you could include a question "How much time do you spend at home, at work, and in public places - during the hours of this survey?" This could be used to calculate a weight for each individual. It would make the results slightly more accurate, but I suspect it would rarely be worth the trouble.

To learn more about audience research click on the link here to access our comprehensive book Know your Audience.