Audience Dialogue

Know Your Audience: chapter 13
Observation

One method of doing research is to ask people questions. This is most people’s concept of what a survey involves. However, it’s possible to do research without asking questions, but simply observing respondents. This is called observation.

There are two ways of doing observation:

(1) Informal observation, also called unstructured or exploratory observation. This is usually done when the research group has little knowledge of a population and its behaviour. The main purpose of informal observation is to create hypotheses to be tested later, in a survey or using formal observation.

(2) Formal observation, also called structured or systematic observation. This is more like a survey, where every respondent is asked the same set of questions. But in this case, questions are not asked. Instead, particular types of behaviour are looked for, and counted.

Informal observation

When I’m sent to a country I’ve never visited before, and asked to organize a research program, usually one of the first things I do is some informal observation. This means watching and listening to people, usually in public places, and trying to form some ideas about their behaviour, knowledge, and beliefs.

This requires an attitude of openness to experience, and not taking anything for granted. It’s often useful to imagine that you are a visitor from another planet, and that you are visiting Earth for the first time. What do you notice about the humans? If you were writing a letter to friends on your home planet, what would you tell them about the humans? Even the most trivial details can be quite revealing.

An example: when I visited New Guinea for the first time in 1991, I visited an adult education class for village women, near the town of Goroka. After I had been there a while, all the women starting packing away their school books. The teacher had said nothing, and none of them wore watches - so how did they know their class was finishing?

I had hardly noticed a distant siren, in Goroka: it is sounded every day at noon. The women had all heard this faint sound, and knew it meant the end of the morning. The instructor told me that people didn’t use clocks or watches here: they divided the day into three parts: morning (before the siren), afternoon (after the siren), and night (when it was dark). That was enough for them.

Shops and markets are a good place for observation. By looking at the goods for sale, you find out what people can buy. In New Guinea, for example, very few clocks and watches were on sale, but cassettes of music by local bands were available everywhere.

In Vietnam, in 1997, it was almost the opposite. Clocks were available everywhere, in many shapes and sizes, and there were many small bookshops. However, only a few shops were selling cassettes and compact discs, and many of these were of American and European musicians.

These simple facts tell you a lot about the differences between Vietnam and New Guinea. I formed the hypothesis that people in Vietnam are much more interested in what time it is. Perhaps they must be, if they have to start work at some exact time.

With informal observation, you are always looking for something that might make a community different from others. Perhaps the difference is in equipment they can buy, perhaps it is the way they treat young children, or perhaps it is their attitude to authority. Because nothing is certain, you need to be like a visitor from Mars, and notice anything that makes a community unusual.

Of course, there are limits to observation. It is easy to observe what goods are on sale in shops and markets, but not nearly so easy to see how people talk to various types of relative. Informal observation always misses many subtle activities - but it is a good way to begin to understand a community. And notice that the subject of observation is usually a community, or a large group of people - not one person, or a single family.

Though it is possible to observe particular people or families, even if they agree to your presence, the fact that you are observing them can make them behave differently from normal, and you are never sure that they would do the same things if they were not being observed.

If a person who already belongs to a community is the observer, this person is much less noticeable, and the behaviour probably will not change. The problem is, though, that people who belong to a community often don’t notice its peculiarities: they assume the rest of the world is the same - though often it is not. So in many ways, an outsider makes a better observer. Perhaps the ideal observer is an insider, who has been specially trained in observation methods.

How to do informal observation

It’s not enough simply to observe people: the results of observation are not useful unless they are recorded and considered. I have found that the best way to do informal observation is to always carry a small notebook and pen. Whenever I see something interesting, I write it down - usually one or two sentences is enough.

When I am doing informal observation, I am usually visiting a place for the first time, and planning to do some detailed research there. Depending on the topic of the research, certain basic questions must be asked - for example, in what ways can research best be done there?

As I form theories, I write these down too, in the same notebook - usually in the form of questions. After visiting the women’s education class near Goroka, I wrote about their lack of clocks and watches, and the division of the day into 3 parts. Later, in the shops at Goroka, I looked for clocks and watches, and found very few on sale.

Whenever you form a theory about some possibility, look for evidence that will either confirm or oppose the theory. If there had been a lot of clocks and watches for sale in Goroka, this would have opposed my theory that people living in nearby villages were not much interested in the time of day - and therefore would not be able to accurately fill in a radio listening diary.

It’s very important, when forming a theory, not to jump to conclusions straight away, and to look for evidence on both sides of the question. Many people will form a theory, and believe it too soon, without searching for opposing evidence. The main difference between observation in normal life, and informal observation for research is that the researcher is always trying to seek the truth, and to find out how far that truth can be extended.

For example, was it only the women who didn’t know the time of day in Goroka? I observed the village men, and saw that very few wore watches. Some men did wear watches, but these were mainly the richer men, and I noticed that several of their watches were not going: the batteries were flat, but they wore them as a status symbol. I asked them (through an interpreter) if they knew the time, and they always answered "morning" or "afternoon". I wasn’t in the village at night.

With informal observation, you are constantly forming theories, and testing them out.

As well as jotting interesting things in the small notebook, I also keep a larger notebook. Every night, I copy items from the small notebook into the large one. In the large notebook, every page is a separate topic. In my large 1991 New Guinea notebook, page headings include Knowledge of Time, House Types, Transport Methods, Shoes on Powerlines, and so on.

Shoes on power lines? The children tie pairs of old shoes together by the laces, and throw them up in the air, trying to make them hang over power cables, one shoe on each side of the cable. I never figured out what this meant, but shoes were expensive in New Guinea, and if they simply wanted to test their throwing skills, they could have tied two sticks together. I don’t know if this was relevant to broadcasting, but it was a common enough sight that I took the trouble of noting it down.

In this way, with each page in the large notebook being on a particular topic, and entries from the small notebook copied there, you can collect the evidence for and against a particular question. In some cases, as with shoes on power lines, I never formed a theory. The only question was "Why do they do it?" and there I recorded the opinions of various people who I asked about this practice.

Notice that for observation to be successful, you must often go beyond observation, and ask people "Why do you do that?" You are not observing purely for the sake of observing, but to find out why people act as they do.

Formal observation

This is sometimes called structured or systematic observation, and is usually done as part of a survey, and all the normal survey processes are followed - except one. No questions are asked. Instead of asking questions, the interviewer (or observer) watches people, and records their behaviour.

In fact, many normal surveys include some observations. There’s usually no need to ask "Which sex are you?" Instead, the interviewer notes whether the respondent is a man or woman. That’s observation.

One difference between a normal survey and formal observation is that observers often watch a number of people at once. For example, in a study of Aborigines in Australia, living in the central desert areas, we found that they often sat in groups, under trees and in cool places, while listening to radio or cassettes. The topics of interest included the size of these groups, what types of program or music caused them to stop talking and listen, and what types of people were in each group. We found that there were often four types of group: older men, older women, younger men, and younger women. The four types of people had quite different tastes in music and radio programs. Instead of one questionnaire per person, the observers filled in one questionnaire for each group.

Instead of having possible answers such as Yes and No, there were spaces where the observers could write the number of people taking part in each activity, or giving each answer.

Sampling with formal observation

When an activity is being observed, you find two types of problem. Either the activity is so unusual that it is hardly ever recorded on the questionnaire, or else it is so common that it’s impossible for the observers to count every occasion. In the latter case, time-sampling is often used. Instead of choosing a person at random (as with a normal survey), an observer will record a behaviour at random times.

This might involve pausing at a particular time in each hour - e.g. a random number of minutes after each hour - and recording how many people in a group were clearly paying full attention to a radio program, how many seemed to be ignoring it, and how many the observer was not sure about. This is called the Snapshot method, because it is similar to taking a photo of each person in a group, and recording their behaviour. In fact, when you are using this snapshot method, it’s often a good idea to take a real photo. But don’t rely on this photo being successful: always take notes as well. A photo cannot tell you everything you need to know.

A different sampling approach from the snapshot method (which involves observing people at a particular moment) is the slice-of-time method, as it is sometimes called. With this, the observer looks for one particular type of behaviour, and counts the number of times it happens in a randomly chosen period - perhaps 5 minutes in each hour, from 37 minutes to 42 minutes past the hour. The observer sets a small alarm to go off at a particular time, and counts one type of activity for a number of minutes. For example, if you are studying people’s reading habits on public transport, the observer could be in a train. He or she would walk through a carriage from 37 to 42 minutes past each hour, counting the number of people who were reading, and also the number who were not reading.

Automatic observation

Often, observation involves counting the occurrence of a particular event. For the data to be usable, it’s often necessary to also count non-occurrences, so two counts need to be made at the same time. Without knowing the total number of people in a group, the number who were doing one type of activity is meaningless.

For example, you might want to count the proportion of people using handphones in a market. To be able to calculate the proportion, you need to count (1) the people using handphones, and (2) the people not using handphones.

It takes a lot of practice to accurately make two counts at the same time - and if the observer is distracted, he or she can easily forget one or both counts. So it’s necessary to record the counts. This can be done several ways: using paper, mechanical counters, calculators, a camera, or on tape.

Paper and clipboard:

The observer needs to carry a clipboard, with a sheet of paper ruled into two areas, divided by a line. Every time a person is seen doing the activity (whatever is being recorded) the observer puts a tick on one side of the line. Every person seen who is not doing the activity, the observer ticks on the other side of the line.

This produces quite accurate results, but in many places an observer with a clipboard is very obvious. The people being observed become worried about this, and may change their behaviour, or go away as fast as possible.

Mechanical counters:

It is less conspicuous if the observer carries two small tally counters, one in each hand. These counters, designed for scoring sports matches and usually sold at sports equipment shops, have a number display - a little like the distance recorder on a speedometer. They also have a large button. Every time the button is pressed, the number increases by one. The counters are usually small enough to be concealed in a person’s hand. The observer presses a button with his or her left hand whenever a person doing the activity is seen. For everybody not doing the activity, the observer presses the button in his or her left hand. The advantage of these counters is that, at the end of each session, there’s no need to count the results: the final count is visible on each counter.

The major practical problem with these counters (at least, for one make I used some years ago) was pressing the Reset button by mistake - this resets the count to zero.

Calculators:

An alternative to a mechanical counter is a small calculator - now cheaper than the counters. A calculator used for this purpose should have an automatic repeat for addition. To test this, press 1 + 1 = and 2 will be displayed. Now press = several more times. If the calculator displays 3, 4, 5, etc (adding 1 every time the = key is pressed) it will be suitable. Some calculators turn themselves off after about 10 minutes of non-use. If this is a problem, you need a cheaper calculator!

However, calculators have lots of buttons, and it’s easier to press the wrong button with a calculator - specially if you’re not looking at it. This will often spoil the results. If you do use calculators, I suggest you buy a model with a hard plastic lid, and cut a small hole in the lid, so that only the = key can be pressed when the lid is closed.

Audio tape:

Another observation method is to use a small cassette recorder with a built in microphone. The observer sets the machine to record - but instead of speaking into the microphone (which would be conspicuous), he or she taps it once for a Yes and twice for a No. Later, the tape is listened to, and the number of single and double taps counted.

Videotape:

The above discussion is about live observation: having an observer count an event while it happens. A more accurate approach, which is sometimes possible, is to use a video camera. It is mounted in a safe place (high up) and set to record for a fixed time. The rarer the activity to be counted, the longer the recording should go for - but many small video cameras can record for less than an hour.

The observation is done when the recording is played back. In this case, there are no problems with recording on paper, because the playback is done in an office. If the observer loses the count, the videotape can be rewound a little. For even greater accuracy, several observers can watch the same tape, and each make their own count. Because it’s possible to replay the tape, a number of different counts can be made: too difficult for a single observer to do accurately. So the video method usually produces much better results than live observation.

Still camera:

For the snapshot method of observation (counting the number of people doing a particular activity at one moment) an ordinary camera can sometimes be used. In most situations, a single photo does not show enough detail, because the people being observed are usually facing different directions.

But sometimes, a still camera can be very effective. For example, some years ago I was asked to find out the reactions of young children to a new TV program. These children were only 3 or 4 years old, too young to interview. So we arranged to visit a kindergarten, and showed the new TV program there - along with another program, which we already knew to be popular. The children were in a large room, with the TV set, and plenty of toys. We had a camera on a tripod behind the TV set, and a mirror at the back of the room. There was a remote-control shutter release, on a cord, so that the photographer would not be too intrusive. We also used high-speed film, so there would be no distracting flashes. Every 30 seconds, a photo was taken, of the children watching TV.

When the films were developed - one for each program - in each photo we simply counted the number of faces looking directly at the camera (and the TV). In the background, the mirror was meant to show what was happening in the program - but in fact, the children’s bodies often blocked the mirror, and the mirrored screen was often too small to make out. Even so, it was clear that the new program didn’t attract them as much as the older program. The children in the photos were more often playing with each other or with toys than watching the new program - while with the old program, most faces were glued to the TV set (and the camera) nearly all the time.

This method could be adapted for many theatrical performances - though these days, I’d use a video camera to tape the audience the whole time, and stop the video every minute or so to count the number of attentive faces at that moment. Of course, there’s no need to focus on the entire audience: as always, a sample can be taken. However, the people in the front seats may not be typical of the whole audience. The video camera’s microphone could be used to relate the audience image to the point in the play.

Self-observation

By self-observation I mean getting respondents to observe themselves. This mainly involves the use of diaries, which respondents are meant to fill in whenever they do a certain activity, such as watching TV, reading a newspaper, or buying something. This is not a very accurate method of observation: most respondents aren’t motivated to fill in the diary every single time they do the activity of interest. Often they mean to fill in the diary, but forget. Sometimes they deliberately lie.

For example, a friend of mine in New Zealand once received a radio listening diary to fill in for a day. He told me that his radio had broken down, but he put in the diary that he listened to his favourite station all day and all night. He was worried that it had a small audience, and might close down if a lot of people didn’t support it. So his ticks in the diary meant "I strongly support this station" - not "today I listened to this station all day and all night".

In fact, if a station has a very small audience, and only a few people say they listen most of the day, this can make a large increase in its audience. For example, if a survey has 500 respondents, and the average person listens to radio for 2 hours a day, and one station has 3% of the total audience, that station will have 30 hours of listening in the whole survey. So if two more people claim to listen to that station, each for 15 hours a day, the station’s audience will double.

Even though self-observation is not as reliable as using observers, it is still generally more accurate than asking people what they usually do, or what they did the previous day, or otherwise relying on their memories, or their beliefs about their usual behaviour.

Domestic observers

Though it’s easily possible to observe people in a public place, it is much more difficult to observe them at home. Even if they give permission to a stranger to visit them and observe them, the presence of the observer is likely to change their normal behaviour - for a while. It is only when an observer is no longer a stranger that domestic behaviour is likely to go back to normal.

In the 1970s the Surgeon-General of the U.S. government commissioned some studies on smoking, with practically unlimited funds. In one of these studies, random members of the public were asked to co-operate with having video cameras put in their living rooms, to record what they did at home. Outside each chosen home, a large van was parked in the street. The video lines led to this van, which was manned around the clock. The observers found that after a week or so, the respondents almost forgot about the camera, and began to do some very private activities, not bothering that these were being recorded. These studies, which as far as I know have never been repeated, produced a wealth of information about the detailed ways in which people watched TV.

On a much more modest scale, I did a similar study in Australia in the 1980s. This was a project for university students in sociology. They were asked to seek permission to go to a friend’s house, and watch TV with their friend for some nights, while they recorded the other things their hosts did while watching TV. They were quite open about the purpose of the research, but because they were visiting people they already knew well, the TV-watching behaviour was probably not affected by this observation. Also, the observation records were anonymous: they showed the name of the student, but not the name or address of the hosts.

No formal questionnaire was used. Instead, each student kept a notebook, showing the time, which people were watching the TV set, how much attention the viewers were paying, and what else they were doing. The students were asked to quietly watch TV and record their hosts’ behaviour, and not to talk with the hosts while they were viewing.

Although this was done on a small scale, with a sample of only 40 households (biased towards richer households) it provided some useful information. One of the most startling findings was that the average viewer was paying full attention to the TV less than half the time.

Combining observation and response

When you are studying people’s behaviour, observation can produce more accurate results than asking respondents what they do. However, observing behaviour usually doesn’t tell you the reasons for people’s actions. So to understand your audience well, the best data comes from combining observation and other research methods.