Audience Dialogue

Know Your Audience: chapter 10
In-depth interviewing

In-depth interviewing is usually done at the beginning of a major research project, when you will be studying a population that you have never researched before. In-depth interviews - also called "semi-structured interviews", or "informal interviews" - are very different from survey interviews. They are much more similar to journalistic interviews. Some of the differences between survey interviewing and in-depth interviewing are:

The reason for these differences between survey interviewing and in-depth interviewing is that their purposes are different. Unlike survey interviewing, in-depth interviewing does not claim to obtain results that can be generalized to a whole population. You normally use in-depth interviewing for collecting background information, so that when you write a questionnaire, you will be able to use questions and wording that are more relevant to the population being studied.

Choosing interviewers

With a survey interview, as long as the questionnaire is well designed and instructions are clear, interviewers can be relatively unskilled.

With in-depth interviews, there are only three rules

1. You need to decide in advance which main topics you want the interview to cover.

2. You need to decide whether everybody will be asked the same questions, or you will change the questions, depending on the respondent.

3. The interview needs to be recorded in some way.

Because there are no fixed rules, there are no standard procedures, so the quality of the interview depends very much on the skill of the interviewer. It's therefore normal to use highly skilled interviewers, who have been working closely with the project leaders, so that the interviewers know the main issues of interest in the study. Often, it is the chief researchers themselves who do the informal interviews, because they have a better knowledge than anybody else of the project's purposes.

If the chief researchers aren't experienced or confident interviewers, trained interviewers can be used, but they should be chosen well in advance, and participate in the development of the research.

Use pairs of interviewers

I've found it's best for interviewers to go out in pairs. At the beginning of each interview, one speaks to the respondent, while the other takes notes and works the tape recorder. Part of the way through the interview, the two interviewers swap their roles. The advantage of this is that different interviewers think of different questions, and often a respondent will say something to one interviewer but not another.

As soon as the interview is finished, the two interviewers can discuss the findings. Before they do anything else, they should write up their notes on the interview. Even if the interview has been taped, some nuances will be forgotten as soon as the next interview is done. Also, having notes on each interview makes it easier to interpret the results, and serves as a backup if the tape recording fails.

After each interview, the two interviewers can also discuss their techniques of interviewing, the wording they use, and decide on ways to improve. In each of the first few interviews, some issues will arise that you had not thought of before, and these will create questions to be asked in later interviews. In fact, it is only when you stop finding new questions that you can be sure the sample was large enough.

Finding respondents

With a survey, you are trying to obtain a true cross-section of the population, and the best way to achieve this is through random sampling. In-depth interviewing is different. It is usually a preliminary exercise, designed to find the most appropriate questions to ask in a later survey. So it's the survey that will provide the representative results: the in-depth interviews by themselves do not produce definitive data.

In-depth interviews need to ensure that many different types of respondent are interviewed. This is best done, not with random sampling, but with maximum-diversity sampling (see section 3 of the chapter on sampling). When the sample size is less than about 30, a random sample will have a high chance of not being fully representative of the population.

If you want to cover the whole population, make sure that your sample includes at least two people in each of these age/sex categories:

Occasionally, other variables are more important than age or sex. By "important" I mean that people in different categories are likely to give quite different answers. Some of these categories are:

If you make sure your sample includes at least two people from each of the above categories, you can be reasonably sure that it will be a good cross-section of the population.

As long as the same pair of interviewers is doing all the interviews, you'll know when the sample is large enough, because different answers will become less and less common. If several pairs of interviewers are working at the same time, they should all discuss their findings after each day's work.

When choosing a sample size, bear in mind that in-depth interviews are much more expensive than normal survey interviews: they use more highly paid staff, two at a time, and take longer: both to interview and to analyse.

Because in-depth interviews are exploratory in nature, and are usually followed by a full survey, and they generate a lot of very detailed data, 20 interviews is often enough, and 50 is usually too many.

I don't recommend doing a set of in-depth interviews as the only fieldwork in a research project. So instead of doing 50 in-depth interviews, I'd suggest 20 interviews, plus a set of 3 consensus groups. The cost will be less, and using two information sources instead of one will create greater confidence in the results.

In case you fear that different research methods will produce contradictory results: this seldom happens. If it does happen, you need to do some more research to try and resolve the differences. Different methods produce slightly different results, but these complement rather than contradict one another.

Question outlines

In-depth interviews don't need elaborate questionnaires. Instead, they use question outlines. These can take two forms.

(1) A list of topics, on a single page. The topics are internal questions, rather than questionnaire questions. (See section 1 of the chapter on questionnaires, which explains the difference.) Usually there are 5 to 10 topics: the main things you want to find out from the research. As more interviews are done, each of the main topics is gradually divided into a number of sub-topics, and these are written in on the question outline.

Respondents' comments are usually written in large hard-backed notebooks: one for each interviewer.

(2) The same list of topics, but in questionnaire format: like a questionnaire consisting almost entirely of open-ended questions. Each question should start on a new page, giving plenty of space to summarize the respondent's comments on that topic. At the end of the questionnaire are several blank pages - to cover topics not included in the original list. As with a normal survey, there is one questionnaire for each respondent.

Method 1: the notebook, with a separate one-page question outline. This method is easier for experienced interviewers. A wide margin is left on each page, and comments are written down in the order they are made. As the topic changes, the interviewer writes the appropriate code in the margin.

Method 2: the open-ended questionnaire. This is easier for beginners, but requires the interviewer to decide which topic is being discussed, and to turn the pages backwards and forwards so that each comment goes on the right page. If a respondent has a lot to say on some topics, extra pages need to be inserted.

Appointments and settings

In a normal survey, the interviewer must keep to a tight time schedule, and interviews usually take less than half an hour. But an informal interview shouldn't be hurried. The interviewers and the respondent should feel that there is plenty of time to discuss the subject.

It's therefore usual to make an appointment before interviewing anybody. An ordinary (i.e. less skilled) interviewer can be sent to find respondents and make appointments, and the pair of in-depth interviewers can meet the respondent at a suitable time.

It's best to do in-depth interviews in a place where the respondent feels most comfortable. This is usually at home, but could be at their workplace, or perhaps even in a public place (e.g. a quiet restaurant or bar) if their home is too crowded and they are unable to talk freely there. Let the respondent choose the place of interview.

Introductions

An introduction for an in-depth interview is much the same as for survey interview. Some research companies make interviewers learn introductions by heart, but to me such introductions always sound unnatural.

A better way to do it is to list some key points which must be covered, and leave it to each interviewer to make those points in the most appropriate way for a particular respondent in a particular setting.

There are two sets of key points: one should be told to all respondents. The other points are answers to questions that some respondents will ask.

All respondents should be told:

1. The purpose of the research - e.g. "This research is being done for Provincial Radio - they're trying to find out how people use their programs, and how to improve them."

2. Why this respondent has been chosen. "We chose this village because it's typical of the whole region, and within the village we've picked 5 houses at random. It doesn't matter if you yourself aren't typical, because when we combine everybody's answers we'll have the opinions of all sorts of people throughout the region."

3. The expected duration of the interview - e.g. "It's partly up to you how long it will last, but probably between 1 hour and 2 hours."

4. Reasons for having two interviewers. "There are two of us because it's difficult to talk to somebody and remember what they say at the same time. First I'll talk to you, and my friend will take notes and work the tape recorder, and later we'll swap over."

Answers to questions that respondents often ask

A full introduction may take up to 10 minutes, but it's important not to rush this stage. It is only when a respondent feels comfortable with the idea of being interviewed that he or she will give full and honest answers.

Informal interviews should be done sitting down, preferably with food or drink. It works well if the interviewers bring some snack for themselves and the respondent to eat. The respondent will then usually provide a drink as well - and with the exchange of food and drink comes an exchange of views.

Interviewing techniques

Unlike a normal survey, the questions in a question outline should not be read out to the respondent. The principle is to introduce the broad topic, and get the respondent to discuss it.

With some respondents, the more talkative ones, it's only necessary to ask a broad question, and they'll talk until they are stopped. The problem with these respondents is that they usually drift off the subject, and need to be occasionally reminded of your area of interest.

Other respondents, not so talkative, may give very short answers, perhaps hoping that the interviewer will do most of the talking. Often this happens when the respondent doesn't fully understand what the interviewer wants to know. I've found the best way to solve this problem is to ask the respondent to recount some of their recent experiences relating to the topic of the research.

The interviewer can prompt the respondent with statements beginning like this: "Tell me about a time when you ...."

Notice that this is not a question: it's an invitation to tell a story. There are many situations and events that people can describe well, because these are recent, and from their own lives. This approach is much more effective than asking a respondent to give opinions, or find out what they think other people do. If a respondent begins to make generalizations, the interviewer should ask for a specific example.

Who's doing the talking?

A common problem with inexperienced interviewers is that they end up doing most of the talking, trying to explain the research, and so on. It's easy to spend a lot of time talking, without realizing - specially as the type of people who apply for interviewing jobs are usually talkative types themselves. Here's an exercise to check the amount of talking being done - assuming the interview has been taped, and your tape recorder has a counter.

Divide the tape into about 20 equal segments, and work out the counter position at the end of each. For example, if the counter reads 000 at the beginning of the tape, and 900 at the end of the interview, one 20th of 900 is 45. That's too difficult: round it to 50. Now fast-forward the tape to 50 on the counter. Who is talking? The interviewer or the respondent? Note it down. Repeat the process, checking at 100, 150, 200, and so on. What you are doing here is making a random sample of the interview: in this case, a sample of 18 snatches of conversation. If the interviewer is talking more than half as much as the respondent, that's too much. You'll probably notice that, toward the end of an interview, the interviewer is usually speaking less, and the respondent is speaking more.

Follow-up interviews

Sometimes a questionnaire can be left with each respondent - perhaps asking them to record their activity over a day. One or both interviewers then return, and a short follow-up interview can be done: 10 minutes is often enough. Frequently, at these second interviews the respondent will have thought of something not mentioned at the first interview.

Combining interviewing and observation

Because in-depth interviews are usually done in respondents' homes, they can often be combined with observation. In other words, you can record what you see, as well as what you are told. If you are studying newspaper and magazine reading, can you see newspapers or magazines in the home? You could ask respondents to show you some publications they have acquired recently. In the case of radio or TV, you could note the prominence of the set in the house. Is it proudly displayed in the main room? Is it covered with a cloth? Is it pushed away into a corner? This can give you clues about the importance that people place on the various media.

Recording in-depth interviews

There are two main ways of recording in-depth interviews: on tape, and on questionnaires.

On tape: interviews can be recorded in full (on audio or video tape), and the tapes then transcribed. This is fine, but before you go ahead with this, consider whether you will ever have time to review the tapes, or read the transcription. (A word-by-word transcription of a one-hour interview usually takes a skilled typist a whole day.)

On paper: as mentioned earlier, this can either be a separate questionnaire for each respondent, or an interviewer can record every respondent's comments in a notebook.

A disadvantage of relying on a tape recorder is that a tape recording can go wrong in all sorts of ways. Batteries go flat (in both microphone and tape recorder), power cuts happen, tapes break or jam, cable connections fail, the volume knob can be set too low or too high, and the Pause button can be left on by mistake. All of these have happened to me!

Recording comments only on paper also has its disadvantages. Writing the answers while the respondent speaks takes less time than transcribing a tape, but it slows the interview down a lot. The respondent will have to keep waiting while the recording interviewer makes notes. Sometimes this is an advantage (because the respondent thinks of more answers while the previous ones are being written down), but more often it is a disadvantage (because the respondent loses his or her train of thought while waiting for the interviewer).

If the interviewers let the respondent talk at full speed, it's almost impossible to keep up, unless the recording interviewer can write shorthand. In this case, most of the respondent's exact words will be lost, and when the summary is read later it could be misunderstood.

I therefore recommend using both methods: writing down a summary of the answers, and also taping the interview. Use a tape recorder with a counter, and set this to zero at the beginning of each tape. If the recording interviewer writes down the counter number every few minutes, it's easy to find a particular place on the tape, if the summary's not clear later, or somebody wants to hear the respondent's actual words.

It's often useful for a different person to listen to the whole tape later, and make their own notes, but it's seldom necessary to transcribe a whole tape. A partial transcription, done by somebody who thoroughly understands the purpose of the research, can be made in about 2 hours for every hour of tape.

Several respondents with one interview

Informal interviews need not be with a single respondent - I have successfully interviewed up to three people at a time - usually members of the same family, on the same subject, such as the family's TV viewing habits. With more than three people, it becomes chaotic: too many people try to speak at once, and you miss some of what is said. It's no longer an interview, but a group discussion.

A combination that often works well is two respondents, with two interviewers. If the two respondents know each other well (e.g. husband and wife) they can cross-check each other's statements. The danger with two respondents and two interviewers is that each respondent will try to talk to a separate interviewer. The rule should be: one conversation at a time. One interviewer asks the questions, while the other takes notes.

Analysing in-depth interviews

Because almost all the information in an in-depth interview is verbatim, it's easy to be overcome by words, and not be able to make out the main sense of the answers. I've known people to spend months trying to make sense of a mountain of text, working out what is most important. If every interview is fully transcribed, there's much more text, and the problem is much worse.

There are two secrets to successful analysis of verbal data. One is to summarize each interview, as soon as possible after it has been finished. If there are two interviewers, they should compile a page or so of notes: the main things said in the interview, and also anything which the respondent was expected to say, but did not.

The other secret is to build a structure, and fit each statement into that structure. Usually this can be drawn in a tree-like diagram, rather like the Logical Framework system described in chapter 1.

Creating a topic tree

If you started the in-depth interviews with, say, 7 broad questions, you probably discovered (in the first few interviews) more broad questions that you should have included: so you often begin the analysis with about 10 broad questions, or major topics. For each major topic, there will be a number of sub-topics - perhaps up to 10. For some sub-topic, there may even be several sub-sub-topics - but not so many, usually no more than 3. With 20 interviews, beginning with 7 broad questions, you'll probably end up with approximately 100 topics.

With 100 unrelated topics, it's almost impossible to analyse the results well, but if these are placed on a tree diagram (as if they were branches) it's much easier to analyse and understand the results.

The first step is to make sure that the codes are written in each interview record, beside the notes they apply to.

A code can be either a question number (e.g. Q1) or an abbreviation (e.g. PROGS for programs preferred). As more and more interviews are completed, you find out that each main topic becomes divided into a number of sub-topics. For example, under PROGS, respondents might tend to talk about programs broadcast at particular times of the day, so you might decide on sub-topics of BF (breakfast time), LM (later morning), MID (middle of the day), AFT (afternoon), and NT (night). When the sub-topics have been decided, you need to go back to the notes on the earlier interviews, and write in the sub-topics, perhaps changing PROGS to PROGS/BF. If people have a lot to say about a particular sub-topic, you can even introduce sub-sub-topics. For example, if a lot of respondents talk about the music in the breakfast program, you could create a sub-sub-topic of PROGS/BF/MUSIC.

Grouping the comments

To begin the grouping, make a copy of the data from each interview. This can include an interviewer's notes from a notebook, the responses written into a questionnaire, and a full or partial transcript of the tape for that interview. The combination of all these words, from one interview, is called the interview record.

Photocopy the interview record, retype it, or type it into a computer and print it out. You could even rewrite it by hand. This may seem like a lot of effort, and a waste of time, but it's not. The copying forces you to read through all the notes, and will produce a much clearer mental picture of the set of interviews.

Use only one side of the paper, start each broad topic on a new page, and leave a broad margin on one side. Give the interview a number, and make sure that the number is on every page of the copy.

Don't attach the copied pages together - except when one topic, from one interview, takes more than a single page.

Sort these pages into the broad topic areas: about 10 heaps. (It helps to have a large table or desk, and a draught-free room.) For each broad topic area in turn, read through the heap and work out what the sub-topics are. Every time a different sub-topic comes up - every few lines, probably - write it in the margin of the text. It's best to do this in pencil, because you'll probably keep changing the categories a lot as you go through the heap.

If several people are working on this, it's best if they do it together, to make sure they are all using the same sub-topics.

When everybody is satisfied with the list of topics and sub-topics, and all the answers have been coded, write the interview number next to each sub-topic, and cut up every page in the heap, making a separate heap for each sub-topic and sub-sub-topic. There could now be about 50 to 100 heaps, each with about as many pieces of paper as there were respondents. Some respondents don't mention some sub-topics, but others comment several times.

Reporting the findings

Take each small heap in turn, and read through the comments in it. Lay them out on a table, and put the similar ones close to each other. (Make sure you do this with the windows closed - a gust of wind can ruin hours of work!) As soon as you've read through the comments, write a brief summary of them - a brief paragraph is usually enough. Often, part of a comment can be used in this summary.

When you have written a summary paragraph for each heap in one sub-topic, you'll have covered an entire topic: one of the main research questions. Now write a few paragraphs summarizing the other summary paragraphs.

If you have formed some hypotheses or ideas about the population you are studying, these can be tested against the relevant comments. For each hypothesis, look at the comments which might confirm it, and compare these with comments which suggest it's not true. Summarize the comparison, and put this in your report too.

In this way, by going through each of the numerous small heaps and summarizing it, you have created a simple report on the results of the informal research. There's a lot more that skilled analysts can do, but this process of dividing the mass of interview records into sub-topics, and reporting on each, is the basis of qualitative analysis.

And, as you may have noticed, the process is much the same as the intent structure (described in section 2 of chapter 1) used in the Logical Framework approach.

The necessity of analysis

Some beginners with research are tempted to stop after the interviewing, particularly if they have done it themselves. "Now we know it all," they think. "We've spoken to all these people, we know what they said, so why bother to write it down?"

The answer is: you forget. Interview records can be as long or as short as you like, but if you don't make them, or don't analyse them, you don't have a comprehensive view of your research. A few unusual moments will stand out in your memory, as will comments made in very striking ways by people who are probably not typical. If you doubt this, here's an exercise,

Before you have read through all the interview records, and without looking through your notes, write a one-page summary of what you think you found in the research.

Then do the analysis and produce a report based on each sub-topic, as outlined above

Now compare the initial summary with the full report. Not only will you have forgotten a lot when making the initial summary, but many of the conclusions are likely to be different, because the memories that stood out were not always typical of what most respondents said.

Without analysis, without considering every comment made, it's not research, but simply an impression.

Computer analysis of in-depth interview data

Instead of putting the comments in heaps, cutting up the heaps, and sorting them out on a tabletop, you can use a computer to achieve the same purpose. There's special-purpose software for qualitative research - e.g. NVivo, Atlas TI, The Ethnograph, and Verbastat. These programs are powerful, expensive, and take several weeks to learn fully.

Another alternative is to use a database program, such as Filemaker Pro or Microsoft Access. However, these programs take quite a while to learn, and aren't flexible enough to handle text for research purposes: they're designed mainly for accounting. Unless you're planning to analyse huge amounts of text, using these programs could take more time than using simpler methods.

One of these simpler methods is to use a word-processing or text editing program, which can sort paragraphs. Common programs which can do this include Microsoft Word, Word Perfect, and OpenOffice Writer.

The steps involved here are:

1. Type in all the interview notes, on a single file. Start each interview with a new paragraph, beginning with a symbol which isn't going to occur at the start of any other paragraph. The / symbol is a good one to use: a paragraph beginning /01 can mean "This is the beginning of interview number 1". Include that initial 0, because word processing programs sort in alphabetical order, not numerical: 10 comes between 1 and 2, not after 9.

In the first paragraph, also include some details about the person interviewed, the place of interview, the interviewer, the date - and anything else you think is relevant.

Whenever the topic or sub-topic changes, start a new paragraph. Every paragraph, except the one starting a new interview, should begin with a symbol not used in normal punctuation. I use the slash / because you don't have to press the Shift key to type it.

Why begin every paragraph with this symbol? So that after everything is sorted, every paragraph not beginning with / (or whatever you choose) is a mistake to be fixed - usually one that hasn't yet been coded.

After the / (or whatever) symbol, write the code for that topic. For example, if the comment is about music on the breakfast program, begin the paragraph with /PROGS/BF/MUSIC. If one paragraph is about two different subjects, you can copy the whole paragraph, and change the code on the copy. If you want to know which respondent made which comment, you can add the interview number at the end of the heading. For example, if respondent number 6 had made the comment about music on the breakfast program, you could head that paragraph with /PROGS/BF/MUSIC/06. Sometimes a comment only makes sense when you take other information into account. If you are interviewing people in different places, and a respondent makes a comment about "here." it won't make sense out of context. The easiest solution is to add your own explanation, usually in square brackets, for example "here [Bahir Dar]" or "old people like me [age group 45-54]" - and so on.

2. When all the notes from all the interviews have been typed in, save that file, and make a copy of it, with a different name. Take care of this copy, because if you anything goes wrong with the sorting, you'll need to go back to it.

3. Open the copy of the file, and sort its paragraphs. (For example, in Microsoft Word, use the Sort command in the Table menu.) This will sort all the paragraphs into alphabetical order.

Now the reasons for the strange codes become clear. First will come the header paragraphs, summarizing each interview (if you have prefaced them with the / symbol.)

Next, you will see the comments on each subject grouped together in alphabetical order, with sub-sub-topics grouped under sub-topics, and sub-topics grouped into their topics. Within each topic group, comments will be listed in order of interview number.

After the last paragraph beginning with / will come the mistakes, such as part-paragraphs where the typist pressed the Enter key too soon, and paragraphs that haven't yet been coded into topics. You can now fix these mistakes (though you may have to go back to the original file to match the errant part-paragraphs), make a new copy file, and sort that.

When all this has been done, you are up to the same stage as, in manual analysis, when all the little heaps are sorted on the table. With all the comments on each sub-sub-topic grouped together, you can now write a brief summary of each sub-sub-topic, and combine all of these to produce the report.