Program logic also goes under several other names: program theory, logic modelling, program logic models, theory of action, and theory-based evaluation. If you're searching for information about it on the Web, you'll need to look under all those names. The "programs" here are programs of planned action in general - usually not computer programs, radio programs, or TV programs.
Program logic (as I'll call it here) seems to be an extension of the Logical Framework Approach, which was developed in the 1960s for international development projects, and is now widely used for that purpose. (Too widely, perhaps, given its rigidity.)
Around the 1980s came program logic modelling, also known as program theory. Till recently, this approach has been used most in the evaluation of health and social welfare programs. Later offshoots, not covered here, serve the increasing focus on accountability in public service work, such as Results-based Measurement, widely used by the Canadian government. Program logic models, on which this page is focusing, are produced in a wide variety of different forms, with this common basis:
The diagram is very abstract, so let's create an example to show how the model can work: a radio program designed to educate people about how to prevent malaria. (This is a very simple example, with only a single chain. Real examples are a lot more complex.)
But the principle is that you specify what you hope will happen, in the form of a chain, and then test each link in the chain to work out what really did happen. That's program logic modelling in a nutshell. At first it seems like a ridiculously cumbersome way of stating the obvious, but its value in practice lies in that step-by-step testing. If the program didn't work as intended, you can detect the weak links in the logic chain.
To confuse the uninitiated, the labels "impacts" and "outcomes" for the long-term and short-term objectives are sometimes reversed. Also bear in mind that the division of results into two types is arbitrary: there can also be medium-term effects.
The above diagram is represented as a chain, implying that no stage can occur until all previous stages are complete. In that sense, it resembles a chain scenario (or sequence of possible future events): the main difference is that program logic embodies one overriding intention, while a scenario does not. The problem with this type of thinking is that (though its truth seems obvious) the world doesn't always work like that. This "hierarchy of effects" approach, as it's sometimes called, just doesn't correspond with reality. For example, there's the famous AIDA model of marketing. This means that A (awareness) happens first - people become aware of a product. Then I happens: they're interested in it. Then it's D: they Desire it. Finally, they Acquire it. It makes sense, doesn't it? The trouble is that a lot of stages are often skipped. A few years ago we did a study of why students choose to do postgraduate study at a particular university. We interviewed 19 students, and not a single one followed the "obvious" AIDA model.
Given these types of problem, thinking about "impacts" and "outcomes" is being superseded by an approach known as the outcomes hierarchy. Instead of just two levels, with impacts following from outcomes, there's a whole network, showing that effects can occur through different paths. Here's an explanatory report on outcomes hierarchies.
But getting back to the above diagram, and setting aside for the moment the complexities of real-world outcomes, the diagram's simplicity conceals its power - in two ways:
A more complex example is a project I worked on over several years, with public broadcasters in Vietnam. The activity was the introduction of live radio and talkback programs on provincial radio stations. The inputs were funding and training expertise, provided by a Swedish aid agency. The activity was training local radio producers in the principles of live and talkback radio. The outputs were trained producers. The direct outcomes were the new programs they broadcast. The desired broad outcome (by the funding agency) was improved democracy.
(In fact, this project, like much international development work, used the Logical Framework approach, but I have re-expressed it as a program logic model.)
Notice that the above logic model was described from the Swedish point of view. Though the Vietnamese view of the inputs and activities would be much the same, the later stages could be seen quite differently. It would be plausible to view the end-goal of the Vietnamese radio network as survival in the face of the increasing popularity of television - achieved through an intermediate goal of a new, slicker style of radio programming. From a broader point of view still, it would be possible to see the end-goal as educating the Vietnamese public in the possibilities of consumer choice. This in turn could lead to greater transparency in government and business, democratic involvement with the political process, and increased awareness of the power of consumerism - not quite the same as democracy.
The crucial point - of which this Vietnamese case is only one example - is that from each actor's point of view, the logic model would be somewhat different. When a funder and the recipient have different logic models, they might both commission evaluations, which might not concur on the program's achievements.
The further to the right in the sequence, the more difficult it becomes to relate inputs to outputs, or causes to effects. (This is nearly always the case with logic models.) To begin at the left: did providing the resources result in the desired activities? Simple clerical work could answer that question. But whether the outputs of the project - regular live radio programs with talkback - actually led to increased democracy in Vietnam is much more elusive to measure. As this project was only one of many factors in the westernization of Vietnam, how could its contribution to enhanced democracy in Vietnam be accurately assessed? At the right-hand end of the chain, definitions and assumptions become crucial: a small change in the definition of an end-goal might result in a large change in the evaluated success of a project. Sue Funnell's (1997) cascading approach to program logic could help here: a project may not produce its intended broad outcomes unless specific interventions are used to assist in their accomplishment. Implication: produce the model before beginning the activities.
In practice, a program will usually involve multiple activities, with multiple inputs and outputs, all intended to work on multiple target groups towards a broader goal. For example, how would those talkback radio programs contribute to democracy? There might be several different types of impact from a regular talkback program. (Here, the output is defined from the Vietnamese point of view: i.e. the production of a regular radio program.)
|Criterion to be evaluated||Method of evaluation|
|Program audience size, and listeners' awareness of content||A survey, designed to measure the program's audience|
|Acceptance of principle that official actions are accountable to all citizens||Qualitative research, e.g. consensus groups|
|Listeners take part in talkback programs||Radio station keeps records of the number of people calling talkback program|
|Officials hesitate to make undemocratic decisions||"Before" and "after" unattributed interviews with officials|
|Elected representatives take up themes mentioned on the programs||Content analysis of parliamentary speeches and government news releases|
But the question of how far those types of activity (even when enumerated) contribute to the enhancement of democracy is very much a political one. The answer to that question would be strongly influenced by the definitions used in the final stage of the logic model.
Program logic has limitations: some intrinsic, and others arising from the context in which it is normally carried out...
The University of Wisconsin offers a free online course on logic modelling. (The terms they use are a little different from those used above.)
The Kellogg Foundation has an online handbook on Program Logic Modelling, using a slightly different approach again (64-page PDF file).
Sue Funnell's 1997 article: "Program logic: an adaptable tool for designing and evaluating programs" in Evaluation News and Comment, vol. 6, issue 1, pages 5-17.
A classic 1997 article by Paul F McCawley, The Logic Model for Program Planning and Evaluation, from the University of Idaho Extension.
Two other web pages on the Audience Dialogue site extend the principles of program logic: one on website effectiveness, and one on business models. Our page on rethinking social marketing is also relevant.