These are commonly referred to as cluster or portfolio evaluations. New Directions for Evaluation, 120, 55-69. The theory of change has, to some extent, been tested. Alternative theories of change may emerge and other influencing factors may be identified. Funnell is a director of Performance Improvement, a consulting company, and the former president of the Australasian Evaluation Society. Funnell is a director of Performance Improvement, a consulting company, and the former president of the Australasian Evaluation Society. Purposeful Program Theory shows how to develop, represent, and use program theory thoughtfully and strategically to suit your particular situation, drawing on the fifty-year history of program theory and the authors' experiences over more than twenty-five years.
Realistic evaluation of an illicit drug deterrence programme. Figures, Tables, and Exhibits x Acknowledgments xv The Authors xvii Introduction: The Promise and Risks of Using Program Theory xix Part One Key Ideas in Program Theory Chapter 1: The Essence of Program Theory 3 Evaluation Without Program Theory 3 Evaluation with Program Theory 6 Summary 13 Exercises 13 Chapter 2: Variations of Program Theory over Time 15 A Short History of Program Theory 15 Terminology in Program Theory 23 Key Ideas in Program Theory 30 Summary 34 Exercises 35 Chapter 3: Common Myths and Traps 37 Some Common Myths 37 Traps to Avoid When Developing and Using Program Theory 41 Summary 52 Exercises 52 Part Two Assessing Your Circumstances Chapter 4: Scoping Intended Uses 55 Why Intended Use Matters 55 Using Program Theory 58 Summary 67 Exercises 68 Chapter 5: The Nature of the Situation and the Intervention 69 Simple, Complicated, and Complex 70 Focus 74 Governance 80 Consistency 82 Necessariness 84 Sufficiency 85 Change Trajectory 88 Summary 90 Exercise 91 Part Three Developing and Representing Program Theory Chapter 6: Processes to Identify or Develop a Program Theory 95 Process Options for Developing Program Theory 95 Decision 1: Who Should Be Involved in Developing a Program Theory, and How? On the right side of the box is an arrow that points toward the results chain. The Changing Landscape of Youth Workers pp. Realistic Evaluation Realistic evaluation is a form of theory-based evaluation developed by Pawson and Tilley 1997, 2006. Footnote 4 Generic policy strategies are often grouped into three categories: incentives i.
The theory of change at this stage is the prior-hypothesized theory of how the designers of the intervention expected it work. My only reservation relates to the flip-side of the level of detail contained in the book. Reflections on a quarter-century of Extension evaluation practice. The findings from the individual evaluations can then be aggregated and synthesized to arrive at findings and conclusions about the policy. First, the primary importance of specifying a programme's ultimate objective i.
This section discusses how theory-based approaches can be applied to these endeavours. Horizontal Initiatives From a methodological perspective, horizontal initiatives are quite similar to the policy evaluation case described above; the horizontal initiative can be seen as the overarching policy and the individual program components as the equivalent of the program interventions being implemented to fulfill the policy goals. Chapter 10: Critiquing Program Theory. The theory of change can be used to test — with evidence — the assumed causal chain of results with what is observed to have happened, checking each link and assumption in the process to verify the expected theory. Chapter 13: Logic Models Resources.
But until now no book has examined these multiple uses of program theory in a comprehensive, understandable, and integrated way. Journal of Youth Development, 6 3 , 82-94. Evaluating anti-corruption initiatives: Underlying logic and mid-term impact of a World Bank Program. To account for this variation, the frameworks were grouped into three categories based on the science and context in which they were developed. This perspective becomes useful when the various program interventions use different policy mechanisms.
Jane Davidson, PhD, Real Evaluation Ltd. Similarly, an increase in the use of results information can lead to an increase in demand for good results information. A dialogical, story-based evaluation tool: The most significant change technique. This book was one of, if not the, most useful texts I was able to access whilst completing my dissertation and research project. Canadian Journal of Program Evaluation, 18 2 , 91—113.
Performance measurement: Getting results 2nd ed. From then to now: Emerging directions for youth program evaluation. A theory-based perspective is especially suited for this type of analysis and facilitates learning in the cluster. Theories of change are referred to by a variety of names, including program theories, impacts pathways, and pathways of change. Each causal link box contains a description of the risks and assumptions that exist between the levels of the results chain.
For realistic evaluators, the key evaluation questions are, What works? Figures, tables and exhibits are also included. At the same time, specific interventions may also contribute to other theories of change within a department. Practical program evaluation: Assessing and improving planning, implementation, and effectiveness. Chapter 7: Developing a Theory of Change. At the same time, theory-based approaches seek to identify and assess any significant influencing factors i. The right-hand side of Figure 2 identifies the causal links between the outputs and outcomes i. These can be used as a means to develop, or further develop, a theory of change, and as a way to determine and identify evidence on the extent to which the theory of change was realized in practice.
As an evaluator, this is particularly useful in my day-to-day work. Results management: Can results evidence gain a foothold in the public sector? Applied Developmental Science, 7, 172-180. Evaluation Review, 21 4 , 501—524. Chapter 6: Processes to Identify or Develop a Program Theory. My research topic was Theories of Change approaches to strategic planning in the context of a philanthropic grant-making foundation. Advances in Child Development and Behavior, 41, 197-230. This promises to be a breakthrough book, valuable to practitioners, program designers, evaluators, policy analysts, funders, and scholars who care about understanding why an intervention works or doesn't work.
Context must be part of the evaluation framework, and specific contexts, whether within or outside the control of the intervention, can enhance or detract from how well the intervention works. Chapter 13: Logic Models Resources. Now widely used in government and not-for-profit organizations, program theory provides a coherent picture of how change occurs and how to improve performance. This arrow represents the impact that external influences can have on any of the elements of the results chain. The practice and the theory: Lessons from the application of a theories of change approach. All kids are our kids: What communities must do to raise caring and responsible children and adolescents. In some cases, theories of change are subdivided into two components: the intervention theory, which outlines the underlying behavioural assumptions the mechanisms behind the intervention, and the implementation theory, which identifies how an intervention is expected to operate and trigger these mechanisms.