Finding the impact in a messy intervention: Using an integrated design to evaluate a comprehensive citywide health initiative

Beth C. Weitzman, Tod Mijanovich, Diana Silver, Charles Brecher

Research output: Contribution to journalArticle

Abstract

This article uses the evaluation of the Robert Wood Johnson Foundation's (RWJF) Urban Health Initiative (UHI), a 10-year effort to improve health and safety outcomes in distressed cities, to demonstrate the strength of an evaluation design that integrates theory of change and quasi-experimental approaches, including the use of comparison cities. This paper focuses on the later stages of implementation and, especially, our methods for estimating program impacts. While the theory of change was used to make preliminary identification of intended outcomes, we used the sites' plans and early implementation to refine this list and revisit our strategy for estimating impacts. Using our integrated design, differences between program and comparison cities are considered impacts only if they were predicted by program theory, local plans for action, and early implementation. We find small, measurable changes in areas of greatest programmatic effort. We discuss the importance of the integrated design in identifying impacts.

Original languageEnglish (US)
Pages (from-to)495-514
Number of pages20
JournalAmerican Journal of Evaluation
Volume30
Issue number4
DOIs
StatePublished - Dec 1 2009

    Fingerprint

Keywords

  • Child health
  • Comprehensive community initiatives
  • Quasi-experimental designs
  • Theory of change
  • Urban health

ASJC Scopus subject areas

  • Business and International Management
  • Social Psychology
  • Health(social science)
  • Education
  • Sociology and Political Science
  • Strategy and Management

Cite this