The Health Profession Opportunity Grant (HPOG) Impact Study: A Behind-the-Scenes Look at Rigorous Evaluation in Practice
Presenter: Laura Peck, Ph.D.
Date: March 10th, 2021, 12-1 pm PST
Abstract: In 2010, the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services awarded the first round of five-year Health Profession Opportunity Grants (HPOG 1.0) to 32 organizations in 23 states. The purpose of the HPOG Program is to provide education and training to Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals for occupations in the healthcare field that pay well and aim to meet local areas’ healthcare sector labor shortages. To assess its effectiveness, an experimental evaluation design assigned eligible program applicants at random to a “treatment” group that could access the program or a “control” group that could not. Beyond the impact analysis, the evaluation also probed questions about what drove program impacts, using various strategies to do so.
This workshop will discuss how the HPOG 1.0 impact study was designed and implemented and introduce participants’ to the various design and analysis choices the investigators used, in partnership with the government funder to address varied research questions. Specific topics will include: experimental design, multi-armed experimental design, experimental impact analysis, planned variation, natural variation, endogenous subgroup analysis, evaluation in practice.Additional discussion can consider the optimal conditions for using an experimental evaluation design, as was the case for HPOG and as might be the case for future evaluations.
Western Positive Psychology Conference
The Nature of Knowledge in Complex Dynamic Systems: Evaluating and Conducting Applied Research During a Global Pandemic, Social Unrest, and Economic Turbulence
Presenter: Michael Quinn Patton, Ph.D., CGU Professor of Evaluation Practice
Host and Facilitator: Stewart I. Donaldson, Ph.D., Distinguished University Professor & Executive Director, Claremont Evaluation Center (CEC) & The Evaluators’ Institute (TEI).
Dates: Sept 16th, Sept 30th, Oct 13th, Nov 3rd, Nov 10th, Nov 17th, Dec 1st, 2020
During this 7 session series, Dr. Michael Quinn Patton explored key issues such as the nature of systems transformation, the role of evaluation in creating systemic change, lessons learned from the COVID-19 Pandemic, and how evaluation can be used to interpret the 2020 presidential election. Please see below for a description of what was covered in during each webinar.
Wednesday, Sept 16
Title: Evaluation Fundamentals: Coronavirus Insights and Lessons
Description: Responses to the pandemic require evidence-based judgments, the territory of evaluation. This session will examine how, and how well, evaluation logic and fundamental concepts have held up throughout the pandemic. What we will find is both discouraging and hopeful, but not in equal dosages.
Wednesday, Sept 30
Title: Systems transformations and paradigm shifts: what they are, how they happen, why they matter, and how they are unfolding right now in multiple dimensions and arenas.
Description: The language of “systems transformations” and “paradigm shifts” can sound academic and esoteric. But these notions are fundamental to how change occurs. This webinar will examine micro and macro examples, personal, institutional, societal, and scientific transformations and paradigm shifts. You won’t think about these things the same way after this webinar.
Tuesday, October 13
Title: Systemic racism, structural racism, institutional racism: What does evaluation bring to the global social justice uprising clamoring for systemic change?
Description: What light can evaluation shed on what is meant by “systemic racism”? How do evaluative thinking and systems thinking intersect in examining, monitoring, and illuminating the nature, scope, and implications of systemic racism? Traditionally, evaluation and evaluators have believed that credibility depended on being above and outside the fray. Post-normal evaluation reframes evaluation and evaluators as very much part of the fray, whether acknowledged or not. This webinar will include examination of evaluation’s historic and current engagement with issues of racism and social justice, both within evaluation practice and in the larger society. Reflective practice question: How far dare evaluators go in attempting to change the world?
Tuesday, November 3
Title: Election day anticipation: Bringing an evaluation lens to preparing to understand and interpret the 2020 presidential election.
Description: This webinar will be an exercise in meaning-making and sense-making by asking: how do we prepare to make sense of the election results evaluatively? How do we interpret what has happened? How can and does evaluative thinking inform meaning-making and sense-making on the big issues of our time, both for us personally and professionally, and institutionally and societally?
Tuesday, November 10
Title: Seeing, understanding, and acting on interconnections, interrelationships, and interdependencies
Description: Systems evaluation spotlights interconnectedness. This webinar will engage participants in connecting major global trends and examine the implications of those interconnections. This means moving beyond the silos and territorialities of agencies, philanthropic foundations, government ministries, funding streams, and, not incidentally, evaluation. This webinar aims to change your worldview of what is happening in the world.
Tuesday, November 17
Title: Epidemiology and Evaluation: what these fields, together, teach us about applied science in the real world, in real time, on matters that matter
Description: Evaluators are inherently epidemiologists. Epidemiologists are inherently evaluators. Neither group understands and appreciates the nature and implications of the two professions shared and overlapping engagement processes, methods, and worldviews. This webinar will illuminate those interconnections for the benefit of both fields.
Tuesday, December 1
Title: The future of the university: what role can and should evaluation play in supporting the transformation of higher education?
Description: This webinar will feature evaluative thinking as a form of futuring using universities as the focus of transformation.