Courses – Claremont Evaluation Center

Brief Orientation Courses for UN Diplomats

The Power of Evaluation: A Framework for Success in Achieving the SDGs

Deborah Rugg, PhD

Leaders from all sectors – from government officials and legislators, to UN senior officials to non-governmental, philanthropic and private-sector organizations – have to show that their policies, programs, and services are making a difference for their citizens, clients and customers. Evaluation offers a systematic way to understand issues and perspectives, and uses evidence to explore different types of strategies and actions for policies and stakeholders. This course will provide an introductory understanding of:

The difference between project-based performance evaluation and “systems-thinking” evaluation approaches
How evaluation turns “facts” into a “story,” and “story” into a “compelling argument”
The essential role of evaluation in determining policy or program impact and learning how and why things work or don’t work, and ways to improve
How to read and use an evaluation report like a pro

SDG-Responsive Evaluation Fundamentals for Diplomats and Politicians

Deborah Rugg, PhD

We now operate in a world where all governments have committed to the Sustainable Development Goals (SDGs), and they have to report on them. This course will explain how evaluation can help governments and civil society be accountable to their constituents, including the United Nations, for their progress toward the SDGs.

Evaluation as a core strategy in an SDG era
How complexity-thinking can make you smarter on the SDGs

Building Evaluation Capacity at Country Level

Deborah Rugg, PhD

In 2015, all countries committed to building evaluation capacity at country level. This course will discuss the benefits of country-led evaluation to strengthen country-led negotiations and strategies. We will explore the key elements of an evaluation system at country level, and how to identify what you already have and build it.

What is required by UN Resolution A/RES/69/237?
Models of evaluation country capacity
How to get started on building evaluation capacity at country level: existing resources and key steps

In-Depth Leadership Certificate Program in Evaluation and SDG Fall 2017 courses

The Power of Evaluation: A Framework for Success in Achieving the SDGs

Deborah Rugg, PhD

The difference between project-based performance evaluation and “systems-thinking” evaluation approaches
How evaluation turns “facts” into a “story,” and “story” into a “compelling argument”
The essential role of evaluation in determining policy or program impact and learning how and why things work or don’t work, and ways to improve
How to read and use an evaluation report like a pro

Key Issues for National Evaluation Agendas in the SDG Era

Zenda Ofir, PhD

The MDG era highlighted important weaknesses in how evaluation had been used (or not) to help improve development. Now, in both the Global North and the Global South, we have to ensure that national evaluation agendas are designed mindful of those qualities necessary for success in the SDG era. This course will therefore focus on the following:

Lessons learned during the MDG era about the role, value and foci of evaluation
Key characteristics of evaluation that support development
The implications for national evaluation agendas

Equity and Responsiveness in Monitoring and Evaluation

Katrina Bledsoe, PhD

Monitoring and evaluation are key components to understanding the unique factors that influence policies, programs, governments, and societies. They are also tools that can be used to ensure equity and cultural responsiveness. Equitable and responsive evaluation focuses on who should be involved in decision-making; what questions should be asked and understanding how these questions are contextually grounded; what indicators are most representative of the context; what kinds of data will ensure a more accurate assessment; and in what manner data should be analyzed to enable all voices to be heard. Although most stakeholders will often be commissioners and consumers of evaluations, they will also be the people who are most in a position to guide the evaluation in ensuring equity and responsiveness.

This half-day, interactive course includes some necessary foundation topics, as well as plenty of discussion and opportunities to think about equity and responsiveness in monitoring and evaluation. We will examine:

What is meant by equity and responsiveness
Developing evaluation questions with an eye towards equity
Developing context-appropriate indicators of progress and success
Collecting and analyzing data so that it is representative of the myriad of groups that policies and programs cover
Becoming a savvy consumer of evaluation

Evaluation as a Leadership Function

Michael Quinn Patton, PhD

Leadership development has become one of the most highly valued forms of capacity-building. The challenges of leading modern, complex organizations have generated great interest. At the same time, organizations of all kinds are facing increasing demands for accountability and evaluation. This course puts these two trends together by addressing evaluation as a key leadership function.

Evaluation is much more than compliance reporting, mandated paperwork, and meeting funders’ monitoring demands. Evaluation as a leadership function places the emphasis where it should be: ensuring that programs, organizations, and initiatives are having desired impacts in accordance with their mission. This unique course will introduce: reality-testing, results-oriented, learning-focused leadership for sustainable development.

The reality-testing dimension focuses on the challenges of overcoming selective perception, biases, misinformation, and other barriers that leaders experience in attempting to assess whether what they think is happening in their organizations is what is actually happening.
The results-oriented focus concerns how leaders are involved in setting an organization’s direction and then doing reality-testing to determine the extent to which desired results are being attained.
The learning-focused dimension concerns how leaders create a culture of learning and model learning in their own behaviors.

Being committed to reality-testing, staying results-oriented, and being serious about learning are grounded in evaluative thinking and meaningful performance measurement. Reality-testing, results-oriented, learning-focused leaders are essential for creating and sustaining effective learning organizations that make a difference in the world. In examining evaluation as a leadership function, the course incorporates social science research and evaluation evidence on factors related to organizational effectiveness. The process draws on both research and experience to provide concrete, practical guidance about how to use evaluation to create a learning organization for increased organizational effectiveness.

Leadership for Transformative Change in the SDG Era

Michael Quinn Patton, PhD

We inhabit a world in which an increasing sense of urgency calls for transformational change. The compelling vision represented by the Sustainable Development Goals, the latest data on accelerating climate change, and the global problems that transcend national borders are challenges that require new ways of thinking, acting, and evaluating. Slow, incremental change is not sufficient. But what does transformational change mean? How is it done? How would we know? Evaluation is no longer just a back-end activity in which evaluators come in to examine what happened after an initiative is over. Evaluators are now involved at the front-end, participating in the design of initiatives, bringing to bear accumulated evaluation knowledge about what works and what doesn’t, and ensuring that evaluative thinking and evaluability are built into new endeavors from the beginning.

This leading edge course is for those who share that sense of urgency that everything we do must accelerate the pace of sustainable development solutions – including evaluation. This course will examine:

Theories of change related to transformative change
Dimensions of transformative change that constitute an evaluation framework

Framing the Right Questions and Picking the Right Approach: An Overview

Tarek Azzam, PhD

Every time we try something new, we often ask ourselves, “Is it better? What makes it good, what is its value, and what impact is it having?” These are common questions that are often used in evaluations of anything from small interventions to large-scale policies. The identification of the most important questions in evaluation is the most important first step in the evaluation design process, because it forms the foundation of a useful, credible, and rigorous evaluation. It is important to make sure the questions are driving the choice of evaluation methods and not the other way around. Once questions have been identified then the evaluation must integrate three facets that impact the design:

Program context (e.g., stakeholders, politics, maturity of the program, complexity of the program, etc.)
Evaluators (e.g., level of expertise, theoretical perspectives, competency, etc.)
Evaluation methods (e.g., type of design, interviews, surveys, case studies, RCTs, etc.)

Navigating the Leadership Labyrinth: Parts 1 & 2

Michelle Bligh, PhD

There is no question that leadership profoundly affects our lives through our roles in various types of organizations. As an area of study, leadership is a very complicated and elusive subject to understand and explain, making it ideally suited to approach from different disciplines and perspectives. Throughout history, the successes and failures of individuals, groups, organizations, and societies have been attributed to leadership. Leadership is more than just a collection of tools and tips, or even skills and competencies; the essence of an individual’s leadership is fundamentally shaped by her or his values, philosophies, and beliefs. In this course, we will examine:

The leadership labyrinth – the various challenges and opportunities that uniquely face each individual rooted in gender, race, ethnicity, age, class, citizenship, ability, and experience
How to define effective leadership, and how can you develop your identity both as a leader and as a proactive follower
How to develop inclusive leadership abilities to respond to the new challenges and changing demands of a global world
What successful 21st-century leaderships looks like, drawing on theories of philosophy and ethics, charismatic and transformational leadership, and followership
Constructs critical to practicing inclusive leadership in modern organizations, including empowerment, authenticity, accountability, courage, influence, and humility

Ethical Considerations in the New Global Evaluation Landscape

Michael Scriven, PhD

Course description to come.

Additional Courses: Leadership Certificate Program in Evaluation and SDG Spring 2018

Adaptive Learning – A Pivotal Competence

Chris Soderquist

When a process to develop and execute public health strategies fails to achieve desired impacts, it’s often assumed the process was wrong or the right people weren’t involved. In many cases, it might be something else. Just like soccer players need to know how to dribble, pass and shoot the ball before running an offensive play, those involved in improvement efforts and strategy development needs certain skills in order to succeed. These skills (known as enabling competencies) are: systems thinking, conversational capacity, and “yes to the mess.” During the course – through several real world examples from community and public policy initiatives – participants will learn the following:

Why these competencies are overlooked in developing and implementing community strategy
How a systems thinking approach assists in finding high-leverage solutions
How to facilitate conversations that spark more learning than defensiveness
How to step into the current mess and create a process of rapid learning
How these adaptive learning competencies create a collective capacity that leads to insight and system improvement

Developing High-Leverage Strategies

Chris Soderquist

Individuals and organizations working to improve the public good – who are engaged in addressing challenges of sustainability, equity, and health – are faced with myriad intractable issues that seem impervious to any efforts to address them. What’s required is a more holistic, long-term focused approach. This course exposes participants to:

A systemic process and tools they can use to better identify high-leverage policies and strategies
Practical techniques (including a multi-question framework and visual tools) that can generate the required learning, especially when used by teams and groups of diverse stakeholders and subject matter experts

Empowerment Evaluation

David Fetterman, PhD

Empowerment evaluation builds program capacity and fosters program improvement. It teaches people to help themselves by learning how to evaluate their own programs. Key concepts include: a critical friend, cycles of reflection and action, and a community of learners. Principles guiding empowerment evaluation range from improvement to capacity-building and accountability. The basic steps of empowerment evaluation include: 1) establishing a mission or unifying purpose; 2) taking stock – creating a baseline to measure growth and improvement; and 3) planning for the future – establishing goals and strategies to achieve objectives, as well as credible evidence to monitor change. A dashboard is used to compare annual goals with quarterly progress. The role of the evaluator is that of a coach or facilitator in an empowerment evaluation, since the group is in charge of the evaluation itself. This course will:

Introduce participants to the theory, concepts, principles, and steps of empowerment evaluation as well as the technological tools to facilitate the approach
Highlight how empowerment evaluation produces measurable outcomes with case examples ranging from high-tech companies such as Google and Hewlett-Packard to work in rural Arkansas and squatter settlements in South Africa
Employing lecture, activities, demonstration and discussion

Ethnography and Qualitative Methods

Michael Quinn Patton, PhD

Quality evaluation requires both qualitative and quantitative methods. Ethnography is a credible and well-established qualitative approach in the social sciences and evaluation. It is the art and science of description. The key, however, is to describe, for example, a classroom, business office, or homeless encampment from the participant’s world view. Perceptions (of reality) shape behavior. Key concepts guiding the approach include: an emic (or insider) perspective of reality; nonjudgmental orientation; contextualization; holistic orientation; and a cultural perspective. Specific methods concerning data collection include: fieldwork; participant observation; and key informant interviewing. Tools for analysis include: pattern identification; triangulation; and qualitative data analysis software. In addition, the role of a cultural interpretation is described (helping the evaluator distinguish between a “wink” and a “blink”). Ethics inform much of ethnographic work. Deception is not appropriate or necessary. Ethnography involves extend periods on site, building trust, allowing for patterns to emerge, and thus minimizing “company behavior.” This course will focus on:

Ethical discussions: permission, honesty, trust, pseudonyms, and reciprocity
Ethical dilemmas, including dirty hands and guilty knowledge
Technological tools of the trade, ranging from online surveys to Google Hangouts

Tech Tools in Evaluation

Tarek Azzam, PhD

Technology can facilitate and enhance an evaluation. Online surveys, for example, can be used to rapidly poll entire communities with little to no expense. Google Hangouts, FaceTime, and Skype are used for a personal touch, maintaining communication while away from the community or when trying to reach people in remote locations. Google documents, sheets, and slides can be used to encourage collaboration, improving the quality of data collection and reporting.

Blogs are used to record ongoing group or community activities, which can be analyzed and used to document community or project progress and dynamics. YouTube can be used as data to analyze videos produced by the community; identifying community concerns, interests, and priorities. They can also be used to build local evaluation capacity. It is a free training tool.

Infographics and data visualization are recognized as instrumental tools in evaluation. They contribute to effective communication and impactful reporting.

This workshop will be hands-on and experiential in nature. In addition to the tech tools discussed above, the group will share and experiment with tech tools recommended by workshop participants.

Online Courses

Sustainable Development Goals (SDG) Evaluation: What We Need and Why We Don’t Have It Yet

Deborah Rugg, PhD

The adoption of the 2030 Agenda for Sustainable Development and the important and visible role that the agenda assigns to evaluation to inform the agenda’s follow-up and review process will require efforts to further strengthen evaluation practices in developed and developing countries alike. The aspirational nature and interconnectedness of many of the agenda’s targets will require those conducting evaluation, as well as those commissioning evaluation, to have a thorough understanding of the agenda and its goals, targets and indicator framework. This workshop will describe the background of the development of the Sustainable Development Goals, why it is unprecedented, some current approaches to “SDG-responsive evaluation” and why we have a very long way to go. Consideration of a rights-based approach to performance-based versus systems-based approaches will be highlighted. We will conclude with a discussion of the current status of evaluation as a strategic tool to help achieve the SDGs, and what we now need to do better.

Participants will specifically learn the following:

What are the SDGs, how did they develop, why are they unprecedented, how did evaluation come to be included, and what are the current challenges?
How are the SDGs being monitored and evaluated nationally and globally? What are the SDG targets and indicators and how do they relate to evaluation?
What is SDG-responsive evaluation, and what are some of the different approaches, such as performance-based vs. systems-based approaches?
What is the status of evaluation in the current global context? What is the 2020 Global Agenda for Evaluation? What is needed now, how to learn more and become involved?