August 17 – 25, 2022

Daily Schedule

9:00 am PT: Workshops Begin
10:30 am PT: Break
12:00 pm PT: Lunch Break
2:30 pm PT: Break
4:00 pm PT: Workshops Conclude

All workshops will be conducted online. Please note that all times listed are in the Pacific time zone. Each workshop lasts one full day, from 9:00 am to 4:00 pm. 

2022 Workshops

Wednesday, August 17

Thursday, August 18

Friday, August 19

Monday, August 22

Tuesday, August 23

Wednesday, August 24

Thursday, August 25

Workshop Descriptions

Wednesday, August 17

Introduction to Evaluation & Applied Research
Stewart I. Donaldson
and Christina A. Christie

headshot of Stewart Donaldson headshot of Christina Christie

This workshop will provide participants with an introduction to core concepts in contemporary evaluation and applied research practice. Key topics will include the various uses, purposes, and benefits of conducting evaluations and applied research, basics of validity and design sensitivity, evaluation theory, theories of change, and the strengths and weaknesses of a variety of common applied research methods. In addition, participants will be introduced to a range of popular evaluation approaches including the transdisciplinary approach, theory-driven program evaluation, experimental and quasi-experimental evaluations, culturally responsive evaluation, equity-focused evaluation, appreciative and strengths-focused evaluation, empowerment evaluation, utilization-focused evaluation, and developmental evaluation. This workshop is intended to provide participants with a solid introduction, overview, or refresher on the latest developments in evaluation and applied research, and to prepare participants for intermediate and advanced level workshops in the series.

Recommended reading:

Donaldson, S. I. (2022). Introduction to theory-driven program evaluation: Culturally responsive and strengths-focused applications. Routledge (Chapters 1-3).

Questions regarding this workshop may be addressed to stewart.donaldson@cgu.edu or tina.christie@ucla.edu

Thursday, August 18

Qualitative Methods
Kendall Cotton Bronk

headshot of Kendall Bronk
This workshop is designed to introduce you to qualitative research methods. The session will focus on how qualitative research can be effectively utilized in applied research and evaluation contexts. We’ll talk about how to devise qualitative research questions, how to select purposive samples, and what kinds of data to collect for qualitative investigations. We’ll also discuss a few approaches to analyzing qualitative findings, we’ll explore strategies for enhancing the validity of qualitative studies, and we’ll discuss the types of claims qualitative researchers can make based on their methods. Finally, we’ll dedicate time in the afternoon to addressing specific issues class participants are having with qualitative work they’re currently doing or plan to do.

Questions regarding this workshop may be addressed to kendall.bronk@cgu.edu.

Note: Please be advised that this workshop will NOT be recorded.

Longitudinal Research Methods: Building and Maintaining Participant Commitment
Anna Woodcock

Many psychological and behavioral processes unfold over time, necessitating longitudinal research designs. Spanning a week, month, year, or decades, longitudinal research poses a host of methodological challenges, foremost of which is participant attrition. This workshop introduces the Tailored Panel Management (TPM) approach to explore how psychological research informs recruitment and retention strategies in longitudinal studies. Using examples and case studies from more than a decade of research, we will focus on practices regarding compensation, communication, consistency, and credibility that promote sustained commitment to longitudinal research participation. This workshop is intended to provide an in-depth understanding of the TPM approach that participants can apply to future longitudinal research projects.

Recommended reading:

Estrada, M., Woodcock, A., & Schultz, P. W. (2014). Tailored panel management: A theory-based approach to building and maintaining participant commitment to a longitudinal study. Evaluation Review, 38, 3-28. doi: 10.1177/0193841×14524956

Questions regarding this workshop may be addressed to anna.woodcock@cgu.edu.

Friday, August 19

Quasi-Experimental Methods
William D. Crano

headshot of William Crano

Quasi-experimental (QE) methods often are an afterthought for many trained largely or exclusively on experimental methods. This can prove a limiting factor in their research, as many important issues do not transition well into the experimental space. QE methods open the field to a host of issues that simply cannot be stuffed into experimental designs. One supposed weakness of QE research is its defining feature, which is that such designs never involve random assignment, and so cannot support causal inferences. The first part of the statement is true, the second may not be true. In this workshop, we will discuss research designs using non-random participant samples that produce results as instructive causally as any true experiment could – and that could not have been accomplished in experimental settings. In other instances, the QE design will not produce bullet-proof evidence of causation but will show how the results provide strong insights into probable causes that encourage the more precise (if less generalizable) lens of the experiment.

In our exploration of QE designs and research, we will discuss basics of all good science, which involve the quality (reliability & validity) of measures, a comparison of “true“ vs. quasi-experimental designs, important threats to internal validity in QE designs (& how to minimize them), and the regression artifact, a special problem in all non-experimental research. We will then move to what I call “primitive” QE designs, case control (or ex post facto) designs, slightly less primitive designs, matching and propensity analysis, interrupted time series analyses, and regression/discontinuity analysis. Throughout, we will consider interesting and difficult problems to help expand your views regarding the utility of QE methods in your own research lives.

Questions regarding this workshop may be addressed to william.crano@cgu.edu.

Applications of Correlation and Multiple Regression: Mediation, Moderation, and More
Dale E. Berger

Dale Berger

Multiple regression is a powerful and flexible tool that has wide applications in evaluation and applied research. Regression analyses are used to describe relationships, test theories, make predictions with data from experimental or observational studies, and model complex relationships. In this workshop we’ll explore preparing data for analysis, selecting models that are appropriate to your data and research questions, running analyses including mediation and moderation, interpreting results, and presenting findings to a nontechnical audience. The presenter will demonstrate applications from start to finish with SPSS and Excel. In recognition of the fact that it is difficult to remember everything in a presentation, participants will be given detailed handouts with explanations and examples that can be used later to guide similar applications.

Questions regarding this workshop may be addressed to dale.berger@cgu.edu.

Approaches and Methods for Promoting DEI in the Workplace
Gloria González-Morales and Jennifer Feitosa

Every year the Society of Industrial-Organizational Psychology surveys its members to highlight the top ten workplace trends. Not surprisingly, diversity, equity, and inclusion (DEI) has consistently been considered one of the top #3 workplace trends. With that in mind, we will be addressing this exciting and evolving topic using a critical perspective. We will work together to leverage our resources -diverse identities and perspectives- to appreciate the complexity of DEI issues and how they apply to evaluation, research, and practice.

This workshop will start by providing an overview of conceptualizations and operationalizations of DEI in organizations. In addition, we will draw from multiple organizational and DEI frameworks to understand the theory, concepts, and methods of DEI measurement. In the second part of the workshop, we will dive into the practice and evaluation of DEI with applied scenarios and activities. Lastly, participants will discuss the limitations and challenges of measuring DEI along with opportunities to integrate DEI in our evaluation practices moving forward.

Questions regarding this workshop may be addressed to gloria.gonzalez@cgu.edu or jfeitosa@cmc.edu

Note: Please be advised that this workshop will NOT be recorded.

Monday, August 22

Designing Well-being Interventions & Programs
Stewart I. Donaldson

headshot of Stewart Donaldson

This workshop is designed to provide participants with an in-depth understanding of how to practically design evidence-based well-being interventions and programs. We will begin by reviewing the accumulated scientific evidence on the efficacy and effectiveness of well-being interventions and programs provided by the Centers for Disease Control and from more than two decades of the scientific study of positive psychology interventions. This research will be placed in the context of a relatively new framework, PERMA+4, which summarizes the building blocks of well-being typically targeted including physical health, positive mindset, creating positive environments, economic security, positive emotions, engagement, relationships, meaning and purpose, and achievement. You will learn how to design interventions and programs relevant for addressing contemporary topics and societal issues through mini-lectures, small group discussions, and by designing an innovative well-being intervention or program based on your personal and/or career interests.

Recommended reading:

Donaldson, S. I., Cabrera, V., & Gaffaney, J. (2021). Following the science to generate well-being: Using the highest quality experimental evidence to design interventions. Frontiers in Psychology. DOI: 10.3389/fpsyg.2021.739352. (Special Issue on Positive Psychological Interventions Beyond Weird Contexts: How, When, and Why They Work)

Questions regarding this workshop may be addressed to stewart.donaldson@cgu.edu

Tuesday, August 23

Bibliometric Methods
Becky Reichard

headshot of Becky Reichard

One rite of passage as an emerging scholar, including Ph.D. students, is to write a review paper on a given topic or field. This task is daunting, and for a good reason. With over 90+ million entries in the web of science, living in the information age is a gift. Still, the sheer amount of information we must sift through can be incredibly overwhelming. During this process, you may find yourself wondering questions such as: what are the most impactful papers in this field? Which authors are folks listening to the most? Since I’m relatively new to this field, how can I ultimately be sure that I have not missed any essential work? Despite facing such questions, emerging scholars must still find a way to orient themselves and quickly become knowledgeable in our chosen field of study. Thankfully, due to advances in technology developed to help us handle big data, we now have a new, easy-to-use, and robust analytical tool to help us with this task. That’s where this workshop comes in. Bibliometric methods provide us with an efficient way to cut through the clutter and objectively identify the key papers and authors in a particular field. In essence, bibliometric methods provide a clear, systematic structure for reviewing published literature on any topic.

Not only can this help you in quickly getting up to speed on your research topic, but it can also result in a publishable review paper! Using simple, free software and the Web of Science, bibliometric methods allow us to (1) examine citation indices to identify the essential documents in a particular field and (2) generate a visual network map of how individual publications cluster and relate to one another. The purpose of this workshop is to provide you with an introduction to the various bibliometric methods (e.g., histography, co-citation analysis, and bibliometric coupling). We will review the logic and objectives of bibliometric methods, demonstrate how to use the free software, and help you understand how to interpret the network maps. With the tool of bibliometric methods at your disposal, you will be well-positioned to understand the ongoing narrative within a field of study and craft an innovative, data-driven review paper.

This workshop is for anyone interested in upgrading their ability to understand a particular field of research. Additionally, the workshop is highly recommended for PhD students, who will be aided in learning a valuable tool in assisting their review paper portfolio requirement.

Questions regarding this workshop may be addressed to becky.reichard@cgu.edu

Wednesday, August 24

Survey Research Methods
Jason T. Siegel

Jason Siegel

Dr. Jason Siegel has given his survey writing workshop to large organizations such as NBC TV/Universal and the American Evaluation Association, as well as smaller ones such as Gutcheckit.com and the Energy Center of Wisconsin. He has served as a survey consultant for organizations such as the Bill and Melinda Gates Foundation and the U.S. Department of Labor. His workshop is now going virtual.

Developing surveys is like taking wedding pictures. Everyone thinks it is easy, but it takes an experienced professional to do it right. Creating an effective survey requires a complete understanding of the impact that item wording, question order, and survey design can have on a research effort. Only through adequate training can a good survey be discriminated from the bad.

Dr. Siegel’s one-day workshop will cover all basics of writing and ordering survey items. This workshop will teach you how to create surveys in a way that minimizes error and maximizes the validity of your data. Additionally, the workshop will discuss why it is important to increase respondents’ motivation to provide accurate answers and how easy it is to accidentally de-motive your respondent. Once the workshop is over, you will never look at a survey the same way again.

Questions regarding this workshop may be addressed to jason.siegel@cgu.edu.

Thursday, August 25

Social Network Analysis
Wallace Chipidza

Wallace-Chipidza

This workshop explores social network analysis – a suite of methods for describing, visualizing, and modeling networks of relationships. The workshop has both theoretical and hands-on components. We first present an overview of key concepts in network science and their potential applications. We then explore network data collection, preparation, and visualization. We proceed to investigate networks with respect to i) centrality i.e., asking who occupies positions of prominence within a given network and ii) community detection, i.e., identifying densely connected regions of the network. The workshop emphasizes applications of SNA to social science and evaluation.

Questions regarding this workshop may be addressed to wallace.chipidza@cgu.edu

Contact Us

Claremont Evaluation Center

Claremont Graduate University
175 E. 12th Street
Claremont, CA 91711
lakshmi-krishna-sahiti.bhaskara@cgu.edu