December 21, 2020
As 2020 draws to a close, we are reflecting on the year that marked CCHE’s 30th anniversary of conducting community-based evaluations. We started off by compiling and sharing with you an overview of our project work and evolution through this interactive timeline. Throughout the year we shared learnings and highlighted some of the tools and resources that we have created, including population dose, the Measuring What Matters evaluation capacity building curriculum, our coalition model, and policy spectrum. We also highlighted some of our recent work and results from our evaluations on initiatives that build quality improvement capacity in the safety net, enhance the skills of public health leaders, support trauma- and resilience-informed care, and confront systemic racism.
This year pushed us in new ways as we shifted to an entirely virtual work environment, adapted our evaluation plans to reflect the needs of communities as they respond to COVID-19, and made explicit our commitment to racial justice. We are pleased to share our reflections in this brief, Evaluating Ourselves, which shares some of our resources and the lessons that have shaped how we do our work, including:
Looking ahead, we remain committed to living these values. We commit to showing up with curiosity, being willing to adapt, and having the patience to listen. We believe this approach to evaluation is critical to ensure that evaluation findings can be used to drive changes toward a better world.
December 7, 2020
Foundations, government agencies, and nonprofits recognize that public and organizational policy change is a pathway to sustainable improvements in community health. However, it can be challenging to evaluate the impact of broad, complex policy initiatives. To help understand and untangle the dynamic and fluid aspects of policy and advocacy evaluations, CCHE has developed the policy spectrum, a tool that evaluators and funders can use to inform their investments and assess progress across multiple sites and myriad aspects of complex initiatives.
CCHE’s policy spectrum:
Learn more about the policy spectrum framework and how we've used it to support policy and advocacy initiatives.
November 13, 2020
The dual pandemics of racism and COVID-19 have underscored the urgency of effectively responding to trauma and promoting resiliency in pediatric primary care. At the same time, research about what works is still in process and best practices have not yet been standardized. Since 2017, CCHE has contributed to learning in the field by evaluating programs that support health care organizations in adopting practices to be more healing-centered, including screening and responding to adverse childhood experiences. In doing this work, we’ve learned about the importance of meeting teams where they are in their journeys to becoming healing organizations and the opportunities for transformational change within organizational cultures.
We discovered that becoming a healing-centered organization requires complementary efforts. This starts at the top with meaningful support and buy-in from organizational leadership, and continues with commitment across the organization, from front-line staff to clinical care teams to custodial and security staff. It requires shifting the way people conduct their work at every level from the visible clinic environment to the way care is delivered within and beyond the clinic walls.
The evaluation of the Resilient Beginnings Collaborative (RBC) was an opportunity to capture lessons and promising practices from those at the forefront of this work in the San Francisco Bay area. RBC is a partnership between the Center for Care Innovations (CCI) and Genentech Charitable Giving and is part of Genentech’s Resilience Effect initiative. The two-year program launched in June 2018 and supported seven safety net organizations in strengthening their capacity to address childhood adversity and promote resiliency in pediatric care. The foundational work of RBC strengthened the participants’ ability to respond to the traumas of 2020 from the COVID-19 global pandemic and police violence towards Black people to the wildfires that raged through several RBC communities. The experience and lessons of RBC are contributing to a new program funded by Genentech and implemented by CCI called the Resilient Beginnings Network, which is an opportunity to expand and deepen this work.
For more information about what we learned in RBC, read our final evaluation brief.
October 8, 2020
Over the past 30 years, one theme that has emerged is the importance of dedicating time for reflection to drive learning and improvement. Reflection is important at CCHE not only for our personal health and wellbeing, but also for sparking insights for improving our work with each other, with you, and communities.
For CCHE’s annual retreat in September, we reflected on our 30-year anniversary. We examined themes from interviews with CCHE founders and harvested additional reflections from current staff about our evolution, culture, approach, and impact. We reflected on such questions as: What projects or moments would you call out as being significant to CCHE’s learning? What lessons would you highlight about our approach, community orientation, or commitment to social justice? What stories would you highlight about how CCHE’s work has an impact? Here are a few quotes from staff:
“In one initiative, we provided individualized evaluation coaching to the grantees to build a logic model and evaluation plan. In my first meeting with one organization, the Executive Director was dismissive, referring to “illogic models.” I continued to work with others on the staff for several years and watched their capacity and comfort with evaluation planning increase significantly. At the end, they noted that they had greater success when their grant proposals included a well thought out logic model and evaluation plan. Through this I learned the importance of long-term partnerships to build sustainable evaluation capacity.”
“We partnered with a community health foundation to develop criteria for and award $1 million to several organizations to improve access to care for kids in our state. After the grants were awarded, we worked with the grantees to develop simple evaluation plans and helped them document their progress and impact. It was a fun project and I got to meet some wonderful people across the state, and learned a lot about working with communities.”
“Early in my career at CCHE, I was so encouraged to be enabled by CCHE leadership to spend significant time in communities, just visiting people. This emphasized the importance of relationship building and trust, and illustrated what shared decision making and a truly participatory evaluation looks like.”
CCHE’s culture and our work is shaped by our people. This year, we welcomed four new staff to our team: Tia Vue, Crystal Dinh, Abbie Lee, and Natasha Arora.
September 11, 2020
Communities know they need to collaborate – that no organization acting alone can hope to “move the needle” on complex systemic issues. In our work, we see both the recognition that collaboration is essential and also a desire for greater understanding of what it takes to work effectively together.
In response, CCHE developed a collaboration model that is grounded in our 25 years of experience evaluating multi-sector collaborative initiatives. The model includes six essential elements for effective collaboration and defines indicators of success for each element:
Learn more about the collaboration model and see how CCHE has used it in evaluation to support collaboration across the country.
July 23, 2020
We've learned over the past 30 years about the central role that health care safety net providers (e.g., Federally Qualified Health Centers, free clinics, public hospital systems) play in supporting the health and well-being of people living in underserved communities. As a result, we have committed to partnering with funders and other organizations on initiatives that support safety net improvements and innovations to improve community health.
Our evaluations are designed to identify promising practices that support peer learning, provide timely feedback for initiative improvement, and use data to inform future investments. Currently, we’re seeing our safety net partners stretch in new ways as they build capacity and systems for delivering virtual care and build or maintain a strong focus on equity and reducing disparities in access and health outcomes.
For more than 10 years, CCHE has partnered with Kaiser Permanente’s (KP) Community Health Programs to evaluate its safety net initiatives. We recently released a new report on the impact of PHASE (Preventing Heart Attacks and Strokes Everyday). PHASE is a long-term initiative in Northern California that focuses on spreading a KP evidence-based, population management approach for patients most at-risk for heart attacks and strokes. Through PHASE, KP has provided grants and technical assistance to its safety net partners since 2006. PHASE currently engages six health center organizations, four regional clinic consortia representing 32 health center organizations, and four public hospitals from 18 counties across Northern California.
Through our evaluation of PHASE over the past decade, we have learned that:
These lessons have guided PHASE in its current pivot to supporting the safety net’s ability to care for patients at risk of heart attacks and strokes while responding to COVID-19.
June 5, 2020
At CCHE, we join our communities in mourning the murders of George Floyd, Breonna Taylor, Ahmaud Arbery, and far too many others across our country. CCHE stands in solidarity with people in this country who have experienced injustice for far too long. CCHE recognizes that police violence against black, indigenous, and people of color is a symptom of deeply rooted institutionalized racism and a systemic problem that we cannot ignore.
In quality improvement work we often talk about how “every system is perfectly designed to get the results it gets.” The painful reality is that our systems are achieving exactly what they were designed to do—protect the privileged at the expense of everyone else. Racism is a well-documented, well-researched public health crisis, and we at CCHE have a responsibility to speak up and be part of the solution. Community leaders and advocates have tried for decades to mend our systems, and that’s not working; we need to dismantle and rebuild these systems if we want to truly interrupt institutional racism at its source.
CCHE is committed to equity and we believe that equity needs to be at the center of all public health and health care investments if we want to improve the health and well-being of everyone. As evaluators, we recognize that we have power and influence to advance the conversation and prompt action to rebuild more equitable and just systems. We recognize that while we don’t have the answers for the best way to do this work, we haven’t pushed ourselves and our partners enough to think about and prioritize equity. We will make mistakes as we try to do better at living our values— knowing it will be difficult and awkward at times does not mean we should choose not to act.
We are committed to deepening our reflection, learning, and efforts to use our power and privilege to name the inequity and racism that we see in the systems around us and contribute to building more just systems. In our evaluation work we will:
We have a lot of work to do. It won’t be easy, and we are grateful for our community health partners going forward.
Photo by Maria Oswalt on Unsplash
May 28, 2020
Over the past 30 years, one consistent evaluation theme that emerges is the importance of leadership. A strong program or initiative without a strong leader and leadership buy-in cannot achieve its goals. Leadership can manifest in many different ways and sometimes comes from unexpected people or entities who have a commitment and passion for the work.
For nine years, CCHE has been partnering with the Public Health Institute’s Center for Health Leadership and Practice to evaluate its National Leadership Academy for the Public’s Health (NLAPH). NLAPH is designed to build leadership capacity and strengthen multi-sector collaboration for community health initiatives throughout the United States. Using skills and tools learned through NLAPH, participants are better able to support the systems changes necessary to improve community health and achieve health equity.
Within eight NLAPH cohorts, we’ve found that the program has created significant and sustainable changes in the leadership of its participants—enabling them to grow as individual leaders, develop as a team, and make meaningful progress on a project in their communities. A survey and interviews with alumni showed that most are still using skills and tools from NLAPH and feel that participation positioned them to take on and lead other community health improvement efforts. Read more about the lasting impact of participating in NLAPH in our alumni report.
We’ve also learned about which qualities of individuals and teams position them to have a positive impact on the health of their communities. These include having individuals that are mission-driven and committed to the work and team members who were in a position to navigate power dynamics and political processes to make change. Learn more about the factors that influenced teams’ ability to have an impact at the individual, team, and community level in our report on the impact of NLAPH.
May 5, 2020
During our 30 years of evaluation experience, CCHE has seen time and time again that many organizations don’t have the tools they need to meet funder requirements for showing results or to understand whether their programs are achieving their goals.
In response, 23 developed the Measuring What Matters curriculum and toolkit to support nonprofits, community collaboratives, and funders who want to understand how their programs are making a difference and communicate the results.
Based on CDC's Framework for Program Evaluation in Public Health, our self-service toolkit breaks down the six key phases of evaluation into understandable steps that nonprofits can use to see if they are making progress, learn how to improve programs in real time, and share results with their stakeholders, including funders. Each section includes templates that users can adapt for their own programs and initiatives. CCHE also uses the toolkit for in-person or online training to build evaluation capacity with organizations that want to dig in deeper.
We created Measuring What Matters because we believe that strong evaluation capacity is a cornerstone of building the community infrastructure that organizations need to promote equitable solutions to some of today’s most complex problems.
Download the complete toolkit or individual components from the CCHE Measuring What Matters page.
April 8, 2020
One thing we’ve learned at CCHE over the course of 30 years, is that change happens when communities come together and partner to identify and implement solutions that fit for their community. Because of this power of coming together, we find ourselves missing the time and space to be in-person connecting, learning, and innovating with all of you.
We also know the negative impacts of crises are not born equally or equitably, particularly by people of color and those who are low income, which makes our work together to improve health equity, during this pandemic, more important than ever.
We see these two themes—the benefits of coming together and the call to address health equity—converging in our work to evaluate SPARCC, a complex systems change initiative, recently published in a report on SPARCC’s blog. SPARCC has always been about building relationships and connections, and that shows up in effective collaborations in its six sites. Since its formation, SPARCC has had an ambitious goal—help shift decades of racial discrimination in housing and community development, affecting equity, health, and climate resilience.
February 25, 2020
Community health initiatives seek to improve policies, programs, and neighborhood environments. Over the past 30 years, we’ve learned that we need multiple strategies to combat complex problems. These can include focused programs targeting relatively few people, such as classes on how to cook healthy meals, combined with policy and environmental ones that reach everyone in a community, like increasing the amount of healthy food in grocery stores. How can we combine the effects of different strategies to strengthen their overall impact?
Dose methods allow us to add up the impact of very different strategies using a common yardstick. Dose combines reach—the number of people affected by a strategy—and strength—the degree to which those people change their behavior.
Using dose we can compare, for example, building sidewalks to increase walkability (high-reach, low-strength) to a daily walking group (low-reach, high-strength). Dose also helps focus attention on ways of increasing effect of initiatives—greater scale (reach) and greater impact (strength) on each person reached. We’ve distilled our learnings in a toolkit that can be used to plan and evaluate complex community initiatives, particularly those focused on healthy eating and active living.
Learn more about dose and download the dose toolkit
January 31, 2020
This is a landmark year for us—the 30th year since we began evaluating community-based health initiatives. It’s been a wonderful journey partnering with an incredible diversity of communities, organizations, funders, and others to further our collective mission of improving community health.
We began in 1990 with a contract awarded to the Group Health Center for Health Promotion to evaluate a national health promotion initiative. Through this and subsequent evaluations, the evaluation team, which would later form CCHE, began learning about what works to conduct a meaningful evaluation of community health improvement strategies.
Along the way we have shared what we were learning with the field about the importance of engaging stakeholders in all phases of an evaluation, focusing on realistic outcomes that matter and can be expected to change during the course of the initiative, using data collection methods that balance rigor and burden on participants, andregularly sharing back information and insights with all partners.
This continuous learning informs how we approach evaluation. As we reflect on our 30 years of work, we want to thank each of you for your partnership in improving the health of our communities.
Over the past 30 years, we have:
We look forward to continuing to learn with you the best ways to improve community health.