I've been collecting stories of Most Significant Change as part of the evaluation of a systems-change initiative in the healthcare sector.  It's innovative, complex and it's the first time anything of this scale is being attempted by my client. 

The Most Significant Change method calls for collecting stories within a certain domain of change, so we've been focusing on the area where there has been the greatest impact since the inception of the initiative in the fall of last year.  Stories have come from physicians, patients, partners and allied health professionals.  Up until now I was recording the stories, but there still wasn't clear buy-in about the process of using stories as part of the evaluation.

The magic happened last week when we went through the story selection process.  For the first time, the Evaluation Working Group had the opportunity to hear the stories of Most Significant Change in people's lives, and they realized what a difference they were making.  Not only did the stories show the incredible challenges of patients seeking care for complex conditions and care providers struggling to support patients who had social barriers clearly impacting their health but outside of the realm of what they could address, but they got to see how the changes came about and why they were significant to the storytellers.  The group discussed at length the significance of each story and it didn't take long before they identified themes in the stories that reflected their original motivation for getting involved in the initiative. 
As the conversation touched on the raw experiences of the initiative, there was an opportunity for deep reflection.

This was one of the most satisfying meetings I've had all year.  There was a "click" where the hard data started to take on faces and experiences, guiding us through the journey of change that has been happening.  The stories illustrated the change in a way that made our survey statistics and care data come to life. 

Here are some things to consider when using Most Significant Change:

1. Stories help illustrate the context.  It's complementary, though, and is most valuable when presented with hard data that shows the bigger picture.
2. Gather stories from a diversity of respondents.  The target interview groups should ideally be identified as part of the evaluation plan.
3. Be ready to facilitate!  The selection process is rewarding but needs guidance to maintain a safe, open space and help nudge the group towards a decision using a process they feel comfortable with.

Enjoy the process!

 
 
Picture
How do you deal with the complexity of collaborating organizations that are on different timelines, with power differentials, and varying levels of data quality?  Krishna Belbase of the Evaluation Office of UNICEF introduced the Resource Pack on Joint Evaluations developed by the UN Evaluation Group, at the CES 2015 conference in Montreal.  He suggested that it is structured for UN agencies, but could be adapted to suit other organizations.  The Resource Pack is a rich resource not only because of the simple yet comprehensive guide it provides for evaluation, but also because of the way it details the governance structures needed to support evaluation in organizations working together on evaluation. 

In today’s world many evaluations are done with some element of collaboration, and the Guidance Document and Toolkit that make up the Resource Pack can be used to help define the key functions, structures, and questions to ask when determining how to govern evaluation. 

The Guidance Document helps tease out the various functions like communication, management, technical input, and logistics.  The Toolkit then walks you through the steps from deciding to work together on an evaluation, preparing for the evaluation, implementing the evaluation, to utilizing the outcomes.  It addresses sticky issues like readiness and buy-in, and provides advice at every stage from developing terms of reference to disseminating findings.

Do you need a steering committee, management group, reference group, stakeholder group, or advisory group?  The Toolkit lays out the considerations for making important decisions about the most appropriate governance structure for your situation.  Overall, the Resource Pack on Joint Evaluations is a great resource for any organization looking to support decision-makers and leaders in structuring their governance, and provides tools such as checklists, examples and good practices to evaluation practitioners.

Check out this amazing resource: Resource Pack on Joint Evaluations


 
 
As planners and evaluators we have an important and influential role in supporting decision-making,  but we are ultimately advisors. How can we exercise leadership in our role while respecting the role of final decision-makers?  I've found a few approaches that help:

1. Help identify decision-makers
2. Facilitate the development of terms of reference, scope of decisions or decision-making criteria
3. Ensure that organizational values are reflected in the plans and evaluation framework
4. Articulate evaluation findings and recommendations in a language and format that meets the needs of decision-makers
5. Time work to accommodate upcoming decisions and information needs

Good governance is important to everyone and we are responsible for contributing to a sound decision-making process.  We can take leadership in strengthening the decision-making process in an advisory role.