I’ve been hearing a lot about failure lately: “I don’t want to admit that I have failed”, “Let’s re-frame these results so it doesn’t seem like we failed”.  Most of these comments have come from what I consider to be successful people, running successful programs and businesses.  Why is this conversation coming up about failure?

People are trying new things.  We live in a complex world that requires innovation to adapt and improve our approaches.  It’s courageous to try something new, especially when we’ve received funding and feel like we’re accountable for results.  Innovation requires a certain kind of risk, because we are doing something we aren’t actually sure will work.  If we already new what would work, and if our context never changed, we could confidently continue what we are already going. 

Why is innovation so important?

We live in a changing world with new challenges, complexity, and ever-shifting influences.  Innovation allows us to imagine new solutions.  This requires a lot of leadership and willingness to learn.

Why is innovation so hard?

We may understand the need to change, simply because we know our approach needs to improve or because we can envision a better way.  Knowing change is needed, however, isn’t the same as knowing what to do about it.  Until we try something, we don’t know if it will work: we are operating in a constellation of needs, stakeholders, funding, relationships and other pressures that will impact the best-laid plans.  We can’t know how something will work until we try it.  Trying something new means exposing ourselves to a world of unknowns.  Our challenge is to act with our best information, intentions and approach.  Then we need to reflect, because there will be nuances to our experience that can teach us a great deal about how we might move forward effectively.  Listening carefully is the key to our ability to learn and improve, bringing a clear understanding of what didn’t work forward, just as much as what did work. 

I am much more worried about failure of imagination, failure to act, and failure to reflect, than I am about hearing “this completely failed, let’s learn from it.”  The very reason that we tried something new was to see if it worked.  If it didn’t, let’s not repeat it, and let’s understand why. 

How do we as evaluators create a safe space to talk about failure? It’s a conversation that helps us evolve and grow as a profession.  It’s key to supporting our clients to benefit from their experience; saying something failed shouldn’t be about admitting weakness, it should be about celebrating a new approach and building collective wisdom around how it worked, what didn’t work, and what lessons can be learned and shared.

What can we do to support talking about failure?

·         Create a safe space for the conversation

·         Make it clear from the beginning that learning is the goal

·         Focus on the experiment, not the success or failure of the organization carrying it out

If we knew exactly how to do something, it wouldn’t be innovation. We can create the opportunity to build on our failures through innovation, action, and reflection.

 
 
How Evaluation Saved my Family Christmas

A few weeks ago, my dad sat me down and started asking about whether we’d be having stockings for Christmas, and who would do it, and suggesting that maybe we shouldn’t have stockings anymore.  My sister called me the same night to talk about whether we should have turnips in the mashed potatoes or not.  I could feel the start of it…Christmas drama. 

Instead of engaging in the usual backroom negotiations and lobbying for my preferences, I used appreciative inquiry to think about how I could make this the best Christmas ever.  What do we all really want?  I know that deep down we all want to have a joyful holiday that brings us together to eat good food and enjoy one another’s company.  The worry about what kind of gifts to get or who should cook the gravy using what technique was distracting us from what is truly important. 

I spent some time thinking about the tools I use in my daily work, and I decided to try a Christmas survey.  What better way to find out what everyone wants?  To be successful, this would need to incorporate some important principles:

1.       Full engagement-            Everyone would have to participate to be heard, and the experience would have to be fun to inspire their participation

2.       Transparency-                   The raw results would have to be shared to build trust

3.       Simplicity-                          The survey would have be quick and simple

4.       Actionable recommendations- I’d have to translate the raw data into actionable items that everyone could understand and participate in

Full Engagement

I did my best to make the invitation to the survey fun, and to make the survey itself an enjoyable experience.  I incorporated the following elements:

·         Casual, fun language in the survey invite

·         Explicit statement of goal in the invite: “We want to make this the best Christmas ever”

·         Everyone is asked to click on “I will help make this the best, most fun Christmas ever” at the end of the survey

·         The survey is highly visual with fun and symbolic Christmas images like trees and ornaments

·         The survey questions are short and simple

Transparency

The full results are presented so that everyone can see the raw data.  On top of the raw results, I help with interpretation by clearly showing which choices were most and least popular.  This was important in an environment where one person in a family might speak for another.  For example, in one of the comments, someone said: “Thanks for organizing! Absolutely no purchased items for me or anyone in our family :-) Can't wait for cookie day - I vote sugar cookies”.  Needless to say, the other family members actually voted that they wanted stockings and gifts.  The survey allowed each individual to express their wishes without going through a family representative who might alter their response.  Transparency was key to establishing trust in the process and further supporting buy-in.

Simplicity

The survey has only two questions:

·         How do you feel about the following activities?

·         What do you want to eat?

The scales were simple and fun:

·         Activities:            Love it!    *     Um…maybe.   *     Kill me now.

·         Food:                    Yum J     *     If I must…        *     Gross.

I incorporated images into the survey and the “report” to keep it entertaining. 

Actionable Recommendations

The results were important to share, but in order for people to take the data and act on it, the recommendations needed to be clear.  In the case of Christmas, this took the form of a list of activities, menus for dinner and dessert, and a budget for gifts and stockings. 

Several people made new suggestions in the activities section, so we had a one-question ranking survey sent out to follow up, and we are now starting a new family tradition- we’ll be watching James Bond together.

This is the Best Christmas ever!

I’m happy to say that I’m more convinced than ever that the tools of evaluation can help us to encourage engagement, build relationships, and create a shared sense of purpose and agreed process.  We’ll be having English trifle for dessert, which is my favourite, and everyone has been completely on-task and excited about Christmas.

Happy Holidays!

 
 
Picture
How do you deal with the complexity of collaborating organizations that are on different timelines, with power differentials, and varying levels of data quality?  Krishna Belbase of the Evaluation Office of UNICEF introduced the Resource Pack on Joint Evaluations developed by the UN Evaluation Group, at the CES 2015 conference in Montreal.  He suggested that it is structured for UN agencies, but could be adapted to suit other organizations.  The Resource Pack is a rich resource not only because of the simple yet comprehensive guide it provides for evaluation, but also because of the way it details the governance structures needed to support evaluation in organizations working together on evaluation. 

In today’s world many evaluations are done with some element of collaboration, and the Guidance Document and Toolkit that make up the Resource Pack can be used to help define the key functions, structures, and questions to ask when determining how to govern evaluation. 

The Guidance Document helps tease out the various functions like communication, management, technical input, and logistics.  The Toolkit then walks you through the steps from deciding to work together on an evaluation, preparing for the evaluation, implementing the evaluation, to utilizing the outcomes.  It addresses sticky issues like readiness and buy-in, and provides advice at every stage from developing terms of reference to disseminating findings.

Do you need a steering committee, management group, reference group, stakeholder group, or advisory group?  The Toolkit lays out the considerations for making important decisions about the most appropriate governance structure for your situation.  Overall, the Resource Pack on Joint Evaluations is a great resource for any organization looking to support decision-makers and leaders in structuring their governance, and provides tools such as checklists, examples and good practices to evaluation practitioners.

Check out this amazing resource: Resource Pack on Joint Evaluations