‘The willingness to experiment, it turns out, is the chief indicator of how innovative a person or company will be.’ (Hal Gregorson)

Test and Learn is an experimental, adaptive technique, used to address complexity, uncertainty and innovation. It’s useful in situations where, say, past experience isn't a reliable guide for future action because e.g. critical conditions have changed. It’s also useful when moving into new, unchartered territory where the evidence needed for sound decision-making can only be generated by, ‘let's suck it and see’. It shares a lot in common with action research: create a tentative hypothesis, step forward, observe the results, try to make sense of them, refine the hypothesis, take the next step.

Test and Learn is used in fast-paced, fluid environments, such as by rapid-onset disaster response teams where conventional strategizing and planning isn't realistic or possible. By the time a detailed plan is formulated, things have moved on - and the paper it's written on is sent for recycling before the ink has dried. Test and Learn is also used by marketing teams when testing new products or services or seeking to penetrate new or not-yet-known markets. It provides tangible evidence based on customer responses which, in turn, enables change or refinement before investing further.

What psychological, relational and cultural conditions enable Test and Learn to work?

1. Leaders who believe and overtly support the idea that learning is a key to success, both from experiments that go well and those that fail.
2. Shared opportunities for ideation, hypothesis formulation, testing of assumptions and critical sense-making: e.g. via action learning.
3. Experiments that are sufficiently specific and observable to draw out key evidence and learning.
   
4. Meaningful recognition, celebration and reward for experimentation and learning, and for the progress that results from it.

When have you used Test and Learn? How did you do it? What difference did it make?

(See also: Unpredictable; Adaptive)

​What’s your theory of change? What issues are you trying to address? What creates and sustains those issues? What kind of interventions and when are most likely to prove successful? What would success look and feel like, and for whom? What is your overall goal? These are some of the questions we looked at on a Theory of Change workshop I took part in yesterday. Theories of change are becoming increasingly commonplace in the third sector, paralleling e.g. strategy maps in other sectors. There are a number of reasons for this. Charities and NGOs are under increasing scrutiny from supporters and funders to demonstrate how their resources are being used to achieve optimal impact. This has created a whole industry in impact evaluation.

The third sector is maturing too. No longer driven into action by empathy or altruistic instinct alone, organisations in this sector  have more experience, more evidence of what works and what doesn’t and more analysis and understanding of why. The issues have turned out to be more complex than some had originally imagined, making significant and sustained progress challenging. Against this backdrop, a theory of change can prove valuable. It aims to clarify goals and outcomes and to work back to activities and other factors that will enable the outcomes to be achieved. In articulating these things clearly and succinctly (often in simple graphic flowchart form), underlying assumptions and causal links can be surfaced, explained and tested.

At heart, a theory of change answers questions such as ‘What are we trying to achieve?’, ‘What is necessary for the goal to be achieved?’ and ‘What’s the rationale behind our intervention strategy?’ In doing so, it makes the organisation’s focus, operations and use of resources transparent, accountable and more open to challenge and improvement as new research and evidence emerges. I find myself particularly drawn to the critical-reflective aspects. For instance, one NGO I worked with conducted a  fundamental strategy review starting with these same principles, asking questions such as, ‘Why are people poor?, ‘What causes and sustains poverty?’, ‘What interventions make the greatest difference?’, ‘What is our optimal contribution?’

One of the interesting challenges for a third sector organisation is whose voice is represented in framing and answering such questions, e.g. donors, beneficiaries, trustees, staff, volunteers. A charitable organisation I work with currently conducted a strategy review recently, inviting feedback from beneficiaries using surveys, focus groups etc. to find out what they struggle with and aspire to and what role they would want to see the organisation playing in helping them address or achieve these issues. The needs and aspirations that surfaced have been summarised as ‘I’ rather than ‘we’ or ‘they’ statements in clear and colloquial language, keeping the focus on what each individual as beneficiary wants to experience as a result of the organisation’s actions.

This is a sharp contrast with some experiences I’ve had in the past. In one instance, a third sector organisation I worked with set up a drop-in project providing advice and support for long-term unemployed people. The Local Authority provided funding using ‘number of people using the service’ as its key success criterion. Paradoxically, the more successful the service was in enabling local people to find employment, thereby reducing the number of people who needed to access the service, the more the service was deemed statistically by the Local Authority to be failing. A theory of change can help surface such outcomes and assumptions at an early stage, enabling more constructive dialogue and agreement between agencies and stakeholders.

I believe the potential for theory of change extends beyond third sector organisations aiming to articulate their vision, strategy, plans and reasons behind them. I’ve used similar methodologies to explore and articulate an organisation development strategy within a third sector organisation. We started by exploring a number of questions with diverse stakeholders and groups such as, ‘What kind of organisation are we trying to develop?’, ‘Where are we now?’, ‘Why are things as they are?’, ‘What drives or sustains how things are?’, ‘What matters most to people here?’, ‘Who or what influences change?’, ‘What would it take to achieve the changes?’ This enabled us to create a map showing goals, activities, assumptions and causal relationships.

The same principles can be applied at team and individual levels too, e.g. for leadership, coaching, mentoring, training and counselling purposes. It enables dialogue between different parties and keeps rationale and assumptions explicit. If assumptions are clear to all parties, they can be challenged and revised in light of different preferences, perspectives, realities and evidence. I’ve used adaptations of this approach with people and organisations where Christian beliefs have been held as important and integral, developing the model as a theology of change. A theology of change may surface and articulate e.g. God’s purpose, values, presence and activity in the world, the role of the Spirit and Christians, discerning a sense of ‘calling’.

In my experience, the language and methods of applying theory or change need to be adapted for different purposes and audiences. It represents a logical-rational paradigm that is likely to work well for some people and cultures but not so well for others. Using Honey & Mumford’s learning styles as one possible frame of reference, theory of change (as the name implies) may appeal most to people, teams or cultures with a theorist orientation. Reflectors may be attracted most by its emphasis on  surfacing underlying assumptions, activists by the evidential dimensions and pragmatists by its focus on outcomes. Perhaps the key lies in using the principles it embodies flexibly and sensitively in the context of real human dialogue and relationship.