How do you evaluate an experiment?


Inclusive Creativity is a pilot project, which draws on the skills and experience of three partner organisations (Walled City Music, St Magnus International Festival and Drake Music), in order to carve out new directions for us all. Drake’s role in the project is evaluation – to examine, question and articulate the value of our work.

A rainbow disappearing into a cloud, over the runway at Kirkwall Airport, OrkneyThis means that the evaluation is a significant part of the project – weighted equally with the creative elements – and, as such, is also under investigation. Will the evaluation work? Will it be valuable? And what, exactly, is it doing?

In this pilot project, we are focusing on process. We expect there will also be creative ‘products’ –improvised jamming sessions between musicians, new approaches to notation from the composers – but these will arise organically from the quality our working together. How do performers and composers collaborate? How do they work with new instruments and technologies? How do three organisations working at different ends of the United Kingdom, come together over time and distance?

This interest in process also means that, while I have some questions to ask, the framework for the evaluation needs to be created by the people taking part. Rather than define what will happen in advance, I will try to measure the effect of the project by asking people what they expected to happen at the beginning, and comparing this to how they feel at the end.

The first stage of the evaluation process is a questionnaire, asking people to do just this: to write down their own expectations and aspirations for the project. The questionnaire is based loosely on the Theory of Change model, a tool for understanding ideas as events change, which makes it particularly useful for evaluating experimental work.

The second stage of the evaluation process is interviews. These short, informal conversations are designed to delve a bit deeper than the questionnaire, and to find out how it feels to be involved in the midst of the creative work.

There is also a third, informal stage of the evaluation process in the form of this blog,, where Frank has been blogging about the workshops in Derry, and where I hope some of the other participants might want to blog about their experiences.  This is also where I’ll collate and publish my evaluation data at the end of the project.

As a whole, I hope the blog, the questionnaires and the interviews will help us all understand the detail of our personal experiences.  I also hope they will create a culture of reflection and sharing, which will, in turn,  feed from and contribute to the process of working together with openness and innovation. Whether or not this works in practice is yet to be seen – and, indeed, evaluated – at the end of the project.

Which means that in answer to my original question, how do you evaluate an experiment? The simple answer is, of course: experimentally.

Leave a Reply