 |
Project update: evaluating advocacy campaigns
What is the link between our advocacy activities and policy outcomes? Is it possible to establish a causal relationship between a vote in Parliament and campaign tactics? In mid-2007, two of the Change Agency team initiated the advocacy evaluation project to:
|
 | Collect and synthesise resources and literature on campaign evaluation including case studies | |
Identify evaluation frameworks that are simple, powerful and road-tested | |
Develop a suite of resources for participatory workshops that support activists to evaluate their own campaigns | |
Trial and revise resources | |
Identify opportunities for ongoing action research |
|
Our research has uncovered the excellent work of the Harvard Family Research Program (HFRP). Julia Coffman and her associates developed their composite logic model to help advocates, funders, and evaluators articulate an advocacy or policy change strategy or theory of change. The model comprises five sets of factors: inputs, activities and tactics, interim outcomes, policy outcomes and impacts. By considering each set of factors, advocates are challenged to specify a theory of change. What would success look like? How do we believe we’ll we get there? The HFRP team have a useful set of questions to help advocates engage with the model and develop specificity. These questions readily form the basis for a participatory workshop.
|
The online advocacy logic model tool was developed by Continuous Progress to make the composite logic model more accessible to campaigners. This model allows advocates to select elements from each of the HFRP categories and automatically generates an evaluation framework. The Continuous Progress website is an Aladdin’s cave of resources for evaluating social change campaigns.
|
The most recent edition of their update invited readers to participate in an online survey to share experiences with advocacy evaluation and coalition-based advocacy
|
Justin had previously developed and applied evaluation frameworks to recent campaigns of Australian social movement organisations including the effort to block elements of the Work Choices industrial relations legislation and the Fair Wear campaign to uphold human rights for outworkers. Papers from these evaluations are included in tCA’s collection of articles and case studies, as is an extensive literature review of the field.
|
More recently, Justin has completed an evaluation of the Australian Anti-War movement and its mobilisations prior to the most recent invasion of Iraq. Justin used theoretical frameworks from social movement theory and nonviolent action theory to analyse that campaign in relation to both internal and external factors influencing success. This should be available online soon.
|
James and Sam’s work with the Big Switch climate change campaign created an opportunity to apply and test-drive the HFRP composite logic model. As a prospective (rather than retrospective) evaluation framework, the composite logic model challenges advocates to clarify their theory of change in advance. Justin facilitated a workshop with the campaign team during our initial strategising phase. We started by identifying the campaign inputs: the available organisational capacity, funding, leadership and staffing. A sobering activity, given that our small team were attempting to create a useful contribution to development of national climate change policy within six months with a budget of less than $30,000. We then identified the required preparation and planning: data collection; problem identification; policy identification, goal setting and strategy development. These phases of our campaign required focused time and would impact on our outcomes and impact.
|
Following the composite logic model sequence, we then considered campaign tactics. From the wide range of activities that were possible, we focused on: message and materials development; policy proposal development; media advocacy and outreach; candidate interaction; coalition and network building; voter education and civic engagement; and grassroots organising and mobilising. The Big Switch campaign objectives and preferred policy outcomes had already been developed by a coalition of environment and social justice groups.
|
The next set of factors we considered were interim outcomes. The six-month campaign forms part of the ongoing advocacy of the member groups of the Big Switch coalition, so one interim outcome to focus on was organisational capacity. Success in this campaign would rely on building capacity. We resolved that from the menu of interim outcomes in the composite logic model, we would focus especially on partnerships and alliances, collaboration and alignment; new advocates and champions; media coverage and salience; campaign visibility and recognition; constituency or support base growth; audience engagement and mobilisation; changes in attitudes or beliefs; and increased public will or support.
|
One particularly useful consequence of applying the model early in the campaign’s development was that we were compelled to specify the policy outcomes we would ideally result from our activities. Did we expect to see policy implementation, monitoring and impact, or simply policy adoption? Or was that premature? Were we seeking to put preferred climate change policies on the agenda?
|
During the last few months, we’ve been implementing our strategy. The shared understandings that emerged from first developing an evaluation framework have been so important. They have created a discipline that guides our choice of tactics on a daily basis. If I do X rather than Z, will it move us in the right direction? When we evaluate, will this action have helped achieve our success indicators?
|
As the first phase of the Big Switch is drawing to an end, we’re now applying the evaluation framework we developed together. So we’ll be able to report soon on the experience.
|
Another small step in this project was a workshop that Justin and James led at the NSW environmental educators conference in early November. Our discussions with community educators confirm the interest in and importance of tools for advocacy evaluation.
|
Let us know if you or your organisation are interested in collaborating with us on this project. We plan to develop some useful tools along the way and will be sure to add them to the Change Agency’s online resources.
James and Justin
|
Back to tCA news - november 2007
|
|
Back
|
|
