What gets measured gets done: The New Zealand Resilience Index
11/12/2018
By Dr Joanne Stevenson, Chris Bowie, Ellie Kay and Dr John Vargo
The New Zealand Resilience Index will help decision-makers stay on track as we find ways to build disaster resilience in our communities, environments, and economies. This index, developed by the Resilience to Natures’ Challenges Trajectories Toolbox, is part of the holistic national strategy to ensure long-term community wellbeing despite inevitable disruption and environmental change.
Measuring resilience helps make an abstract concept visible. Once resilience is visible, we can do something about it. As noted in the National Disaster Resilience Strategy(NDRS) currently out for public consultation, “Holding ourselves to account is paramount,” and part of this accountability is systematically measuring and monitoring progress (NDRS 2018, p.30). The Trajectories Toolbox team, led by Dr Joanne Stevenson, a principal consultant with Resilient Organisations Ltd., has worked along-side the development of the NDRS to create a composite indicator of resilience that conceptually aligns with the multi-capital approach laid out in the strategy.
The New Zealand Resilience Index (NZRI) uses quantitative indicators to assess place-based communities’ resilience across the six ‘capitals’ that compose society: social, cultural, economic, built, natural environment, and governance/ institutional capital.
To enable comparisons between places and across time, the data for the NZRI needed to be quantitative, consistently available at a standard geographic area, and collected at regular time intervals. To make the index manageable and cost effective in the long-term, the researchers decided to aim for cost-free secondary data (i.e., data collected for another purpose like the Census) that is published publicly. Repeatability allows researchers, decision makers, and communities to track progress, assess whether interventions are making a difference, and check with the indicators can predict post-disaster outcomes.
The data for the New Zealand Resilience Index needed to be available at the territorial authority or census area unit (CAU) level. CAUs were selected as the preferred ‘place’ as they are likely to best reflect a community of people and organisations at a local level. CAUs generally coincide with suburbs and have a population of 3,000 to 5,000 people. These areas are also small enough to identify the many differences between places and groups of people that shape the community’s resilience to disruptions and could be addressed by community-based interventions. After a rigorous selection process, including the development of a ‘bank’ of over 1,000 indicators drawing on indices from around the world, we identified a number of common resilience concepts to focus the NZRI indicators within.
The concepts are as follows: community networks and sense of belonging, economic sector diversity, household capacity to cope with economic disruption, networked infrastructure resilience, functionality and safety of buildings following a disruption, household emergency preparedness, community access to shelters and welfare centres, heritage and culture is valued and preserved, resilience capacity of individuals, quality of legislation and planning addressing hazards, health system response capacity, and availability of natural buffers to hazards. Selecting indicators that measure each of these concepts using available data and then combining these into a single composite indicator allows us to create a relative resilience ‘score’.
Not all indicators are created equal. We realised that some indicators will have a more significant impact on a communities’ ability to survive and thrive in the face of disaster than others. As a result, the team employed an approach called conjoint analysis, which allowed us to determine weights for the different indicators of resilience using expert knowledge. To do this, we employed a novel survey software developed in New Zealand called 1000 Minds. Experts in New Zealand’s disaster resilience across theory and practice engaged in the 1000 Minds exercise making trade-offs between various indicators to reveal their ‘preferences’ for certain indicators over others. Research over the last 30 years has shown that people generally have a poor understanding of how they make trade-offs during decision making, making it useful to have a structured way to help us make consistent judgements when considering multiple objectives or criteria. 1000 Minds provides such a structure, using the PAPRIKA (Potentially All Pairwise RanKings of all possible Alternatives) method to rank all pairs of of the criteria being considered. 89 experts from across New Zealand with extensive knowledge across a range of specialist areas from social science to engineering completed the 1000 Minds exercise.
The results of the 1000 Minds analysis show that networked infrastructure resilience along with community (inter-personal) networks and long-term residency, used as a proxy for place-attachment, are likely to have the biggest influence on a place’s disaster resilience. The smallest influence on disaster resilience is attributed to heritage preservation. The results revealed, however, that all of the indicators will have some influence on resilience.
The research team has spent considerable time and effort considering other datasets and exploring ways to assess other dimensions of disaster resilience. For example, two indicators were included in the 1000 Minds exercise that are not included in the current baseline calculation of the NZRI. The proportion of total commercial prone buildings that are considered “earthquake prone” was considered an important indicator of resilience of the built environment, but the best dataset available does not have adequate coverage for the entire country. Similarly, we asked experts to rate the importance of sheltering and welfare centre data availability as a proxy for local civil defence and emergency management preparedness and resourcing. This dataset has been omitted from the calculation of the NZRI due to incompleteness and significantly different policy approaches to the provisioning of civil defence centres by local and regional authorities.
There are many high-quality datasets collected and maintained by local and regional authorities, and the team is exploring ways to integrate additional datasets into the national NZRI baseline.
Going forward we will finalise the statistical analysis of the index, circulate the initial results to our co-creation partners at MCDEM, Wellington Regional Emergency Management Office, and the Auckland Emergency Management – Auckland Council. We will then work with these co-creation partners to understand how the calculated NZRI aligns with their experiences, and plan how we can better integrate local data into the national baseline. A wider release of the results will be coordinated with MCDEM in 2019.
The New Zealand Resilience Index will be a useful addition to New Zealand’s resilience building took kit, but we must be careful about how we interpret and use it. It will not, for example, be a good tool for making definitive choices about funding allocations. It is, however, a useful tool for ensuring that we think beyond buildings and consider the way our communities, institutions, and the natural environment contribute to resilience. Taking a holistic view of resilience will improve our chance of delivering resilience building programmes that account for the many interactions that make up our society.
