How to choose a cost‐effective indicator to trigger conservation decisions?

Effective biodiversity conservation requires responding to threats in a timely fashion. This requires understanding the impacts of threats on biodiversity and when management needs to be implemented. However, most ecological systems face multiple threats, so monitoring to assess their impacts on biodiversity is a complex task. Indicators help simplify the challenge of monitoring but choosing the best indicator(s) to inform management is not straightforward. We provide a decision framework that can help identify optimal indicators to trigger management in a system faced with multiple threats. The approach evaluates indicators based on criteria spanning monitoring efficiency, management outcomes and the economic constraints for decision‐making. Critical decision factors (or parameters) are identified and detailed in a six‐step process to estimate the cost‐effectiveness of alternate indicators, including threat impacts, sensitivity of indicators to detect change, and the benefits, costs and feasibility of alternative indicators and management actions. Using the Kimberley as a case study, we evaluate 18 indicators for informing management of three key threats in the region: fire and grazing, feral cat predation, and weeds. We show that indicator selection based on our approach can help improve the expected outcome of management decisions under limited resources. By accounting for multiple factors in estimating benefit and costs of monitoring, our approach improves on common approaches that select indicators based only on whether they are sensitive to change and/or cheap to monitor. We also identify how uncertainty in decision factors influences indicator selection. Although cost‐effectiveness analyses are gaining popularity, ours is the first study to integrate multiple selection criteria using a return on investment framework to compare indicators for monitoring multiple threats and triggering management.


| INTRODUC TI ON
Monitoring is an important element in the conservation of threatened biodiversity; it helps demonstrate outcomes, raise awareness, learn about complex environments and trigger management interventions (McDonald-Madden et al., 2010;Nichols & Williams, 2006).
Numerous conservation efforts rely on indicators for monitoring change in ecological systems when dealing with broad objectives for management such as restoration (e.g. Catterall et al., 2012). While the use of indicators in conservation has drawn mixed reviews (e.g. Seddon & Leech, 2008;Sibarani et al., 2019), their practical utility has made them standard monitoring practice in ecological systems (Siddig et al., 2016). A challenge remains, however, in deciding which indicator(s) to monitor to trigger timely and cost-effective interventions for biodiversity conservation. Indicators bear different monetary costs and may provide information capable of changing the course of management decisions in various ways (Hill et al., 2016).
Despite this, the literature on how to select cost-effective indicators for management decision-making is sparse.
A variety of approaches have been used to select indicators for monitoring (Bal, Tulloch, Addison, et al., 2018). For example, biological, physical or social-ecological attributes of the environment may be selected based on their response to environmental change, feasibility of monitoring, ability to communicate observed environmental trends to a broad audience (Barton et al., 2020;Game et al., 2017), public appeal as in the case of charismatic species (Heink & Kowarik, 2010), or based on their cultural significance to a community (Sterling et al., 2017).
Despite the recognised lack of funding for conservation endeavours, few approaches for selecting indicators explicitly consider costs, focusing instead on an indicator's ability to represent the system of concern (Peck et al., 2014). Fewer still consider the management benefits of monitoring indicators, which should be related to the choice of alternative management actions and ultimately improving management outcomes (Bal, Tulloch, Chadès, et al., 2018;Tulloch et al., 2011).
When systems are faced with multiple threats, for example, deforestation, illegal hunting and climate change, monitoring decisions are fraught with additional uncertainty, complexity and trade-offs between indicator choices. In such situations, managers need indicators to inform on the status of multiple threats and when interventions for threat mitigation must be mobilised. Optimal indicator selection needs to be explicit about the management context and the economic constraints on decisions. Such choices are complex, involving multiple decision criteria, and are precisely the situations that require a systematic approach to decision-making (Possingham et al., 2001).
Here, we propose a return on investment framework  to help conservation practitioners choose indicators that can inform management for multiple threats. Our framework estimates cost-effectiveness of alternative indicators based on only a few decision factors (or parameters) that can be elicited from experts or estimated from existing literature or data. These factors span threat impacts, sensitivity of indicators to detect change, and the benefits, costs and feasibility of alternative indicators and management actions.
We show how to combine these factors in a logical six-step process.
We illustrate our framework with a case study from the Kimberley region of northwest Australia, a landscape facing degradation and biodiversity declines due to changing fire regimes, and predation and competition from invasive species (Carwardine et al., 2012). We also test the sensitivity of indicator selection to uncertainty in the decision factors used in cost-effectiveness calculations.

| Six-step framework for indicator prioritisation
Our decision framework comprises of six steps for selecting costeffective indicators for threat management. It prompts the decision-maker to explicitly state (1) what are the threats, their impacts and triggers for management?; (2) what are the indicators for each threat, and how feasible and costly are they?; (3) how likely are the indicators to detect the management triggers?; (4) if management is triggered, how feasible and costly is the implemented action?; (5) what is the expected benefit of management in response to those triggers? and finally to (6) estimate the cost-effectiveness to make a decision (i.e., choose an indicator). In addressing these questions, we identify critical decision factors used in the cost-effectiveness calculations ( Figure 1). This framework allows (a) a given indicator to detect one or more threats; (b) threat management to be triggered by multiple indicators and (c) only one management action per threat (or a suite of actions collectively considered as a single intervention). Below we detail the six steps of the framework and the decision factors (parameters) therein: 1. What threats are operating in the system, what are their impacts on biodiversity and the associated triggers for management?
We first formulate a list of the potential threats, j, which may be acting in a region. For each threat, we establish their impact on 4. Although cost-effectiveness analyses are gaining popularity, ours is the first study to integrate multiple selection criteria using a return on investment framework to compare indicators for monitoring multiple threats and triggering management.

K E Y W O R D S
cost-effectiveness, indicator selection, indicator species, livestock grazing, management, multiple threats, prescribed burning, prioritisation biodiversity, for example, probability of persistence of a species under the threat, and an explicit definition of the level of change (in threat or its impact) that is required to trigger management. Such trigger points or decision triggers are often used to indicate when a system is moving to an undesirable state and inform managers when to intervene to ensure timely actions (Cook et al., 2016). Trigger points may be specified as the level of the threat itself or their impacts on the system and established from long-term monitoring data using methods such as control charts (Morrison, 2008), or based on expert judgement (Martin, Burgman, et al., 2012). Management thresholds used in fisheries (biomass), agriculture (fruit fly numbers) and water quality management (contaminant concentrations) offer guidance on the use of triggers to inform management over short timeframes (Dominiak et al., 2011;McClanahan et al., 2011).
In our framework, we consider the probability of threat j causing an impact of magnitude equivalent to a pre-defined threshold to trigger management, d j . Establishing trigger points explicitly will help favour indicators that can detect threats that are known to have a significant impact in the region (based on prior knowledge).
2. What are the indicators for each threat, and how feasible and costly are they to monitor?
For each threat that is perceived to be acting on the system currently or in the future, indicators, i, for this threat are identified. This may be based on some functional relationship between a state variable of that indicator and the impact of the threat to the broader system. For each indicator considered in the prioritisation, we consider how best to monitor the indicator (e.g. aerial surveys vs. camera trapping for large mammals), the probability that monitoring will be implemented and the costs associated with the monitoring. There may be cases where monitoring is not implemented due to budget constraints, community opposition, lack of political will, conflict or due to the lack of specialised skills (Field et al., 2007;Zamora-Gutierrez et al., 2016). Alternatively, some indicators may be relatively easy to monitor due to well-established monitoring methods and/or availability of experts (McGregor et al., 2013). We capture this as the feasibility of monitoring an indicator, F i , which is the probability that monitoring of an indicator happens. We also specify the relative costs of monitoring alternative indicators, R i . Unlike absolute costs, we expect the relative costs to remain fairly constant if the approach is applied across different regions (Carwardine 2019, pers. comm., 29 March). However, costs are expected to change as technological advances make sophisticated monitoring methods, such as, sensors and drones, more accessible and applicable across many indicators (Lyons et al., 2019).
3. How likely is the indicator to detect the management triggers identified for each threat?
To fully evaluate the benefits of an indicator for triggering management across multiple threats, we must then ask for each indicator-threat combination 'what is the probability that indicator i will detect the trigger (i.e. level of change required to trigger management) for threat j, given that threat j is present?' This probability, p ij , implies that the indicator has adequate statistical power to inform F I G U R E 1 Estimating the expected benefit and cost of monitoring an indicator, i, for a given threat, j, in our cost-effectiveness approach. The framework identifies critical decision factors (or parameters) to evaluate the cost-effectiveness of indicators: d j , F i , p ij , q ij , G j and A j are described in the figure above. R i is the cost of monitoring an indicator relative to the costs of monitoring the other indicators, and M j is the cost of the management action. Here, q ij denotes a Type I error (false positive) and 1 -p ij denotes a Type II error (false negative). Appendix S1 provides the full list of parameters used in the prioritisation decision-making (Field et al., 2004;Legg & Nagy, 2006). There will always be some probability of observation errors when using indicators to detect trigger points for management (Mapstone, 1995), resulting in cases when an indicator either fails to detect a real change in the system (a Type II error, represented as 1 -p ij ) or when an indicator falsely detects a change when there is none (a Type I error, represented as q ij ). q ij denotes a probability parameter and allows us to penalise cases where monitoring an indicator suggests that management of a threat must be implemented when the threat is in fact not present.
The ability of the indicator to detect a trigger point can be estimated through the use of power analysis (Maxwell & Jennings, 2005;Thomas, 1997). Even so, factors like p ij and q ij are not straightforward to elicit because these parameters are conditional on a number of other constituent probabilities of how the system responds.
Mental models (Jones et al., 2011) or Bayesian belief networks may be useful in such cases to represent cause-effect dynamics before expert knowledge is sought to estimate the underlying prior and conditional probabilities (Lauritzen & Spiegelhalter, 1988;Martin et al., 2005). An iterative processes to elicit and update information can also help improve such parameter estimates over time (Hemming et al., 2018).

What management action is triggered, and how feasible and costly is it to implement?
If an indicator triggers management, we represent the feasibility of management for threat j as G j . This is defined as the probability that management is successful in achieving the expected benefit multiplied by the probability that the management is implemented (Carwardine et al., 2019). We assume that a feasible action for threat management (single or a suite of actions) has already been identified using decision tools such as multi-criteria decision analysis (Rout & Walshe, 2013), project prioritisation protocol (Joseph et al., 2009), priority threat management (Carwardine et al., 2019) or optimisation (Auerbach et al., 2014). A management action with low feasibility will decrease the overall benefit from monitoring an indicator that triggers that management. We also define the cost of implementing the management action, M j .

What is the expected benefit of the management action for biodiversity?
The potential or the expected benefit of the chosen management action for threat j, A j , helps gauge the effectiveness of the action in achieving (or coming close to achieving) the management objective. Commonly used management objectives for biodiversity conservation generally seek to maximise some quantifiable measure of system benefit, for example, improving species persistence (Nicholson et al., 2013), growth rates (Maxwell et al., 2015) or abundance (Rout et al., 2009). A j is usually specified as the difference between the outcomes of a scenario with the management action, and the outcomes of the alternative scenario in which the action did not occur (Maron et al., 2013). For instance, this latter scenario, also known as the 'counterfactual', can be indicated by the persistence of a species under a threat as was established in the first step of the process. Although such data may not be readily available, methods to estimate probabilities under counterfactuals and the associated confidence intervals based on expert knowledge are being continually tested and improved upon, as evidenced by existing large-scale prioritisations for threat managements (Brazill-Boast et al., 2018;Carwardine et al., 2012). Establishing such scenarios explicitly through structured elicitation is central to improving outcomes from decision-analytic approaches (see Joseph et al., 2009).

Make a decision
The above questions guide the decision-maker to identify key factors required to calculate the cost-effectiveness of indicators.
We first define the expected benefit, B ij , of monitoring indicator i for threat j (Figure 1) as: d j is the probability of threat j causing an impact of magnitude equivalent to a pre-defined trigger point for implementing management; F i is the probability that monitoring indicator i is feasible; p ij is the probability that indicator i can detect the trigger point for threat j given that threat j is present; G j is the probability that the management action taken in response to threat j is feasible and A j is the expected benefit from implementing management for all species considered.
The expected cost, C ij , is the sum of the cost of managing and monitoring threat j using indicator i (Figure 1): where q ij is the probability that indicator i incorrectly detects the trigger for threat j when threat j is absent (Type I error); R i is the cost of monitoring indicator i and M j is the cost of the management action for threat j.
Cost-effectiveness of indicator i is the expected net benefit of monitoring that indicator across all threats, relative to its expected cost of monitoring, C i for all threats, that is, where ΣB ij is the expected net benefit of monitoring indicator i across all threats and ΣC ij is the expected cost of monitoring indicator i for all threats.
Finally, indicators are ranked in decreasing order of cost-effectiveness, enabling the decision-maker to prioritise based on diminishing returns. Indicators may be selected till a cut-off point after which additional indicators result in <20% change in the cumulative expected benefit relative to the cumulative expected cost of monitoring ('point of diminishing returns'), that is, (2) In this way, our approach approximates optimal indicator(s) selection under specific constraints (here, a threshold for diminishing returns).
Alternately, budget constraints may be used such that indicators are selected until an allocated budget for monitoring is exhausted.

| Case study
We demonstrate this approach for indicator selection using a case study from the Kimberley region of northwest Australia (Figure 2).
The region has a number of endemic wildlife species, many under observed or predicted declines due to multiple threats (Woinarski et al., 2011). We considered three main threats for the Kimberley as per the threat prioritisation study by Carwardine et al. (2012): (a) altered fire regimes and introduced herbivores; (b) feral cats and (c) invasive weeds. To estimate the impact of threats, we considered threatened biodiversity occurring in non-rugged (relatively flat ecosystems used for grazing by pastoral properties) and rugged (ecosystems on rugged sandstone, providing refuge from fires for wildlife) savannahs. Species were grouped based on resource (habitat and food) use, into 12 ecological groups, for example, litter dwelling animals or predators, as groups were expected to have similar responses to threats and their management. We assumed that the management objective for a manager was to maximise the persistence of these target species groups. Information on the best (suite of) management action for each threat, and the probability of persistence for species groups under management of each threat was obtained from Carwardine et al. (2012). Expected benefit of threat management was calculated as the difference in the probability of persistence of species groups with and without management (see Equation S1 in Appendix S3). We listed 18 potential indicators, for example, time since fire or perennial grass cover. Indicator-specific information including probability of detecting change, feasibility and cost of monitoring were obtained from two experts with extensive knowledge on the ecology, conservation and management of the Kimberley following a structured elicitation process (see Appendix S2 for data elicitation spreadsheet) and estimates were validated through feedback and open discussion to ensure consistency (Hemming et al., 2018;Martin, Burgman, et al., 2012). Data used in this study are publicly available (Bal, 2020) and are also included in Appendix S3, along with further details on parameter estimation for the case study.

| Sensitivity analysis
We conducted a sensitivity analysis to evaluate the effect of uncertainty (or error) in parameter estimates on indicator selection.

| RE SULTS
Indicator rankings for the Kimberley show that four out of the six highest-ranked indicators detect one threat only, that is, fire and grazing ( Figure 3). Of these, two indicators, perennial grass cover and weed cover, provided information on an additional threat, that is, weeds, although the probability of detecting this threat is low ( Figure 4a). Interestingly, understorey and ground nesting birds has low cost-effectiveness despite its ability to detect all three threats. All indicators for cat predation, including understorey and ground nesting birds, incur high expected cost of monitoring leading to overall low cost-effectiveness (Figure 4b). Four out of the 18 indicators did not inform on any threats and were removed from the analyses.
Cost-effectiveness of the highest ranked indicator for the Kimberley, namely bare ground cover, is six times that of any of the other indicators (Figure 3). This high cost-effectiveness is driven by the low cost of monitoring bare ground cover, which costs 10 times less than the next best indicator, perennial grass cover, even though both indicators have comparable expected benefit (Figure 4b).
Overall, indicators incurring low relative cost and high feasibility tend to be ranked higher in the prioritisation.

| D ISCUSS I ON
Choosing indicators for informing management decisions is complicated because different indicators vary in how much they cost to monitor, how easy they are to monitor and in the information they provide, naturally leading to the question of which indicators to select. Selection methods need to identify indicators that deliver the greatest benefit per unit cost (Mace & Baillie, 2007;Nichols & Williams, 2006). Although some studies have evaluated the cost-effectiveness of alternate indicator choices (Peck et al., 2014;Tulloch et al., 2011), approaches to do so when indicators need to inform on multiple threats and their management are lacking. In this study, we propose that informed indicator choice when multiple threats are present requires knowing about factors related to the monitoring efficiency, management outcomes and the economic constraints for decision-making (Figure 1). We present a novel approach to systematically integrate multiple factors for indicator selection in a single decision framework that can quantitatively assess competing indicator choices. Applying this approach to the Kimberley, we find that a cost-effectiveness approach is likely to deliver better outcomes than considering individual factors like cost or probability of detecting a change when selecting indicators ( Figure 5).
Our decision framework shows that, all else being equal, an indicator is better if (a) it can provide reliable information on at least some threats in the system (even if not all); (b) it is cheap and easy to monitor and (c) it can trigger feasible management actions with high expected benefits. Though these insights may be intuitive, ours is a logical and quantitative approach to derive the same, hence providing greater confidence in decision-makers' intuition. Overall, we find that the cost-effectiveness of an indicator is a result of the trade-offs between the multiple factors that drive selection. Interestingly, an indicator need not inform on multiple threats to be cost-effective; it can outperform others if it has a high probability of correctly detecting change to trigger management for at least one threat. For instance, four out of the six indicators prioritised in the case study can detect only one threat, that is, p ij of 1 for fire and grazing (Figure 3). Monitoring exotic herbivore scats has lower p ij value but was selected in the prioritisation owing to its high feasibility and low relative cost (see Table S5). It follows therefore that low cost-effectiveness can arise if the benefit of monitoring an indicator is traded-off against another factor such as high cost, for example, exotic herbivore (aerial transect), or low feasibility of monitoring, for example, arboreal animals (spotlighting).
We acknowledge that this approach assumes that all parameters are independent, which may not be true when indicators provide information on the same threat or on one another (Tulloch et al., 2016). To opt for complementary indicators that maximise the number of threats represented in the selection, we can choose selectively down the rankings to pick indicators only if they detect additional threats not already represented in the selection. For Kimberley however, bare ground is likely to be selected despite dependencies because it is significantly more cost-effective than the other indicators, but correlations among parameters may change rankings for some of the other indicators. Indicators may also have shared monitoring costs (Auerbach et al., 2015) when they can be monitored simultaneously, for example, estimating cover for bare ground, perennial grass and weeds, or spotlighting for cats or arboreal animals. Using a greedy algorithm, akin to the current approach, but where expected costs and the cost-effectiveness are recalculated after each new indicator is selected, could be used to account for complementary monitoring costs across indicators (Possingham et al., 2000). F I G U R E 5 Indicator prioritisation for the Kimberley based on costeffectiveness, E i (black line), relative cost of monitoring an indicator, R i (purple line) or the average probability of detecting triggers for all three threats by monitoring an indicator, ∑ 3 j = 1 p ij (green line). Relative (or rescaled) costs range between 1 and 100 representing least and most expensive indicators, respectively. Diminishing returns threshold represents the point after which monitoring additional indicators results in <20% gain in expected benefit relative to the cost of monitoring F I G U R E 6 Change in mean value of four sensitivity metrics (y-axes) plotted against the error level in parameters (x-axes). See Appendix S1 for parameter descriptions. Mean values were calculated using 1,000 realisations for each parameter value obtained from a uniform distribution using Monte Carlo simulations. Coloured lines correspond to the parameter (see legend) being perturbed for the analysis. Dotted line represents the 30% error level (see corresponding Figure S2). Since metric 1 is a correlation, smaller values indicate higher sensitivity, unlike for metric 2, metric 3 and metric 4 where smaller values indicate lower sensitivity Almost all conservation prioritisation tasks are challenged by uncertainty and incomplete information on species responses to threats and management, and the costs and feasibility of decisions (Regan et al., 2005;Williams, 2001). For a decision-making model to be credible, we need to identify the relative contributions of different sources of uncertainty in model inputs to the model output (Norton, 2015;Saltelli et al., 2008). In our framework too, uncertainties in management-related factors, like prior knowledge of threat impacts and the feasibility and benefit of management actions, are more likely to influence decisions than uncertainties in other factors ( Figure 6). This further emphasises the need to consider indicators capable of mobilising management rather than opting for indicators that focus only on information gain (McDonald-Madden et al., 2010) so that critical conservation opportunities are not lost Martin, Nally, et al., 2012). Furthermore, certain sensitivity metrics are more useful than others for decision-making. For instance, in our view, change in the identities of selected indicators (metric 4) is of highest utility as it indicates variability in decisions due to uncertainty. On the other hand, the Spearman rank correlation between indicator rankings across prioritisations (metric 1) is of least utility because it considers change in the lowest ranked indicators as important as change in the highest ranked indicators. This is not sensible because we rarely care about indicators with low cost-effectiveness as these will never be selected for monitoring.
We acknowledge that decisions are not solely driven by cost-effectiveness considerations. Cultural importance of indicators (Sterling et al., 2017), technical capacity of land managers/custodians to implement monitoring and how compelling a decision-maker finds an indicator are an important part of how likely an indicator is to be adopted. We simplify the prioritisation problem by encompassing these elements within the feasibility of monitoring (F i ) such that a feasibility of close to 1 represents an indicator that is considered fit for purpose and assumes that there is good capacity for monitoring it. If decisions around management require involving a range of stakeholders with different values and priorities, as is likely in most cases, then thinking through what information speaks best to desired management objectives as well as a particular audience is important (Game et al., 2017). Selecting indicators within a socialecological context therefore needs due consideration when parameterising the prioritisation analysis. Finally, although we account for risk in decision-making by specifying probabilities for making errors in detection using indicators (Type I and Type II errors), assessing risk preferences of decision-makers within the prioritisation needs further attention. For instance, we may impose thresholds on parameters q ij and p ij , indicating acceptable levels of risk and use these to penalise or remove indicators from the prioritisation when exceeded.
Indicator selection must ultimately be based on what is most relevant to the decision context, for example, accurate information on individual threats, higher return on investment across all threats or prioritising for cultural importance of indicators to engage local communities in monitoring programmes. Our framework provides a logical approach to consider multiple decision factors that can influence optimal indicator selection in a system faced with multiple threats. The simplicity of the underlying analysis makes our approach easily adaptable to real-world conservation decisions that may involve a large number of threats, species, management actions and potential indicators. S.L. and J.C. provided the expert advice on the case study and the data; P.B. led the writing of the manuscript. All authors contributed to the drafts and gave final approval for publication.

PEER R E V I E W
The peer review history for this article is available at https://publo ns. com/publo n/10.1111/2041-210X.13532.

DATA AVA I L A B I L I T Y S TAT E M E N T
R code and the data supporting this article are available at https:// zenodo.org/recor d/4158771 (Bal, 2020).