# Why shouldn't I collect more data? Reconciling disagreements between intuition and value of information analyses

**Handling Editor:**Sarah Goslee

## Abstract

- Value of information (VoI) analysis is a method for quantifying how additional information may improve management decisions, with applications ranging from conservation to fisheries. However, VoI studies frequently suggest that collecting more data will not substantially improve management outcomes. This often contradicts the intuition of ecologists and managers who usually believe new information is critical for management. This inconsistency is exacerbated by the perception that VoI is a black-box method. A lack of understanding as to why VoI is usually lower than ecologists expect is hampering on-ground uptake. There is an urgent need to identify the factors that drive VoI methodology to produce low values.
- Here, we use a rigorous approach to provide insights into why VoI values are often low. We first derive analytic solutions and upper bounds for a VoI problem with two uncertain states, two actions, and four management outcomes. We show how VoI changes with respect to the benefit (i.e. utility) of implementing actions in each state, and the probability the system is in each state. We apply our formulation to a published frog population management case study and extend the results numerically to 10 million randomly generated larger-sized problems.
- Zero VoI occurred half of the time in our two-action two-state simulations, corresponding to when one action is best, or equal best, across all states. Even when VoI values were positive, they were typically low. However, on average, VoI tended to increase with the number of states and actions. Our analytic expression for VoI, in the case where VoI is positive, demonstrates that VoI is characterized by the state probabilities and, the utility gaps, that is the difference in utility of deploying each action in each state. Our derived bounds reveal that, in all two-action two-state systems, VoI cannot be larger than half the largest utility gap.
- Our simple, yet powerful, analysis provides precious insight into the important factors that drive VoI analysis. Our work provides an essential stepping stone towards increasing the interpretability of VoI analysis in more complex settings, ultimately empowering managers to use VoI to help inform their decisions.

## 1 INTRODUCTION

Actions are urgently needed to halt declines in biodiversity (Ceballos et al., 2020). Given limited resources and incomplete information on the effectiveness of alternative management actions, applied ecologists and practitioners face a dilemma about whether to act quickly or wait to collect more data (Canessa et al., 2015). Value of information (VoI) analysis offers a solution to this dilemma. It is a framework for identifying data collection strategies that optimize management outcomes (Schlaifer & Raiffa, 1961). Over the last 30 years, VoI has been applied to many problems in ecological management (reviewed in Bolam et al., 2018), including fisheries (Costello et al., 1998; Kuikka et al., 1999), agricultural pests and diseases (Li et al., 2021; Shea et al., 2014) invasive species management (D'evelyn et al., 2008; Johnson et al., 2017; Moore et al., 2011; Sahlin et al., 2011) and the protection of threatened and endangered species (Bennett et al., 2018; Hanson et al., 2022).

While the results of VoI analyses are context-dependent, an emerging pattern from the literature suggests that, for many ecological systems, there is limited ability for new data to improve management outcomes (e.g. Bal et al., 2018; Hanson et al., 2022; Johnson, Jensen, et al., 2014; Maxwell et al., 2014; Runge et al., 2011; Tulloch et al., 2017). For example, Maxwell et al. (2014) found that to maximize Koala population growth rates across a wide range of scenarios, only 0.1%–1.7% of management budgets should go towards resolving population dynamic uncertainty, whereas 98.3%–99.9% should go directly towards deploying management interventions. Other studies have found similar results managing overabundant populations and harvested species (Johnson, Jensen, et al., 2014; Tulloch et al., 2017).

Such results starkly contrast with the budgets of many of the world's governments and environmental NGOs. A sample of countries across the globe showed half of their threatened species management budgets are allocated towards information gathering and monitoring to improve scientific knowledge rather than on-ground actions (Buxton et al., 2020). As far as we are aware, these agencies have not conducted formal VoI analyses to identify monitoring needs, and therefore, it is unclear if they are overspending on monitoring. However, if they are over-monitoring, it could have harmful consequences. Ten species have gone extinct while being monitored (Lindenmayer et al., 2013). For one of these species, the Christmas Island Pipistrelle (*Pipistrellus murrayi*), excessive spending on monitoring at the expense of quickly setting up a program to breed the last individuals in captivity has been implicated in the species's extinction (Martin, Nally, et al., 2012).

So why are monitoring budgets so high? Why do ecologists and decision-makers believe information gathering is so critical for making good management decisions (McDonald-Madden et al., 2010; Sanchirico et al., 2014), yet many times when VoI analyses are deployed, they often suggest that additional information is unimportant for improving outcomes? One possibility is that people have inaccurate expectations about the role of information gathering and its ability to improve decisions, on average. After all, the vast majority of environmental monitoring budgets are decided without any formal VoI analysis (Maxwell et al., 2014; McDonald-Madden et al., 2010). If VoI analyses were performed, perhaps some money would be redirected from knowledge acquisition to management interventions. Another possibility is that the mathematical construction and assumptions underpinning the VoI framework favour results that indicate information gathering has low value. Finally, low values could be driven by biases in the reported literature. There are indeed case studies where VoI analyses have shown that more data collection can substantially improve management outcomes (e.g. Bouma et al., 2011). Additionally, management objectives are value-based; small changes in a measured benefit may appear numerically small but could be substantial to a manager. Therefore, it is hard to compare VoI across the literature, as variable reporting and differing objectives make results challenging to synthesize and generalize (Bolam et al., 2018). In addition to the lack of empirical understanding on the frequency of low and high values of information, there is also limited mechanistic theory that puts these values in a context that is useful for managers to understand and trust.

In this paper, we step towards a more mechanistic understanding of the VoI framework by (i) presenting the theoretical maximum value we can expect across all VoI analyses in ecological management; (ii) identifying the two archetypal cases, of which only one provides a non-zero expected change in outcomes; (iii) describing mechanistic drivers of high VoI and (iv) performing a simulation study to understand how information gathering could improve conservation management when starting with no information on the ecological system. We first examine a general—yet simple—two-state, two-action system to provide a clear foundation. We then extend our simulations to an arbitrarily large number of possible states and actions to verify that our findings generalize to more complex scenarios. Our results provide a way to transparently understand and interpret VoI analyses and is one step closer to explaining the main drivers between low and high VoI in ecological systems.

## 2 METHODS

In its simplest form, VoI is calculated as the expected value of perfect information (EVPI), which is the expected improvement in management outcomes if all uncertainty were resolved by means of data collection (reviewed in Canessa et al., 2015). Before defining the problem mathematically, it is useful to walk through a biological example to help understand the method.

### 2.1 Motivating example

One of the most well-cited case studies for VoI analysis is on the fungus *Batrachochytrium dendrobatidis* (Chytrid), which has been implicated in the extinction and decline of many amphibian species worldwide (Skerratt et al., 2007). Canessa et al. (2015) present a detailed VoI analysis considering the management of a population of 100 frogs with the objective of maximizing the frog's population size after 10 years (Table 1). We summarize the analysis here to provide a concrete introduction to VoI calculations before presenting the mathematical theory. In (Canessa et al., 2015), half of the frogs can either be translocated to a new area or left in their current site. If the manager does nothing, the frog population is expected to remain constant. If the manager moves half the frogs to a new area, and Chytrid is absent at the new site, the new population will increase by 35 frogs across the two sites. If Chytrid is present, the total population declines by 45 frogs. A manager who tests for Chytrid (assuming the test is perfectly accurate) would know with certainty whether Chytrid is present at the new site. If the manager does not have this information, they can state their prior belief, as a probability, that Chytrid is present. Canessa et al. (2015) consider a 0.5 probability of Chytrid presence. In this example, the EVPI is 17.5 frogs. We illustrate our theoretical findings with this example throughout the paper.

Chytrid absent (50% chance) | Chytrid (50% chance) | |
---|---|---|

Translocate | 135 frogs | 55 frogs |

Do nothing | 100 frogs | 100 frogs |

We will benchmark this EVPI against a theoretical maximum across possible parameterizations. We will demonstrate that the EVPI reported (Canessa et al., 2015) is nearly 78 percent of the maximum possible EVPI that could be achieved and that this Chytrid problem would be in the top one percent of general two-action two-state EVPI problems, with parameters generated from uniform probability distributions.

### 2.2 Problem formulation

The first term in Equation (1) is the expected utility if the manager selects the best action, given that they have fully resolved the uncertainty. The second term is the expected utility if the manager selects the best action under uncertainty. Hence, EVPI is the difference in utility between selecting an action without uncertainty and under uncertainty.

To illustrate the use of Equation (1), consider the frog translocation problem summarized above and in Table 1. If the manager tested for Chytrid, and it was absent, they would translocate the frogs, achieving 135 frogs. If the test revealed that Chytrid was present, they would not translocate, achieving 100 frogs. Since the prior probability Chytrid is present is 0.5, the manager expects to achieve 117.5 frogs if they test for Chytrid presence ($135\times 0.5+100\times 0.5=117.5$). However, if the manager does not obtain this information, and therefore does not know whether the site has Chytrid in advance, then if they translocate, they would achieve 135 frogs, with a 50 percent chance or 55 frogs also with a 50 percent chance, leading to 95 frogs in expectation ($135\times 0.5+55\times 0.5=95$). Therefore, they should not translocate and achieve 100 frogs ($100\times 0.5+100\times 0.5=100$) because $100>95$. The EVPI is, therefore, 17.5 frogs ($117.5-100$). This means that gaining perfect information would net an expected 17.5 additional frogs after management, relative to an initial population size of 100 frogs.

EVPI is a powerful metric because it is relatively simple to calculate and is an upper bound for other forms of VoI analyses. For example, one might be interested in designing an experiment or field study to learn the population growth rate of a species. Knowing the population growth rate may help inform a management decision, but it might also be valuable for other reasons, such as to gain general fundamental scientific knowledge. However, if the only reason this quantity needs to be known is to inform a decision, first the manager should ask: What is the expected value of knowing this quantity perfectly, given their current belief, Equation (1)? If the answer is low, then there might be no need to design the experiment, because an experiment, which will usually only resolve the quantity with some level of uncertainty, cannot deliver a value any higher than EVPI, Equation (1). If the value is high, an experiment may be worthwhile. In such a case, the researcher can calculate the expected value of sample information (EVSI), which accounts for the fact that the experiment does not fully resolve all uncertainty. EVSI measures the value of collecting a sample of data for improving a management decision. Similarly, a manager also might have several sources of uncertainty and may wish to determine which source is more valuable to resolve. To do so, a manager could calculate the Expected Value of Perfect Partial Information (EVPPI) for each source of uncertainty (sometimes referred to as the Expected Value of Partial X Information, EVPXI). All of these calculations use logic similar to what is presented in Equation (1) (see Canessa et al., 2015, for a review). However, EVPI will always be greater than or equal to both EVSI and EVPPI. Therefore, if EVPI is low, at best, EVSI and EVPPI will also be low.

- can specify uncertainty in system states through prior probabilities,
- can state their values as quantifiable objectives (i.e. utilities),
- can quantify how actions affect objectives (e.g. by predicting management costs and management action outcomes with uncertainty in each state),
- is risk neutral, such that maximizing an expected value is desirable,
- would correctly account for all uncertainty when deciding on an action in the absence of new information.

Assumptions 1–3 can often be satisfied through expert elicitation (Martin, Burgman, et al., 2012) and structured decision-making workshops (Gregory et al., 2012). System states can be literal descriptions of the system, but they can also be hypotheses describing system dynamics under different actions. Therefore, the decision maker need not know the utility of actions with certainty. For example, if a manager does not know how much a frog population will increase/decrease after a translocation, the manager can specify prior probabilities for different levels of increase/decrease. This uncertainty can be incorporated by expanding the problem. Each level of increase becomes a different state. System states can be any type of system description, for example, healthy and collapsed population statuses. States can also be alternative mechanistic descriptions of how the world works, defined as a set of candidate models, hypotheses or scenarios (Johnson, Jensen, et al., 2014).

Even for risk-averse managers, assumption four can be satisfied by modifying the utilities. For example, in the Chytrid example, if utility was measured in ${\mathrm{log}}_{10}$ frogs (rather than absolute frog abundance), then 100 frogs declining to nine represents a greater change in utility than 100 frogs increasing to 1000.

Assumption 5 is at the heart of VoI theory. If the decision maker is unable or unwilling to take the best action after correctly accounting for uncertainty, EVPI will underestimate the value of collecting information. For example, consider a government deciding whether to conduct a fishery stock assessment. A stock assessment is a report that usually includes population dynamic modelling and data analysis to provide decision-makers with information on the state of a fishery (Hansen & Jones, 2008; Holden et al., 2024). This information is then used to ensure sustainable future harvest. If a stock assessment reveals population declines, governments typically can implement more stringent fishing regulations. In many regions, regulations require the government to conduct a stock assessment before modifying a harvest policy. Therefore, in this example, the decision-maker can only act optimally with new information. They can only enforce smaller harvest limits by conducting an expensive stock assessment, even if the government believes the fishery is in a poor state and would almost certainly benefit from such restrictions. Other examples may include invasive species and pest management, where a manager is not allowed to deploy potentially harmful pest control without detecting pest presence with certainty (Holden et al., 2016).

### 2.3 Deriving mechanistic insights for a general two-action, two-state system

- the utility of one action is greater than or equal to the utility of the other action no matter the state, that is ${u}_{1,1}\ge {u}_{2,1}$ and ${u}_{1,2}\ge {u}_{2,2}$ (or vice versa; ${u}_{2,1}\ge {u}_{1,1}$ and ${u}_{2,2}\ge {u}_{1,2}$), or
- a different action is optimal in each state, more precisely (${u}_{1,1}>{u}_{2,1}$ and ${u}_{2,2}>{u}_{1,2}$) or (${u}_{2,1}>{u}_{1,1}$ and ${u}_{1,2}>{u}_{2,2}$).

In case one, where one action is best (or equal best) regardless of the state of the system, EVPI is zero. There is no value in resolving uncertainty in the state of the system if it does not change what action you will take in either state. So the lower bound for EVPI is zero, which is widely known (Canessa et al., 2015). However, what is the maximum possible value for EVPI (upper bound)? In the Appendix S1, we derive an analytic expression for EVPI in case two, where the optimal action flips depending on the true state. Runge et al. (2023) also recently presented a similar derivation in a different context, independently arriving at the same expression. We use this analytic expression to understand why low EVPI has often been reported in the literature.

In this way, we have fully described the EVPI of a two-state, two-action system only in terms of these utility gaps and the probability of being in each state.

### 2.4 Simulation methods

To provide additional insights into VoI analysis and verify our analytic results and theoretical upper bounds (presented in the Analytic Results section), we simulate EVPI for 10 million randomly generated two-action, two-state problems. Here, we test the utilities of each action in each state being uniformly distributed on 0 to $m$ (where $m$ is initially set to two so that the mean is one, but varied from 0.001 to 1000 in a sensitivity analysis), exponentially distributed with a rate parameter of 0.001 to 1000, and normally distributed with mean one and variances between 0.001 to 1000. For the two-action, two-state problem we drew all four utilities ${u}_{a,s}$ independently from the specified distribution using the standard base functions runif, rnorm and rexp in R (R Core Team, 2023).

We consider the probability that the state is in state one, $p$, fixed at a value of 0.5, and also uniformly and beta-randomly distributed with both parameters equal to five and 0.2. These four distributions on $p$ represent four degrees of uncertainty in the random simulation. Fixing $p$ to 0.5 only yields random draws with maximum uncertainty in the state (the system is equally likely to be in state one or state two). Beta-distributed $p$, with alpha and beta parameters equal to five, yields draws where the system is likely highly uncertain ($p$ has a mean of 0.5 and the distribution is bell-shaped). Uniformly distributed $p$ generates cases where all levels of uncertainty are equally common. Finally, beta-distributed $p$, with alpha and beta parameters equal to 0.2, yields a $U$-shaped distribution, where the system's state is more certain.

For all simulations, we report the frequency of EVPI values and compare the values to our theoretical upper bounds in two-action two-state systems. We then extend our analysis to problems with $n$ states by $k$ actions, with $n$ and $k$ varying between two and 10,000. We randomly draw utilities from uniform and exponential distributions (as described above). To choose a uniform vector of probabilities, which are guaranteed to sum to one, we use the Dirichlet distribution with $n$ concentration parameters equal to one. Dirichlet draws were computed using the rdirichlet function from the package LaplacesDemon (Statisticat and LLC, 2018). For code, see (Holden, 2024).

## 3 RESULTS

### 3.1 Analytic results

In the two-action, two-state case where EVPI is non-zero, the analytic solution shows that EVPI is uniquely determined by the utility gaps and the probability the system is in state one. This means our expectations should always be tempered in the context of how much worse one outcome is than the other and not directly by the utility values themselves. The analytic solution for EVPI, equation (2), also highlights when EVPI takes on high and low values.

High EVPI arises when $p$ is close to ${p}^{*}$. The quantity ${p}^{*}$ is the threshold probability of being in state one for which a manager, in the absence of new information, switches between choosing actions one and two. The fact that $p={p}^{*}$ maximizes EVPI can be seen directly from the analytic solution in Equation (2). If $p<{p}^{*}$, then EVPI $=p{\beta}_{1}$, meaning EVPI increases with $p$, but if $p>{p}^{*}$, EVPI $=\left(1-p\right){\beta}_{2}$, which decreases with $p$. So, the expression is maximized when $p={p}^{*}$. Below, we illustrate this concept with two examples.

One way $p$ can be close to ${p}^{*}$ is if the utility gaps are similar and both states are equally likely. This is the case for our frog example in Table 1, presented in detail in Canessa et al. (2015). Here, $p=0.5$ and ${\beta}_{1}=135-100=35$, and ${\beta}_{2}=100-55=45$. In this case, we have similarly high utility gaps of $35$ and $45$ frogs gained by choosing the best action in each state. The threshold probability that would cause the manager to switch actions is ${p}^{*}=45/\left(35+45\right)=0.56$. Since $p=0.5$ is close to ${p}^{*}=0.56$ we obtain a relatively high EVPI of $17.5$ frogs, close to half of the largest utility gap ($45$ frogs).

However, $p$ can be close to ${p}^{*}$, even when one state is far more likely. Let's consider the Chytrid example, but where the population triples to 300 across the two sites if the frogs are translocated and Chytrid is absent instead of increasing to 135 frogs. Assume all other values remain the same. In this case, ${\beta}_{1}=300-100=200$, and ${\beta}_{2}$ is still $45$. Here, a manager wishing to maximize the expected number of frogs would do the translocation, without additional information, as long as the probability of Chytrid absence is greater than ${p}^{*}=45/\left(200+45\right)\approx 0.18$. Given the new utility gap, ${\beta}_{1}$, EVPI is largest when $p=0.18$, rather than $p=0.56$. This means, in this case, maximum uncertainty (believing each state is equally likely) leads to lower EVPI than if the manager believes Chytrid is likely present.

As a benefit gap increases, EVPI can only increase or stay the same. To see why, note that if $p<{p}^{*}$ then $\text{EVPI}=p{\beta}_{1}$, so small increases in ${\beta}_{1}$ increase EVPI. However, note that as ${\beta}_{1}$ increases ${p}^{*}$ decreases. If ${\beta}_{1}$ increases enough such that ${p}^{*}$ decreases below $p$, then EVPI becomes $\left(1-p\right){\beta}_{2}$. At the exact point where ${p}^{*}$ collides with $p$, we have $p{\beta}_{1}=\left(1-p\right){\beta}_{2}$. After this collision, since $\text{EVPI}=\left(1-p\right){\beta}_{2}$, EVPI no longer depends on ${\beta}_{1}$, and therefore continuing to increase ${\beta}_{1}$ does not affect EVPI (see the middle column of Figure S1 for an illustration of this phenomenon in the Chytrid example).

The above mathematical analysis helps formalize the reasoning for when EVPI values are high, but it also helps us explain when EVPI values are low. In many situations, choosing the optimal action *accounting for* (rather than *resolving*) uncertainty can yield good outcomes. To understand this principle, consider the case where the consequences of choosing the wrong action are severe in state two but mild in state one. An example of this would be if translocating frogs to a site with Chytrid caused immediate extinction at all sites, but translocating frogs to a Chytrid-absent site only increased the population by a few frogs. Intuition might suggest that in such a case, there is a high benefit in collecting information about whether the translocation site has Chytrid because there is the potential for extinction. However, examining the analytic solution shows that this intuition is incorrect. There is little value in collecting information because translocation has a similar utility to doing nothing when Chytrid is absent. Therefore, the manager can act as if Chytrid is present regardless of the state, and not translocate, obtaining a relatively good outcome across all states. In such a case ${p}^{*}\approx 1$, meaning the manager has to be nearly certain Chytrid is absent to translocate.

Generalizing this example, when choosing the right action has major consequences in one state, but only small consequences in the other state, a manager correctly accounting for uncertainty will err on the side of choosing the action as if the system is in the state where the consequences of choosing the wrong action are high. This means that if one action is risky, under a low probability event, the manager will avoid this action in the absence of information anyway. Resolving the uncertainty is not important because the two actions have similar utility in the high probability state, and in the low probability state, with the catastrophic consequences, the manager has chosen the action that is best in that state anyway.

Interpreting EVPI will always, to some degree, be context-dependent. However, to summarize, the analyic solution has revealed that EVPI will always be higher when benefit gaps are high and $p={p}^{*}$. Putting all of these facts together, cases where both benefit gaps are large and equal (meaning ${p}^{*}=0.5$) and where uncertainty is the largest ($p=0.5$, and hence equal to ${p}^{*}$) will yield the highest possible EVPI. In Figure 1, we show this graphically by plotting the analytic solution for EVPI against $p$ and utility gap one when the other utility gap is (a) low, 0.1 (b) medium, 0.5 and (c) high, one. Note if all utility gaps are one and $p={p}^{*}$, then the maximum EVPI achieved is one-half.

Finally, it is important to note that we derived the analytic solution for the cases where EVPI is non-zero. EVPI will always be zero when the utility of one action is greater than or equal to the utility of the other action no matter the state. This would be expected to occur half of the time in randomly generated two-action, two-state systems (with independent, identically distributed, continuous utilities) since there are only 24 orderings of the four ${u}_{a,s}$ utility values and 12 of these orderings would yield one action always being best regardless of the state.

### 3.2 Simulation results

Our simulations confirm our analytic results. Half (50.0%) of the random two-action, two-state systems result in zero EVPI, regardless of the distributions from which we draw the parameters. For the remainder of these results, we focus on the 50% of simulations where EVPI values were non-zero. Low (non-zero) EVPI values were far more common than high EVPI values in the simulations where the state one probability was uniformly distributed (see Figure S2 for a distribution of raw EVPI values). Since low EVPI simulations may be driven by low simulated random utilities, we rescale EVPI by dividing it by the maximum utility gap (producing a quantity always less than or equal to 0.5 for two-state, two-action systems). The rescaling allows us to examine how often and to what degree EVPI is close to a theoretical benchmark in randomly generated systems.

Low EVPI values were common compared to their theoretical maximum when the state one probability, $p$, was uniformly distributed (Figure 2a–c). Higher values of EVPI occurred less frequently. This pattern was roughly the same regardless of the distribution the utilities were drawn from (Compare Figure 2a–c), and the parameters governing the distributions (see Figures S4–S6).

EVPI values tended to be higher in the case of maximum uncertainty, that is where $p$ was fixed at 0.5, meaning both states were equally likely (compare Figure 2a–c to d,e). EVPI was close to its maximum value (given the utilities drawn) in a notable portion of these random systems (furthest-right bars in Figure 2d,e). This is consistent with our analytic result that EVPI is at its largest possible value in the case where $p={p}^{*}=0.5$.

When $p$ is fixed at 0.5, EVPI is lower more frequently when the utilities are distributed exponentially (Figure 2e) than uniformly (Figure 2d) or normally (Figure 2f). This is consistent with our insights from the analytic solution. Given the exponential distribution's heavier tail and higher variance, it might be reasonable to assume the opposite: higher EVPI with exponentially distributed utilities. However, the distribution of ${p}^{*}$ in our simulations (Figure 3) drives the surprising result: the heavy tail leads more often to utility gaps that are far apart, creating ${p}^{*}$ values more likely to be far away from 0.5 (see more bars near zero and one in Figure 3b than in 3a and 3c).

EVPI was nearly always near zero for distributions on $p$ representing minimal uncertainty (e.g. a bimodal beta distribution with rate and shape parameters of 0.2, representing peaks at $p=0$ and $p=1$, see Figure S3). In all cases, regardless of the distributions chosen, for any of the parameter values, EVPI was always below the theoretical, analytically derived upper bound we calculated in Equation (5). This serves as independent validation of the analytic bound on EVPI in two-action, two-state systems.

In higher dimensional systems, more states and actions generally led to increased EVPI values (Figures 4 and 5), with non-zero medians (black curves in Figure 4), compared to medians of zero in two-action two-state systems (intercepts in Figure 4a,c). Given uniformly drawn utilities, median EVPI values increased and then levelled off with the number of actions (Figure 4b,d). However, EVPI decreased as the number of states grew very large (greater than 20, Figure 4a,b). More actions led to decreased variation in simulated EVPI values (narrowing shaded regions in Figure 4).

The distributions that the utilities were drawn from affected EVPI values in the higher dimensional systems (compare Figures 4 with 5), in contrast with the two-action two-state simulations (see columns in Figure 2). Interestingly, exponentially distributed utilities (i.e. unbounded utilities, with more frequent extreme values) led to higher maximum EVPI when fixing the number of states and increasing the number of actions compared to the uniform, bounded utility case (compare Figures 4a,b to 5a,b).

EVPI begins to decline as the number of actions in a two-state system increases to infinity (Figure 4a,b). To explain why, it is helpful to consider the case where there are two states and several actions. EVPI is zero when one action produces the highest utility in both states—this action maximizes utility regardless of the true state. With two actions the chance of this happening randomly is 50%. This is less likely as the number of actions increases, so median EVPI initially increases from zero with more actions. However, increasing the number of actions further means drawing more and more utility values uniformly on $\left[0,2\right]$. With enough random utilities across enough actions, eventually, one action will have a utility close to two in both states. For example, with random utilities and 1000 actions, action number 320 might have utility 1.93 in state one and 1.95 in state two. While it is unlikely that this action is the best out of 1000 actions in both states, choosing it would result in very good utility. Therefore, EVPI will be low relative to the maximum utility gap, but still non-zero. This explains why EVPI decreases when increasing the number of actions above 100 in Figure 4a,b. With more states (e.g. 100 instead of 2), the chance that a single action has high utility compared to other actions across many states is relatively low. Thus, the decline starts further to the right and is more gradual in Figure 4b compared to 4a.

When instead the number of actions is fixed but the number of states is increasing, EVPI divided by the highest utility gap steadily increases (Figure 4c,d). Without resolving uncertainty, all actions are equally good if the number of states is large because we are drawing a large number of utilities randomly. However, if the uncertainty is resolved, the best action is chosen for each state. On average, the corresponding utility of that action is an increasing function of the number of states, since drawing more random variables gives a greater chance of drawing high values (see Fisher & Tippett, 1928, for a classic reference on the distributions of the maximum of random variables for technical details). Since the maximum utility gap is bounded by two in the uniform $\left[0,2\right]$ case, EVPI divided by the maximum utility gap never decreases in Figure 4b,d. While the above logic also holds for exponentially distributed utility, the maximum utility gap heads to infinity as the number of actions approaches infinity because the exponential distribution is unbounded. Thus EPVI divided by the maximum utility gap eventually decreases as the number of states goes to infinity (Figure 5c,d).

Unlike uniformly distributed utilities (where the highest points of the shaded regions in Figure 4 never exceed 0.5), exponentially distributed utilities resulted in maximum EVPI values that did exceed half the greatest utility gap in 0.001 proportion of simulations. This only occurred with roughly 100 or more actions and 10–100 states (see vertical axis in Figure 5b,d). These results suggest that systems with utilities unbounded by physical constraints lead to a higher VoI than systems with bounded utilities. This result does not contradict the analytic derivations in the previous section, as those derivations only apply to a two-action, two-state system. In the two-action two-state simulations, EVPI values were always equal to or below half the maximum utility gap (Figure 2). However, our simulations confirm that it is possible to achieve higher EVPI values in systems with many states and actions (Figure 5b,d).

## 4 DISCUSSION

In this paper, we used analytic and simulation approaches to show why the Value of Information (VoI) for environmental decision-making is often calculated to be lower than an ecologist would intuitively expect. We then explored the key mechanistic drivers of low and high VoI. We focussed on the value of resolving all uncertainty in the system, called the Expected Value of Perfect Information (EVPI) because it is the simplest VoI calculation, and also an upper bound for all other related quantities in the literature (e.g. the expected value of sample information, EVSI, which assumes the manager collects a data sample with some noise and therefore only partially resolves uncertainty). We showed that in two-action two-state systems (1) the EVPI is often zero, (2) even when EVPI is non-zero, it is frequently low, (3) it is mathematically impossible to achieve EVPI greater than half the maximum difference in utilities across states (utility gaps) and (4) EVPI is highest in cases where utility gaps, in each state, are similar, large, and uncertainty is high. Principle four means that a massive difference in utility in one system state (e.g. extinction vs. no population change) combined with high levels of state uncertainty alone are not enough to lead to high EVPI values, because in such cases $p$ is not close to ${p}^{*}$ in Equation (3).

Insight (4) highlights a major principle of VoI theory. Choosing the optimal action accounting for (rather than resolving) uncertainty often yields good outcomes. If managers just care about yielding good outcomes and not learning for knowledge's sake they may not need to resolve uncertainty. But how do they account for uncertainty? All they need to do is be able to state their beliefs in different system states (states can be descriptions, scenarios, or models) using prior probabilities. There are several structured decision-making protocols on how to do this well, which usually involve expert elicitation workshops (Gregory et al., 2012; Martin, Burgman, et al., 2012).

While a two-action problem might seem like a major limitation of our work, ecological management problems are often posed as dichotomies between two actions. For example, protect an area or not based on the likely presence of a sensitive species (Bennett et al., 2018), expand a protected area or better manage it (Adams et al., 2019; Timms & Holden, 2024), restrict fishing effort or create marine protected areas (Erm et al., 2023; McGowan et al., 2018), increase anti-poaching enforcement or reduce demand for wildlife products (Holden et al., 2019), ban wildlife trade or regulate it (Holden & Lockyer, 2021), translocate a species or not (Canessa et al., 2015), or start a captive breading program or not (Martin, Nally, et al., 2012). In one 2-action example, Bouma et al. (2011) calculated how uncertainty in the spatial variation in sediment run-off might influence two abatement approaches for Great Barrier Reef management. They found the maximum EVPI was 55 million AUD, or approximately 40% of the difference between the most expensive and least expensive options. While 40% may initially only seem respectable, our analysis puts this number in context. It is actually exceptional VoI, close to the upper bound of 50% and close to putting it in the top 0.001% of randomly generated systems. Mäntyniemi et al. (2009) illustrate an example where the decision to fish offshore or in an estuary (two-actions) depends on which location hosts the greatest proportion of the population. The maximum EVPI in this example equalled 50% of the value of the largest utility gap, which is the highest VoI possible—higher than all two-action two-state systems in our seven million random simulations. As we can see, casting VoI results in terms of bounds and distributions can completely reframe how we view the results of these studies, and place them in context.

Despite these exceptional examples, EVPI values tend to be low in the literature. For example, Runge et al. (2011) investigated seven management actions and eight hypotheses of reproductive failure in the endangered whooping crane (*Grus americana*). They found that restoring meadows produced the highest benefits among all alternative management actions irrespective of the cause of reproductive failure. Therefore, the optimal management action made without information was the same as the one with information, making EVPI zero. Johnson, Jensen, et al. (2014) found that, when equipped with a perfect understanding of population dynamics, the optimal actions for managing the pink-footed goose (*Anser brachyrhynchus*) could only perform 6.2% greater than current management actions. Similarly, when resolving several hypotheses regarding the decline of a harvested quail species, the northern bobwhite, Johnson, Hagan, et al. (2014), found that resolving all uncertainty across 11 hypotheses for the decline would only improve management performance by 3.5%. All of these examples used EVPI, and involved managers and stakeholders directly in the structured decision-making process. The results from these studies are in line with our simulation results, which demonstrate that EVPI is likely to be low, especially in systems with only a few actions and states.

However, substantially increasing the number of actions and states did increase the median EVPI in our study. Particularly we showed that it is even possible for EVPI to exceed 50% of the maximum utility gap (our bound in two-action two-state systems) when utilities are generated from an exponential distribution and there are roughly 100 actions. The fact that many VoI analyses in conservation only consider a few management options (Bolam et al., 2018), is one possible reason why EVPI may often be low in the conservation literature. However, not all conservation problems are low dimensional, such as fine-scale reserve site selection problems, which consider astronomically large, combinatorial sets of actions and states (e.g. Cho et al., 2023; Hanson et al., 2022; Popov et al., 2022; Shah et al., 2017). Our findings show that VoI could indeed be high when moving beyond low-dimensional case studies found in the literature. The derivation of bounds for these high-dimensional problems remains an open problem that will become increasingly important as examples in the literature expand in size.

While our analysis helps reveal that the dimensions of decision problems could help explain the frequency of low VoI in the literature, there are also other hypotheses. Without or before conducting a formal VoI analysis, decision-makers may fundamentally misunderstand the VoI for management because they are unaware of how well the best action, accounting for uncertainty, is expected to perform relative to resolving or ignoring uncertainty. Alternatively, some decision-makers may value the knowledge beyond its ability to improve outcomes for a specific decision problem. For example, a decision-maker might conduct statistical analyses and design a field study that ensures an estimated population growth rate is both accurate and precise (e.g. using optimal design theory, Akinlotan et al., 2024). They may see the potential for learning the growth rate accurately to improve scientific knowledge or improve management decisions in other contexts. These values are not captured in traditional VoI analyses. If a decision-maker conflates these different values with the value of outcomes achieved for a specific decision problem, their intuition will not match VoI calculations.

Another possible driver of the disagreement between intuition and VoI analysis is that managers may be risk-averse. VoI theory requires specifying value under uncertainty in terms of expected benefits. Expectations are averages that do not account for a manager's potential tolerance to risk, for example their preference to avoid extreme negative outcomes. While risk-avoiding managers may benefit from information more than risk-neutral managers (Xiao et al., 2019), the logic behind our analysis likely holds in a risk-avoiding context as well. This can be seen by simply defining an objective function that punishes low values such as the logarithm or square root of population size, rather than absolute abundance. Similarly, to risk preferences, our approach could also be extended to multiple objectives. This is commonly done via utility functions that add up each objective corresponding to an elicited stakeholder weighting (Johnson, Hagan, et al., 2014), for which all of our analyses would directly apply. However, it is also possible to incorporate multiple objectives by examining trade-offs (Nicol et al., 2018), for which calculating VoI bounds and mechanistic insights may be more complex.

One particular limitation of our simulation approach is that we assumed utilities were independently distributed across action-state combinations. This is unlikely to be true for decision problems where the action is a continuous variable, for example harvested biomass in a fisheries management problem (Filar et al., 2024; Holden & Conrad, 2015). In such a case, the utility of harvesting 10% of the biomass would likely be correlated with the utility of harvesting 11% of the biomass. So our simulations do not correspond to this case. However, including such correlations would likely make EVPI behave similarly to systems with fewer independent action-state pairs if the system were more coarsely discretized. Therefore, our study may shine a light on continuous systems as well.

There are a growing number of ecological applications of VoI analysis. While many of these studies can provide case-specific reasons why EVPI is low (or in some instances, high), attempts to generalize and understand such results in a broader context have fallen short. The archetypical scenario where one action is always best no matter the state of the system is widely known to produce zero EVPI (Johnson, Hagan, et al., 2014; McDonald-Madden et al., 2010; Runge et al., 2011), however, there have been no general explanations for why EVPI is often low, even when the best management action does depend on the true state of the system. In this paper, we have provided overarching insights across systems based on the mathematical structure of the VoI formulation and demonstrated why EVPI will typically be low in many ecological problems. The results allow decision-makers to manage expectations and transparently reflect on VoI Analysis assumptions. Our study also provides a much-needed approach for evaluating, benchmarking and comparing VoI metrics, ultimately informing the age-old problem of whether to collect more data or act immediately.

## AUTHOR CONTRIBUTIONS

Matthew H Holden: Conceptualization, writing—original draft, formal analysis, visualization, software, supervision, and project administration. Morenikeji D. Akinlotan: Conceptualization, formal analysis, visualization, software, writingreview and editing. Allison D. Binley: Conceptualization, writing—review and editing. Frankie H. T. Cho: Conceptualization, writing—review and editing. Kate J. Helmstedt: Conceptualization, writing—review and editing. Iadine Chadès: Conceptualization, formal analysis, project administration, supervision, writing—reviewing and editing.

## ACKNOWLEDGEMENTS

We thank Jeffrey Hanson for feedback on previous versions of this manuscript and Joseph Bennett, Hugh Possingham, Eve-McDonald Madden and Michael Runge for useful conversations that improved this work.

## CONFLICT OF INTEREST STATEMENT

The authors have no conflicts to disclose.

## Open Research

# DATA AVAILABILITY STATEMENT

All simulated data is generated using code available via Zenodo at https://doi.org/10.5281/zenodo.12770810 (Holden, 2024).