The Data‐Limited Methods Toolkit (DLMtool): An R package for informing management of data‐limited populations

Abstract

1. A simulation‐based approach known as management strategy evaluation (MSE) is increasingly used by resource managers to identify management procedures that are robust to uncertainties in system dynamics.
2. The majority of global fish populations are data limited and there is large uncertainty over their population and exploitation dynamics.
3. The Data‐Limited Methods Toolkit (DLMtool) is an R package that allows for rapid and flexible MSE specification. The package consolidates a large number of existing data‐limited management procedures and allows for rapid MSE testing of new approaches.
4. The DLMtool package has supported transparent and rigorous decision‐making for a number of data‐limited populations, identifying robust management procedures and revealing performance trade‐offs.

1 INTRODUCTION

Management strategy evaluation (MSE) is a risk‐based approach increasingly used in the management of exploited populations that accounts for uncertainties in population and exploitation dynamics (Bunnefeld, Hoshino, & Milner‐Gulland, 2011; Butterworth & Punt, 1999; Punt, Butterworth, de Moor, De Oliveira, & Haddon, 2016). MSE involves the simulation of the exploited system—the operating model—that encompasses plausible hypotheses for population, exploitation, observation, and management implementation dynamics. MSE can be used to test management procedures (MPs—a model or algorithm providing management recommendations from data) and fixed management policies (e.g., a constant harvest level) over a projected time period accounting for feedback with the simulated system represented by the operating model.

Over 90% of fish populations are data limited: there are insufficient data to conduct a conventional population assessment (Costello et al., 2012) and in most cases there is considerable uncertainty over population status and trajectory. In order to meet national and international guidelines for sustainability it is necessary to demonstrate that MPs that are proposed for management are robust to such uncertainties. MSE offers a powerful tool for informing management of data‐limited populations: even if the performance of an MP cannot be established explicitly through a population assessment, performance may be evaluated implicitly by simulation. For example, a size limit or spatial closure that does not itself inform population status may consistently achieve management objectives over a wide range of simulated conditions.

The MSE approach has been used to evaluate a range of data‐limited MPs (Carruthers et al., 2014, 2015). The modelling framework was formalized in the DLMtool (Carruthers & Hordyk, 2018) (‘the package’) with the overarching aim to use the MSE in support of transparent and rigorous decision‐making in data‐limited fisheries. The package is intended for fishery scientists providing strategic advice to fishery managers. Given a fully specified operating model and explicit management performance objectives, the package can evaluate the performance of alternative management approaches, identify the most effective fishery control types (e.g., catch limits, time–area closures), determine appropriate models for assessing populations, and quantify the critical data gaps.

The first practical applications of the package were in U.S. Federal fisheries (e.g., Sagarese, Rios, Cass‐Calay, & Cummings, 2018). It has since been applied to four fisheries by the California Department of Fish and Wildlife (CDFW) (Hordyk, Newman, Carruthers, & Suatoni, 2017). Additional background on the package and its implementation can be found online (Anon, 2018). Although the origin of the package is the management of fisheries, its versatility has been demonstrated recently by the development of operating models for marine mammals and terrestrial sheep populations. A complete list of applications is available in Appendix S1 (Table App.A.1) and a range of fully documented operating models are available online (Anon, 2018). This paper describes the DLMtool software, its design, features, and intended use in reference to MSE best practices.

2 MATERIALS AND METHODS

The design and features of DLMtool are described here using the application to four CDFW fisheries as an example (Californian Halibut, Paralichthys californicus; barred sea bass, Paralabrax nebulifer; red sea urchin, Strongylocentrotus franciscanus; and warty sea cucumber, Parastichopus parvimensis) (Hordyk et al., 2017) (see Table 1 for a list of software features). The DLMtool approach is presented here in six steps that match the guidelines for MSE best practice of Punt et al. (2016) (Figure 1, see Table 2 for an example code).

Table 1. DLMtool features

Primary features

Specify operating models for a wide range of fishery types Reveal management performance trade‐offs among MPs Select robust MPs for data‐limited fisheries Prioritize data collection Identify critical operating model uncertainties Provide management recommendations from a wide range of MPs Determine what MPs can be applied for a given set of data Identify what additional data are required to use a particular MP Rapidly develop and test new MPs Construct operating models from existing Stock, Fleet, Obs, and Imp objects Describe operating models and visualize results Establish a suitable update interval for management advice Evaluate the relative efficacy of models of management such as catch limits, effort control, size limits, and time‐area closures Determine the feasibility of MPs based on their data availability and the types of managements that are possible (e.g., size limits, TAC controls). Allow for custom control of parameter distributions and parameter cross correlation for all operating model parameters Use parallel processing and cluster computing to speed up computationally intensive MSE analysis

Secondary features

Automatically specify operating models from existing assessments Diagnose MSE convergence (have sufficient simulations been carried out?) Investigate the impact of time‐varying population parameters on MP performance Evaluate the relative additional value of data‐rich MPs Use demographic meta‐analyses to impute missing life‐history data Identify suitable indicators of stock status Prioritize stocks for management based on status quo risks Investigate the impacts of discarding and catch over‐ and underages on MP efficacy Frame performance in terms of plausible outcomes (what are realistic best and worst case scenarios?) Evaluate feasibility of management objectives (can they be achieved given zero exploitation or “optimal” exploitation) Calculates common reference points for calculation of performance metrics such as those relating to maximum sustainable yield and minimum viable population size. Characterize the theoretical performance of MPs subject to perfect information.

Figure 1

DLMtool design, workflow and the MSE process. MSE recursively tests MPs over a projected time period by simulating data and then modelling the impact of MP management recommendations on the simulated system (represented by the operating model that combines population, exploitation, observation and implementation dynamics). Based on performance in the MSE simulations a management procedure may be selected and used to calculate management advice. Red text represents DLMtool object classes (e.g. Imp, Data) that contain the necessary parameters to specify the related dynamics. Blue text represents the 6 steps of ‘best practice’ MSE described by Punt et al. (2016) and are numbered in the order they are presented in that paper

Table 2. Example DLMtool code for the six steps of MSE described by Punt et al. (2016)

Install and load DLMtool package
install.packages(‘DLMtool’) library(DLMtool)
Download extra online resources and load these
DLMextra() library(DLMextra)
Set up DLMtool parallel processing
setup()
1. Identify performance metrics (class ‘PM’)
avail(‘PM’) myPMs = list(‘AAVY’, ‘LTY’, ‘P50’)
2. Selection of uncertainties (specification of operating models)
avail(‘OM’) myOM = Blue_Shark_IO_IOTC
3. Identification of candidate management procedures
avail(‘MP’) myMPs = c(‘DD’, ‘AvC’, ‘DCAC’)
4. Simulation of the application of the management strategy (MSE)
myMSE = runMSE(myOM, myMPs, PPD = T)
5. Presentation of results and selection of a management strategy
TradePlot(myMSE, PMlist = myPMs) Summary(myMSE, PMlist = myPMs)
6. Formal MP review (plot projected abundance index data for the MP ‘DD’)
MPno = match(‘DD’, myMPs) index = myMSE@Misc[[MPno]]@Ind matplot(t(index), type = ‘l’, xlab = ‘Year’, ylab = ‘Relative abundance index’)

2.1 Step 1. Selection of objectives and performance metrics

It is desirable to identify management objectives and quantifiable performance metrics prior to MSE analyses. Given the feedback from a wide range of stakeholders, functions of class PM (Performance Metric) are defined that summarize MSE outputs in terms of quantities of interest, for example, average yield and the probability of the population dropping below a lower limit (see Figure 2 for two CDFW performance metrics). Since the simulated system is known exactly, a wide range of metrics are available, including probability of meeting conservation and economic objectives. Once defined, functions of class PM are compatible with a range of standard outputs including MSE performance tables and figures (see MSE step 5).

Figure 2

Maximum Sustainable Yield (MSY) based performance metrics used in the DLMtool application by the Californian Department of Fish and Wildlife for four populations: Californian halibut, barred sea bass, red sea urchin and warty sea cucumber (Hordyk et al., 2017). Coloured lines represent individual simulations of yield and biomass over 30 projected years for which a management procedure was applied iteratively. Long‐Term Yield (LTY) is the average yield over the final 10 years of the projection, relative to that of MSY exploitation rate. Probability of low biomass (PLB) is the fraction of simulation years (black points) in which population biomass drops below 50% of MSY biomass

2.2 Step 2. Selection of uncertainties/specification of operating models

The DLMtool‐operating models include four central components of a real exploited system: population dynamics (e.g., growth, reproduction, migration), exploitation dynamics (e.g., spatial targeting, size vulnerability), observation processes (e.g., bias and precision in a population survey), and management implementation (e.g., catch limit overages).

The DLMtool makes extensive use of Object‐Oriented Programming to construct operating models. The user specifies objects for each of the four major components and then combines them into a single operating model object of class OM (see Appendix S1B for a full account of DLMtool design including operating model specification, for all model equations relating to population, exploitation, observation, and implementation dynamics see Appendix S3C–F). This approach allows for rapid operating model building and the ability to borrow dynamics from other exploited populations (e.g., construct a new operating model borrowing the population dynamics from another operating model of a similar species).

To make the process of operating the model construction simpler, the package contains a number of helper functions. For example, in the case of the CDFW application, operating model components were specified in a single annotated Microsoft Excel file which was then imported to R in the correct format. Similar functions allow users to specify operating models using stochastic stock reduction analysis (Walters, Martell, & Korman, 2006) or directly from the outputs of stock assessment packages such as Stock Synthesis (Methot & Wetzel, 2013) and iSCAM (Martell, 2017).

2.3 Step 3. Identification of candidate management procedures

The package consolidates over 80 MPs that have been designed for managing data‐limited fisheries (Appendix S3: Tables App.G.1‐2). These include model‐based MPs such as stock assessment models that explicitly estimate population and fishing dynamics, and empirical MPs that are simpler algorithms providing management advice from streamlined data such as a single index of abundance. To maximize extensibility, the package defines a standardized data object Data and a standardized management recommendation object Rec. Users can write their own functions that accept a Data object as an argument and return a Rec object, which can then be tested as MPs in the package. In the CDFW application, several of the standard MPs included in the package were adapted into 38 new MPs that were evaluated against the stated performance objectives.

The DLMtool includes functions for MP feasibility analysis where MPs are identified that can run given the types of data available (e.g., annual catches, recent length samples) and the modes of management that are possible (e.g., only catch limits are legally permitted).

2.4 Step 4. Simulation of the application of the management strategy (MSE)

Once an operating model has been specified and a range of candidate MPs have been selected, an MSE may be conducted. Since MSE is computationally intensive, DLMtool makes use of a faster C++ code for recursive operations and is compatible with parallel computing over multiple cores on a local workstation or numerous processors using cloud computing. MSE is carried out with a single‐function call, runMSE, that produces an object of class MSE containing a wide range of information about the MSE simulations.

2.5 Step 5. Presentation of results and selection of a management strategy

The package contains numerous graphing and tabulation functions for summarizing the results of MSE analysis. There are two purposes to these outputs: (a) to provide a general audience of stakeholders with an intuitive description of what occurred during the simulations (e.g., the CDFW projection plots of Figure 3) and (b) summarize performance for the selection of an MP, including uncertainty in outcomes (e.g., Canadian Department of Fisheries and Oceans status plots, Figure 4) and examine performance trade‐offs (e.g., CDFW trade‐off plots, Figure 5). In the CDFW‐barred sea bass case study, trade‐off plots revealed a number of MPs that were desirable and could obtain close to the highest yields with substantial improvements in biological risk (e.g., an MP‐modifying fishing effort based on trends in mean length data “LstepCE2,” Figure 5).

Figure 3

An example of MSE biomass projections for three management procedures (see Appendix S3: Table App.G.1) tested in the California halibut MSE (Hordyk et al., 2017; Figure 4 of that paper). The top row shows a projection for a single simulation. The middle row of panels shows the projection of 50 simulations. The lines are coloured according to the level of biomass in the final projection year where red simulations show depletion to low stock sizes. The bottom panel shows the quantiles of all 300 simulations. The grey shaded area is the 80% probability interval, the bold black line is the median and the thin black line is a single simulation. The horizontal dashed grey line represents reference spawning biomass at MSY levels. This plot is designed to provide an intuitive representation of the simulated system and MP dynamics over the projected time period

Figure 4

Spawning biomass and fishing mortality rate relative to MSY levels at the end of the MSE projection for two MPs (see Appendix S3: Table App.G.1 & 2 for details) tested by a preliminary MSE for BC arrowtooth flounder (Carruthers 2017, based on Figure 21 of that report). The regions of each plot represent stock classifications identified by the Canadian Department of Fisheries and Oceans. The solid and dashed contour lines encompass 50% and 90% of simulations. The black numbers represent the fraction of simulations that fall in each of the sectors. This plot is designed to communicate uncertainty in MP outcomes among simulations

Figure 5

A Californian Department of Fish and Wildlife performance trade‐off plot for barred sea bass (Hordyk et al., 2017; Figure 7 of that report) describing biological risk and long‐term yield among various management procedures (see Appendix S3: Table App.G.1 & 2 for details)

2.6 Step 6. Formal MP review

Once an MP has been selected and is being used for management of the fishery, it is important to evaluate whether it is operating as expected. In applications of MSE by the International Whaling Commission and the Commission for the Conservation of Southern Bluefin Tuna, this has been determined by the similarity of future observations of relative abundance indices with those simulated in the projections of the MSE. To support such analyses, the MSE object also records a wide range of posterior predicted data from the MSE simulations including relative abundance indices and size composition data.

2.7 Calculating management advice

Since MPs operate on the standard Data object, rather than simulated data, users can populate a Data object with real data and calculate management advice using a selected MP. In this way, the same MP equations tested in MSE are used in the provision of advice. The package contains numerous tabulation and graphing functions for management recommendations. For example, two MPs that were shortlisted in the CDFW case study of the red sea urchin, Islope1 and Islope4, use historical landings and the slope in recent catch‐per‐unit‐effort data, to set a catch limit. In this example, both the input data and the MP management recommendations were presented together (Figure 6).

Figure 6

Management recommendations of two management procedures that were shortlisted using MSE for the Californian Department of Fish and Wildlife red sea urchin case study (Figure 23 of Hordyk et al., 2017). Historical landings and catch‐per‐unit‐effort (CPUE) data are presented in panels a and b, respectively. The most recent landings and the gradient in CPUE (represented by the blue line) are used by the MPs to calculate current (2017) total allowable catch (TAC) advice (panel c)

3 DISCUSSION

Through various applications of DLMtool and wider experiences applying MSE, it is clear that the behaviour of MPs within the feedback‐control loop of an MSE simulation is often nonintuitive (Carruthers et al., 2014, 2015). MP selection can be strongly dependent on one or more factors, including: (a) management objectives, (b) the status of the stock, (c) population life‐history characteristics, (d) fishery dynamics, (e) data quality, (f) implementation error, and (g) feasibility of various types of management controls (data availability or management constraints, for example, input and output controls). While traditionally a data‐rich concept, MSE may be the most coherent approach for selecting MPs in data‐limited settings where uncertainty is high regarding many system attributes, including population status.

The development of the package has underlined the large potential benefit of establishing a standard for organizing fishery data. As described above, DLMtool requires a Data object class so that multiple MPs can be applied simultaneously to the same simulated or real fishery data. Establishing a fishery data standard (or database model) should be a priority for global fisheries management. If methods of fishery data processing and analysis were compatible with such a standard, scientists and managers could more easily benefit from the endeavours of the wider science community.

Other software packages exist for the MSE testing of management procedures. The most notable is Fisheries Library in R (FLR; Kell et al., 2007) which is an extensive open resource for a large range of population assessment and MSE tasks. The original concept for DLMtool was to use FLR‐operating models. Instead, the operating model of the package was written from scratch because using FLR, it was not straightforward to add new MPs or new observation models that could generate the types of data required by these MPs. An additional concern was that FLR is not available on the Comprehensive R Archive Network and would leave DLMtool dependant on other softwares that did not adhere to a rigid standard.

The package brings together many existing data‐limited MPs. Users can rapidly develop an operating model for their fishery management system, simultaneously test and select from the available MPs, and then calculate real management advice using the fishery data that are available. The package offers a flexible platform for designing and testing new MPs, calculating custom performance metrics, and developing new operating models. By including a high level of flexibility in operating model dynamics, DLMtool allows users to identify management options that are robust to uncertainties for a diverse range of exploited populations.

ACKNOWLEDGEMENTS

Many thanks for the ongoing support of the Natural Resources Defense Council, and, in particular, DLMtool team members David Newman and Lisa Suatoni who have provided input at every stage. The development of DLMtool has been funded by the Gordon and Betty Moore Foundation, the Packard Foundation, Fisheries and Oceans Canada, the Walton Foundation, Resources Legacy Fund, the Natural Resources Defense Council, the United Nations Food and Agricultural Organization, and the California Department of Fish and Wildlife.

AUTHORS' CONTRIBUTIONS

T.R.C. and A.R.H. contributed equally to both this manuscript and the development of the DLMtool package.

DATA ACCESSIBILITY

The DLMtool code and documentation is hosted at https://github.com/DLMtool/DLMtool/ and the package is available at https://cran.r-project.org/src/contrib/DLMtool_5.2.2.tar.gz. The operating models used this paper are included in the DLMtool R package (Carruthers & Hordyk, 2018).