How do we study resilience? A systematic review
Yann le Polain de Waroux, Marie-Claude Carignan and Olivia del Giorgio co-first authors.
Abstract
- The concept of resilience has gained immense popularity as a way to frame social and environmental challenges. However, its empirical operationalization and the integration of social and ecological dimensions continue to present difficulties.
- In this paper, we conduct a systematic review of existing empirical studies of resilience in social, ecological and social-ecological systems (SESs) and examine how and to what extent these studies have achieved the operationalization of the concept of resilience.
- We evaluate the operationalization of resilience in 463 papers based on whether they define the system of interest and disturbances, whether they define resilience, whether they evaluate resilience, and for papers focusing on SESs, whether that evaluation integrates social and ecological dimensions.
- We find that 51% of empirical studies do not meet at least one of these operationalization criteria, and that even those that do often lack key features for effective operationalization, such as clear system boundaries and baseline state or an effective integration of social and ecological dimensions. Of the papers examining SESs and evaluating resilience, only 54% integrate social and ecological dimensions in that evaluation.
- Building on these findings, we propose some design guidelines for operationalizing future empirical studies of resilience.
Read the free Plain Language Summary for this article on the Journal blog.
1 INTRODUCTION
The concept of ecological resilience, originally defined by Holling (1973) as a ‘measure of the persistence of systems and of their ability to absorb change and disturbance and still maintain the same relationships between populations or state variables’, has gained immense popularity as a way to frame a variety of social and environmental challenges (Capdevila et al., 2021). In five decades, resilience has expanded from a relatively bounded concept referring to the ability of an ecological system to bounce back or to the magnitude of disturbance it can withstand without undergoing a state shift, to a broader, more normative concept no longer limited in its focus to ecological systems. This theoretical abstraction has underpinned the widespread uptake and application of the concept in other domains, such as disaster management, international development and urban planning.
Despite its popularity, resilience remains challenging to capture. This may be because the concept's nature itself as a high-order abstraction presents particular challenges to operationalization. Indeed, questions have been raised as to whether its usefulness lies in its empirical application or rather in its use as a ‘boundary object’—a word that, by virtue of its general and poorly specified nature, provides a meeting ground for various fields, that is, a useful metaphor (Brand & Jax, 2007). Additionally, despite a growing emphasis in both research and policy on the need to integrate social and ecological dimensions of global change, and an increasingly widespread use of ‘social-ecological resilience’ as a way of framing sustainability issues, social and ecological components often appear disconnected in empirical studies of resilience.
In this paper, we take stock of the existing empirical studies of resilience in social, ecological and social-ecological systems (SESs). We ask how and to what extent these studies have operationalized the concept of resilience, that is, applied it empirically in a way that allows them to be used to inform and test theory as well as form a basis for policies and management strategies. Our premise is that the concept of resilience has value beyond its use as a boundary object and that the development of resilience theory is worthwhile, but that in order for that development to be possible, empirical research about resilience needs to meet a certain degree of rigour and comparability. That rigour and comparability in operationalizing resilience can then enable researchers, policymakers and practitioners to translate theoretical ideas into actionable measures and strategies that can be implemented in the real world, and to assess the effectiveness of these strategies. We address the question of operationalization by focusing on four dimensions: the definition of systems (resilience of what), the definition of disturbances (resilience to what), the definition of resilience itself, and the ways in which resilience is evaluated. Additionally, for studies of SESs (Ostrom, 2009), we consider whether the evaluation of resilience integrates social and ecological dimensions.
We conducted a systematic literature review of studies that claim to examine resilience, with particular attention to the empirical application of the concept—something that, to our knowledge, has not been done before. We demonstrate that only about half of the empirical studies meet minimum operationalization criteria across all these dimensions, and that those that do often lack key features, such as clear system boundaries and baseline state, clear characterization of disturbances or an effective integration of social and ecological dimensions. Building on these findings, we conclude with some design guidelines for operationalizing future empirical studies of resilience.
2 BACKGROUND
The origins of the resilience concept as used today can be traced back to ecological studies in the 1960s and early 1970s that explored ecological stability through the analysis of population dynamics and predator–prey interactions (e.g. May, 1971, 1972, 1973; Murdoch, 1969; Vandermeer, 1973). Central to the theoretical underpinnings of ecology at the time was the notion that ecological communities were structured around a single, optimal equilibrium (Beisner et al., 2003; Bodin & Wiman, 2004). This implied that the carrying capacity of an ecological community was fixed to this single state, and that, consequently, variability (i.e. change, disturbance and shock) caused deviation from an optimal configuration and should therefore be minimized (Folke, 2006; Vayda & McCay, 1975). In 1973, C.S. Holling, in the seminal paper “Resilience and Stability of Ecological Systems”, presented the possibility of multiple stability domains, which were identified by applying modified versions of the equations previously developed for the classification of predation responses to model population dynamics (see email transcript in Folke, 2006).
To describe the behaviour of ecological systems within this proposed multi-state reality, Holling adopted the notion of ‘resilience’, long used in the material sciences as well as in psychology and medicine (Hellige, 2018), (re)defining it as the capacity of a system to absorb change and thus persist within a given stability domain (Holling, 1973, 1985). Holling, although the first to propose a process model of multi-equilibrium theory (which fuelled the rapid uptake of the resilience concept, Hellige, 2018), was not the first to propose resilience as a property of a multi-equilibrium system. In fact, other ecologists, namely Errington (Errington, 1946; Errington et al., 1940), Person (1960) and May (1974), had presented similar postulations. Because of the lack of empirical support for the multi-state hypothesis, however, ecologists continued working on the assumption of a single equilibrium state (Folke, 2006), and early resilience studies focused on measuring or explaining the ability of an ecological system to maintain optimal function and recover to a pre-disturbance state (Carpenter et al., 1992; Dayton et al., 1984; DeAngelis, 1980; DeAngelis et al., 1989; Harrison, 1979; Pimm, 1984). Holling dubbed this notion of resilience as the return time to equilibrium after disturbance engineering resilience (Hellige, 2018; Holling, 1996).
Proof of the existence of multiple stable states first came from ecosystem ecology, through experimental and model-based studies of community-level interactions (Ludwig et al., 1978; Scheffer & Carpenter, 2003; Walker et al., 1981). The examination of cross-scalar and slow–fast system interactions, a shift in perspective spearheaded by the emergence of the sub-discipline of landscape ecology, further helped overcome the hurdle of empirically demonstrating the existence of multiple stable states (Newman et al., 2019; Peters et al., 2004; Turner, 1989, 1990; Wiens, 2002). By moving away from narrow temporal and spatial scales towards the analysis of processes occurring over long time periods and across landscapes, state changes that had previously been overlooked began to be captured. Growing empirical evidence for the multi-state hypothesis shifted attention onto complex system behaviours (self-organization, internal and external interactions, and feedbacks between components at multiple scales, Holling, 2001; Levin, 1998; Westoby et al., 1989), motivating interpretation of resilience according to system properties of renewal, transformation and re-organization (Carpenter, 2003). Stemming from this shift, the concept of ecological (or ecosystem) resilience emerged as the magnitude of disturbance that a system can absorb before it changes state (Gunderson, 2000; Gunderson & Holling, 2002; Walker et al., 2004).
In the social sciences, increasing attention to the sustainability of human–environment interactions led to the co-option of the resilience perspective from ecology (Bryant, 1992; Vayda & McCay, 1975). Departing from ideas of balance, regulation and harmony with nature (Arrow et al., 1995; Weinschenck, 1991), the emergence of a ‘new ecology’ that rejected the notion of an optimal ecological equilibrium translated in the social sciences to an increasing understanding that variability, while commonly entrenching the negative behaviours of social systems, could also possibly produce social innovation and progress (Scoones, 1999). The concept of social resilience consequently took shape as ‘the ability of groups or communities to cope with external stresses and disturbances as a result of social, political, and environmental change’ (Adger, 2000). This broadening of the concept contributed to its rapid adoption across multiple fields, such as in the development and climate change adaptation literatures, where it has become ubiquitous (Carr, 2019). Even so, the application of the concept of resilience to social systems remains contested, and many social scientists reject it (Olsson et al., 2015).
Beginning in the 1990s, growing awareness of human activity as the principal driver of emerging global environmental challenges instigated a re-direction in system studies from the disjointed conceptualization of social and ecological systems to their consolidation as a single, social-ecological whole, that is, as systems characterized by complex human–environment interactions and an inherent ability to self-organize (Berkes et al., 1998, 2002). Entwined from the origins of this integrated conceptualization of SESs was the notion of resilience which, by serving to describe the response of these complex systems to change, offered the seemingly perfect lens through which to examine those rapidly stacking environmental challenges. The rise of the integrative SES framework contributed to further extending the resilience perspective beyond persistence and buffering capacity, to include properties of transformation, learning and agency (Davidson-Hunt & Berkes, 2002). Resilience, as applied in the context of SES, has thus been increasingly interpreted as underpinning the adaptive capacity to both sustain function and develop in the face of change (Gunderson, 2002; Smit & Wandel, 2006).
The operationalization of the concept of resilience, meanwhile, has long been recognized as limited (Carpenter et al., 2001), in part due to the challenge in identifying and selecting meaningful indicators posed by its abstract, multi-dimensional nature (Cumming et al., 2005; Grimm & Calabrese, 2011). Carpenter et al. proposed in 2001 that overcoming this challenge and effectively converting “metaphor to measurement” would require careful research design involving a number of critical structuring elements (Carpenter et al., 2001). First, they argued that operationalizing resilience calls for clear delineation of the system of study—resilience of what. Second, the form and scale of the disturbance (e.g. categories of hurricanes) should be reported—resilience to what. Third, resilience itself should be defined to get at the questions ‘to what?’ and ‘of what?’ in relation to what specifically is meant by resilience. Finally, indicators should be selected to evaluate the resilience of the system when exposed to the specific disturbance in question. The motives of selection of these indicators should be provided, stating how each relates to, and captures, a certain aspect of resilience as defined for the study.
In what follows, we review the ways in which empirical studies have tackled the definition of the system of interest, of disturbances, and of resilience, and the evaluation of resilience. In addition, for studies that examine SESs, we argue that operationalizing resilience in the context of SES effectively requires some degree of integration between social and ecological dimensions in the evaluation, and we therefore examine the extent to which that is the case.
3 METHODS
All steps of the analysis are reported in a protocol that follows the RepOrting standards for Systematic Evidence Syntheses (Table S1). We started by generating a list of papers having to do with social and/or ecological resilience, broadly construed. To do this, in November 2019, we first conducted a search in the Scopus database using a search string that included a series of combinations of the word ‘resilience’ with relevant qualifiers. In addition to the search terms ‘ecological resilience’, ‘social resilience’ and ‘social-ecological resilience’, we also included other combinations of the word resilience with a qualifier (e.g. ‘vegetation resilience’, ‘community resilience’ or ‘livelihood resilience’) to capture a greater number of relevant papers. To avoid irrelevant results, we filtered for subject areas typically associated with social and ecological resilience (environmental sciences, agricultural and biological sciences, social sciences, earth and planetary sciences, arts and humanities, economics, econometrics and finance, or multidisciplinary), leaving out some subject areas that returned results on resilience but were outside our scope, such as psychology or computer science. We also limited the search to peer-reviewed papers and book chapters. This initial filtering yielded a total of 3499 documents published before 2020.
To obtain the final inclusion set, we screened the abstracts of these papers and filtered for whether they include social, ecological or social-ecological resilience as an object of study (i.e. whether they appeared to meaningfully engage with resilience, as opposed to a passing mention), allowing us to eliminate 1687 articles (Figure 1). Then, using the full texts, we screened articles on that criterion again plus whether they were empirical (i.e. based on the observation of real phenomena in a defined spatio-temporal context, as opposed, e.g. to modelling or conceptual papers), and whether papers included resilience specifically in their objectives or questions. This allowed us to eliminate another 1339 papers, for a final list of 463 papers. The person screening the abstracts conducted the exclusion based on full text, and two other people then verified it for consistency. Hereafter, we call ‘social-ecological papers’ as those in which the system studied includes social and ecological components that are acknowledged in the description of the system, and “social papers” or “ecological papers” as the ones where the system of analysis is solely social or ecological.
In order to examine the extent to which the concept of resilience has been successfully operationalized, we collected information on the selected papers along a series of variables, summarized in Table 1. The list of papers was divided in three parts, each assigned to a different person for analysis. In addition to continuous exchanges between analysts to ensure common encoding standards, one analyst reviewed all entries by the other two. The set of variables we extracted were meant to capture whether the papers met minimum requirements for operationalization (do they explicitly name the system(s) and disturbance(s) studied? Do they provide a definition of resilience? Do they provide an evaluation of resilience?) and, for those that did meet these requirements, to evaluate how the systems, disturbances, definitions and evaluations were characterized. For social-ecological papers, we also looked at whether they integrated social and ecological dimensions in the evaluation of resilience.
| Variable | Explanation |
|---|---|
| Framing | |
| Objectives | What are the objectives of the study with respect to resilience? |
| Focus | Does the study focus on social, ecological or social-ecological systems? |
| System characteristics | |
| System of interest (Y/N) | Does the study answer the question ‘resilience of what’? |
| System type | Typology of systems of interest |
| Disturbance characteristics | |
| Disturbance (Y/N) | Does the study answer the question ‘resilience to what’? |
| Disturbance type | Typology of disturbances |
| Resilience definition | |
| Definition (Y/N) | Does the study define resilience? |
| Definition | Typology of definitions of resilience |
| Evaluation of resilience | |
| Methods | Does the study use quantitative, qualitative or mixed methods overall? |
| Evaluation (Y/N) | Is resilience evaluated? |
| Evaluation method | Typology of evaluation methods |
| Integration | For social-ecological papers, are both social and ecological dimensions considered in the evaluation of resilience? |
To reduce the number of possible values for variables and facilitate analysis, the three lead authors created a series of typologies. For example, for the variable ‘system type’ (the answer to the ‘resilience of what?’ question), entries such as ‘secondary tropical dry forest ecosystem’ or ‘rural communities of region X’ were simplified to ‘ecosystem’ or ‘community’. For definitions of resilience, we looked for recurring patterns in how these definitions were structured by capturing the central concept or ‘classifier’ used in the sentence (i.e. an ‘ability’ to do something, a ‘process’ or an ‘amount’) and the associated responses (e.g. ‘recover’, ‘buffer’ and ‘transform’), grouping words with similar meanings under common labels (e.g. ‘degree’ or ‘quantity’ are grouped under ‘amount’). To build these typologies, one of the three lead authors defined a set of categories based on a first reading of all entries and then all three reviewed and organized them together to form a coherent framework. The resulting categories are provided in Table S2. For all typologies, one study can belong to several categories, meaning that the total percentage of categories can sum up to more than 100%. Finally, where comparisons of frequencies between groups were needed, we employed simple statistics such as Pearson's t-test.
4 RESULTS
Of the 463 papers reviewed, 42% were classified as ‘social’, 30% as ‘ecological’ and 28% as ‘social-ecological’ (Figure 2). Most of these studies pursued objectives related to assessing resilience in systems, understanding mechanisms associated with resilience, or developing and testing metrics and frameworks of resilience (see Table S3 for details), and the majority (>90%) were published after 2010. Both the regions studied and the authors were primarily in and from the United States, China and Australia. These studies were published in a total of 245 outlets, most frequently PLoS One and Journal of Vegetation Science for ecological studies, the International Journal of Disaster Risk Reduction and Natural Hazards for social studies, and Ecology and Society for social-ecological studies.
4.1 Definition of the system
All studies met the minimum operationalization requirement of naming a system—that is, it was possible, at the most basic level, to answer the question ‘the resilience of what’? Ecological papers overwhelmingly focused on ecosystems, ecological communities and/or ecological populations (Figure 3). Social papers focused mostly on the resilience of communities, specific demographic groups and/or whole populations. Social-ecological papers largely focused on resource–user systems, that is, systems defined by their relationship with a resource base (e.g. ‘forest-based community’, ‘fishery-based community’, ‘smallholder farming communities’). Like social papers, social-ecological papers also showed a strong emphasis on the resilience of communities, frequently overlapping with the resilience of resource–user systems (in 41% of cases). Almost half (47%) of the papers classified as social-ecological explicitly described the system under study as a SES, while the remaining half defined the system with both social and ecological components without designating it explicitly as social-ecological.
In terms of precision, a qualitative examination of the systems studied showed wide variation in how clearly papers defined system boundaries. Not all papers specified geographical boundaries for their systems, and we were unable to identify a spatial scale in many of them, especially social or social-ecological ones. Ecological papers generally appeared better at reporting spatial scale as well as temporal span and resolution. Furthermore, of all the social and social-ecological papers' study systems, we noted that papers studying ‘communities’ (per our typology, see Table S2) tended to be less precisely defined, either spatially or in terms of other social parameters, whereas ‘populations’ or specific demographic groups were more rigorously bound. For example, some articles included results from multiple case studies with varying or absent definitions of community (Keating et al., 2017; e.g. Haggard et al., 2019). In another instance, Sherrieb et al. (2012) left the interpretation of the ‘community’ to key informants, asking high school principals from a number of counties throughout the United States to evaluate the resilience of the community served by their school. Finally, some communities were defined by institutional spaces that were shared by many actor groups unevenly, without explicitly addressing the question of ‘resilience of whom’ in the analysis (e.g. Lucio & McFadden, 2017).
Additionally, while it would be reasonable to expect social-ecological papers to discuss how social and ecological components relate with each other to form a system, we found that only a minority did so. The papers that did usually described the natural and social components as being mutually interconnected and intricately linked. This was most frequent when nature-based livelihood systems were considered, such as in Sapkota et al.'s (2019) paper on the resilience of community forestry in Nepal, which defines a community forestry system as ‘the processes and outcomes of interactions in community forestry’, the latter comprising the environment, society, and the various formal and informal institutions that mediate their interactions (Sapkota et al., 2019, p. 181). Some papers also described a more unidirectional relationship where one sub-system shapes the other, namely through natural resource reliance and dependency for livelihood provision and resource management.
4.2 Definition of disturbances
Most studies (about 94%) specified a disturbance. Here, we use the term ‘disturbance’ to refer to any factors mentioned in the papers as disrupting the system's reference state, whether they be ecological disturbances, shocks, stresses, events or other forms of adversity. In order to facilitate interpretation, we classified disturbances into anthropogenic or natural, based on whether they directly involved human action or not, meaning that, as an example, climate change or invasive species were classified as natural disturbances in spite of their human origins since the direct agent of disturbance in the systems studied was not human. A majority of studies (69%) looked at natural disturbances, consisting mostly of natural hazards (e.g. floods, earthquakes and wildfires), climate change and other environmental variability or change (e.g. changes in water salinity, coral bleaching). Anthropogenic disturbances were examined in 32% of the studies (natural and anthropogenic disturbances were sometimes considered jointly) and consisted in most cases of events like human interventions on the environment, socio-economic and political changes, or livelihood changes.
A small but non-negligible number of papers (about 1% of ecological, 5% of social and 13% of social-ecological papers) failed the minimum operationalization requirement of specifying a disturbance. The majority of these studies were in fact ones that framed resilience as a quality that exists independently of specific types of shocks—they examined resilience to ‘disturbances’ in general, sometimes specifying ‘such as X or Y’—as for example in one study where farmers were asked to describe their livelihood resilience in the face of ‘dynamic changes in both sudden and long-term events’ and engage in ‘scenario analysis based on plausible disturbances’ (Panpakdee & Limnirankul, 2018, p. 415). Among ecological studies, most were non-longitudinal studies that base their evaluation of ecosystem resilience on characteristics described in existing literature as resilience-enhancing, such as Knudby et al. (2013, p. 1323) who mapped ‘selected aspects of the local reef community that are likely to confer resilience’. While about a third of the studies not specifying disturbances also did not evaluate resilience, those that did tended to concentrate their evaluation, like this one, on system characteristics thought a priori to be associated with resilience (see below).
4.3 Definition of resilience
Most papers (80%) provided a definition of resilience. In 82% of these papers, the definition focused on the ‘ability’ of the system to perform a function. For ecological studies, ‘ability’ was most commonly associated with the response ‘to recover’, followed by the ability to maintain, absorb, transform or cope. Resilience was also frequently defined in terms of rate of recovery. Social and social-ecological studies similarly emphasized the ability of systems to perform certain functions but emphasized a wider array of responses: recover, adapt, absorb and cope were the dominant responses, but responses of transformation, maintenance, resistance or persistence were also commonly mentioned. Importantly, definitions used in social studies comprised concepts that did not appear in ecological studies, such as the ability to learn, anticipate or self-organize. A smaller number of definitions in social papers also centred on quantities rather than abilities (e.g. rate of recovery or amount of disturbance), and a few defined resilience as a process rather than in terms of a characteristic of the system.
The remaining 20% of the papers we examined did not fulfil the minimum operationalization requirement of providing a definition of resilience. This gap was more prevalent in ecological studies (about 27% of these did not have a definition) than in social or social-ecological ones (18% and 15%). Many of the papers with no definition discussed resilience in a way that implied that it was a universally known term that did not need defining, and so tended to qualify rather than define it, as for example in Olds et al. (2012, p. 1196), which describes the key role that herbivores play in ‘sustain[ing] resilience by enhancing the capacity of [coral] reefs to absorb perturbations and regenerate without slowly degrading or changing state’. Some of these papers remained vague altogether on what they meant by resilience—sometimes on purpose, as in cases where authors intended to let an understanding of resilience emerge from the data (e.g. Gale & Bolzan, 2013). Others implied a definition of resilience through their description of a metric or by breaking resilience down into its components, such as Thrush et al. (2008), who evaluated resilience through a disturbance recovery experiment to test the propensity of estuarine communities to cross equilibrium thresholds. Another group of papers used resilience mostly in framing the study and interpreting results but did not connect that discussion explicitly to the empirical part of the study. This was common for papers mentioning resilience in the context of other concepts such as vulnerability, adaptive capacity or recovery. Papers without definition were also less likely than others to evaluate resilience (56% without evaluation vs. 24% for papers with definition, t-test p-value <0.001).
4.4 Evaluation of resilience
About 70% of the papers provided an evaluation of resilience, using either a quantitative (61% of these papers), a qualitative (20%) or a mixed qualitative and quantitative approach (19%) (Figure 3). Most ecological studies adopted exclusively quantitative designs (96%), whereas only 54% of social studies and 38% of social-ecological studies did so, giving more space to qualitative (social: 26%; social-ecological: 30%) or mixed methods designs (social: 20%; social-ecological: 32%).
We found that quantitative approaches to resilience fell along three broad categories (Table 2). The first of these encompasses metrics that capture system attributes, which refer to characteristics of the system already linked in the existing literature with resilience, meaning that systems for which these characteristics are present are assumed to be more resilient (e.g. ‘connectivity in the system’). These metrics, often presented as indices, group multiple dimensions of the system (e.g. financial, social, environmental, institutional, infrastructural), each represented by several indicators, the value of which can come from surveys or be derived from secondary data. We found this approach to be most frequently used in the disaster resilience literature.
| Categories | Description | ||
|---|---|---|---|
| Quantitative | Evaluation of attributes associated with resilience | Primary data | Index composed of system attributes associated with enhanced resilience in the literature and used to collect data. (e.g. access to resources, assets and capital) |
| Secondary data | Index composed of system attributes associated with enhanced resilience in the literature, with data from secondary sources | ||
| Evaluation of resilience itself | Proxy | Use of a proxy variable representing system attributes for which the relationship with resilience has been demonstrated in previous literature. Proxies are only considered in this category when the relationship between the variable and resilience is explicitly mentioned and/or explained rather than only implied | |
| Subjective | Indices or indicators based on surveys asking informants for their perception of resilience. These surveys, often based on Likert scales, include questions that ask subjective questions, such as ‘I feel like my community is prepared for disasters’, or ‘I know that when things look hopeless, I do not give up’ | ||
| Evaluation of outcomes reflecting resilience | Disturbance | Magnitude of disturbance tolerated before a system shift | |
| Variation | Magnitude of variation in a system variable over time | ||
| Recovery | Recovery time or recovery rate of a variable after disturbance | ||
| Residual | Residual impact of the disturbance on a system variable | ||
| Qualitative | Deductive evaluation of resilience | Evaluation of multiple attributes | Qualitative evaluation of attributes associated with resilience or dimensions of resilience defined a priori and used in thematic analysis (e.g. capacity to persist, adaptive capacity and transformative capability). This category also includes studies that evaluate resilience on a qualitative scale (e.g. low, moderate, and high physical and social resilience) |
| Informant accounts | Analysis of lived accounts with key informants (people who are part of the system, such as a community's members) where resilience is evaluated based on themes and attributes determined a priori as associated with resilience | ||
| Inductive evaluation of resilience | Evaluation of multiple attributes | Qualitative evaluation of attributes based on system dimensions (e.g. natural, physical, economic, human and financial capital) or phases (e.g. pre-disaster and post-disaster). These attributes are used to break down the system to evaluate its resilience ‘in parts’, and often emerge a posteriori with resilience being the central theme of thematic analysis | |
| Informant accounts | Analysis of informant accounts through individual or group interviews from which resilience is evaluated inductively through the analysis of mechanisms, narratives, expertise and other forms of personal experiences | ||
| Longitudinal assessment of changes | Qualitative assessment of resilience through the description of changes occurring in the system and its dimensions over time. These papers emphasize the transitions and mechanisms that occur through time in response to described disturbances and trends | ||
A second category contains metrics that aim to capture resilience itself, that is, variables that can be considered substitutes for the underlying quality (rather than the source) of resilience. This is done in different ways. Some papers approach resilience through proxy variables, functional trait diversity being a frequent example in ecological studies (e.g. De Lange et al., 2013; Laliberté et al., 2010; Sterk et al., 2013). Others captured resilience through subjective assessments of a system's capacities that are considered as constitutive of resilience (e.g. ‘ability to recover’), for instance, by surveying individuals on their perception of risk and their ability to cope, learn, plan and reorganize (e.g. Mulyasari & Shaw, 2013; Wyche et al., 2011).
The third category we identified encompasses metrics that represent system outcomes in the face of disturbances, the implication being that the response of a system to disturbances reveals its resilience. We found outcome metrics to usually be unidimensional; to mostly focus on the magnitude of variation in a system variable following a shock (e.g. Lipoma et al., 2016; Vasilakopoulos & Marshall, 2015), the magnitude of disturbance tolerated by the system before it shifts (e.g. Buma & Wessman, 2011), the recovery time of a variable after a shock (e.g. Chang & Wen, 2017; Downing & Leibold, 2010) or the residual impact of a shock on a variable (e.g. Yang et al., 2016); and to be most commonly used in ecological studies. Some papers also use a combination of metrics described above to measure resilience. For instance, Wei et al. (2019) combine a shrubland ecosystem's resistance to a disturbance (variation in water use efficiency during a drought) and its recovery (residual change following the drought) into one all-encompassing resilience metric.
Qualitative evaluations of resilience can be broadly divided into deductive and inductive approaches. Deductive approaches examine data based on dimensions of resilience defined a priori, either as characteristics that are thought of as constitutive of resilience (e.g. capacity to persist) or as ones that are considered resilience enhancing. They can be based on the exploration of multiple attributes, for example, ones noting the presence or the absence of certain characteristics in a systematic manner. This is well demonstrated by Sapkota et al. (2019), who evaluated the manifestations of known ecological resilience attributes (diversity, redundancy, modularity and feedback loops) in a community forestry SES. Or, they can be based on the thematic analysis of accounts by informants, for example, the coding of open or semi-structured interviews with a pre-determined set of themes, as in Ruiz-Ballesteros and Ramos-Ballesteros (2019), where extensive ethnographic data were examined with an explicit focus on the following dimensions of SES resilience: demographic behaviour, productive practices, local knowledge/learning capacity, governance, place attachment and attitude change.
Inductive approaches start with a broad interest in resilience but without defining a priori categories of interest. These may be, for example, studies exploring multiple attributes that are derived from the data, or studies using informant accounts to draw insights on resilience without relying on a pre-established ‘codebook’. A good example of this is Panpakdee and Limnirankul (2018) who used data from interviews with organic rice farmers to derive four key SES properties and subsequent indicators to evaluate rice-farming resilience. The inductive approach also includes some studies looking at longitudinal changes in a system through a variety of sources.
Close to a third of the papers we analysed (30%) did not provide any qualitative or quantitative evaluation of resilience. This proportion was somewhat higher among ecological studies (35%) than among social (29%) or social-ecological studies (28%), and it was also higher among qualitative (50%) than quantitative (23%) or mixed methods papers (21%). Papers that did not evaluate resilience were in part studies whose objectives were to understand whether and how a factor influenced the resilience of a system (49%) or to assess the resilience of a system (16%), which makes the lack of evaluation surprising.
Among the papers classified as not evaluating resilience, some presented a disconnect between what was evaluated and the concept of resilience. These papers generally discussed resilience at length in the conceptual section but did not make it clear how the empirical part tracked with understanding resilience. In some of these papers, there was an implicit claim of a link between parameters that are measured empirically and resilience, but that claim was not made clearly and therefore left room for ambiguity. For example, multiple papers measured something that could be considered a proxy for resilience without explicitly linking the two, or discussed responses, like recovery, persistence or vulnerability, in a way that suggested they equated to resilience, yet without explaining how they do. In others, there was no discernible link between the empirical design and the concept of resilience. Another group of papers not evaluating resilience used resilience as a lens to discuss results but did not mobilize it in the methods. Finally, a few papers had objectives that were not congruent with an evaluation of resilience—for example, ones that aimed to examine resilience as a discourse.
For social-ecological papers, we argued that evaluation should also take into account both social and ecological dimensions. However, only 54% of those social-ecological papers providing an evaluation of resilience integrated social and ecological dimensions in that evaluation, while the others only considered ecological or, in most cases, social parameters. The proportion was even lower (42%), surprisingly, for papers that explicitly refer to their systems as social-ecological. Most of the social-ecological papers evaluating resilience did so through quantitative measures using metrics of system attributes, based on either primary or secondary data. This contrasts with studies that did not integrate social and ecological dimensions in their evaluation of resilience, which tended to be more qualitative and adopt inductive approaches. Indeed, quantitative metrics based on system attributes, by virtue of their multidimensional nature, allow for the combination of variables related to both social and ecological characteristics. For example, Cao et al. (2018), in an assessment of resilience of two grasslands in the Tibetan Plateau, used a series of 20 indicators of the grazing system (space, water source and transhumance), the ecological system (vegetation and soil characteristics), the economic system (income and infrastructure) and the social system (equity, health and social relations) to evaluate resilience as a weighted sum of scores along these different indicators, based on empirical data.
Other studies used more standardized resilience indices, an evaluation method that is particularly prevalent in disaster research. For example, Smith et al. (2019), in an attempt to characterize coastal community resilience to natural disasters in the United States, used a re-scaled version of the Climate Resilience Screening Index, ‘a composite measure for characterizing the resilience of SES in the context of governance and risk to natural hazard events’, that includes five domains (Risk, Governance, Society, Built Environment and Natural Environment) made up of 20 indicators and 117 metrics. Here too, the measure of resilience is based on a weighted sum of a series of simple metrics that include social and ecological dimensions, this time based on secondary data at the county scale. Some papers also integrated social and environmental dimensions through qualitative evaluations, most often using deductive or inductive evaluations of attributes, which offer a relatively straightforward way to combine social and ecological parameters. For example, Spies (2018), in a study of food system resilience in the Karakoram mountain range of Pakistan, used a case study design to generate field data on livelihood and food system change that is then interpreted in terms of capacity to transform, persist and adapt. This allowed the author to integrate social indicators, such as livelihood diversification, as well as ecological ones, such as decline in soil productivity, to produce a qualitative account of resilience.
5 TAKING STOCK OF THE OPERATIONALIZATION OF RESILIENCE IN EMPIRICAL STUDIES
5.1 Challenges and limitations
Over 20 years ago, Carpenter et al. (2001) drew out the requisites for converting resilience, a concept shrouded in abstraction, from metaphor to measurement. Since then, the concept has been applied in hundreds of empirical studies, gaining a prominent place in the lexicon used to frame the mounting crises we face globally. Yet the extent to which resilience has been effectively converted from an allegory for ‘toughness’, to a measured system property, is not evident. The results of our review serve to explain the lack, still, of a clear consensus on the empirical usefulness of the concept, considering that 51% of the studies we assessed did not meet minimum operationalization criteria derived from Carpenter et al. (2001). Instead, many used resilience as a way to frame their conceptual and discussion sections. This metaphorical application of the term is not a problem per se—on the contrary, the phenomenal uptake of resilience as a concept (as indicated by the high number of articles yielded by our initial search, pre-filtering) serves to highlight its usefulness. However, our results draw attention to several gaps along the four dimensions of operationalization that underlie the deficient empirical application of the concept.
First, while we were able to identify, at a basic level, the system under observation for all the studies assessed, the precision with which studies delimited the boundaries of those systems varied greatly. In this regard, studies of SESs stood out as those for which the answer to the question ‘resilience of what?’ was most often obscure. For many of these studies, the spatial scale of the system was not identifiable, its boundaries were unspecified, and its components were not detailed. This not only makes it difficult to measure resilience but also hampers any measurement of change—an issue raised almost 20 years ago by Cumming and Collier (2005). Second, a small but non-negligible number of papers did not answer the question of resilience ‘to what’ with specificity. Here again, social-ecological studies stood out, with a notably higher proportion not defining the disturbance. Third, one-fifth of all papers did not provide a definition of resilience, rendering the questions of resilience ‘of what’ and ‘to what’ somewhat irrelevant. These papers often discussed resilience in a way that implied there was no need to define it. In the case of ecological studies, the ones that most frequently did not define the concept, this may be explained by a disciplinary view of the concept as ubiquitous and self-explanatory, which might make it seem unnecessary to spell out its meaning. This tendency towards implication, which also accounted for the lack of specificity in the definition of disturbances and in a large number of studies, thus seems to be at fault for many papers failing to operationalize the concept in a way that allows for effective empirical evaluation. Finally, close to a third of the papers we analysed did not provide any qualitative or quantitative evaluation of resilience. Surprisingly, a large portion of those were the ones whose objectives were explicitly to understand whether and how a factor influenced the resilience of a system or to assess the resilience of a system. Potential difficulty aside, several characteristics stood out for papers that did not evaluate resilience: some presented a disconnect between what was evaluated and the concept of resilience; some had objectives that were not congruent with an evaluation of resilience; and, tying back to the metaphorical use of the term, others merely used resilience as a lens through which to discuss results, without mobilizing the concept in the methods.
Overall, while issues of operationalization were present across all study types, social-ecological papers appeared to face the most challenges in the empirical application of the resilience concept. These challenges to operationalization likely relate to the added complexity presented by the integration of social and ecological systems and, in parallel, the necessary complexification of the resilience concept as pertaining to these unified systems. The higher tendency of social-ecological studies, as compared to ecological and social studies, not to define the boundaries of the system under observation as well as the disturbances to which that system is exposed to, might relate to the challenge posed by the inherent ambiguity of these systems' boundaries (Partelow, 2018). This ambiguity arises in part from differences in system ontology between the social and natural sciences, which creates difficulty in delineating a cohesive system ‘unit’, as well as because SES are autopoietic, meaning that system boundaries are determined by the system itself (Grimm & Calabrese, 2011; Olsson et al., 2015). While social-ecological studies fared better than ecological and social ones with respect to defining resilience itself, the fact that almost half of those that evaluated resilience did not integrate social and ecological dimensions into that evaluation suggests that the presence of a social-ecological resilience definition is not synonymous with a functional definition—meaning one with specificity enough to guide a study's methodological design.
This common failure to integrate ecological and social components into the study design also points to the difficulty of selecting and subsequently evaluating observable indicators of social-ecological resilience. This may be due to the different characteristics, and thus methodological requirements, of the system components. For one, while ecological indicators are typically captured quantitatively (i.e. measured), social indicators (e.g. assets, flexibility, social organization, learning, socio-cognitive constructs and agency, Cinner & Barnes, 2019) often require qualitative approaches. The challenge of reconciling different types of indicators is compounded by the difficulty of defining SES baselines—so-called ‘initial undisturbed states’—and assessing, from there, SES state changes or thresholds. Additionally, the interdisciplinary nature of the approach needed to achieve it represents an epistemic and methodological challenge for researchers. As noted by Carpenter et al. (2005), while deliberate manipulations of ecosystems and before/after studies of large disturbances are possible in the natural sciences, ‘in interdisciplinary science, it may be impossible, unethical, or both, to induce a threshold-crossing in an SES’ (Carpenter et al., 2005, p. 941), though it is possible to examine what happens in instances where thresholds are crossed naturally.
5.2 Examples of effective operationalization of resilience
Still, about half of the papers assessed did meet all the four minimal operationalization criteria. A group-level comparison of these papers with those that did not in terms of their characteristics offers only limited insights (see analysis in Supporting Information) but looking more closely at some of these studies can give us some clues as to what makes for effective operationalization of resilience. We discuss some examples of effective operationalization below. Note that each of the cases discussed relies on expert knowledge for the choice of variables, scales or definition of the system. We do not evaluate these choices, but rather discuss these papers as cases where the operationalization criteria were met.
In a study from the disaster literature, Moreno et al. (2019) examined the aftermath of the 2010 tsunami in a small fishing community in Chile, aiming to understand the ‘resilience capacities’ that were activated in the course of the event and allowed the community to survive the first few days after the tsunami. The social system here was taken as the ‘community’, defined along geographical lines (the town), and the disturbance was a single event, the 2010 tsunami. Resilience was defined as ‘the ability of a social system to respond to and recover from disasters and includes those inherent conditions that allow the system to absorb impacts and cope with an event, as well as the post-event’, which in this case meant the ability of the community to subsist in place after the disaster. In order to understand what allowed this community to do so, the authors used inductive qualitative coding of interviews with local informants centred on the impacts of the damage, the collective actions and the capacities thought to be associated with resilience. In this case, the response to questions of the resilience ‘of what’ and ‘to what’ is clear and bounded, and a rigorous methodology that clearly relates to the definition of resilience allows exploring that question.
In another example, this time from ecology, Herrero and Zamora (2014) aimed to examine the resilience of a mountain ecosystem to an extreme drought that occurred in 2005. Specifically, their goal was to compare the resilience of two tree and two shrub species in the Sierra de Baza natural park in Andalucia, Spain, to that of drought. The authors defined resilience as ‘the capacity to restore pre-disturbance structure and function’ and fused established indices (developed by Lloret et al., 2011) of resilience and relative resilience to compare shoot growth and needle length for these different species, two variables that, the authors argue, can be used as indicators of plant responses to water supply, ‘providing a straightforward field sampling measure to analyze short term responses to extreme climatic events’ (Herrero & Zamora, 2014, p. 2). The indices compared growth performance before, during and after the disaster, thus focusing on the residual outcomes of the drought for these species as an approximation of their resilience. Based on field measurements, these numerical indices allowed the authors to compare the response of different species to the drought and draw inferences about the evolution of vegetation dynamics.
In a third example, a study focusing on SESs by Delgado-Serrano et al. (2018) sought to understand the role of community-based natural-resource management initiatives in the resilience of three communities in Latin America. The systems were understood as these three communities, defined in geographical terms, and comprising both social and ecological dimensions, which were described for each case. Here, rather than a single event or disturbance, the authors were interested in a multiplicity of site-specific disturbances, including deforestation, illegal resource extraction, internal conflicts and rainfall variability. Resilience was defined following existing definitions as the capacity of the system ‘to sustain human well-being in the face of change, both by buffering shocks, but also through adapting or transforming in response to change’, and the study used both quantitative and qualitative methods to evaluate attributes of the systems pertaining to this ability to ‘sustain human well-being’. Specifically, the authors developed a set of indicators adapted from Bergamini et al. (2013) that related to multiple dimensions of resilience: ecosystems, biodiversity, knowledge, learning and innovation, governance and equity, infrastructures, health and education, and livelihoods. They then looked at the history of community-based natural resource management implementation in the three sites in relation to indices related to these dimensions, in order to make inferences about its role in increasing the capacity/capability to maintain human wellbeing.
The multiplicity of indices of resilience, in this case, was also essential to enable an approach that integrated social and ecological dimensions in its evaluation. Indeed, this study adopted an ‘integrated’ approach at all levels considered in this review, defining systems in both social and ecological terms, paying attention to both natural and anthropogenic disturbances, evaluating resilience in an integrated manner, and considering anthropogenic and natural sources of resilience. Other studies that effectively integrated social and ecological dimensions tend to adopt similar methodological approaches. For example, in Guillotreau et al. (2017), the authors examined the resilience of marine SESs to mass mortalities of bivalves. The focus was on marine production systems, understood as comprising natural, social and governance sub-systems. This was reflected in an empirical approach that targeted multiple axes of resilience (e.g. static socio-economic resilience and dynamic socio-economic resilience) each referring to both different components of the system (e.g. the marine ecosystem and the economy) and also to different measures (e.g. speed or recovery or the magnitude of change tolerated). Different study sites were given ratings along these different axes, which were then summarized numerically as one index, allowing inference about the causes for differentiated responses between these sites.
6 CONCLUSION: SOME BEST-PRACTICE CONSIDERATIONS
We explored to what extent resilience has been operationalized in empirical studies of social, ecological and SESs. While half of the studies assessed met the minimum operationalization criteria, our findings point to some persistent pitfalls that limit the ability of even these studies to contribute to general insights about resilience. Taking these pitfalls as points of departure, we propose several considerations for future studies that aim to empirically assess resilience.
6.1 Meeting the minimum operationalization criteria
At the very minimum, empirical studies of resilience should clearly state (1) the system(s) examined, (2) the disturbance(s) for which resilience of the system(s) is assessed, (3) the definition of the resilience as considered in the study, and (4) the method of evaluation. Additionally, for social-ecological studies, the method of evaluation should (5) consider both social and ecological dimensions. The ambiguity found in many of the papers we reviewed in delimiting and/or defining some of these elements hampers the ability to make general claims about resilience and increases the likelihood that the concept will continue to be used without a full understanding of its workings.
6.2 Clear system boundaries and baseline state
Although a system was identifiable for all the studies assessed, the lack of precision with respect to the system limits compromised, across many studies, the ability to answer the question of resilience as a property of exactly what. Beyond describing, in broad terms, the characteristics of the system examined, the system should be as clearly bounded as possible. This means (1) delimiting the spatial boundaries in geographical terms, (2) outlining what this delimitation represents in terms of both spatial scale (i.e. extent of the area of study) and operational scale (the ‘grain’ of the study), (3) stating the amount of time over which the system behaviour was examined (temporal scale, e.g., days, weeks and years) and (4) describing the baseline state.
6.3 Clear disturbance characterization
What exactly the ‘disturbance’ entails should be clearly outlined, otherwise, the system's resilience evaluation becomes essentially meaningless. This description should include not only the form of disturbance (e.g. hurricane) but ideally also the intensity of the disturbance (e.g. Category 2), and its duration (e.g. over the course of a day). In cases where multiple types of disturbances are considered jointly, the range of types of disturbances should be discussed in detail.
6.4 Coherent definition
The definition of resilience should be congruent with the system behaviour examined (i.e. an engineering definition should be used where ‘amount of time to recovery’ is the variable evaluated). Where a new definition is used (i.e. where no reference is made to a previous study for the definition), the rationale for it should be presented. Similarly, if an existent definition of resilience is used but modified to suit the study, the differences should be clearly stated and justified.
6.5 Meaningful evaluation
The failure by many of the studies that we assessed to provide any qualitative or quantitative evaluation seems to be, in large part, due to the difficulty in choosing meaningful indicators of resilience as a system property. That choice should be informed by an understanding of the system and disturbances in their complexity and interactions. Studies are more likely to be successful at operationalization if the choice of evaluation method is justified and tailored to the nature of the system and disturbances. Each indicator of resilience that is used should be selected, consequently, on the basis of a knowledge of the system boundaries, baseline state and disturbance definition.
6.6 Effective integration of social and ecological dimensions
For papers looking at SESs, the evaluation should, at the very least, explicitly address social and ecological dimensions. Many articles studying SESs do not actually end up integrating these dimensions. Also worth noting, we could not identify any papers in our selection that meaningfully discussed the relationship between the ecological resilience of a system and its social resilience. When framing a study in social-ecological terms, it is necessary to carefully consider whether or not the study is based on the indicators of ecological and social resilience that are sufficiently linked to each other, or that allow a socio-ecological approach to be efficiently addressed.
Additionally, although we did not consider it as a criterion for operationalization in this study, we align ourselves with recent literature in suggesting that future studies should also consider answering the question of ‘resilience for whom’ (Jarvis & Wyborn, 2014; Meerow & Newell, 2019). This entails, on the one hand, a transparent process of reflexivity by researchers on how their positionality influences what they choose to examine (e.g. the choices made in terms of the minimum operability criteria) as well as their interpretation of the results (Cretney, 2014; Herrera, 2017). On the other hand, the question of ‘resilience for whom’ prompts consideration on the structures of power which create the inequitable distribution of both vulnerability and resilience—a question then of whose resilience is privileged and, by extension, whose is not (Cutter, 2016).
The present value of resilience as a boundary object and as a general narrative framing for ecological, social or socio-ecological studies, is undeniable. However, for the concept to evolve from framework to theory, some of the assumptions and hypotheses associated with it need to be subjected to empirical testing. This means that we need empirical studies that either directly test these assumptions or are framed in such a way that they can contribute to a body of evidence that will make it possible for that testing to be done. The fact that such a limited proportion of studies so far have done this satisfactorily is testament to the many practical difficulties involved. Where the intention is to evaluate resilience empirically, it is our hope that the best-practice considerations proposed above will help assist the design of studies in a way that will ultimately improve our ability to draw generalizable insights on resilience.
AUTHOR CONTRIBUTIONS
Conceptualization and writing—review and editing: Yann le Polain de Waroux, Marie-Claude Carignan, Olivia del Giorgio, Leandro Díaz, Lucas Enrico, Pedro Jaureguiberry, María Lucrecia Lipoma and Sandra Díaz. Methodology: Yann le Polain de Waroux, Marie-Claude Carignan and Olivia del Giorgio. Formal analysis and supervision: Yann le Polain de Waroux; Investigation: Marie-Claude Carignan and Olivia del Giorgio. Writing—original draft: Yann le Polain de Waroux, Marie-Claude Carignan and Olivia del Giorgio. Project administration: Flavia Mazzini. Funding acquisition: Sandra Díaz and Yann le Polain de Waroux.
ACKNOWLEDGEMENTS
This research was supported by the Inter-American Institute for Global Change Research (grant #SGP-HW-090). The authors thank the Editor, Associate Editor and the two reviewers for their constructive comments.
CONFLICT OF INTEREST STATEMENT
The authors declare no conflict of interest.
Open Research
DATA AVAILABILITY STATEMENT
The data for this paper are accessible under https://doi.org/10.5683/SP3/U3OXGU.
