Collaboration and term usage dynamics in agricultural buffer strip research: A research weaving protocol

Abstract

1. Research weaving is an approach that combines systematic mapping methods with bibliometric and scientometric analyses, shedding light on how research in a systematic map is connected or disconnected. Given its novelty, few examples exist that demonstrate methods for research weaving or highlight its value in the evidence synthesis context.
2. Here, we seek to fill this gap by applying a research weaving analysis to a previously published systematic map on the roles of buffer strips in agricultural fields. This work identified a lack of studies addressing multifunctional roles of buffer strips and a remarkable array of terminology used by researchers to describe buffer strips. Using network visualization, such as co-authorship and bibliographic coupling networks, as well as content and text analyses, we aim to build on these findings addressing questions related to collaboration, disciplinary contributions and citation and term usage patterns.
3. As a result of this work, we aim to provide workflows, tools, and recommendations for the application of research weaving across a wide range of evidence maps in any domain. We will discuss the unique challenges of conducting bibliometric analysis in an evidence synthesis context and will propose solutions to address these challenges.

1 INTRODUCTION

Evidence maps (including formalized methods, such as evidence and gap maps and systematic maps) bring together scattered knowledge from existing research to provide decision-making tools for evidence-based practice and policy (Snilstveit et al., 2016). They often present findings in the form of interactive visualizations that allow decision-makers to more easily navigate current evidence based on intervention characteristics, geography, outcomes and other factors. Evidence maps also help to identify research gaps and well-researched areas ripe for a formal systematic synthesis of evidence. Evidence mapping has become more prevalent in environmental science and ecology in particular, in part due to organizations like the Collaboration for Environmental Evidence (CEE; https://www.environmentalevidence.org/), which has produced numerous evidence maps and has helped to advance the methods for this type of research.

Common across most evidence maps is the application of systematic approaches for finding research across many databases, repositories and websites. Search results are screened against predefined inclusion criteria. The analysis of included studies in evidence maps typically involves a thematic approach, categorizing studies by characteristics related to study context, the intervention, measured outcomes and other features relevant to decision-makers. These results are presented in two- or three-dimensional matrices, bubble plots or geographic maps (Saran & White, 2018). Authors often include a basic bibliometric analysis to understand general trends in publishing such as number of publications over time, an analysis of journal titles or trends related to authorship such as number of authors per paper.

More recently, Nakagawa et al. (2019) proposed an approach called ‘research weaving’ to address an important question, which an evidence map does not attend to: How are pieces of evidence in such a map connected or disconnected? Research weaving brings together the rigour of evidence mapping (specifically, Nakagawa and colleagues refer to systematic mapping as defined by CEE) with a deep bibliometric and network analysis to visualize and elucidate the impact and influence of research and authors. The full suite of bibliometric and network analysis tools is extensive and includes co-authorship networks, co-word and topical analyses, and networks based on citation patterns within a body of work. Together with evidence mapping, these methods can provide a more profound understanding of what we know about a practice- or policy-relevant topic, providing insight into the underlying social mechanisms that drive collaboration and scientific knowledge forward.

Bibliometrics and network approaches can be applied before the screening phase in an evidence map or after screening. Before screening, analyses like co-word mapping and topic and text analyses can highlight concepts and keywords that can be used to search for relevant literature (Nunez-Mir et al., 2016). Citation and co-authorship networks also have potential for use as a research discovery tool (Cowhitt et al., 2019) or to guide source selection for searches. After the screening phase, researchers can use the included studies identified in the evidence mapping process to build network visualizations and conduct topic modelling that can aid in study coding and synthesis (van Eck et al., 2010). Moreover, network and topic visualizations can provide an added dimension upon which the literature can be analysed; namely, a dimension based on the social structure of the research community defined by collaboration and citation behaviour. This type of analysis has been increasingly used in many contexts to understand collaboration in research – for example, to inform strategic planning to meet public health and research capacity building goals in neglected disease research (Morel et al., 2009), to shed light on the drivers of research siloing in health disparities research (Collyer & Smith, 2020), to understand trends in global collaboration in fisheries research to challenge current thinking in policy and science (Syed et al., 2019) and to interrogate biases of non-independence in meta-analytic studies (Moulin & Amaral, 2020). This research demonstrates the potential of these network and bibliometric tools to contribute insights toward the evidence mapping goals of informing research priorities and identifying and filling research gaps.

These research weaving methods present a powerful new way to look at the research gathered in the process of evidence mapping. However, no single software or platform is yet available to facilitate this type of analysis (Nakagawa et al., 2019) and navigating the vast and complex literature on bibliometric and network applications can be daunting. This paper aims to provide a set of concrete workflows, software tools and recommendations for applying these methods in an evidence mapping context, particularly for post-screening application. Through the inspection of network visualizations, basic network statistics and cluster-level analyses, we will demonstrate how these methods can be used to address specific questions that arise from the evidence maps themselves, and effectively integrate information derived from both the maps and the networks to reveal new knowledge.

2 OBJECTIVES

The current work aims to apply research weaving approaches using evidence mapping data (specifically, through a systematic map case study) to generate a variety of bibliographic networks and analyses. These will include co-authorship networks and citation networks and will integrate topic modelling approaches. The techniques employed herein, like co-authorship analysis and topic modelling, will make use of data readily available in a well-documented systematic map (Haddaway et al., 2018), namely author names, titles/abstracts for each included study and data about study characteristics coded as part of the mapping process. The data needed to construct citation networks will be gathered from bibliographic databases like Web of Science Core Collection and Scopus, or open access scholarly search tools like Lens.org.

The body of work making up an evidence map is typically defined by practice- or policy-relevant questions, often involving stakeholder input. Interestingly, this differs from most bibliometric applications, which tend to focus on work defined by a discipline, domain or emerging topic. Thus, we anticipate that there may be considerable siloing (i.e. disconnected network components), with research groups working in distinct domains but addressing questions that help define a practice or policy direction. The visualization of these networks can provide insight into the nature of this siloing and perhaps guide the establishment of future collaborations to bridge domain gaps and foster interdisciplinary research toward common goals. Together, these techniques will shed light on the dynamics of collaboration and citation behaviour within a research community and can serve as an additional tool for identifying topics and search terms for use in updating evidence maps or developing future systematic reviews.

Taking Haddaway et al. (2018) as a case study, we will carry out these different types of analyses to demonstrate the added value of these techniques in the evidence mapping context. Haddaway et al. describe the evidence related to the impacts of vegetated strips in boreo-temperate farming systems. These features, including various types of field margins, buffer strips (used interchangeably with vegetated strips throughout this paper) and hedgerows, and their ongoing management are used to mitigate the negative impacts of agricultural operations and are highly variable in purpose, from preventing pesticide runoff to increasing biodiversity or preventing erosion. Figure 1 provides a conceptual model of the role of vegetated strips in the agricultural landscape. This systematic map was developed to bring together evidence on these multi-functional roles.

FIGURE 1

Conceptual model of pathways to impact for vegetated strips within or around fields. Illustration: Neal Haddaway (from Haddaway et al., 2018)

The authors of this systematic map carried out a number of analyses to collate the existing evidence and identify knowledge gaps and clusters. The authors explored this body of literature based on factors like geographic location of the study, types of vegetated strip used and outcomes measured. Visualizations related to the number of publications per year, the spatial scale of the studies and types of interventions, among others, were provided. They also carried out a text analysis producing tree maps of the terminology used to describe vegetated strips, finding a remarkable 205 different terms in use across the 1072 studies included in the map. In addition, the authors found a lack of research addressing multiple outcomes together, despite the fact that vegetated strips designed for one purpose may have unintended negative consequences in other aspects of the ecosystem (Haddaway et al., 2018).

We aim to take this work a step further by developing bibliometric network and topic analyses to shed light on the nature of collaborations and co-authorships (and conversely, where research is siloed and disjointed), and the topics and sub-disciplines contributing to this body of work. Specifically, this work will seek to address the following questions about the evidence on the role of vegetated strips in agricultural fields:

1. Is this research community siloed or well-integrated and collaborative? If siloed, how are these silos constructed?

2. Who are the key authors in this research community and have authors shifted among sub-communities over time? To what extent have authors and authorship groups worked on multiple different outcomes (e.g. biodiversity and pollution)?

3. What sub-disciplines contribute knowledge to this systematic map? Have authors shifted between sub-disciplines, or are they historically siloed by collaboration as well as subject area?

4. Do these sub-communities use distinct terminology to describe vegetated strips?

Given the novelty of using studies generated from systematic mapping for bibliometric and network analysis, an additional aim of the current work is to understand the challenges with this approach and develop tools and methods to address these challenges. For example, the literature included in evidence maps differs from that of most bibliometric studies in that it is comprehensive for a topic, thus including gray literature not indexed in bibliographic databases as well as older research. The lack of available structured meta-data for these types of literature presents a unique challenge for conducting bibliometric and network analyses, and we will attempt to identify sources, methods and processes for filling these data gaps to create a sufficient representation of the research community. Furthermore, we will provide a reproducible, step-wise process for constructing and analysing these network visualizations using open source tools that can be applied to any systematic mapping project in any domain.

3 MATERIALS AND METHODS

3.1 Dataset and data preparation

The dataset used in this study comes from Haddaway et al. (2018), a systematic map on the role of vegetated strips in agricultural fields. As with other evidence maps, the published report provides a database of all studies that met the inclusion criteria: the database includes detailed meta-data (descriptive information about each study) regarding the characteristics of the population, intervention, study context, outcomes, etc. This systematic map includes both primary studies and review articles. The current work will focus on the 1072 primary studies and will not include the review articles in the analyses.

To build citation networks, bibliographic records with structured data of each study's cited references are needed. We will investigate the feasibility of gathering these data from bibliographic citation databases including Web of Science Core Collection, Scopus and Lens.org.

3.2 Data analysis

To address our objectives and examine the ways in which the systematic map studies are connected or disconnected, we will build and analyse networks based on similarities in authorship and cited references, described in more detail below. Packages in R (such as bibliometrix (Aria & Cuccurullo, 2017) or igraph (Csardi & Nepusz, 2006)) and open source network visualization tools such as Gephi (Bastian et al., 2009) and vosViewer (van Eck et al., 2010) will be explored for this purpose. Figure 2 provides an overview of the methodological workflow we will use to address the study objectives.

FIGURE 2

Schematic representation of the methodological workflow. Objectives 1–4 will be addressed using a combination of co-authorship networks, bibliographic coupling networks and content analysis. The network visualizations were generated with the bibliometrix package in R using a preliminary version of our dataset

3.2.1 Collaboration and co-authorship

To explore Objectives 1 and 2 related to co-authorship, collaboration and the level of integration or siloing of this research, we will construct a co-authorship network using author names in our dataset. A co-authorship network visualizes connections between authors (i.e. nodes) based on shared authorship.

Standard network statistics like network density and centralization will be calculated to explore changes in the cohesiveness of the co-authorship network over time. Network density is the proportion of all potential connections in a network that are actual connections and provides a measure of the level of cohesion in a graph. Centralization is a graph-level descriptive of how cohesive a network is around particular points. Taken together, density and centralization, and changes in these measures over time, provide information about the connectedness of researchers within the network and can indicate the efficiency of knowledge exchange across the network (Phelps et al., 2012).

Degree centrality and betweenness centrality are two network statistics that provide node-level information, in this case, information about individual authors. These centrality measures can indicate the level of importance of individual authors in the network. Highly central authors collaborate with many other authors in the network and authors with high betweenness centrality act as knowledge bridges or gatekeepers between otherwise disconnected authors or sub-communities (Ghajar-Khosravi & Chignell, 2017).

We will identify the major components of the co-authorship networks, or those connected groups including the largest proportions of the complete author list. These components will be further explored based on factors such as the geographic location of the authors, the terminology used to describe buffer strips and the buffer strip functional roles studied (e.g. pesticide runoff, biodiversity, erosion, etc.). This will help shed light on whether authors study different functional roles over time and whether or not functional roles studied by collaborating author groups are singular or diverse.

3.2.2 Contributing topics and sub-disciplines

To address Objective 3, we will build networks using cited reference data for the studies included in the systematic map. Two types of citation networks are commonly used, namely bibliographic coupling networks and co-citation networks. We will use a combination of a bibliographic coupling network and content analysis to identify the various topics and sub-disciplines that contribute to the existing evidence on the roles of buffer strips. Bibliographic coupling networks are networks of individual papers (i.e. papers included in the systematic map) that are connected when they cite the same paper (Martyn, 1964). Approaches that combine bibliographic coupling and textual content analysis have been found to perform well in analysing topical clusters in a body of work (Boyack & Klavans, 2010). We will use the approach taken in Lamb et al. (2018), which applied topics derived from automatic content analysis of study abstracts to a bibliographic coupling network to explore the epistemic communities contributing to research on the mitigation of climate change in urban environments.

The bibliographic coupling networks will also be used to further explore the lack of research on multi-functional roles of buffer strips found in Haddaway et al. (2018). A community detection algorithm will be used to identify clusters in the network based on similarities in the cited literature. These clusters will be compared based on the functional roles addressed in individual papers in the clusters to identify whether differences in buffer strip functional roles are reflected in the different knowledge communities defined by citation patterns.

3.2.3 Use of buffer strip terminology

With Objective 4, we seek to explore the great diversity of terminology used in the literature to refer to vegetated strips. Studies in the dataset may use one or more terms for vegetated strips. These terms will be extracted from abstracts and full text and will provide another study-level attribute on which to assess author groups and topic clusters in the previously described networks. Furthermore, we will explore the use of these terms to build co-word networks that link studies based on shared terminology.

4 DISCUSSION

One goal of this work is to illustrate the types of questions that can be addressed by bibliometrics and network analysis techniques in an evidence map context, providing insights beyond those of standard analytical practices. We will describe tools, methods and workflows that can be used to apply these techniques across a broad range of systematic and evidence and gap maps in any domain. Moreover, we will provide guidance on making a priori decisions about analyses based on research questions and will highlight challenges related to conducting research weaving, proposing future directions for building tools and workflows that can make research weaving more accessible.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

AUTHORS’ CONTRIBUTIONS

S.Y. wrote the manuscript with support from all other authors. All authors provided critical feedback and helped shape the research objectives and analysis plan. N.R.H. provided the data.

Open Research

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1002/2688-8319.12084

DATA AVAILABILITY STATEMENT

No data were used in the generation of this Stage 1 report, with the exception of the network graphics in Figure 2 used for illustrative purposes only. Any data used in the subsequent Stage 2 report will be made available in Open Science Framework or another open access data repository.