Will your paper be used in a meta‐analysis? Make the reach of your research broader and longer lasting

Ecological and evolutionary research increasingly uses quantitative synthesis of primary research studies (meta‐analysis) for answering fundamental questions, informing environmental policy and summarizing results for decision makers. Knowing how meta‐analysis works is important for researchers so that their research can have broader impact. Meta‐analytic thinking encourages scientists to see single primary research studies as substantial contributions to a larger picture. To facilitate inclusion in a meta‐analysis, relevant primary research studies must be found and basic information about the methods and results must be thoroughly, clearly and transparently reported. While many published papers provide this information, it is common for essential data to be omitted, leading to study exclusion from meta‐analyses. We provide guidelines for correctly reporting basic data needed from primary studies in ecology and evolutionary biology so that they can be included in meta‐analyses, together with examples that show how data should be reported to enable calculation and analysis of effect sizes, and how data should be made accessible. These guidelines are important for reporting research results in general, whether or not results are included in subsequent meta‐analyses, because they are necessary for the interpretation and assessment of study outcomes. Increased implementation of these guidelines by authors, editors and publishers, and reinforcement by funders, will foster higher quality and more inclusive syntheses, further the goals of transparency and reproducibility in science, and improve the quality and value of primary research studies.


Introduction
Researchers in ecology and related disciplines increasingly use quantitative synthesis of primary research studies for answering fundamental questions, testing hypotheses, informing environmental policy and summarizing results for decision makers (Cadotte, Mehrkens & Menge 2012;Pullin 2012;Mengersen, MacNeil & Caley 2015). To accomplish these goals, there are formal guidelines and statistical methods using meta-analysis to summarize results of independent studies and analyse general trends, as well as for evaluating factors that may cause heterogeneity in outcomes among studies (Borenstein et al. 2009;Koricheva, Gurevitch & Mengersen 2013).
As meta-analyses have become increasingly common and important in ecology and evolutionary biology, it is useful for all researchers to understand how a meta-analysis works. This knowledge not only enables scientists to interpret and evaluate published meta-analyses, but also allows them to make their own research accessible for meta-analysts so that their results can be incorporated and interpreted in the broader context of research on the questions they are addressing. Meta-analytical thinking allows scientists to see single primary research papers as essential contributions to a larger picture within a research topic (Schmidt 1992;Nakagawa & Cuthill 2007). Using meta-analysis, researchers can consider how the results of an individual study relate to or contrast with others from different geographical areas or ecological systems, how new results complement or contradict earlier findings, and how they will be seen in the context of future findings. Thus, meta-analytical thinking increases the intellectual impact of individual studies and makes them more long-lasting. This is particularly compelling considering that about 27% of published individual research studies in natural sciences and engineering are never cited (Larivi ere, Gingras & Archambault 2009).
Here, we suggest specific guidelines for future primary research studies in ecology and evolutionary biology, amplifying and highlighting previous calls for higher publication standards including improved protocols for reporting procedures and outcomes of ecological studies (Hillebrand & Gurevitch 2013;Zuur & Ieno 2016). We also provide examples of how certain types of data should be reported to allow calculation of effect sizes from them, which is essential for meta-analyses. By implementing these guidelines for reporting research results, higher quality syntheses and meta-analyses of the current state of knowledge will be enabled. In this way, future meta-analyses will contribute to developing and answering new research questions in the domains of ecology, evolutionary biology, conservation biology and environmental science.

The principles of meta-analyses
In order to be jointly analysed in a meta-analysis, the outcome of each study must be expressed on a common scale. This measure of outcome, called 'effect size', includes information on the direction and magnitude of an effect of interest from each study. 'Effect size thinking' has been encouraged by various authors even in reporting primary research study results because it emphasizes outcome magnitude and direction, in contrast to P-values, which indicate neither (Arnqvist & Wooster 1995;Nakagawa & Cuthill 2007). The sampling variance of the effect size expresses the precision with which the effect is estimated. These effect size measures can then be combined across studies, taking into account the precision with which each is estimated. One can then estimate an overall mean effect and confidence intervals around that mean effect, test whether the overall effect differs significantly from zero, assess whether the outcomes of the studies are heterogeneous, and if so, test hypothesized categorical or continuous covariates (if any) to account for that heterogeneity.
Originally, meta-analysis was developed in the social sciences and medicine in the late 1970s and first introduced into ecology in the early 1990s (Koricheva, Gurevitch & Mengersen 2013). Meta-analyses in ecology and evolutionary biology not only share many aspects with those in medicine and the social sciences, but also differ substantially, for example in the types of research questions addressed and the data structures typically encountered in primary studies. A fundamental difference between meta-analyses in medical and ecological research is that the former are generally focused on specific estimates of the efficacy of a drug or surgical interventions, while the latter are often concerned with summarizing much larger and varied groups of studies to understand factors associated with the heterogeneity of their outcomes. Ecological meta-analyses, thus, are largely concerned with statistical issues involved with accounting for heterogeneity among studies rather than with estimating a single mean effect across all studies. For example, meta-analyses in ecology and evolution often encounter nonindependence among studies due to shared phylogeny (Nakagawa & Santos 2012). Moreover, ecological and environmental primary research is characterized by a diversity of model systems, geographical variability and stochasticity inherent to data collected under less controllable conditions, for example in the field. Thus, ecological data are likely to vary according to factors like climate, geomorphological processes, soil conditions and local or global human activities that might also affect the data used in meta-analyses. Meta-analyses can be a powerful tool in assessing the impact of such factors when results are combined across studies but synthesis can be hampered by lack of transparency in reporting results, unstandardized or missing data descriptions, and missing or inadequate metadata describing data characteristics and collection protocols (Hillebrand & Gurevitch 2013).
Meta-analysis requires a major search and data extraction effort from primary research studies, and crucially depends on primary research studies to contain relevant keywords to enable studies to be located (Côt e et al. 2013), and on the transparency and usability of reported data (Nakagawa & Cuthill 2007;Nakagawa & Parker 2015). Some previous recommendations to increase the quality of ecological studies, their analysis and the clarity with which they are reported, go back decades (Fowler 1990) while more recent efforts have urged study reproducibility and transparency (Goodman, Fanelli & Ioannidis 2016;Parker, Nakagawa & Gurevitch 2016b;Parker et al. 2016a). Numerous reviews in ecology and evolution find that as many as half of published articles lack key pieces of information regarding statistical relationships (Parker et al. 2016a). Moreover, important metadata is often not sufficiently reported (Hillebrand & Gurevitch 2013;Roche et al. 2015;Zuur & Ieno 2016) hampering the use of metadata as covariates in a meta-analyses.
If primary research papers are to be made accessible to meta-analyses, there is a need to promote 'meta-analysis thinking' which goes beyond 'effect-size thinking' (Nakagawa & Cuthill 2007) by considering how the paper will be found in a literature search, and how clearly and completely the procedures, analyses and results are reported, including relevant covariates that characterize the study or were used in the analyses.
Guidelines for meta-analytical thinking The number of studies found can be increased by using more keywords. However, just using a greater number of keywords also increases the number of 'hits' identified that are without actual relevance for the selected research question. Hence, for a meta-analyst, there is a tradeoff between false positives (studies that are not relevant to the questions addressed by the synthesis) and false negatives (relevant studies that are missed by the search).
To increase the likelihood of primary research studies being identified in a literature search, authors should carefully consider the choice of title, abstract content and keywords to find a balance between being broad enough to be found through a keyword search and being specific enough to be identified as relevant. Abstract and title should not solely focus on one key finding, but briefly capture the full experiment, and the use of technical terms not recognized in other (sub)disciplines should be supplemented by generally recognized synonyms (cf. Table 1).
To improve the chances of a study being incorporated in a meta-analysis, open access papers are the gold standard. If a publication cannot be made open access, many journals allow users to host a manuscript in an unedited form (but permission depends on the policy and agreements established by the publisher). Online platforms, such as ResearchGate or Mendeley, substantially reduce the effort required for individual authors to provide full text access.

R E P O R T I N G U S A B L E O U T C O M E S O F P R I M A R Y R E S E A R C H T O E N A B L E C A L C U L A T I O N O F E F F E C T S I Z E S
Accurate data extraction from primary research studies is essential to calculate effect sizes and is one of the most timeconsuming parts of a meta-analysis. The outcome and legitimacy of a meta-analysis depends on accurate and complete reporting of study outcomes. Effect sizes should summarize the results of each study on the same scale and in an unbiased manner. Many different effect size metrics can be used, from well-known to less familiar, each suitable for specific purposes and data conditions (Osenberg, Sarnelle & Cooper 1997;Nakagawa & Cuthill 2007;. Data needed from each study to calculate effect sizes depend on the research question, the data structure and the specific metric of effect size chosen. The most commonly used effect sizes in ecology are standardized mean differences, response ratios and correlation coefficients (using Fisher's Z transformation). Response ratios and (standardized) mean differences are often used in ecological research syntheses when the goal is estimating the magnitude of the effect of an experimental treatment on a continuous response variable (such as biomass). This is the case where one wishes to compare means for experimental and control groups from studies where means and standard deviations are reported. These measures can also be used for non-experimental data where one is comparing means of two groups as a measure of 'effect'. Odds and risk ratios can be used for binary response variables (alive/dead, pollinated/not pollinated) but are much less common in ecology than in medical meta-analysis. If the relationship between two continuous variables is of interest, correlation coefficients are often appropriate; slopes from simple linear regressions are problematic as effect sizes for several reasons, including calculation on very different scales among studies (Rosenberg, Rothstein & Gurevitch 2013). It is necessary to understand the sampling distribution of the effect size metric used so that one can correctly calculate a measure of sampling variance to be used in subsequent analyses, including weighting the outcomes according to their precision (Gurevitch, Curtis & Jones 2001).
Papers reporting on research outcomes from primary research should generally include basic information on means, sample sizes and measures of variation to be useful in a metaanalysis, or if correlation coefficients are reported or calculated, the sample sizes should be provided ( Fig. 1). Ideally, raw data are provided. If raw data are presented in figures, points should be clearly distinguishable and non-overlapping. Study design should be clearly documented, including hierarchical designs and any aggregation of (raw) data (e.g. do the means and confidence intervals correspond to individual organisms or are they plot means?). It is essential to identify the measure of variation reported (e.g. standard error, standard deviation or 95% confidence interval) in the text, as well as in figures and table captions.
In order to understand the outcome of a study, it is valuable to report or be able to calculate effect sizes and their variation. This provides readers with more complete information not only of statistical significance, but also with the magnitude and variation in effect sizes (Nakagawa & Cuthill 2007). Furthermore, it is important to report all of the results, including the statistically non-significant ones. 'Not significant' does not mean 'non-informative'. Reporting only significant results ('p-hacking') create biases ('publication bias'), and those biases are magnified when results are combined across studies (Rothstein, Sutton & Borenstein 2006). It is a widespread misconception that reporting on significant findings may increase the probability of getting a paper accepted (Koricheva 2003), and non-significant results can be of great relevance in a broader context (Parker et al. 2016a). We thus support and amplify the Center for Open Science Transparency and Openness (TOPS) guidelines ('OSF|Tools for Transparency in Ecology and Evolution (TTEE)', 'COS|Openness, Integrity, and Reproduciblity') and the checklist published by Hillebrand & Gurevitch (2013) to include reports on all results, regardless of statistical significance and direction of the effect.
A common goal of ecological meta-analyses is the analysis of effect sizes and causes of variation in study outcomes. Therefore, another crucial step alongside the data extraction for calculation of effect sizes is the data extraction of relevant covariates (moderators). Variation in study outcomes may be Table 1. : Core issues, action-items and reasoning to be considered in primary research studies in ecology and evolution to be beneficial for use in meta-analysis studies.

Common problems found in primary research papers
What to do? Why is this important?
Getting found in a literature search Study is not indexed in major bibliographic databases (while master or doctoral theses are indexed, "grey" literature, e.g. NGO reports, are difficult to find).
Journals such as Research Ideas and Outcomes (RIO) promote publishing any part of the research cycle (project proposals, data, methods, workflows, software, project reports) and consequently provide access to "grey" literature, too. Alternatively, metaanalysts may use less restrictive search engines to Web of Science, like Google Scholar or Scopus.
Studies that are not indexed in major bibliographic databases will usually not be found or considerable effort is needed.
Studies might be overlooked if title, abstract and keywords are too generic.
Consider your abstract and title to be distinctive from a review or opinion paper in the same field. In the title and abstract, use words that describe the main finding of the study, the geographical context, the methodology, and if possible the main covariates used in the analysis Relevant studies might not be identified and do not reveal that codable information is presented.
Studies might be overlooked if title and abstract are too specific, i.e. highlight major findings and omit additional data and tests.
Consider possible broader questions, the findings might contribute to (without overselling). Use this for title or a final synthesis-section or -sentence in the abstract. Be as brief but comprehensive as possible by itemizing tests, hypotheses, or data in the abstract, including non-significant results.
Relevant studies might not be identified, as the presented findings might be seen as out of scope.
A paper cannot be accessed by the metaanalyst's research institution.
If a publication cannot be made open-access, some journals allow users to host a manuscript in an unedited form. Otherwise, respond timely to pdf requests.
The study is identified as relevant, but the results cannot be used, and the paper will be listed as "not accessible" in the meta-analysis study.
Reporting usable outcomes to enable calculation of effect sizes Deficiencies in reporting statistics Present either raw data or summary statistics for the response, including means, variation around the mean and number of samples for both the control and treatment groups. If raw data are presented in figures avoid overlapping points in scatterplots, use transparency and indicate full overlap. Aggregation of (raw) data should be clearly documented (e.g. are the means and confidence intervals the means of individual organisms or plot means). In regressiontype analyses either raw data or regression slopes should be presented along with their confidence intervals, in addition to the coefficient of determination R².
Missing statistic information hampers effect size calculation. Reporting negative or non-significant results prevents publication bias and enables replicability of the analyses.
Deficiencies in describing the study and experimental design.
Include the number and sizes of sampling sites, plots within sampling sites, and replicates within plots. In experimental studies, include the nature of replication; full details of how hierarchical designs were accounted for in the analyses (e.g. blocking, split plot designs, nested designs).
The number of sampling sites and plots may be used in effect size calculation and is useful to understand the primary research study, to make (spatial) comparisons across climates and regions, and facilitates using geographical information in largerscale syntheses. Study design is reported in previous papers by same authors, worst case in an inaccessible technical paper.
Be as specific as possible in describing the study design, use supplementary material section to report details even if you repeat information given in previous papers.
Study results cannot be related to the original plots/observations/data and the corresponding moderators in the meta-analysis. Missing exact geographic location. Include coordinates, geographic system, altitude, and depth as accurately as possible.
Studies cannot be mapped nor related to regional climate data, general vegetation and other mapped data. Study locations may also be needed to check whether the selection of primary research studies are geographically biased or representative with respect to environmental and socio-economic context. Full environmental features of the study area are not reported.
Include information on the climate, soil, elevation, and the exact type of ecosystem (there are many kinds of wetlands or forests) and other study-specific and system-specific results.
Complete reporting of such metadata helps with the interpretation of the results of individual studies. Conditions could be used as a covariate in the meta-analysis or help to group studies. Historical conditions unknown.
If your paper relates to environmental degradation or in general refers to any kind of change processes, describe in detail how the study area or system was affected and when the degradation occurred (e.g., duration, start and end dates,).
Complete reporting of such metadata helps with the interpretation of the results of individual studies. Conditions could be used as a covariate in the meta-analysis or help to group studies. No reference system for species taxonomy/ species names given.
Adapt the species names to an accepted taxonomic list (e.g. the plant list) and state which taxonomy at which time point was adapted.
Reference systems are needed to correctly match datasets on species occurrences and abundances from various primary research studies. Spatial scale is not sufficiently or correctly reported.
Include information about spatial grain (e.g., size of plots), focus and extent. Include the size of the whole experimental area, if exists, and the study area.
Complete reporting of spatial scale facilitates the analysis of scaledependence of results, allows to estimate a "study area" and add additional georeferenced data which might explain observed patterns and explain the geographical reach of the meta-analysis. Temporal scale is not sufficiently reported.
Include information about the exact start and end of observations or experiments.
Temporal information might be needed to analyse whether observed patterns are consistent over time to estimate whether community dynamics are responsible for the observed patterns, and to distinguish short-term from long-term results.
Increasing data accessibility Databases are not easily accessible. Provide data in a digital format in data sheets (e.g., xls, csv, shp) rather than text based formats (pdf).
Increased effort is needed to gain access to the data, which may lead to ambiguity and error in subsequent analysis steps. Data are no longer available or link to data repository does not work.
Ensure that the data are still available in five, 20, or 50 years using public, global repositories that guarantee long-time maintenance, at best keeping track on improved versions and updates. Examples are datadryad.org, figshare.com, pangea.de which also provide digital object identifier (DOI) for data set(s).
Filing data at the authors', publishers' or institutions' websites may not guarantee longer-term availability as they are updated or become unsupported, fail or are taken offline.
Meta-data are not provided or are incomplete.
Ensure that metadata are completely provided, e.g., principal investigator, responsible institution, site location including geographic coordinates, plot size, scale of experiment, year, and environmental conditions. Available meta-data standards (i.e. Dublin Core) can provide guidelines.
Alongside results, various aspects of a study are of relevance that further help to use and analyse the study results. A study might even drop out from a meta-analytic analysis if important information is missing, which cannot be imputed. Data requests are not responded to or responses are delayed.
Respond to data requests as soon as possible.
Researchers have an obligation to their funding and salary sources to report their results fully and accurately, and to the organisms and systems they study to make their results available to science.
Timely responses to data requests allow having meta-analytical databases of higher quality, with substantially more information than with few or delayed responses.
due to biologically meaningful and important covariates (organism traits, climate, population density, etc.) or to methodological covariates (study duration, experimental conditions, source of material used). Potential confounding of covariates must be directly discernible (e.g. all woody species were grown in the ground, all herbaceous species in pots, so one cannot separate growth form from growth conditions). Detailed description of experimental methods and study design should include the number and sizes of sampling sites, plots within sampling sites, replicates within plots and study duration. Description of study area should include exact geographical location (as geographical coordinates or as a map) as it may be needed for various reasons: First, to access regional climate data, general vegetation and other mapped data. Second, to map study locations and check whether the selection of primary research studies are geographically biased (Martin, Blossey & Ellis 2012) or representative with respect to environmental and socio-economic context (Margulies et al. 2016). Further information should be provided on the environmental context such as climate, soil, elevation, and the specific type of ecosystem and other study-specific and system-specific results.
Other contextual features, such as land-use history, might be useful to interpret results and make them comparable with other sites or studies. If a study investigates effects of environmental change, describing how the study area or system was affected (duration of the impact and start and end date, any measure of the intensity of the alteration) helps with the interpretation of the results of individual studies and may be useful as a covariate in meta-analysis or help to group studies. To correctly match datasets on species occurrences and abundances from primary research studies authors must report species names according to a documented taxonomic authority (e.g. the plant list) and state which taxonomy at which time point was adapted.
In ecology, the spatial scale of observation deserves specific treatment (Whittaker 2010;Chase & Knight 2013). However, primary research studies frequently fail to report the details of the spatial scale of the study by incorrectly distinguishing between spatial scale components grain, focus, and extent, and even erroneously reporting the spatial scale to which the summary statistics belong (Whittaker, Willis & Field 2001). Grain denotes the size of the analytical unit. Focus is the area or inference space represented by each data point and thus represents the scale at which the grains are aggregated or the scale at which a mean is calculated. Extent is the scale at which the entire set of sample-units is analysed (Scheiner et al. 2000). While grain is frequently reported, focus and extent are often missing or descriptions do not allow clear distinctions among the spatial scale components. Such ambiguity hampers the analysis of spatial scale effects, and makes among-study comparisons less reliable.

I N C R E A S I N G D A T A A C C E S S I B I L I T Y
Complete access to all data in published studies has been a much-debated issue (e.g. Gewin 2016). For meta-analyses, however, it is extraordinarily useful. Although most results in primary research studies are provided in tables or figures and can be extracted by the meta-analysts, the underlying data should be made available online, because they may reduce effort, ambiguity and error in subsequent analysis steps. Access to archived data is guaranteed if digital object identifiers are provided and data stored in global repositories, like figshare (https://figshare.com), Dryad (https://www.datadryad.org) or Pangaea (https://www.pangaea.de). In contrast, filing data at the authors', publishers' or institutions' websites may not guarantee longer term availability as they are updated or become unsupported, fail or are taken offline. It is critical to include sufficient metadata along with the archived data. The paper on the publisher's homepage should be always linked with this repository. This 'Open Science' approach is now widely supported and is requested by many journals and funding agencies. However, it still lacks full support in the academic reward system (Nosek et al. 2015).
To obtain the required data missing from published papers, the last option for a meta-analyst is to request the data directly from the study authors. This is usually the most time consuming strategy as responses are often delayed, ignored or data access is only granted in exchange for co-authorship. However, researchers have an obligation to their funding and salary sources to report their results fully and accurately, and to the organisms and systems they study to make their results available to science. As data reporting standards in primary research studies improve in the future, the need for data requests will likely diminish.

Discussion
Comprehensive reporting of results is important for understanding the primary research study and also facilitates future syntheses, as it will provide a more general understanding across different systems or experimental replicates. This may be seen as an opportunity for primary research authors as it will increase study's transparency and clarity, and increase the options to use a study's data and findings, thereby increasing citations and broaden its impact. It could also stimulate the willingness of scientists to conduct more meta-analyses once selecting and extracting the data seem less daunting. We estimate that the effort required to address these recommendations would not impose a major burden to the authors compared to the effort required to collect, analyse and publish the data.
There are also implications for editors, publishers and funders. To make the value of primary research longer lasting and have broader impact, journal editors, reviewers and publishers should provide published guidelines that require the listed information on the results of the study be included upon initial submission. For example, studies should report the size of experiment, constraints, environment and scale of investigation (some journals like 'Nature' and 'Nature Communications' already have these basic standards). Further, journals should require that data supporting the results in papers published in its journals will be archived in an appropriate public archive (e.g. 'Methods in Ecology and Evolution'). Editors and publishers should consider treating every study used in a meta-analysis as a full citation by including a second list of references either within the main paper after the primary reference list in a reduced font size (e.g. 'Global Ecology and Biogeography') or in the online version of the paper (e.g. 'Nature') instead of listing them in the appendices or supplementary material. This will ensure that they are indexed and earn the deserved citation (McGill, Dornelas & Field 2016). This is, however, highly controversial in the scientific publication world. Space limitations should not be a prohibitive factor in an era in which printed journals have become a rarity and data syntheses are increasingly essential.
Data-based synthesis will foster ecological research and provide decision makers with valuable knowledge, institutionalized in synthesis centres in ecology, biology, biodiversity and other fields (see the Synthesis Consortium; http://synthesis-consortium.org). Other forms of meta-studies such as systematic reviews would also surely benefit from better result reporting as suggested by our guidelines (Magliocca et al. 2015). Scientists need to acknowledge that each piece of research could be more than just a publication on a list; it could be an important part of a larger picture. Finally, the more studies that can be included in meta-analyses, the more likely we will be able to glean a more comprehensive understanding of the generalities and tests of theory that meta-analyses in ecology often seek to build.

Data accessibility
Manuscript does not include any data.  Report actual P-values, test statistics and sample sizes, rather than "> 0·05" or asterisks. Report test-statistics alongside P-values.
Report all outcomes, significant as well as those that are not statistically significant; avoid "n.s.". Alternatively, some results can be shown in the supplementary material.
Show means with variation (and identify nature of measure of variation) in figures.
Always report exact location as coordinates (preferably decimal degrees) and give Northing/Easting and units (°, ', ''). If exact locations are not revealed on purpose (i.e. to protect endangered species) provide the reason in the paper.

Intensified
Report all data separately for all treatments and treatment combinations and observation units. Explain what the means represent (e.g. means of individual plants, of plots, etc.), and report how replicates were assigned and number of independent replicates (i.e., sample size in which the means were calculated).

*
If significance is labeled with asterisks in figures, provide the exact P-value in text or table as well Species A Species B, C, D

**
Provide detailed information on sampling area of study (especially for observational or experimental work conducted in the field).
Avoid overlapping points in scatterplots, use transparency and indicate full overlap.