138
Some Cochrane risk of bias items are not important in osteoarthritis trials: A meta-epidemiological study based on Cochrane reviews
Twenty reviews including 126 OA trials with a high degree of heterogeneity was included (τˆ2=0.1247). Among RoB domains only patient blinding had an impact on the results (reducing heterogeneity according to τˆ2 <7%). Inadequate blinding of patients yielded larger effects (SMDDifference = 0.15; 95% CI: 0.11 to 0.29, P=0.035). The most important study characteristic was trial size (heterogeneity reduced by 25%), with small trials reporting larger effects (SMDDifference = 0.29; 95% CI: 0.16 to 0.42, P<0.001).
Science  RiskOfBias 
6 days ago
Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis
When considering the index (primary or first reported) meta-analysis of each SR, just over half (62/110 [56%]) used the random-effects model, but few (5/62 [8%]) interpreted the meta-analytic effect correctly (as the average of the intervention effects across all studies). A statistical test for funnel plot asymmetry was reported in 17/110 (15%) SRs, however, in only 4/17 (24%) did the test include the recommended number of at least 10 studies of varying size. Subgroup analyses accompanied 42/110 (38%) index meta-analyses, but findings were not interpreted with respect to a test for interaction in 29/42 (69%) cases, and the issue of potential confounding in the subgroup analyses was not raised in any SR.
Science  StatsMethods 
6 days ago
Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics
Our interpretation of the results is limited by imprecise estimates. There is some indication that between-trial heterogeneity could be partially explained by reported design characteristics, and hence adjustment for bias could potentially improve accuracy of meta-analysis results.
Science  RiskOfBias 
6 days ago
Differences between protocols for randomized controlled trials and systematic reviews
How to game a protocol? Although we fully acknowledge the potential benefits of protocols for SRs, it is important to point out that there is a clear difference between protocols for SRs when compared to protocols for RCTs.
Science  SRStandards  Protocol 
6 days ago
Potential Developmental and Reproductive Impacts of Triclocarban: A Scoping Review
Scoping reviews are showing signs of improving in value through use of systematic methods to identify and characterise evidence so as to reliably identify health endpoints for future systematic review and to make evidence-based recommendations for future research. This scoping review found 32 relevant studies in humans, rodents, fish, invertebrates, and in vitro. Based on the robustness of the literature, the review identifies estrogenic activity, androgenic activity, and offspring growth as outcomes for potential systematic review.
Science  SystematicMap 
12 days ago
Association between Exposure to p,p′-DDT and Its Metabolite p,p′-DDE with Obesity: Integrated Systematic Review and Meta-Analysis
We classified p,p′-DDT and p,p′-DDE as “presumed” to be obesogenic for humans, based on a moderate level of primary human evidence, a moderate level of primary in vivo evidence, and a moderate level of supporting evidence from in vivo and in vitro studies.
Science  SRCaseStudy 
13 days ago
Make reviews public, says peer review expert Irene Hames
This is a nice read, about how peer-review might be improved without having to completely overhaul the system. Good comments and responses from Irene Hames below the line.
News  PeerReview 
18 days ago
Evaluation of a new method for librarian-mediated literature searches for systematic reviews
Single-line search strategies were prepared in a text document. Term completeness was ensured with a novel optimization technique. Macros in MS Word converted the syntaxes between databases and interfaces almost automatically. The ESM searches differed by using two times more databases, retrieving 44% more references, including 20% more studies in the final systematic review, but the time needed for the search was 8% of that of the control group.
Science  SearchMethods 
18 days ago
Systematic Review and Meta-Analysis of Early-Life Exposure to Bisphenol A and Obesity-Related Outcomes in Rodents
Overall summary estimates indicated significant positive associations between BPA and fat weight [SMD=0.67 (95% CI: 0.53, 0.81)], triglycerides [SMD=0.97 (95% CI: 0.53, 1.40)], and FFA [SMD=0.86 (95% CI: 0.50, 1.22)], and a nonsignificant positive association with leptin levels [MD=0.37 (95% CI: -0.14, 0.87)] and a significant negative association with body weight were estimated [MD=-0.22 (95% CI: -0.37, -0.06)]. Subgroup analyses revealed stronger positive associations for most outcome measures in males and at doses below the current U.S. reference dose of 50μg/kg/d compared with doses above the reference dose. It should be noted that there was substantial heterogeneity across studies for all outcomes assessed and that there was insufficient information to assess risk of bias for most studies.
Science  SRCaseStudy 
18 days ago
Acute effects of fine particulate matter constituents on mortality: A systematic review and meta-regression analysis
Our meta-analysis suggests that (a) combustion elements such as EC and K have a stronger association with mortality, (b) single lag studies underestimate effects, and (c) estimates of PM2.5 and constituents differ across regions. Accounting for PM mass in constituent's health models may lead to more stable and comparable effect estimates across different studies.
Science  SRCaseStudy 
18 days ago
Weight of evidence approaches for the identification of endocrine disrupting properties of chemicals: Review and recommendations for EU regulatory application
This paper reviews WoE approaches to distil key recommendations specifically for the evaluation of potential ED properties of chemicals. In a manner, which is consistent with existing, published WoE frameworks, the WoE evaluation of ED properties can be divided into four phases: 1) Definition of causal questions and data gathering and selection, 2) Review of individual studies, 3) Data integration and evaluation, and 4) Drawing conclusions based on inferences. Recommendations are made on how to conduct each phase robustly and transparently to help guide the WoE evaluation of potential endocrine disrupting properties of chemicals within a European regulatory context.
Science  WeightOfEvidence 
18 days ago
Guidelines for performing systematic reviews in the development of toxicity factors
The Texas Commission on Environmental Quality (TCEQ) developed guidance on conducting systematic reviews during the development of chemical-specific toxicity factors. Using elements from publicly available frameworks, the TCEQ systematic review process was developed in order to supplement the existing TCEQ Guidelines for developing toxicity factors (TCEQ Regulatory Guidance 442).
Science  SRStandards 
18 days ago
Citation analysis is also useful to assess the eligibility of biomedical research works for inclusion in Living systematic reviews. - PubMed - NCBI
In this research letter, I explain how the analysis of the citation network and of the co-citation network of a given topic and the use of the snowballing techniques can be useful to verify the eligibility of a paper for inclusion in the Living Systematic Review (LSR) of that topic.
SearchMethods  Science 
27 days ago
Systematic Review and Meta-Analysis of Early-Life Exposure to Bisphenol A and Obesity-Related Outcomes in Rodents
Overall summary estimates indicated significant positive associations between BPA and fat weight [SMD=0.67 (95% CI: 0.53, 0.81)], triglycerides [SMD=0.97 (95% CI: 0.53, 1.40)], and FFA [SMD=0.86 (95% CI: 0.50, 1.22)], and a nonsignificant positive association with leptin levels [MD=0.37 (95% CI: −0.14, 0.87)] and a significant negative association with body weight were estimated [MD=−0.22 (95% CI: −0.37, −0.06)]. Subgroup analyses revealed stronger positive associations for most outcome measures in males and at doses below the current U.S. reference dose of 50 μg/kg/d compared with doses above the reference dose. It should be noted that there was substantial heterogeneity across studies for all outcomes assessed and that there was insufficient information to assess risk of bias for most studies.
Science  SRCaseStudy 
9 weeks ago
Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms
Data extraction is one of the most time-consuming tasks in performing a systematic review. Extraction is often onto some sort of form. Sharing completed forms can be used to check quality and accuracy of extraction or for re-cycling data to other researchers for updating. However, validating each piece of extracted data is time-consuming and linking to source problematic.In this methodology paper, we summarize three methods for reporting the location of data in original full-text reports, comparing their advantages and disadvantages
Science  DataExtraction 
10 weeks ago
Gray literature: An important resource in systematic reviews
Gray literature's diverse formats and audiences can present a significant challenge in a systematic search for evidence. However, the benefits of including gray literature may far outweigh the cost in time and resource needed to search for it, and it is important for it to be included in a systematic review or review of evidence.
Science  SearchMethods 
10 weeks ago
Cleaning and asthma: A systematic review and approach for effective safety assessment
This proposed WoE method organizes diverse lines of data (i.e., asthma, sensitization, and irritation information) through a systematic, hierarchical framework that provides qualitatively categorized conclusions using hazard bands to predict a specific product or ingredient's potential for asthma induction. This work provides a method for prioritizing chemicals as a first step for quantitative and scenario-specific safety assessments based on their potential for inducing asthmatic effects. Acetic acid is used as a case study to test this framework.
Science  SRCaseStudy  RiskAssessment 
10 weeks ago
Review of reviews on exposures to synthetic organic chemicals and children's neurodevelopment: Methodological and interpretation challenges
Seventy-four relevant reviews were identified, and these were evaluated with respect to four methodological characteristics: (1) systematic inclusion/exclusion criteria and reproducible methods for search and retrieval of studies; (2) structured evaluation of underlying data quality; (3) systematic assessment of consistency across specific exposure-outcome associations; and (4) evaluation of reporting/publication bias. None of the 74 reviews fully met the criteria for all four methodological characteristics. Only four reviews met two criteria, and six reviews fulfilled only one criterion. Perhaps more importantly, the higher quality reviews were not able to meet all of the criteria owing to the shortcomings of underlying studies.
Science 
10 weeks ago
Sloppy reporting on animal studies proves hard to change
Papers from the “treatment group” improved on only two ARRIVE items: husbandry and housing. For all others, authors had duly filled out the checklist but their papers were not actually more compliant. “Apparently, a checklist alone doesn't help much,” Ritskes-Hoitinga says. Another reason may be that PLOS ONE editors didn't enforce compliance.
News 
10 weeks ago
Developmental PBDE Exposure and IQ/ADHD in Childhood: A Systematic Review and Meta-analysis. - PubMed - NCBI
Our meta-analysis of four studies estimated a 10-fold increase (in other words, times 10) in PBDE exposure associated with a decrement of 3.70 IQ points (95% confidence interval: 0.83, 6.56). We concluded there was sufficient evidence supporting an association between developmental PBDE exposure and reduced IQ. Preventing developmental exposure to PBDEs could help prevent loss of human intelligence.
Science  SRCaseStudy 
12 weeks ago
Cumulative effects of prenatal-exposure to exogenous chemicals and psychosocial stress on fetal growth: Systematic-review of the human and animal evidence
We found that despite concern for the combined effects of environmental chemicals and stress, this is still an under-studied topic, though limited available human studies indicate chemical exposures exert stronger effects than stress, and this effect is generally larger in the presence of stress.
Science  SRCaseStudy 
12 weeks ago
Estimating the health benefits of environmental regulations
Assessing health benefits of policies addressing environmental contaminants is important for decision-making and for informing the public about how policy affects their welfare. Benefits analysis, one side of benefit-cost analysis (BCA), can be relatively straightforward when sufficient data are available on dose-response relationships, changes in exposure expected from a proposed policy, and other key inputs. But despite progress, benefits analysis for health effects is needlessly constrained by analytic practices that are scientifically outdated and inconsistent with economic theory.
Science 
12 weeks ago
Most systematic reviews of high methodological quality on psoriasis interventions are classified as high risk of bias using ROBIS tool. - PubMed - NCBI
We classified 139 intervention SRs as displaying high/moderate/low methodological quality, and as high/low risk of bias. A high risk of bias was detected for most SRs classified as displaying high or moderate methodological quality by AMSTAR. When comparing ROBIS result profiles, responses to domain 4 signalling questions showed the greatest differences between bias risk assessments, while domain 2 items showed the least.
Science  RiskOfBias 
september 2017
Favorable and Publicly-Funded Studies are More Likely to be Published: A Systematic Review and Meta-Analysis
The probability of publication was significantly higher for studies whose characteristics were favorable (OR=2.04, 95%CI: 1.62, 2.57) or statistically significant (OR=2.07, 95%CI: 1.52, 2.81), had a multicenter design (OR=1.32, 95%CI: 1.16, 1.45), and were of later regulatory phase (3/4 vs 1/2, OR=1.34, 95%CI: 1.14, 1.49). Industry funding was modestly associated with lower (OR=0.81, 95%CI: 0.67, 0.99) probability of publication. An exploratory analysis of effect modification revealed that the effect of the study characteristic 'favorable results' on likelihood for publication was stronger for industry-funded studies.
Science  SRStandards 
august 2017
Guidelines for reporting meta-epidemiological methodology research
Meta-epidemiological studies adopt a systematic review or meta-analysis approach to examine the impact of certain characteristics of clinical studies on the observed effect and provide empirical evidence for hypothesised associations. The unit of analysis in meta-epidemiological studies is a study, not a patient. The outcomes of meta-epidemiological studies are usually not clinical outcomes. In this guideline, we adapt items from the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) to fit the context of meta-epidemiological studies.
Science  BetterScience 
august 2017
Inter-rater reliability of AMSTAR is dependent on the pair of reviewers. - PubMed - NCBI
Inter-rater reliability varies widely depending on the pair of reviewers. There may be some shortcomings associated with conducting reliability studies with only two reviewers. Further studies should include additional reviewers and should probably also take account of their level of expertise.
Science  SRAppraisal 
august 2017
The risk of bias in systematic reviews (ROBIS) tool showed fair reliability and good construct validity. - PubMed - NCBI
There is a movement from generic quality checklists towards a more domain based approach in critical appraisal tools. This study aims to report on a first experience with the newly developed risk of bias in systematic reviews (ROBIS) tool and compare it with A Measurement Tool to Assess Systematic Reviews (AMSTAR) that is the most common used tool to assess methodological quality of systematic reviews while assessing validity, reliability and applicability.
Science  SRAppraisal 
august 2017
Testing the effectiveness of simplified search strategies for updating systematic reviews
SSS performed well for clinically focused topics and, with a median sensitivity of 100%, could be a viable alternative to a conventional comprehensive search strategy for updating this type of systematic reviews particularly considering the budget constraints and the volume of new literature being published. For broader topics, 80% sensitivity is likely to be considered too low for a systematic review update in most cases, although it might be acceptable if updating a scoping or rapid review.
Science  SearchMethods 
august 2017
Reproducibility of Studies on Text Mining for Citation Screening in Systematic Reviews: Evaluation and Checklist
The application of text mining techniques to citation screening in the context of systematic literature reviews is a relatively young and growing computational field with high relevance for software engineering, medical research and other fields. However, our work revealed that it is currently difficult if not impossible to independently reproduce the results published in any of the studies investigated. The lack of information about the datasets used limits reproducibility of about 80% of the studies assessed. Also, information about the machine learning algorithms is inadequate in about 27% of the papers. On the plus side, the third party software tools used are mostly free and available.
Science  SearchMethods 
august 2017
Developmental PBDE Exposure and IQ/ADHD in Childhood: A Systematic Review and Meta-analysis
We concluded there was sufficient evidence supporting an association between developmental PBDE exposure and reduced IQ. Preventing developmental exposure to PBDEs could help prevent loss of human intelligence.
Science  SRCaseStudy 
august 2017
The influence of the team in conducting a systematic review
This commentary highlights a number of relevant articles indicating how the composition of the review team could jeopardise the integrity of the systematic review study and its conclusions. Key biases require closer attention such as sponsorship bias and researcher allegiance, but there may also be less obvious affiliations in teams conducting secondary evidence-syntheses. The importance of transparency and disclosure are now firmly on the agenda for clinical trials and primary research, but the meta-biases that systematic reviews may be at risk from now require further scrutiny.
Science  BetterScience 
august 2017
Producing Cochrane systematic reviews-a qualitative study of current approaches and opportunities for innovation and improvement
The results suggest that improvements to Cochrane's systematic review production models could come from improving clarity of roles and expectations, ensuring continuity and consistency of input, enabling active management of the review process, centralising some review production steps; breaking reviews into smaller "chunks", and improving approaches to building capacity of and sharing information between authors and Review Groups. Respondents noted the important role new technologies have to play in enabling these improvements.
Science  BetterScience 
august 2017
Reconciling disparate data to determine the 'right' answer: A grounded theory of meta analysts' reasoning in meta-analysis
Results show the complexity of meta-analytic reasoning. At each step of the process, participants attempted to reconcile disparate forms of knowledge to determine a 'right' answer (moral concern) and 'accurately' draw a treatment effect (epistemological concern). The reasoning processes often shifted between considering the meta-analysis as if the data were 'whole', and as if the data were discrete components (individual studies). These findings highlight paradigmatic tensions regarding the epistemological premises of meta-analysis, resembling previous historical investigations of the functioning of scientific communities. In understanding why different meta-analysts interpret data differently, it may be unrealistic to expect 'objective' homogenous recommendations based on meta-analyses.
Science  MetaAnalysis 
august 2017
Systematic review of community health impacts of mountaintop removal mining
This systematic review could not reach conclusions on community health effects of MTR mining because of the strong potential for bias in the current body of human literature. Improved characterization of exposures by future community health studies and further study of the effects of MTR mining chemical mixtures in experimental models will be critical to determining health risks of MTR mining to communities. Without such work, uncertainty will remain regarding the impact of these practices on the health of the people who breathe the air and drink the water affected by MTR mining.
Science  SRCaseStudy 
august 2017
Assessing Exposure to Household Air Pollution: A Systematic Review and Pooled Analysis of Carbon Monoxide as a Surrogate Measure of Particulate Matter
Our results suggest that exposure to CO is not a consistently valid surrogate measure of exposure to PM2.5. Studies measuring CO exposure as a surrogate measure of PM exposure should conduct local validation studies for different stove/fuel types and seasons.
Science  SRCaseStudy 
august 2017
History shows that stacking federal science advisory committees doesn't work
Recommendations from “friendly” panels will not win broad support from the scientific community, and I predict the committees will quickly lose their credibility, legitimacy and influence. Consequently, policies and regulations based on the panels’ recommendations will be less likely to withstand public or political scrutiny and be more open to legal challenges than if they were based on more balanced input.
News  ScienceAdvice 
july 2017
Promote scientific integrity via journal peer review data
There is an increasing push by journals to ensure that data and products related to published papers are shared as part of a cultural move to promote transparency, reproducibility, and trust in the scientific literature. Yet few journals commit to evaluating their effectiveness in implementing reporting standards aimed at meeting those goals.
News  BetterScience 
july 2017
Temporal trends in sperm count: a systematic review and meta-regression analysis
This comprehensive meta-regression analysis reports a significant decline in sperm counts (as measured by SC and TSC) between 1973 and 2011, driven by a 50–60% decline among men unselected by fertility from North America, Europe, Australia and New Zealand. Because of the significant public health implications of these results, research on the causes of this continuing decline is urgently needed.
Science  SRCaseStudy 
july 2017
Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals
Endocrine active chemicals (EACs) have raised concerns that traditional toxicity-testing protocols might be inadequate to identify all potential hazards to human health because they have the ability to modulate normal hormone function, and small alterations in hormone concentrations, particularly during sensitive life stages, can have lasting and significant effects. To address concerns about potential human health effects from EACs at low doses, this US NAS report develops a strategy to evaluate the evidence for such low-dose effects.
Science  SRCaseStudy 
july 2017
Global movement on endocrine disruptors
National Academies warns endocrine active chemicals may be harmful at lower doses than the US currently tests for.
News  SRCaseStudy 
july 2017
A systematic review of cardiovascular emergency department visits, hospital admissions and mortality associated with ambient black carbon
We conducted a systematic review to evaluate the potential for cardiovascular morbidity and mortality following exposure to ambient BC, or the related component elemental carbon (EC), in the context of what is already known about the associations between exposure to fine particulate matter (PM2.5) and cardiovascular health outcomes.
Science  SRCaseStudy 
july 2017
Methodological and Reporting Quality of Systematic Reviews Published in the Highest Ranking Journals in the Field of Pain
Endorsement of PRISMA in instructions for authors was not a guarantee of compliance. Methodological and reporting quality of pain-related SRs should be improved using relevant checklists. This can be remedied by a joint effort of authors, editors, and peer reviewers.
Science  SRStandards 
july 2017
Towards a taxonomy of logic models in systematic reviews and health technology assessments: a priori, staged and iterative approaches
The complexity associated with how interventions result - or fail to result - in outcomes, and how context matters is increasingly recognised. Logic models provide an important tool for handling complexity. This proposed taxonomy of logic models offers an improved understanding of the advantages and limitations of logic models across the spectrum from a priori to fully iterative approaches. Choice of logic model should be informed by scope of evidence synthesis, presence/absence of clearly defined PICO elements and feasibility considerations.
Science  ExternalValidity 
july 2017
Systematic review adherence to methodological or reporting quality
Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.
Science  SRStandards 
july 2017
Selecting and implementing overview methods: implications from five exemplar overviews
Overviews are a relatively new methodological innovation, and there are currently substantial variations in the methodological approaches used within different overviews. There are considerable methodological challenges for which optimal solutions are not necessarily yet known. Lessons learnt from five exemplar overviews highlight a number of methodological decisions which may be beneficial to consider during the development of an overview protocol.
Science  SRStandards 
july 2017
Comparison of a full systematic review versus a rapid review approaches to assess a newborn screening test for tyrosinemia type 1
Rapid reviews are increasingly used to replace or complement systematic reviews to support evidence based decision-making. Not enough is known about how this expedited process affects results. This study assesses the difference between rapid and systematic review approaches for a case study of test accuracy of succinylacetone for detecting tyrosinemia type 1.
Science  RapidReviews 
july 2017
Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study
Review finding that prevalence of poor reporting and inadequate methods in clinical trials has decreased over time, especially for sequence generation and allocation concealment. But more could be done, especially in lower impact factor journals. Suggests that awareness of methodological deficits in primary medical research are becoming better-recognised and picked up by at least the higher-impact journals.
Science  BetterScience 
july 2017
Relevance and reliability of experimental data in human health risk assessment of pesticides
This article describes an approach to assess the relevance and reliability of experimental data from guideline-compliant studies as well as from non-guideline studies published in the scientific literature in the specific context of uncertainty and risk assessment of pesticides.
Science  RiskOfBias  RiskAssessment 
july 2017
Meta-Analyses Can Be Credible and Useful
John Ioannidis outlines a strategic programme for improvement of systematic reviews in mental health research, covering meta-analytical methods and changes to primary research, which would also be applicable to the field of environmental health.
Science  BetterScience 
july 2017
Text mining for search term development in systematic reviewing: A discussion of some methods and challenges
We briefly review the literature on applications of text mining for search term development for systematic reviewing. We found that the tools can be used in 5 overarching ways: improving the precision of searches; identifying search terms to improve search sensitivity; aiding the translation of search strategies across databases; searching and screening within an integrated system; and developing objectively derived search strategies. Using a case study and selected examples, we then reflect on the utility of certain technologies (term frequency-inverse document frequency and Termine, term frequency, and clustering) in improving the precision and sensitivity of searches.
Science  SearchMethods 
july 2017
A WEIGHT OF EVIDENCE FRAMEWORK FOR ENVIRONMENTAL ASSESSMENTS: INFERRING QUALITIES
The weighing of heterogeneous evidence such as conventional laboratory toxicity tests, field tests, biomarkers, and community surveys is essential to environmental assessments. Evidence synthesis and weighing is needed to determine causes of observed effects, hazards posed by chemicals or other agents, the completeness of remediation, and other environmental qualities. As part of its guidelines for WoE in ecological assessments, the U.S. Environmental Protection Agency has developed a generally applicable framework. Its basic steps are: assemble evidence, weight the evidence, and weigh the body of evidence. Use of the framework can increase the consistency and rigor of WoE practices and provide greater transparency than ad hoc and narrative-based approaches.
Science  WeightOfEvidence 
june 2017
Evidence and Health Policy: Using and Regulating Systematic Reviews
Systematic reviews could inform policy more effectively with changes to current practices and the assumptions that sustain these practices—assumptions made by researchers and the organizations that employ them, by public and private funders of systematic reviews, and by organizations that finance, set priorities and standards for, and publish them.
Science  ScienceAdvice 
june 2017
Is the staggeringly profitable business of scientific publishing bad for science?
It is an industry like no other, with profit margins to rival Google – and it was created by one of Britain’s most notorious tycoons: Robert Maxwell.
News 
june 2017
Residential exposure to pesticides as risk factor for childhood and young adult brain tumors: A systematic review and meta-analysis
Our findings support an association between residential exposure to pesticides and childhood brain tumors. Although causality cannot be established, these results add to the evidence leading to recommend limiting residential use of pesticides and to support public health policies serving this objective.
Science  SRCaseStudy 
june 2017
Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study
The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.
Science  SRStandards 
june 2017
New approach to weight-of-evidence assessment of ecotoxicological effects in regulatory decision-making
Ecological risk assessments and risk management decisions are only as sound as the underlying information and processes to integrate them. It is important to develop transparent and reproducible procedures a priori to integrate often-heterogeneous evidence. Current weight-of-evidence (WoE) approaches for effects or hazard assessment tend to conflate aspects of the assessment of the quality of the data with the strength of the body of evidence as a whole. We take forward recent developments in the critical appraisal of the reliability and relevance of individual ecotoxicological studies as part of the effect or hazard assessment of prospective risk assessments and propose a streamlined WoE approach.
Science  WeightOfEvidence 
june 2017
The GRADE Working Group clarifies the construct of certainty of evidence
Certainty of evidence is best considered as the certainty that a true effect lies on one side of a specified threshold, or within a chosen range. We define possible approaches for choosing threshold or range. For guidelines, what we call a fully contextualized approach requires simultaneously considering all critical outcomes and their relative value. Less contextualized approaches, more appropriate for systematic reviews and health technology assessments, include using specified ranges of magnitude of effect, e.g. ranges of what we might consider no effect, trivial, small, moderate, or large effects.
Science  GradingEvidence 
june 2017
The Complex and Multi-Faceted Aspects of Conflicts of Interest
COI, a conflict between a professional responsibility and a personal interest, is at one end of this threat spectrum. COI creates a risk of bias. Bias, a prejudice for or against something, is in the middle of this spectrum. If a COI results in bias, the bias may affect a professional judgment. Dishonesty is deceit or fraud. Dishonesty is at the opposite end of this spectrum from COI. Each of these threats exists on a continuum. COI may be present or perceived. Bias may be conscious or unconscious. Dishonesty may be intentional or unintentional. This issue of JAMA contains a series of Viewpoints on the many facets of COIs.
Science  RiskOfBias 
june 2017
Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?
We identify two important dilemmas, for scientists and researchers, that arise from our initial advice. First, effective actors combine evidence with manipulative emotional appeals to influence the policy agenda – should scientists do the same, or would the reputational costs outweigh the policy benefits? Second, when adapting to multi-level policymaking, should scientists prioritise ‘evidence-based’ policymaking above other factors? The latter includes governance principles such the ‘co-production’ of policy between local public bodies, interest groups and service users. This process may be based primarily on values and involve actors with no commitment to a hierarchy of evidence.
Science  ScienceAdvice 
may 2017
Framework for assessing causality of air pollution-related health effects for reviews of the national ambient air quality standards
The United States Environmental Protection Agency (U.S. EPA) develops the Integrated Science Assessment (ISA), which contains evaluations of the policy-relevant science on the effects of criteria air pollutants and conveys critical science judgments to inform decisions on the National Ambient Air Quality Standards. This article discusses the approach and causal framework used in the ISAs to evaluate and integrate various lines of scientific evidence and draw conclusions about the causal nature of air pollution-induced health effects.
Science  RiskAssessment 
may 2017
A new risk of bias checklist applicable to randomized trials, observational studies and systematic reviews was developed and validated to be used for systematic reviews focusing on drug adverse events
The developed checklist examines eight domains: study design and objectives, selection bias, attrition, adverse events information bias, other information bias, statistical methods to control confounding, other statistical methods, and conflicts of interest. The total number of questions varied from ten to 32 depending on the study design. Inter- and intra-rater agreement were fair with Kendall's W of 0.70 and 0.74, respectively. Median time to complete the checklist was 8.5 minutes.
Science  RiskOfBias 
may 2017
Evaluation of a Rule-based Method for Epidemiological Document Classification Towards the Automation of Systematic Reviews
The generated performance of our text-mining approach demonstrated encouraging results for the identification of targeted information from observational epidemiological study abstracts related to environmental exposures. We have demonstrated that rules based on generic syntactic patterns in one corpus can be applied to other observational study design by simple interchanging the dictionaries aiming to identify certain characteristics (i.e., outcomes, exposures). At the document level, the recognized information can assist in the selection and categorization of studies included in a systematic review.
Science  SearchMethods 
may 2017
Addressing Bias and Conflict of Interest Among Biomedical Researchers
Attempting to identify and eliminate the interests that make scientists who they are serves as a distraction to prevent the research community, professionals, and policy makers from dealing with bias related to financial conflicts of interest. The diversity of viewpoints, motivations, and methodological approaches of researchers will advance science, while the narrowing of interests to financial ones will limit it to questions, methods, and reporting that favor the financial interest.
Science  RiskOfBias 
may 2017
Forcing consensus is bad for science and society
Science is a “show me”, not a “trust me”, field. Purporting to speak on behalf of all science, as the Nobel laureates sought to do with golden rice, conflated science, the scientific method and truth.
News  ScienceAdvice 
may 2017
A primer on systematic reviews in toxicology
To provide the toxicology community with a starting point for conducting or understanding systematic reviews, we herein summarized available guidance documents from various fields of application. We have elaborated on the systematic review process by breaking it down into ten steps, starting with planning the project, framing the question, and writing and publishing the protocol, and concluding with interpretation and reporting. In addition, we have identified the specific methodological challenges of toxicological questions and have summarized how these can be addressed.
Science  SRStandards 
may 2017
Importance of the distinction between quality of methodology and quality of reporting
REMARK addresses only how well authors reported key aspects of their methods and the findings of their research. Good reporting helps readers to judge whether the study design and analysis and interpretation were sound. But good reporting is not a measure of methodological quality, so a “REMARK score” cannot validly be used in that way. A similar error has been seen with the reporting of observational studies.
Science  SRStandards 
may 2017
Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research
Although we may anticipate that systematic reviews and meta-analyses not including unpublished or grey literature study results are likely to overestimate the treatment effects, current empirical research shows that this is only the case in a minority of reviews. Therefore, currently, a meta-analyst should particularly consider time, effort and costs when adding such data to their analysis. Future research is needed to identify which reviews may benefit most from including unpublished or grey data.
Science  PublicationBias 
may 2017
Milk and dairy consumption and risk of cardiovascular diseases and all-cause mortality: dose–response meta-analysis of prospective cohort studies
A cheesy meta-analysis finds "neutral associations between dairy products and cardiovascular and all-cause mortality".
SRCaseStudy  Science 
may 2017
An academic researcher's guide to increased impact on regulatory assessment of chemicals
However, in practice, standard tests conducted under GLP and sponsored and provided by industry are predominantly used. Peer-reviewed studies from independent sources are often disregarded or disputed since they often do not comply with regulatory data requirements and quality criteria. To help bridge such a gap, the aim of this paper is to give an overview of the general workings of legislation of chemicals and propose a set of actions to increase the usability of research data.
Science  BetterScience  ScienceAdvice 
may 2017
Methodology of the ‘craft’ of scientific advice for policy and practice
Scientific advice for policy should be seen as a craft that needs professional development as such. A crucial requirement is that, as science itself, science-informed advice for policy and practice can and should only be convincing when it is based on solid methodology. Essential steps include problem formulation, synthesising evidence, and building the bridge to implementation - but not crossing it.
Science  ScienceAdvice 
may 2017
Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews
The majority of SRs searched for non-English and unpublished studies; however, these represented a small proportion of included studies and rarely impacted the results and conclusions of the review. Inclusion of these study types may have an impact in situations where there are few relevant studies, or where there are questionable vested interests in the published literature. We found substantial variation in whether SRs searched for dissertations; in most reviews that included dissertations, these had little impact on results.
Science  PublicationBias 
may 2017
On the Need for Quantitative Bias Analysis in the Peer-Review Process
Reviewers who insist that quantitative bias analysis be incorporated into the design, conduct, presentation, and interpretation of epidemiologic research could substantially strengthen the process. In the present commentary, we demonstrate how quantitative bias analysis can be used by investigators and authors, reviewers, funding agencies, and editors. By utilizing quantitative bias analysis in the peer-review process, editors can potentially avoid unnecessary rejections, identify key areas for improvement, and improve discussion sections by shifting from speculation on the impact of sources of error to quantification of the impact those sources of bias may have had.
Science  RiskOfBias  SRStandards 
may 2017
The albatross plot: A novel graphical tool for presenting results of diversely reported studies in a systematic review.
Meta-analytical methods can only be used if comparable effect sizes can be computed from each study, and this may not be the case due to variation in how the studies were done or limitations in how their results were reported. Other methods, such as vote counting, are then used to summarize the results of these studies, but most of these methods are limited in that they do not provide any indication of the magnitude of effect. We propose a novel plot, the albatross plot, which requires only a 1-sided P value and a total sample size from each study (or equivalently a 2-sided P value, direction of effect and total sample size).
Science  MetaAnalysis 
may 2017
Systematic review of the potential adverse effects of caffeine consumption in healthy adults, pregnant women, adolescents, and children
The results of this systematic review support a shift in caffeine research to focus on characterizing effects in sensitive populations and establishing better quantitative characterization of interindividual variability (e.g., epigenetic trends), subpopulations (e.g., unhealthy populations, individuals with preexisting conditions), conditions (e.g., coexposures), and outcomes (e.g., exacerbation of risk-taking behavior) that could render individuals to be at greater risk relative to healthy adults and healthy pregnant women. This review, being one of the first to apply systematic review methodologies to toxicological assessments, also highlights the need for refined guidance and frameworks unique to the conduct of systematic review in this field.
Science  SRCaseStudy 
may 2017
Bias Analysis for Uncontrolled Confounding in the Health Sciences
We review methods that can be applied during or after data analysis to adjust for uncontrolled confounding for different outcomes, confounders, and study settings. We discuss relevant bias formulas and how to obtain the required information for applying them. Finally, we develop a new intuitive generalized bias analysis framework for simulating and adjusting for the amount of uncontrolled confounding due to not measuring and adjusting for one or more confounders.
Science  RiskOfBias 
april 2017
The science of stakeholder engagement in research: classification, implementation, and evaluation
We propose a classification system with definitions to determine where projects lie on the stakeholder engagement continuum. We discuss the key elements of implementation and evaluation of stakeholder engagement in research posing key questions to consider when doing this work. We commend and critique the work of Hamilton et al. in their multilevel stakeholder engagement in a VA implementation trial of evidence-based quality improvement in women's health primary care.
ProblemFormulation  Science 
april 2017
Can systematic reviews contribute to regulatory decisions?
Systematic reviews may simultaneously produce new findings and summarize existing knowledge, with the potential of informing regulatory decisions more pragmatically and more rapidly than other research designs. We suggest that national and international calls on independent research on drugs should not put primary clinical research against systematic reviews, as it implies a focus on the methods instead of on the questions being asked.
BetterScience  Science 
april 2017
Can abstract screening workload be reduced using text mining? User experiences of the tool Rayyan
After screening half of the search result with Rayyan, 86% to 99% of the references deemed relevant to the study were identified. Of those studies included in the final reports, 96% to 100% were already identified in the first half of the screening process. Users rated Rayyan 4.5 out of 5.
SearchMethods  Science 
april 2017
Statistics Myth Busters: Dispelling Common Misperceptions Held by Readers of the Biomedical Literature
The myths discussed involve 3 common areas of consideration when evaluating any clinical study: assessing the risk of bias from confounding (propensity score analysis and multivariable modeling), interpretation of the main study findings (P values and hypothesis testing), and secondary evaluations (subgroup analyses). Literature examples are used to illustrate each of the topics.
MetaAnalysis  Science 
april 2017
« earlier      
agencysr betterscience controversy critappstudies dataextraction evidencemapping externalvalidity gradingevidence heterogeneity metaanalysis news peerreview problemformulation protocol publicationbias rapidreviews resourcereqs riskassessment riskofbias science scienceadvice searchmethods socioeconomics srappraisal srcasestudy srstandards statsmethods systematicmap teaching weightofevidence

Copy this bookmark:



description:


tags: