112
Guidelines for reporting meta-epidemiological methodology research
Meta-epidemiological studies adopt a systematic review or meta-analysis approach to examine the impact of certain characteristics of clinical studies on the observed effect and provide empirical evidence for hypothesised associations. The unit of analysis in meta-epidemiological studies is a study, not a patient. The outcomes of meta-epidemiological studies are usually not clinical outcomes. In this guideline, we adapt items from the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) to fit the context of meta-epidemiological studies.
Science  BetterScience 
2 days ago
Inter-rater reliability of AMSTAR is dependent on the pair of reviewers. - PubMed - NCBI
Inter-rater reliability varies widely depending on the pair of reviewers. There may be some shortcomings associated with conducting reliability studies with only two reviewers. Further studies should include additional reviewers and should probably also take account of their level of expertise.
Science  SRAppraisal 
2 days ago
The risk of bias in systematic reviews (ROBIS) tool showed fair reliability and good construct validity. - PubMed - NCBI
There is a movement from generic quality checklists towards a more domain based approach in critical appraisal tools. This study aims to report on a first experience with the newly developed risk of bias in systematic reviews (ROBIS) tool and compare it with A Measurement Tool to Assess Systematic Reviews (AMSTAR) that is the most common used tool to assess methodological quality of systematic reviews while assessing validity, reliability and applicability.
Science  SRAppraisal 
2 days ago
Testing the effectiveness of simplified search strategies for updating systematic reviews
SSS performed well for clinically focused topics and, with a median sensitivity of 100%, could be a viable alternative to a conventional comprehensive search strategy for updating this type of systematic reviews particularly considering the budget constraints and the volume of new literature being published. For broader topics, 80% sensitivity is likely to be considered too low for a systematic review update in most cases, although it might be acceptable if updating a scoping or rapid review.
Science  SearchMethods 
2 days ago
Reproducibility of Studies on Text Mining for Citation Screening in Systematic Reviews: Evaluation and Checklist
The application of text mining techniques to citation screening in the context of systematic literature reviews is a relatively young and growing computational field with high relevance for software engineering, medical research and other fields. However, our work revealed that it is currently difficult if not impossible to independently reproduce the results published in any of the studies investigated. The lack of information about the datasets used limits reproducibility of about 80% of the studies assessed. Also, information about the machine learning algorithms is inadequate in about 27% of the papers. On the plus side, the third party software tools used are mostly free and available.
Science  SearchMethods 
2 days ago
Developmental PBDE Exposure and IQ/ADHD in Childhood: A Systematic Review and Meta-analysis
We concluded there was sufficient evidence supporting an association between developmental PBDE exposure and reduced IQ. Preventing developmental exposure to PBDEs could help prevent loss of human intelligence.
Science  SRCaseStudy 
12 days ago
The influence of the team in conducting a systematic review
This commentary highlights a number of relevant articles indicating how the composition of the review team could jeopardise the integrity of the systematic review study and its conclusions. Key biases require closer attention such as sponsorship bias and researcher allegiance, but there may also be less obvious affiliations in teams conducting secondary evidence-syntheses. The importance of transparency and disclosure are now firmly on the agenda for clinical trials and primary research, but the meta-biases that systematic reviews may be at risk from now require further scrutiny.
Science  BetterScience 
15 days ago
Producing Cochrane systematic reviews-a qualitative study of current approaches and opportunities for innovation and improvement
The results suggest that improvements to Cochrane's systematic review production models could come from improving clarity of roles and expectations, ensuring continuity and consistency of input, enabling active management of the review process, centralising some review production steps; breaking reviews into smaller "chunks", and improving approaches to building capacity of and sharing information between authors and Review Groups. Respondents noted the important role new technologies have to play in enabling these improvements.
Science  BetterScience 
15 days ago
Reconciling disparate data to determine the 'right' answer: A grounded theory of meta analysts' reasoning in meta-analysis
Results show the complexity of meta-analytic reasoning. At each step of the process, participants attempted to reconcile disparate forms of knowledge to determine a 'right' answer (moral concern) and 'accurately' draw a treatment effect (epistemological concern). The reasoning processes often shifted between considering the meta-analysis as if the data were 'whole', and as if the data were discrete components (individual studies). These findings highlight paradigmatic tensions regarding the epistemological premises of meta-analysis, resembling previous historical investigations of the functioning of scientific communities. In understanding why different meta-analysts interpret data differently, it may be unrealistic to expect 'objective' homogenous recommendations based on meta-analyses.
Science  MetaAnalysis 
15 days ago
Systematic review of community health impacts of mountaintop removal mining
This systematic review could not reach conclusions on community health effects of MTR mining because of the strong potential for bias in the current body of human literature. Improved characterization of exposures by future community health studies and further study of the effects of MTR mining chemical mixtures in experimental models will be critical to determining health risks of MTR mining to communities. Without such work, uncertainty will remain regarding the impact of these practices on the health of the people who breathe the air and drink the water affected by MTR mining.
Science  SRCaseStudy 
15 days ago
Assessing Exposure to Household Air Pollution: A Systematic Review and Pooled Analysis of Carbon Monoxide as a Surrogate Measure of Particulate Matter
Our results suggest that exposure to CO is not a consistently valid surrogate measure of exposure to PM2.5. Studies measuring CO exposure as a surrogate measure of PM exposure should conduct local validation studies for different stove/fuel types and seasons.
Science  SRCaseStudy 
15 days ago
History shows that stacking federal science advisory committees doesn't work
Recommendations from “friendly” panels will not win broad support from the scientific community, and I predict the committees will quickly lose their credibility, legitimacy and influence. Consequently, policies and regulations based on the panels’ recommendations will be less likely to withstand public or political scrutiny and be more open to legal challenges than if they were based on more balanced input.
News  ScienceAdvice 
19 days ago
Promote scientific integrity via journal peer review data
There is an increasing push by journals to ensure that data and products related to published papers are shared as part of a cultural move to promote transparency, reproducibility, and trust in the scientific literature. Yet few journals commit to evaluating their effectiveness in implementing reporting standards aimed at meeting those goals.
News  BetterScience 
24 days ago
Temporal trends in sperm count: a systematic review and meta-regression analysis
This comprehensive meta-regression analysis reports a significant decline in sperm counts (as measured by SC and TSC) between 1973 and 2011, driven by a 50–60% decline among men unselected by fertility from North America, Europe, Australia and New Zealand. Because of the significant public health implications of these results, research on the causes of this continuing decline is urgently needed.
Science  SRCaseStudy 
24 days ago
Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals
Endocrine active chemicals (EACs) have raised concerns that traditional toxicity-testing protocols might be inadequate to identify all potential hazards to human health because they have the ability to modulate normal hormone function, and small alterations in hormone concentrations, particularly during sensitive life stages, can have lasting and significant effects. To address concerns about potential human health effects from EACs at low doses, this US NAS report develops a strategy to evaluate the evidence for such low-dose effects.
Science  SRCaseStudy 
26 days ago
Global movement on endocrine disruptors
National Academies warns endocrine active chemicals may be harmful at lower doses than the US currently tests for.
News  SRCaseStudy 
26 days ago
A systematic review of cardiovascular emergency department visits, hospital admissions and mortality associated with ambient black carbon
We conducted a systematic review to evaluate the potential for cardiovascular morbidity and mortality following exposure to ambient BC, or the related component elemental carbon (EC), in the context of what is already known about the associations between exposure to fine particulate matter (PM2.5) and cardiovascular health outcomes.
Science  SRCaseStudy 
26 days ago
Methodological and Reporting Quality of Systematic Reviews Published in the Highest Ranking Journals in the Field of Pain
Endorsement of PRISMA in instructions for authors was not a guarantee of compliance. Methodological and reporting quality of pain-related SRs should be improved using relevant checklists. This can be remedied by a joint effort of authors, editors, and peer reviewers.
Science  SRStandards 
26 days ago
Towards a taxonomy of logic models in systematic reviews and health technology assessments: a priori, staged and iterative approaches
The complexity associated with how interventions result - or fail to result - in outcomes, and how context matters is increasingly recognised. Logic models provide an important tool for handling complexity. This proposed taxonomy of logic models offers an improved understanding of the advantages and limitations of logic models across the spectrum from a priori to fully iterative approaches. Choice of logic model should be informed by scope of evidence synthesis, presence/absence of clearly defined PICO elements and feasibility considerations.
Science  ExternalValidity 
26 days ago
Systematic review adherence to methodological or reporting quality
Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.
Science  SRStandards 
4 weeks ago
Selecting and implementing overview methods: implications from five exemplar overviews
Overviews are a relatively new methodological innovation, and there are currently substantial variations in the methodological approaches used within different overviews. There are considerable methodological challenges for which optimal solutions are not necessarily yet known. Lessons learnt from five exemplar overviews highlight a number of methodological decisions which may be beneficial to consider during the development of an overview protocol.
Science  SRStandards 
4 weeks ago
Comparison of a full systematic review versus a rapid review approaches to assess a newborn screening test for tyrosinemia type 1
Rapid reviews are increasingly used to replace or complement systematic reviews to support evidence based decision-making. Not enough is known about how this expedited process affects results. This study assesses the difference between rapid and systematic review approaches for a case study of test accuracy of succinylacetone for detecting tyrosinemia type 1.
Science  RapidReviews 
4 weeks ago
Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study
Review finding that prevalence of poor reporting and inadequate methods in clinical trials has decreased over time, especially for sequence generation and allocation concealment. But more could be done, especially in lower impact factor journals. Suggests that awareness of methodological deficits in primary medical research are becoming better-recognised and picked up by at least the higher-impact journals.
Science  BetterScience 
5 weeks ago
Relevance and reliability of experimental data in human health risk assessment of pesticides
This article describes an approach to assess the relevance and reliability of experimental data from guideline-compliant studies as well as from non-guideline studies published in the scientific literature in the specific context of uncertainty and risk assessment of pesticides.
Science  RiskOfBias  RiskAssessment 
6 weeks ago
Meta-Analyses Can Be Credible and Useful
John Ioannidis outlines a strategic programme for improvement of systematic reviews in mental health research, covering meta-analytical methods and changes to primary research, which would also be applicable to the field of environmental health.
Science  BetterScience 
6 weeks ago
Text mining for search term development in systematic reviewing: A discussion of some methods and challenges
We briefly review the literature on applications of text mining for search term development for systematic reviewing. We found that the tools can be used in 5 overarching ways: improving the precision of searches; identifying search terms to improve search sensitivity; aiding the translation of search strategies across databases; searching and screening within an integrated system; and developing objectively derived search strategies. Using a case study and selected examples, we then reflect on the utility of certain technologies (term frequency-inverse document frequency and Termine, term frequency, and clustering) in improving the precision and sensitivity of searches.
Science  SearchMethods 
6 weeks ago
A WEIGHT OF EVIDENCE FRAMEWORK FOR ENVIRONMENTAL ASSESSMENTS: INFERRING QUALITIES
The weighing of heterogeneous evidence such as conventional laboratory toxicity tests, field tests, biomarkers, and community surveys is essential to environmental assessments. Evidence synthesis and weighing is needed to determine causes of observed effects, hazards posed by chemicals or other agents, the completeness of remediation, and other environmental qualities. As part of its guidelines for WoE in ecological assessments, the U.S. Environmental Protection Agency has developed a generally applicable framework. Its basic steps are: assemble evidence, weight the evidence, and weigh the body of evidence. Use of the framework can increase the consistency and rigor of WoE practices and provide greater transparency than ad hoc and narrative-based approaches.
Science  WeightOfEvidence 
7 weeks ago
Evidence and Health Policy: Using and Regulating Systematic Reviews
Systematic reviews could inform policy more effectively with changes to current practices and the assumptions that sustain these practices—assumptions made by researchers and the organizations that employ them, by public and private funders of systematic reviews, and by organizations that finance, set priorities and standards for, and publish them.
Science  ScienceAdvice 
7 weeks ago
Is the staggeringly profitable business of scientific publishing bad for science?
It is an industry like no other, with profit margins to rival Google – and it was created by one of Britain’s most notorious tycoons: Robert Maxwell.
News 
7 weeks ago
Residential exposure to pesticides as risk factor for childhood and young adult brain tumors: A systematic review and meta-analysis
Our findings support an association between residential exposure to pesticides and childhood brain tumors. Although causality cannot be established, these results add to the evidence leading to recommend limiting residential use of pesticides and to support public health policies serving this objective.
Science  SRCaseStudy 
7 weeks ago
Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study
The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.
Science  SRStandards 
8 weeks ago
New approach to weight-of-evidence assessment of ecotoxicological effects in regulatory decision-making
Ecological risk assessments and risk management decisions are only as sound as the underlying information and processes to integrate them. It is important to develop transparent and reproducible procedures a priori to integrate often-heterogeneous evidence. Current weight-of-evidence (WoE) approaches for effects or hazard assessment tend to conflate aspects of the assessment of the quality of the data with the strength of the body of evidence as a whole. We take forward recent developments in the critical appraisal of the reliability and relevance of individual ecotoxicological studies as part of the effect or hazard assessment of prospective risk assessments and propose a streamlined WoE approach.
Science  WeightOfEvidence 
9 weeks ago
The GRADE Working Group clarifies the construct of certainty of evidence
Certainty of evidence is best considered as the certainty that a true effect lies on one side of a specified threshold, or within a chosen range. We define possible approaches for choosing threshold or range. For guidelines, what we call a fully contextualized approach requires simultaneously considering all critical outcomes and their relative value. Less contextualized approaches, more appropriate for systematic reviews and health technology assessments, include using specified ranges of magnitude of effect, e.g. ranges of what we might consider no effect, trivial, small, moderate, or large effects.
Science  GradingEvidence 
10 weeks ago
The Complex and Multi-Faceted Aspects of Conflicts of Interest
COI, a conflict between a professional responsibility and a personal interest, is at one end of this threat spectrum. COI creates a risk of bias. Bias, a prejudice for or against something, is in the middle of this spectrum. If a COI results in bias, the bias may affect a professional judgment. Dishonesty is deceit or fraud. Dishonesty is at the opposite end of this spectrum from COI. Each of these threats exists on a continuum. COI may be present or perceived. Bias may be conscious or unconscious. Dishonesty may be intentional or unintentional. This issue of JAMA contains a series of Viewpoints on the many facets of COIs.
Science  RiskOfBias 
10 weeks ago
Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?
We identify two important dilemmas, for scientists and researchers, that arise from our initial advice. First, effective actors combine evidence with manipulative emotional appeals to influence the policy agenda – should scientists do the same, or would the reputational costs outweigh the policy benefits? Second, when adapting to multi-level policymaking, should scientists prioritise ‘evidence-based’ policymaking above other factors? The latter includes governance principles such the ‘co-production’ of policy between local public bodies, interest groups and service users. This process may be based primarily on values and involve actors with no commitment to a hierarchy of evidence.
Science  ScienceAdvice 
11 weeks ago
Framework for assessing causality of air pollution-related health effects for reviews of the national ambient air quality standards
The United States Environmental Protection Agency (U.S. EPA) develops the Integrated Science Assessment (ISA), which contains evaluations of the policy-relevant science on the effects of criteria air pollutants and conveys critical science judgments to inform decisions on the National Ambient Air Quality Standards. This article discusses the approach and causal framework used in the ISAs to evaluate and integrate various lines of scientific evidence and draw conclusions about the causal nature of air pollution-induced health effects.
Science  RiskAssessment 
12 weeks ago
A new risk of bias checklist applicable to randomized trials, observational studies and systematic reviews was developed and validated to be used for systematic reviews focusing on drug adverse events
The developed checklist examines eight domains: study design and objectives, selection bias, attrition, adverse events information bias, other information bias, statistical methods to control confounding, other statistical methods, and conflicts of interest. The total number of questions varied from ten to 32 depending on the study design. Inter- and intra-rater agreement were fair with Kendall's W of 0.70 and 0.74, respectively. Median time to complete the checklist was 8.5 minutes.
Science  RiskOfBias 
12 weeks ago
Evaluation of a Rule-based Method for Epidemiological Document Classification Towards the Automation of Systematic Reviews
The generated performance of our text-mining approach demonstrated encouraging results for the identification of targeted information from observational epidemiological study abstracts related to environmental exposures. We have demonstrated that rules based on generic syntactic patterns in one corpus can be applied to other observational study design by simple interchanging the dictionaries aiming to identify certain characteristics (i.e., outcomes, exposures). At the document level, the recognized information can assist in the selection and categorization of studies included in a systematic review.
Science  SearchMethods 
12 weeks ago
Addressing Bias and Conflict of Interest Among Biomedical Researchers
Attempting to identify and eliminate the interests that make scientists who they are serves as a distraction to prevent the research community, professionals, and policy makers from dealing with bias related to financial conflicts of interest. The diversity of viewpoints, motivations, and methodological approaches of researchers will advance science, while the narrowing of interests to financial ones will limit it to questions, methods, and reporting that favor the financial interest.
Science  RiskOfBias 
may 2017
Forcing consensus is bad for science and society
Science is a “show me”, not a “trust me”, field. Purporting to speak on behalf of all science, as the Nobel laureates sought to do with golden rice, conflated science, the scientific method and truth.
News  ScienceAdvice 
may 2017
A primer on systematic reviews in toxicology
To provide the toxicology community with a starting point for conducting or understanding systematic reviews, we herein summarized available guidance documents from various fields of application. We have elaborated on the systematic review process by breaking it down into ten steps, starting with planning the project, framing the question, and writing and publishing the protocol, and concluding with interpretation and reporting. In addition, we have identified the specific methodological challenges of toxicological questions and have summarized how these can be addressed.
Science  SRStandards 
may 2017
Importance of the distinction between quality of methodology and quality of reporting
REMARK addresses only how well authors reported key aspects of their methods and the findings of their research. Good reporting helps readers to judge whether the study design and analysis and interpretation were sound. But good reporting is not a measure of methodological quality, so a “REMARK score” cannot validly be used in that way. A similar error has been seen with the reporting of observational studies.
Science  SRStandards 
may 2017
Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research
Although we may anticipate that systematic reviews and meta-analyses not including unpublished or grey literature study results are likely to overestimate the treatment effects, current empirical research shows that this is only the case in a minority of reviews. Therefore, currently, a meta-analyst should particularly consider time, effort and costs when adding such data to their analysis. Future research is needed to identify which reviews may benefit most from including unpublished or grey data.
Science  PublicationBias 
may 2017
Milk and dairy consumption and risk of cardiovascular diseases and all-cause mortality: dose–response meta-analysis of prospective cohort studies
A cheesy meta-analysis finds "neutral associations between dairy products and cardiovascular and all-cause mortality".
SRCaseStudy  Science 
may 2017
An academic researcher's guide to increased impact on regulatory assessment of chemicals
However, in practice, standard tests conducted under GLP and sponsored and provided by industry are predominantly used. Peer-reviewed studies from independent sources are often disregarded or disputed since they often do not comply with regulatory data requirements and quality criteria. To help bridge such a gap, the aim of this paper is to give an overview of the general workings of legislation of chemicals and propose a set of actions to increase the usability of research data.
Science  BetterScience  ScienceAdvice 
may 2017
Methodology of the ‘craft’ of scientific advice for policy and practice
Scientific advice for policy should be seen as a craft that needs professional development as such. A crucial requirement is that, as science itself, science-informed advice for policy and practice can and should only be convincing when it is based on solid methodology. Essential steps include problem formulation, synthesising evidence, and building the bridge to implementation - but not crossing it.
Science  ScienceAdvice 
may 2017
Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews
The majority of SRs searched for non-English and unpublished studies; however, these represented a small proportion of included studies and rarely impacted the results and conclusions of the review. Inclusion of these study types may have an impact in situations where there are few relevant studies, or where there are questionable vested interests in the published literature. We found substantial variation in whether SRs searched for dissertations; in most reviews that included dissertations, these had little impact on results.
Science  PublicationBias 
may 2017
On the Need for Quantitative Bias Analysis in the Peer-Review Process
Reviewers who insist that quantitative bias analysis be incorporated into the design, conduct, presentation, and interpretation of epidemiologic research could substantially strengthen the process. In the present commentary, we demonstrate how quantitative bias analysis can be used by investigators and authors, reviewers, funding agencies, and editors. By utilizing quantitative bias analysis in the peer-review process, editors can potentially avoid unnecessary rejections, identify key areas for improvement, and improve discussion sections by shifting from speculation on the impact of sources of error to quantification of the impact those sources of bias may have had.
Science  RiskOfBias  SRStandards 
may 2017
The albatross plot: A novel graphical tool for presenting results of diversely reported studies in a systematic review.
Meta-analytical methods can only be used if comparable effect sizes can be computed from each study, and this may not be the case due to variation in how the studies were done or limitations in how their results were reported. Other methods, such as vote counting, are then used to summarize the results of these studies, but most of these methods are limited in that they do not provide any indication of the magnitude of effect. We propose a novel plot, the albatross plot, which requires only a 1-sided P value and a total sample size from each study (or equivalently a 2-sided P value, direction of effect and total sample size).
Science  MetaAnalysis 
may 2017
Systematic review of the potential adverse effects of caffeine consumption in healthy adults, pregnant women, adolescents, and children
The results of this systematic review support a shift in caffeine research to focus on characterizing effects in sensitive populations and establishing better quantitative characterization of interindividual variability (e.g., epigenetic trends), subpopulations (e.g., unhealthy populations, individuals with preexisting conditions), conditions (e.g., coexposures), and outcomes (e.g., exacerbation of risk-taking behavior) that could render individuals to be at greater risk relative to healthy adults and healthy pregnant women. This review, being one of the first to apply systematic review methodologies to toxicological assessments, also highlights the need for refined guidance and frameworks unique to the conduct of systematic review in this field.
Science  SRCaseStudy 
may 2017
Bias Analysis for Uncontrolled Confounding in the Health Sciences
We review methods that can be applied during or after data analysis to adjust for uncontrolled confounding for different outcomes, confounders, and study settings. We discuss relevant bias formulas and how to obtain the required information for applying them. Finally, we develop a new intuitive generalized bias analysis framework for simulating and adjusting for the amount of uncontrolled confounding due to not measuring and adjusting for one or more confounders.
Science  RiskOfBias 
april 2017
The science of stakeholder engagement in research: classification, implementation, and evaluation
We propose a classification system with definitions to determine where projects lie on the stakeholder engagement continuum. We discuss the key elements of implementation and evaluation of stakeholder engagement in research posing key questions to consider when doing this work. We commend and critique the work of Hamilton et al. in their multilevel stakeholder engagement in a VA implementation trial of evidence-based quality improvement in women's health primary care.
ProblemFormulation  Science 
april 2017
Can systematic reviews contribute to regulatory decisions?
Systematic reviews may simultaneously produce new findings and summarize existing knowledge, with the potential of informing regulatory decisions more pragmatically and more rapidly than other research designs. We suggest that national and international calls on independent research on drugs should not put primary clinical research against systematic reviews, as it implies a focus on the methods instead of on the questions being asked.
BetterScience  Science 
april 2017
Can abstract screening workload be reduced using text mining? User experiences of the tool Rayyan
After screening half of the search result with Rayyan, 86% to 99% of the references deemed relevant to the study were identified. Of those studies included in the final reports, 96% to 100% were already identified in the first half of the screening process. Users rated Rayyan 4.5 out of 5.
SearchMethods  Science 
april 2017
Statistics Myth Busters: Dispelling Common Misperceptions Held by Readers of the Biomedical Literature
The myths discussed involve 3 common areas of consideration when evaluating any clinical study: assessing the risk of bias from confounding (propensity score analysis and multivariable modeling), interpretation of the main study findings (P values and hypothesis testing), and secondary evaluations (subgroup analyses). Literature examples are used to illustrate each of the topics.
MetaAnalysis  Science 
april 2017
Rating the certainty in evidence in the absence of a single estimate of effect
Regardless of whether a single pooled effect estimate is generated or whether data are summarised narratively, decision makers need to know the certainty in the evidence in order to make informed decisions. In this guide, we illustrate how to apply the constructs of the GRADE (Grading of Recommendation, Assessment, Development and Evaluation) approach to assess the certainty in evidence when a meta-analysis has not been performed and data were summarised narratively.
GradingEvidence  Science 
april 2017
Using Ontology-based Semantic Similarity to Facilitate the Article Screening Process for Systematic Reviews
We demonstrated using ontology-based semantics to facilitate the identification of relevant articles for SRs. Effective concepts and concept relations derived from UMLS ontologies can be utilized to establish article semantic relationships. Our approach provided a promising performance and can easily apply to any SR topics in the biomedical domain with generalizability.
SearchMethods  Science 
april 2017
Systematic reviews of economic evaluations: how extensive are their searchers?
Economic evaluation (EE) is an accepted element of decision making and priority setting in healthcare. As the number of published EEs grows, so does the number of systematic reviews (SRs) of EEs. Although search methodology makes an important contribution to SR quality, search methods in reviews of EEs have not been evaluated in detail. We investigated the resources used to identify studies in recent, published SRs of EEs, and assessed whether the resources reflected recommendations.
SRStandards  SocioEconomics  Science 
april 2017
Scientists Struggle to Determine Risky Levels of PFCs in Drinking Water
Scientists are having a difficult time determining what levels may be dangerous because potential health effects vary among species in animal studies and are hard to isolate in human studies. No “unsafe” limits have been declared, leaving residents and municipalities uncertain whether to take corrective actions.
News  RiskAssessment 
april 2017
A rapid method to increase transparency and efficiency in web-based searches
Here, we describe novel methods for downloading results from searches of websites and web-based search engines into comprehensive databases as citations using free-to-use software. Citations from web-based search engines can then be integrated into review procedures along with those from traditional online bibliographic databases.
SearchMethods  Science 
march 2017
What leads to bias in the scientific literature? New study tries to answer
In a paper released today, researchers led by Daniele Fanelli and John Ioannidis — both at Stanford University — suggest that the so-called “pressure-to-publish” does not appear to bias studies toward larger so-called “effect sizes.” Instead, the researchers argue that other factors were a bigger source of bias than the pressure-to-publish, namely the use of small sample sizes (which could contain a skewed sample that shows stronger effects), and relegating studies with smaller effects to the “gray literature,” such as conference proceedings, PhD theses, and other less publicized formats.
News  PublicationBias 
march 2017
Meta-assessment of bias in science
Science is said to be suffering a reproducibility crisis caused by many biases. How common are these problems, across the wide diversity of research fields? We probed for multiple bias-related patterns in a large random sample of meta-analyses taken from all disciplines. The magnitude of these biases varied widely across fields and was on average relatively small. However, we consistently observed that small, early, highly cited studies published in peer-reviewed journals were likely to overestimate effects. We found little evidence that these biases were related to scientific productivity, and we found no difference between biases in male and female researchers. However, a scientist’s early-career status, isolation, and lack of scientific integrity might be significant risk factors for producing unreliable results.
PublicationBias  Science 
march 2017
Real-life effectiveness of 'improved' stoves and clean fuels in reducing PM2.5 and CO: Systematic review and meta-analysis
Although effective at reducing indoor air pollution, neither 'improved' solid fuel stoves nor clean fuels (probably due to neighbourhood contamination) achieve PM2.5 concentrations close to 24-hour air quality guidance values set by the WHO. Household energy policy should therefore prioritise community-wide use of clean fuels.
SRCaseStudy  Science 
march 2017
Rapid evidence assessment: increasing the transparency of an emerging methodology
This article discusses the lack of transparency and limited critical appraisal that can occur in REA, and goes on to propose general principles for conducting a REA. The approach that we describe is consistent with the principles underlying systematic review methodology, but also makes allowances for the rapid delivery of information as required while utilizing explicit and reproducible methods at each stage.
RapidReviews  Science 
march 2017
Protocol registration of systematic reviews published in high-impact factor journals: A meta-epidemiological study
Among the 284 included reviews, 60 (21%) protocols were registered. The proportion of registration increased from 5.6% in 2009 to 27% in 2015 (p for trend < 0.001). Protocol registration was not associated with outcome reporting bias (adjusted odds ratio [OR] 0.85, 95% confidence interval [CI] 0.39-1.86). The association between Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) adherence and protocol registration was not statistically significant (OR 1.09, 95% CI 0.59-2.01). Six years after the launch of the PRISMA statement, the proportion of protocol registration in high-impact journals has increased some but remains low.
SRStandards  Protocol  Science 
march 2017
Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry
The mean estimated time to complete the project and publish the review was 67.3 weeks (IQR=42). The number of studies found in the literature searches ranged from 27 to 92 020; the mean yield rate of included studies was 2.94% (IQR=2.5); and the mean number of authors per review was 5, SD=3. Funded reviews took significantly longer to complete and publish (mean=42 vs 26 weeks) and involved more authors and team members (mean=6.8 vs 4.8 people) than those that did not report funding (both p<0.001).
ResourceReqs  Science 
march 2017
A critical appraisal of the methodology and quality of evidence of systematic reviews and meta-analyses of traditional Chinese medical nursing interventions
There is room for improvement in the methodological quality of systematic reviews/meta-analyses of TCMN interventions published in Chinese journals. Greater efforts should be devoted to ensuring a more comprehensive search strategy, clearer specification of the interventions of interest in the eligibility criteria and identification of meaningful outcomes for clinicians and patients (consumers). The overall quality of evidence among reviews remains suboptimal, which raise concerns about their roles in influencing clinical practice.
SRStandards  Science 
march 2017
Systematic Reviews in Burns Care: Poor Quality and Getting Worse
The overall quality of the 44 included burns care systematic reviews was low, with an average methodological quality of 55% and an average compliance with reporting guidelines of 70%. Correlation analysis showed that adherence to reporting guidelines has been relatively stable, but methodological quality has deteriorated (r = -.32, P < .05). Cochrane reviews had lower citation rates than reviews published in other journals, whereas reviews that included meta-analyses had more citations. Quality did not have a significant effect on citation rate. Health professionals working in burns should be able to expect that systematic reviews published in their field are of a high standard. Unfortunately, this is not the case.
SRStandards  Science 
march 2017
Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a prospective study
A very small trial finds significant improvement in between-rater agreement in application of the Cochrane Risk of Bias tool, when novice users are given standardised training.
RiskOfBias  Science 
march 2017
A parsimonious weight function for modeling publication bias
Quantitative research literature is often biased because studies that fail to find a significant effect (or that demonstrate effects in an undesired or unexpected direction) are less likely to be published. This phenomenon, termed publication bias, can cause problems when researchers attempt to synthesize results using meta-analytic methods. Various techniques exist that attempt to estimate and correct meta-analyses for publication bias. However, there is no single method that can (a) account for continuous moderators by including them within the model, (b) allow for substantial data heterogeneity, (c) produce an adjusted mean effect size, (d) include a formal test for publication bias, and (e) allow for correction when only a small number of effects is included in the analysis. This article describes a method that we believe helps fill that gap.
PublicationBias  Science 
march 2017
Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach?
Identifying which individuals benefit most from particular treatments or other interventions underpins so-called personalised or stratified medicine. However, single trials are typically underpowered for exploring whether participant characteristics, such as age or disease severity, determine an individual’s response to treatment. A meta-analysis of multiple trials, particularly one where individual participant data (IPD) are available, provides greater power to investigate interactions between participant characteristics (covariates) and treatment effects. We use a published IPD meta-analysis to illustrate three broad approaches used for testing such interactions.
MetaAnalysis  Science 
march 2017
Cardiovascular risk from water arsenic exposure in Vietnam: Application of systematic review and meta-regression analysis in chemical health risk assessment
A systematic review (SR) and meta-analysis cannot provide the endpoint answer for a chemical risk assessment (CRA). The objective of this study was to apply SR and meta-regression (MR) analysis to address this limitation using a case study in cardiovascular risk from arsenic exposure in Vietnam.
MetaAnalysis  RiskAssessment  Science 
march 2017
ECHA flags widespread chemical toxicity data gaps
A report published on 27 February reveals that the agency had to demand more information from the manufacturers of 168 of the 184 potentially hazardous chemicals (91%) that it screened in 2016.1 Only 16 dossiers (9%) were judged acceptable.
News  RiskAssessment 
march 2017
Response: “Application of the Navigation Guide systematic review methodology to the evidence for developmental and reproductive toxicity of triclosan”
Response to correspondence critiquing the Navigation Guide framework, providing additional guidance in a number of areas which may not be obvious to all readers.
SRCaseStudy  Science 
february 2017
More clarity needed in the Navigation Guide systematic review framework
Correspondence piece critiquing some aspects of the Navigation Guide framework for conducting systematic reviews.
SRCaseStudy  Science 
february 2017
Synthesizing Evidence in Public Policy Contexts: The Challenge of Synthesis When There Are Only a Few Studies
Using examples from the U.S. Department of Education's What Works Clearinghouse as case studies, we conclude with a discussion of Bayesian meta-analysis as a potential solution to the challenges encountered when attempting to draw inferences about the effectiveness of interventions from a small number of studies.
MetaAnalysis  Science 
february 2017
Industry sponsorship and research outcome
Sponsorship of drug and device studies by the manufacturing company leads to more favorable efficacy results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard 'Risk of bias' assessments.
RiskOfBias  Science 
february 2017
Progress in evidence-based medicine: a quarter century on
EBM's enduring contributions to clinical medicine include placing the practice of medicine on a solid scientific basis, the development of more sophisticated hierarchies of evidence, the recognition of the crucial role of patient values and preferences in clinical decision making, and the development of the methodology for generating trustworthy recommendations.
BetterScience  Science 
february 2017
Rules of evidence (Are evidence standards used by chemical regulators excluding solid science?)
A common plastic additive called bisphenol A (BPA) has become the focus of major controversy. It can weakly mimic the human hormone estrogen, and leach out of products—including plastic drinking bottles and medical supplies—in small quantities. As a result, some companies and governments have moved to remove BPA from certain products, even though there is still fierce debate among regulators over BPA's safety. Some researchers say that debate is being ill-served by rules developed in the 1960s and '70s, in response to a chemical testing scandal, that are designed to make sure regulators consider only the strongest evidence. But the critics argue the rules now prevent regulators from considering high-quality, cutting-edge academic studies that don't follow the rules. A U.S. government–funded initiative is now trying to bridge the divide.
News  SRStandards  Controversy 
february 2017
Knowledge Syntheses in Medical Education: Demystifying Scoping Reviews
In this Perspective, the authors examine the nature, purpose, value, and appropriate use of one particular method: scoping reviews. Scoping reviews are iterative and flexible and can serve multiple main purposes: to examine the extent, range, and nature of research activity in a given field; to determine the value and appropriateness of undertaking a full systematic review; to summarize and disseminate research findings; and to identify research gaps in the existing literature. Despite the advantages of this methodology, there are concerns that it is a less rigorous and defensible means to synthesize HPE literature.
RapidReviews  Science 
february 2017
An Evidence-Based Approach to Conducting Systematic Reviews on CKD
This review article will share best practices associated with conducting systematic reviews on the topic of CKD using an 8-step process and an evidence-based approach to retrieving and abstracting data.
SRStandards  Science 
february 2017
« earlier      
agencysr betterscience controversy critappstudies evidencemapping externalvalidity gradingevidence heterogeneity metaanalysis news problemformulation protocol publicationbias rapidreviews resourcereqs riskassessment riskofbias science scienceadvice searchmethods socioeconomics srappraisal srcasestudy srstandards teaching weightofevidence

Copy this bookmark:



description:


tags: