164
How citation distortions create unfounded authority: analysis of a citation network
Paper showing how weak or false claims can become perceived as authoritative in the scientific literature, via mechanisms of citation bias, citation distortion, amplification and invention, with the concepts explained. SR methods in evidence synthesis would remedy much of this.
RiskOfBias 
7 days ago
Stop this waste of people, animals and money
Predatory journals have shoddy reporting and include papers from wealthy nations, find David Moher, Larissa Shamseer, Kelly Cobey and colleagues.
SRStandards 
16 days ago
Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools
We show that CADIMA is the only available open access tool that is designed to: (1) assist throughout the systematic review/map process; (2) be suited to reviews broader than medical sciences; (3) allow for offline data extraction; and, (4) support working as a review team.
Tools 
17 days ago
Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial
Data abstraction, a critical systematic review step, is time-consuming and prone to errors. Current standards for approaches to data abstraction rest on a weak evidence base. We developed the Data Abstraction Assistant (DAA), a novel software application designed to facilitate the abstraction process by allowing users to (1) view study article PDFs juxtaposed to electronic data abstraction forms linked to a data abstraction system, (2) highlight (or “pin”) the location of the text in the PDF, and (3) copy relevant text from the PDF into the form. We describe the design of a randomized controlled trial (RCT) that compares the relative effectiveness of (A) DAA-facilitated single abstraction plus verification by a second person, (B) traditional (non-DAA-facilitated) single abstraction plus verification by a second person, and (C) traditional independent dual abstraction plus adjudication to ascertain the accuracy and efficiency of abstraction.
DataExtraction 
21 days ago
Robust research needs many lines of evidence
Triangulate, because replication alone will get us only so far. In some cases, routine replication might actually make matters worse. Consistent findings could take on the status of confirmed truths, when they actually reflect failings in study design, methods or analytical tools.
Media 
25 days ago
Role of Risk of Bias in Systematic Review for Chemical Risk Assessment: A Case Study in Understanding the Relationship Between Congenital Heart Defects and Exposures to Trichloroethylene
The National Academy of Science has recommended that a risk of bias (RoB; credibility of the link between exposure and outcome) assessment be conducted on studies that are used as primary data sources for hazard identification and dose–response assessment. Few applications of such have been conducted. Using trichloroethylene and congenital heart defects (CHDs) as a case study, we explore the role of RoB in chemical risk assessment using the National Toxicology Program’s Office of Health Assessment and Translation RoB tool.
RiskOfBias 
27 days ago
Facilitating healthcare decisions by assessing the certainty in the evidence from preclinical animal studies
Here we present how the GRADE approach could be used to rate the certainty in the evidence of preclinical animal studies in the context of therapeutic interventions. We also discuss the methodological challenges that we identified, and for which further work is needed. Examples are defining the importance of consistency within and across animal species and using GRADE’s indirectness domain as a tool to predict translation from animal models to humans.
GradingEvidence  ExternalValidity 
28 days ago
Gray literature: An important resource in systematic reviews
This narrative review provides guidance about the benefits of including gray literature in a systematic review, and sources for searching through gray literature. An illustrative example of a search for evidence within gray literature sources is presented to highlight the potential contributions of such a search to a systematic review. Benefits and challenges of gray literature search methods are discussed, and recommendations made.
GrayLiterature  SearchMethods 
28 days ago
Systematic review and evaluation of aspartame carcinogenicity bioassays using quality criteria
Purported SR of quality of primary studies, using Klimisch criteria and a suspiciously ad-hoc approach to critique. Funded by a low-calorie drinks trade group.
SRCaseStudy  PrimaryStudies  RiskOfBias 
28 days ago
We need better animal research, better reported
BMJ editorial summarising key lessons from a selection of recently-published articles about the poor quality of preclinical research.
PrimaryStudies  Media 
29 days ago
Unit of analysis issues in laboratory-based research
Many studies in the biomedical research literature report analyses that fail to recognise important data dependencies from multilevel or complex experimental designs. Statistical inferences resulting from such analyses are unlikely to be valid and are often potentially highly misleading. Failure to recognise this as a problem is often referred to in the statistical literature as a unit of analysis (UoA) issue. Here, by analysing two example datasets in a simulation study, we demonstrate the impact of UoA issues on study efficiency and estimation bias, and highlight where errors in analysis can occur.
StatsMethods 
4 weeks ago
Inclusive development and prioritization of review questions in a highly controversial field of regulatory science
Paper describing how the EU GRACE framework was tested and implemented in the course of 14 systematic reviews or maps conducted on selected review questions spanning potential health, environmental, and socioeconomic impacts of GMOs, placing a particular emphasis on the stakeholder engagement strategy.
ProblemFormulation 
4 weeks ago
What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences
In this paper a typology is proposed of various systematic review methodologies. The review types are defined and situated with regard to establishing corresponding questions and inclusion criteria. The ultimate objective is to provide clarified guidance for both novice and experienced reviewers and a unified typology with respect to review types.
ProblemFormulation  SRStandards 
5 weeks ago
A call for action: Improve reporting of research studies to increase the scientific basis for regulatory decision-making
This is a call for action to scientific journals to introduce reporting requirements for toxicity and ecotoxicity studies. Such reporting requirements will support the use of peer-reviewed research studies in regulatory decision-making. Moreover, this could improve the reliability and reproducibility of published studies in general and make better use of the resources spent in research.
RiskAssessment  Reporting 
6 weeks ago
Refining tools to bridge the gap between academia and chemical regulation: perspectives for WikiREACH
Experiences with WikiPharma show that there is interest in using peer-reviewed studies in regulatory decision-making. However, tools like WikiPharma require constant updates. Hence, as for “WikiREACH”, effective incentives are needed to motivate researchers to feed in relevant data for regulatory assessments. Besides, support by automated processes can aid in the labour-intensive activity of gathering data.
Automation 
6 weeks ago
Declarations and conflicts of interest in WHO guidelines
The majority of WHO guidelines reported their funding sources, and the DOI and COI of external contributors in their guideline documents. However, there is a need for improvement, in particular for reporting of funders and their role, declaration processes, and management of COI.
Interests 
6 weeks ago
Evidence & Gap Maps: A tool for promoting evidence informed policy and strategic research agendas
To provide an accessible resource for researchers, commissioners, and decision makers, EGMs provide thematic collections of evidence structured around a framework which schematically represents the types of interventions and outcomes of relevance to a particular sector. By mapping the existing evidence using this framework, EGMs provide a visual overview of what we know and do not know about the effects of different programs.
SystematicMap 
6 weeks ago
Cochrane systematic reviews of interventions for risk factors correlate weakly with global risk factor burden: A cross-sectional study
Looking at the wrong things: Our study investigated whether Cochrane risk factor SRs align with global risk factor burden, demonstrating a weak positive correlation. Interventions modifying air pollution and dietary risks were sparsely studied, given disease burden.
6 weeks ago
Technology-assisted risk of bias assessment in systematic reviews: A prospective cross-sectional evaluation of the RobotReviewer machine learning tool
Reliability (95% CI) was moderate for random sequence generation (0.48 (0.43, 0.53)), allocation concealment (0.45 (0.40, 0.51)), and blinding of participants and personnel (0.42 (0.36, 0.47)); fair for overall risk of bias (0.34 (0.25, 0.44)); and slight for blinding of outcome assessors (0.10 (0.06, 0.14)), incomplete outcome data (0.14 (0.08, 0.19)), and selective reporting (0.02 (-0.02, 0.05)). Reliability for blinding of participants and personnel (p<0.001), blinding of outcome assessors (p=0.005), selective reporting (p<0.001), and overall risk of bias (p<0.001) differed by topic. Sensitivity and specificity (95% CI) ranged from 0.20 (0.18, 0.23) to 0.76 (0.72, 0.80) and from 0.61 (0.56, 0.65) to 0.95 (0.93, 0.96), respectively.
Automation 
6 weeks ago
A systematic decision-making process on the need for updating clinical practice guidelines proved to be feasible in a pilot study
The pilot study comprised 1) limited searches in Pubmed to identify new potentially relevant evidence, 2) an online survey among the members of the CPG group to assess the need for update and 3) a consensus conference for determination and prioritization of guideline sections with a high need for update. Subsequently, we conducted a second online survey to evaluate the procedure.
ProblemFormulation 
6 weeks ago
Recommendations for Assessing the Risk of Bias in Systematic Reviews of Health Care Interventions
We developed recommendations for assessing the risk of bias of studies of health care interventions specific to framing the focus and scope of risk-of-bias assessment; selecting risk of bias categories; choosing assessment instruments; and conducting, analyzing, and presenting results of risk-of-bias assessments. Key recommendations include transparency and reproducibility of judgments, separating risk of bias from other constructs such as applicability and precision, and evaluating risk of bias per outcome. We recommend against certain past practices, such as focusing on reporting quality, relying solely on study design or numerical quality scores, and automatically downgrading for industry sponsorship.
RiskOfBias 
7 weeks ago
Systematic reviews with published protocols compared to those without: more effort, older search
Time from search to submission was longer for SRs with published protocols, while at the same time SRs with published protocols were better elaborated and reported. As quality, transparency and currency are cornerstones of SRs, we suggest critically discussing the current practice of publishing SR protocols.
Protocol  SRStandards 
7 weeks ago
Fostering responsible research practices is a shared responsibility of multiple stakeholders
Commentary on drivers and solutions to the "reproducibility crisis" in biomedicine. "The big picture seems to be that scientists experience dilemmas and conflicts of interest in daily practice that arise from the fact that what is good for the validity and reliability of science is not always good for their personal career."
SRStandards 
7 weeks ago
No difference in knowledge obtained from infographic or plain language summary of a Cochrane systematic review: three randomized controlled trials
Although the infographic format was perceived as more enjoyable for reading, it was not better than a traditional, text-based plain language summary in the translation of knowledge about findings from a Cochrane systematic review.
SRStandards 
7 weeks ago
Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review
Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.
SRStandards 
8 weeks ago
A retrospective comparison of systematic reviews with same-topic rapid reviews
It seems this review determined that most systematic reviews are methodologically no better than rapid reviews. So while RRs might give the same results as SRs, it's not saying much if the comparator SRs are giving bad results.
RapidReviews 
9 weeks ago
Some Cochrane risk of bias items are not important in osteoarthritis trials: A meta-epidemiological study based on Cochrane reviews
Twenty reviews including 126 OA trials with a high degree of heterogeneity was included (τˆ2=0.1247). Among RoB domains only patient blinding had an impact on the results (reducing heterogeneity according to τˆ2 <7%). Inadequate blinding of patients yielded larger effects (SMDDifference = 0.15; 95% CI: 0.11 to 0.29, P=0.035). The most important study characteristic was trial size (heterogeneity reduced by 25%), with small trials reporting larger effects (SMDDifference = 0.29; 95% CI: 0.16 to 0.42, P<0.001).
RiskOfBias 
10 weeks ago
Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis
When considering the index (primary or first reported) meta-analysis of each SR, just over half (62/110 [56%]) used the random-effects model, but few (5/62 [8%]) interpreted the meta-analytic effect correctly (as the average of the intervention effects across all studies). A statistical test for funnel plot asymmetry was reported in 17/110 (15%) SRs, however, in only 4/17 (24%) did the test include the recommended number of at least 10 studies of varying size. Subgroup analyses accompanied 42/110 (38%) index meta-analyses, but findings were not interpreted with respect to a test for interaction in 29/42 (69%) cases, and the issue of potential confounding in the subgroup analyses was not raised in any SR.
StatsMethods 
10 weeks ago
Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics
Our interpretation of the results is limited by imprecise estimates. There is some indication that between-trial heterogeneity could be partially explained by reported design characteristics, and hence adjustment for bias could potentially improve accuracy of meta-analysis results.
RiskOfBias 
10 weeks ago
Differences between protocols for randomized controlled trials and systematic reviews
How to game a protocol? Although we fully acknowledge the potential benefits of protocols for SRs, it is important to point out that there is a clear difference between protocols for SRs when compared to protocols for RCTs.
SRStandards  Protocol 
10 weeks ago
Potential Developmental and Reproductive Impacts of Triclocarban: A Scoping Review
Scoping reviews are showing signs of improving in value through use of systematic methods to identify and characterise evidence so as to reliably identify health endpoints for future systematic review and to make evidence-based recommendations for future research. This scoping review found 32 relevant studies in humans, rodents, fish, invertebrates, and in vitro. Based on the robustness of the literature, the review identifies estrogenic activity, androgenic activity, and offspring growth as outcomes for potential systematic review.
SystematicMap 
11 weeks ago
Association between Exposure to p,p′-DDT and Its Metabolite p,p′-DDE with Obesity: Integrated Systematic Review and Meta-Analysis
We classified p,p′-DDT and p,p′-DDE as “presumed” to be obesogenic for humans, based on a moderate level of primary human evidence, a moderate level of primary in vivo evidence, and a moderate level of supporting evidence from in vivo and in vitro studies.
SRCaseStudy 
11 weeks ago
Make reviews public, says peer review expert Irene Hames
This is a nice read, about how peer-review might be improved without having to completely overhaul the system. Good comments and responses from Irene Hames below the line.
PeerReview  Media 
11 weeks ago
Evaluation of a new method for librarian-mediated literature searches for systematic reviews
Single-line search strategies were prepared in a text document. Term completeness was ensured with a novel optimization technique. Macros in MS Word converted the syntaxes between databases and interfaces almost automatically. The ESM searches differed by using two times more databases, retrieving 44% more references, including 20% more studies in the final systematic review, but the time needed for the search was 8% of that of the control group.
SearchMethods 
11 weeks ago
Systematic Review and Meta-Analysis of Early-Life Exposure to Bisphenol A and Obesity-Related Outcomes in Rodents
Overall summary estimates indicated significant positive associations between BPA and fat weight [SMD=0.67 (95% CI: 0.53, 0.81)], triglycerides [SMD=0.97 (95% CI: 0.53, 1.40)], and FFA [SMD=0.86 (95% CI: 0.50, 1.22)], and a nonsignificant positive association with leptin levels [MD=0.37 (95% CI: -0.14, 0.87)] and a significant negative association with body weight were estimated [MD=-0.22 (95% CI: -0.37, -0.06)]. Subgroup analyses revealed stronger positive associations for most outcome measures in males and at doses below the current U.S. reference dose of 50μg/kg/d compared with doses above the reference dose. It should be noted that there was substantial heterogeneity across studies for all outcomes assessed and that there was insufficient information to assess risk of bias for most studies.
SRCaseStudy 
11 weeks ago
Acute effects of fine particulate matter constituents on mortality: A systematic review and meta-regression analysis
Our meta-analysis suggests that (a) combustion elements such as EC and K have a stronger association with mortality, (b) single lag studies underestimate effects, and (c) estimates of PM2.5 and constituents differ across regions. Accounting for PM mass in constituent's health models may lead to more stable and comparable effect estimates across different studies.
SRCaseStudy 
11 weeks ago
Weight of evidence approaches for the identification of endocrine disrupting properties of chemicals: Review and recommendations for EU regulatory application
This paper reviews WoE approaches to distil key recommendations specifically for the evaluation of potential ED properties of chemicals. In a manner, which is consistent with existing, published WoE frameworks, the WoE evaluation of ED properties can be divided into four phases: 1) Definition of causal questions and data gathering and selection, 2) Review of individual studies, 3) Data integration and evaluation, and 4) Drawing conclusions based on inferences. Recommendations are made on how to conduct each phase robustly and transparently to help guide the WoE evaluation of potential endocrine disrupting properties of chemicals within a European regulatory context.
WeightOfEvidence 
11 weeks ago
Guidelines for performing systematic reviews in the development of toxicity factors
The Texas Commission on Environmental Quality (TCEQ) developed guidance on conducting systematic reviews during the development of chemical-specific toxicity factors. Using elements from publicly available frameworks, the TCEQ systematic review process was developed in order to supplement the existing TCEQ Guidelines for developing toxicity factors (TCEQ Regulatory Guidance 442).
SRStandards 
11 weeks ago
Citation analysis is also useful to assess the eligibility of biomedical research works for inclusion in Living systematic reviews. - PubMed - NCBI
In this research letter, I explain how the analysis of the citation network and of the co-citation network of a given topic and the use of the snowballing techniques can be useful to verify the eligibility of a paper for inclusion in the Living Systematic Review (LSR) of that topic.
SearchMethods 
november 2017
Systematic Review and Meta-Analysis of Early-Life Exposure to Bisphenol A and Obesity-Related Outcomes in Rodents
Overall summary estimates indicated significant positive associations between BPA and fat weight [SMD=0.67 (95% CI: 0.53, 0.81)], triglycerides [SMD=0.97 (95% CI: 0.53, 1.40)], and FFA [SMD=0.86 (95% CI: 0.50, 1.22)], and a nonsignificant positive association with leptin levels [MD=0.37 (95% CI: −0.14, 0.87)] and a significant negative association with body weight were estimated [MD=−0.22 (95% CI: −0.37, −0.06)]. Subgroup analyses revealed stronger positive associations for most outcome measures in males and at doses below the current U.S. reference dose of 50 μg/kg/d compared with doses above the reference dose. It should be noted that there was substantial heterogeneity across studies for all outcomes assessed and that there was insufficient information to assess risk of bias for most studies.
SRCaseStudy 
october 2017
Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms
Data extraction is one of the most time-consuming tasks in performing a systematic review. Extraction is often onto some sort of form. Sharing completed forms can be used to check quality and accuracy of extraction or for re-cycling data to other researchers for updating. However, validating each piece of extracted data is time-consuming and linking to source problematic.In this methodology paper, we summarize three methods for reporting the location of data in original full-text reports, comparing their advantages and disadvantages
DataExtraction 
october 2017
Gray literature: An important resource in systematic reviews
Gray literature's diverse formats and audiences can present a significant challenge in a systematic search for evidence. However, the benefits of including gray literature may far outweigh the cost in time and resource needed to search for it, and it is important for it to be included in a systematic review or review of evidence.
SearchMethods 
october 2017
Cleaning and asthma: A systematic review and approach for effective safety assessment
This proposed WoE method organizes diverse lines of data (i.e., asthma, sensitization, and irritation information) through a systematic, hierarchical framework that provides qualitatively categorized conclusions using hazard bands to predict a specific product or ingredient's potential for asthma induction. This work provides a method for prioritizing chemicals as a first step for quantitative and scenario-specific safety assessments based on their potential for inducing asthmatic effects. Acetic acid is used as a case study to test this framework.
SRCaseStudy  RiskAssessment 
october 2017
Review of reviews on exposures to synthetic organic chemicals and children's neurodevelopment: Methodological and interpretation challenges
Seventy-four relevant reviews were identified, and these were evaluated with respect to four methodological characteristics: (1) systematic inclusion/exclusion criteria and reproducible methods for search and retrieval of studies; (2) structured evaluation of underlying data quality; (3) systematic assessment of consistency across specific exposure-outcome associations; and (4) evaluation of reporting/publication bias. None of the 74 reviews fully met the criteria for all four methodological characteristics. Only four reviews met two criteria, and six reviews fulfilled only one criterion. Perhaps more importantly, the higher quality reviews were not able to meet all of the criteria owing to the shortcomings of underlying studies.
october 2017
Sloppy reporting on animal studies proves hard to change
Papers from the “treatment group” improved on only two ARRIVE items: husbandry and housing. For all others, authors had duly filled out the checklist but their papers were not actually more compliant. “Apparently, a checklist alone doesn't help much,” Ritskes-Hoitinga says. Another reason may be that PLOS ONE editors didn't enforce compliance.
Media 
october 2017
Developmental PBDE Exposure and IQ/ADHD in Childhood: A Systematic Review and Meta-analysis. - PubMed - NCBI
Our meta-analysis of four studies estimated a 10-fold increase (in other words, times 10) in PBDE exposure associated with a decrement of 3.70 IQ points (95% confidence interval: 0.83, 6.56). We concluded there was sufficient evidence supporting an association between developmental PBDE exposure and reduced IQ. Preventing developmental exposure to PBDEs could help prevent loss of human intelligence.
SRCaseStudy 
september 2017
Cumulative effects of prenatal-exposure to exogenous chemicals and psychosocial stress on fetal growth: Systematic-review of the human and animal evidence
We found that despite concern for the combined effects of environmental chemicals and stress, this is still an under-studied topic, though limited available human studies indicate chemical exposures exert stronger effects than stress, and this effect is generally larger in the presence of stress.
SRCaseStudy 
september 2017
Estimating the health benefits of environmental regulations
Assessing health benefits of policies addressing environmental contaminants is important for decision-making and for informing the public about how policy affects their welfare. Benefits analysis, one side of benefit-cost analysis (BCA), can be relatively straightforward when sufficient data are available on dose-response relationships, changes in exposure expected from a proposed policy, and other key inputs. But despite progress, benefits analysis for health effects is needlessly constrained by analytic practices that are scientifically outdated and inconsistent with economic theory.
Media 
september 2017
Most systematic reviews of high methodological quality on psoriasis interventions are classified as high risk of bias using ROBIS tool. - PubMed - NCBI
We classified 139 intervention SRs as displaying high/moderate/low methodological quality, and as high/low risk of bias. A high risk of bias was detected for most SRs classified as displaying high or moderate methodological quality by AMSTAR. When comparing ROBIS result profiles, responses to domain 4 signalling questions showed the greatest differences between bias risk assessments, while domain 2 items showed the least.
RiskOfBias 
september 2017
Favorable and Publicly-Funded Studies are More Likely to be Published: A Systematic Review and Meta-Analysis
The probability of publication was significantly higher for studies whose characteristics were favorable (OR=2.04, 95%CI: 1.62, 2.57) or statistically significant (OR=2.07, 95%CI: 1.52, 2.81), had a multicenter design (OR=1.32, 95%CI: 1.16, 1.45), and were of later regulatory phase (3/4 vs 1/2, OR=1.34, 95%CI: 1.14, 1.49). Industry funding was modestly associated with lower (OR=0.81, 95%CI: 0.67, 0.99) probability of publication. An exploratory analysis of effect modification revealed that the effect of the study characteristic 'favorable results' on likelihood for publication was stronger for industry-funded studies.
SRStandards 
august 2017
Guidelines for reporting meta-epidemiological methodology research
Meta-epidemiological studies adopt a systematic review or meta-analysis approach to examine the impact of certain characteristics of clinical studies on the observed effect and provide empirical evidence for hypothesised associations. The unit of analysis in meta-epidemiological studies is a study, not a patient. The outcomes of meta-epidemiological studies are usually not clinical outcomes. In this guideline, we adapt items from the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) to fit the context of meta-epidemiological studies.
BetterScience 
august 2017
Inter-rater reliability of AMSTAR is dependent on the pair of reviewers. - PubMed - NCBI
Inter-rater reliability varies widely depending on the pair of reviewers. There may be some shortcomings associated with conducting reliability studies with only two reviewers. Further studies should include additional reviewers and should probably also take account of their level of expertise.
SRAppraisal 
august 2017
The risk of bias in systematic reviews (ROBIS) tool showed fair reliability and good construct validity. - PubMed - NCBI
There is a movement from generic quality checklists towards a more domain based approach in critical appraisal tools. This study aims to report on a first experience with the newly developed risk of bias in systematic reviews (ROBIS) tool and compare it with A Measurement Tool to Assess Systematic Reviews (AMSTAR) that is the most common used tool to assess methodological quality of systematic reviews while assessing validity, reliability and applicability.
SRAppraisal 
august 2017
Testing the effectiveness of simplified search strategies for updating systematic reviews
SSS performed well for clinically focused topics and, with a median sensitivity of 100%, could be a viable alternative to a conventional comprehensive search strategy for updating this type of systematic reviews particularly considering the budget constraints and the volume of new literature being published. For broader topics, 80% sensitivity is likely to be considered too low for a systematic review update in most cases, although it might be acceptable if updating a scoping or rapid review.
SearchMethods 
august 2017
Reproducibility of Studies on Text Mining for Citation Screening in Systematic Reviews: Evaluation and Checklist
The application of text mining techniques to citation screening in the context of systematic literature reviews is a relatively young and growing computational field with high relevance for software engineering, medical research and other fields. However, our work revealed that it is currently difficult if not impossible to independently reproduce the results published in any of the studies investigated. The lack of information about the datasets used limits reproducibility of about 80% of the studies assessed. Also, information about the machine learning algorithms is inadequate in about 27% of the papers. On the plus side, the third party software tools used are mostly free and available.
SearchMethods  Automation 
august 2017
Developmental PBDE Exposure and IQ/ADHD in Childhood: A Systematic Review and Meta-analysis
We concluded there was sufficient evidence supporting an association between developmental PBDE exposure and reduced IQ. Preventing developmental exposure to PBDEs could help prevent loss of human intelligence.
SRCaseStudy 
august 2017
The influence of the team in conducting a systematic review
This commentary highlights a number of relevant articles indicating how the composition of the review team could jeopardise the integrity of the systematic review study and its conclusions. Key biases require closer attention such as sponsorship bias and researcher allegiance, but there may also be less obvious affiliations in teams conducting secondary evidence-syntheses. The importance of transparency and disclosure are now firmly on the agenda for clinical trials and primary research, but the meta-biases that systematic reviews may be at risk from now require further scrutiny.
BetterScience 
august 2017
Producing Cochrane systematic reviews-a qualitative study of current approaches and opportunities for innovation and improvement
The results suggest that improvements to Cochrane's systematic review production models could come from improving clarity of roles and expectations, ensuring continuity and consistency of input, enabling active management of the review process, centralising some review production steps; breaking reviews into smaller "chunks", and improving approaches to building capacity of and sharing information between authors and Review Groups. Respondents noted the important role new technologies have to play in enabling these improvements.
BetterScience 
august 2017
Reconciling disparate data to determine the 'right' answer: A grounded theory of meta analysts' reasoning in meta-analysis
Results show the complexity of meta-analytic reasoning. At each step of the process, participants attempted to reconcile disparate forms of knowledge to determine a 'right' answer (moral concern) and 'accurately' draw a treatment effect (epistemological concern). The reasoning processes often shifted between considering the meta-analysis as if the data were 'whole', and as if the data were discrete components (individual studies). These findings highlight paradigmatic tensions regarding the epistemological premises of meta-analysis, resembling previous historical investigations of the functioning of scientific communities. In understanding why different meta-analysts interpret data differently, it may be unrealistic to expect 'objective' homogenous recommendations based on meta-analyses.
MetaAnalysis 
august 2017
Systematic review of community health impacts of mountaintop removal mining
This systematic review could not reach conclusions on community health effects of MTR mining because of the strong potential for bias in the current body of human literature. Improved characterization of exposures by future community health studies and further study of the effects of MTR mining chemical mixtures in experimental models will be critical to determining health risks of MTR mining to communities. Without such work, uncertainty will remain regarding the impact of these practices on the health of the people who breathe the air and drink the water affected by MTR mining.
SRCaseStudy 
august 2017
Assessing Exposure to Household Air Pollution: A Systematic Review and Pooled Analysis of Carbon Monoxide as a Surrogate Measure of Particulate Matter
Our results suggest that exposure to CO is not a consistently valid surrogate measure of exposure to PM2.5. Studies measuring CO exposure as a surrogate measure of PM exposure should conduct local validation studies for different stove/fuel types and seasons.
SRCaseStudy 
august 2017
History shows that stacking federal science advisory committees doesn't work
Recommendations from “friendly” panels will not win broad support from the scientific community, and I predict the committees will quickly lose their credibility, legitimacy and influence. Consequently, policies and regulations based on the panels’ recommendations will be less likely to withstand public or political scrutiny and be more open to legal challenges than if they were based on more balanced input.
ScienceAdvice  Media 
july 2017
Promote scientific integrity via journal peer review data
There is an increasing push by journals to ensure that data and products related to published papers are shared as part of a cultural move to promote transparency, reproducibility, and trust in the scientific literature. Yet few journals commit to evaluating their effectiveness in implementing reporting standards aimed at meeting those goals.
BetterScience  Media 
july 2017
Temporal trends in sperm count: a systematic review and meta-regression analysis
This comprehensive meta-regression analysis reports a significant decline in sperm counts (as measured by SC and TSC) between 1973 and 2011, driven by a 50–60% decline among men unselected by fertility from North America, Europe, Australia and New Zealand. Because of the significant public health implications of these results, research on the causes of this continuing decline is urgently needed.
SRCaseStudy 
july 2017
Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals
Endocrine active chemicals (EACs) have raised concerns that traditional toxicity-testing protocols might be inadequate to identify all potential hazards to human health because they have the ability to modulate normal hormone function, and small alterations in hormone concentrations, particularly during sensitive life stages, can have lasting and significant effects. To address concerns about potential human health effects from EACs at low doses, this US NAS report develops a strategy to evaluate the evidence for such low-dose effects.
SRCaseStudy 
july 2017
Global movement on endocrine disruptors
National Academies warns endocrine active chemicals may be harmful at lower doses than the US currently tests for.
SRCaseStudy  Media 
july 2017
A systematic review of cardiovascular emergency department visits, hospital admissions and mortality associated with ambient black carbon
We conducted a systematic review to evaluate the potential for cardiovascular morbidity and mortality following exposure to ambient BC, or the related component elemental carbon (EC), in the context of what is already known about the associations between exposure to fine particulate matter (PM2.5) and cardiovascular health outcomes.
SRCaseStudy 
july 2017
Methodological and Reporting Quality of Systematic Reviews Published in the Highest Ranking Journals in the Field of Pain
Endorsement of PRISMA in instructions for authors was not a guarantee of compliance. Methodological and reporting quality of pain-related SRs should be improved using relevant checklists. This can be remedied by a joint effort of authors, editors, and peer reviewers.
SRStandards 
july 2017
Towards a taxonomy of logic models in systematic reviews and health technology assessments: a priori, staged and iterative approaches
The complexity associated with how interventions result - or fail to result - in outcomes, and how context matters is increasingly recognised. Logic models provide an important tool for handling complexity. This proposed taxonomy of logic models offers an improved understanding of the advantages and limitations of logic models across the spectrum from a priori to fully iterative approaches. Choice of logic model should be informed by scope of evidence synthesis, presence/absence of clearly defined PICO elements and feasibility considerations.
ExternalValidity 
july 2017
Systematic review adherence to methodological or reporting quality
Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.
SRStandards 
july 2017
Selecting and implementing overview methods: implications from five exemplar overviews
Overviews are a relatively new methodological innovation, and there are currently substantial variations in the methodological approaches used within different overviews. There are considerable methodological challenges for which optimal solutions are not necessarily yet known. Lessons learnt from five exemplar overviews highlight a number of methodological decisions which may be beneficial to consider during the development of an overview protocol.
SRStandards 
july 2017
Comparison of a full systematic review versus a rapid review approaches to assess a newborn screening test for tyrosinemia type 1
Rapid reviews are increasingly used to replace or complement systematic reviews to support evidence based decision-making. Not enough is known about how this expedited process affects results. This study assesses the difference between rapid and systematic review approaches for a case study of test accuracy of succinylacetone for detecting tyrosinemia type 1.
RapidReviews 
july 2017
Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study
Review finding that prevalence of poor reporting and inadequate methods in clinical trials has decreased over time, especially for sequence generation and allocation concealment. But more could be done, especially in lower impact factor journals. Suggests that awareness of methodological deficits in primary medical research are becoming better-recognised and picked up by at least the higher-impact journals.
BetterScience 
july 2017
Relevance and reliability of experimental data in human health risk assessment of pesticides
This article describes an approach to assess the relevance and reliability of experimental data from guideline-compliant studies as well as from non-guideline studies published in the scientific literature in the specific context of uncertainty and risk assessment of pesticides.
RiskOfBias  RiskAssessment 
july 2017
Meta-Analyses Can Be Credible and Useful
John Ioannidis outlines a strategic programme for improvement of systematic reviews in mental health research, covering meta-analytical methods and changes to primary research, which would also be applicable to the field of environmental health.
BetterScience  Media 
july 2017
Text mining for search term development in systematic reviewing: A discussion of some methods and challenges
We briefly review the literature on applications of text mining for search term development for systematic reviewing. We found that the tools can be used in 5 overarching ways: improving the precision of searches; identifying search terms to improve search sensitivity; aiding the translation of search strategies across databases; searching and screening within an integrated system; and developing objectively derived search strategies. Using a case study and selected examples, we then reflect on the utility of certain technologies (term frequency-inverse document frequency and Termine, term frequency, and clustering) in improving the precision and sensitivity of searches.
SearchMethods  Automation 
july 2017
A WEIGHT OF EVIDENCE FRAMEWORK FOR ENVIRONMENTAL ASSESSMENTS: INFERRING QUALITIES
The weighing of heterogeneous evidence such as conventional laboratory toxicity tests, field tests, biomarkers, and community surveys is essential to environmental assessments. Evidence synthesis and weighing is needed to determine causes of observed effects, hazards posed by chemicals or other agents, the completeness of remediation, and other environmental qualities. As part of its guidelines for WoE in ecological assessments, the U.S. Environmental Protection Agency has developed a generally applicable framework. Its basic steps are: assemble evidence, weight the evidence, and weigh the body of evidence. Use of the framework can increase the consistency and rigor of WoE practices and provide greater transparency than ad hoc and narrative-based approaches.
WeightOfEvidence 
june 2017
Evidence and Health Policy: Using and Regulating Systematic Reviews
Systematic reviews could inform policy more effectively with changes to current practices and the assumptions that sustain these practices—assumptions made by researchers and the organizations that employ them, by public and private funders of systematic reviews, and by organizations that finance, set priorities and standards for, and publish them.
ScienceAdvice  Media 
june 2017
Is the staggeringly profitable business of scientific publishing bad for science?
It is an industry like no other, with profit margins to rival Google – and it was created by one of Britain’s most notorious tycoons: Robert Maxwell.
Media 
june 2017
« earlier      
agencysr automation betterscience controversy critappstudies dataextraction evidencemapping externalvalidity gradingevidence grayliterature heterogeneity interests media metaanalysis peerreview primarystudies problemformulation protocol publicationbias rapidreviews reporting resourcereqs riskassessment riskofbias scienceadvice searchmethods socioeconomics srappraisal srcasestudy srstandards statsmethods systematicmap teaching tools weightofevidence

Copy this bookmark:



description:


tags: