social_science   460

« earlier    

Understanding and Misunderstanding Randomized Controlled Trials
RCTs would be more useful if there were more realistic expectations of them and if their pitfalls were better recognized. For example, and contrary to many claims in the applied literature, randomization does not equalize everything but the treatment across treatments and controls, it does not automatically deliver a precise estimate of the average treatment effect (ATE), and it does not relieve us of the need to think about (observed or unobserved) confounders. Estimates apply to the trial sample only, sometimes a convenience sample, and usually selected; justification is required to extend them to other groups, including any population to which the trial sample belongs. Demanding “external validity” is unhelpful because it expects too much of an RCT while undervaluing its contribution. Statistical inference on ATEs involves hazards that are not always recognized. RCTs do indeed require minimal assumptions and can operate with little prior knowledge. This is an advantage when persuading distrustful audiences, but it is a disadvantage for cumulative scientific progress, where prior knowledge should be built upon and not discarded. RCTs can play a role in building scientific knowledge and useful predictions but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not “what works,” but “why things work”.
experiments  intervention  public_policy  social_science  methods  critique  angus.deaton 
4 weeks ago by rvenkat
Zachary M. Jones
Author of "Exploratory Data Analysis using Random Forests". Interested in reproducible research in general.
blog  social_science  computational_science  statistics  Reproducible_Research 
6 weeks ago by jkglasbrenner

« earlier    

related tags

?  abortion  acacdemic  academia  academic  academic_publishing_complex  addiction  administrative_state  agent_based_model  ai  algorithms  amos_tversky  amusing  analysis  andrew.gelman  angus.deaton  artificial_intelligence  arts  bayesian  behavior  behavioral_economics  bias  big_data  biology  blog  book  books  burning_man  cars  cass.sunstein  cass_sunstein  causal_inference  causality  cds_101  children  christianity  cognitive_bias  cognitive_science  commentary  common  complexity  computational_science  computer_science  conspiracy  course  courses  crime  criminal_justice  criminology  critique  cs  css  culture  daniel_kahneman  data  discussion  drugs  economics  education  ego_depletion  elite_opinion  epigenetics  epistemology  ethics  ethnography  everything  experimental_design  experiments  explanation  fake_news  federal_reserve  film  flynn_effect  food  for_friends  gafa  gaming  genetics  gerrymandering  governance  great_recession  health  henry.farrell  history  insightful  interesting  internet  intervention  iq  is  japan  lang:en  lindy_effect  machine_learning  macroeconomics  marketing  marriage  math  measurement  medicine  mental_health  meso  meta-analysis  metaphysics  metaresearch  methods  michael_lewis  modeling  moral_philosophy  must  nassim_taleb  networked_public_sphere  networks  neuroscience  new_media  nostalgia  nytimes  observational_studies  obvious  online_experiments  paper  philosophy  philosophy_of_technology  political-science  political_science  politics  pornography  prediction  privacy  probability  programming  pseudoscience  psychology  public_opinion  public_perception_of_science  public_policy  public_sphere  python  reference  regulation  religion  replication_of_studies  reproducible_research  research  research_practice  richard_thaler  romance  science  science_stuff  sense  sex  social_behavior  social_construction_of_knowledge  social_epistemology  social_media  social_systems  socialpsychology  societies  sociology  sociology_of_science  spatial_statistics  statistics  steven_pinker  strength_training  studies  study  teaching  technology  tutorials  us_supreme_court  usa  writing  zoology 

Copy this bookmark: