nhaliday + gelman   45

The Gelman View – spottedtoad
I have read Andrew Gelman’s blog for about five years, and gradually, I’ve decided that among his many blog posts and hundreds of academic articles, he is advancing a philosophy not just of statistics but of quantitative social science in general. Not a statistician myself, here is how I would articulate the Gelman View:

A. Purposes

1. The purpose of social statistics is to describe and understand variation in the world. The world is a complicated place, and we shouldn’t expect things to be simple.
2. The purpose of scientific publication is to allow for communication, dialogue, and critique, not to “certify” a specific finding as absolute truth.
3. The incentive structure of science needs to reward attempts to independently investigate, reproduce, and refute existing claims and observed patterns, not just to advance new hypotheses or support a particular research agenda.

B. Approach

1. Because the world is complicated, the most valuable statistical models for the world will generally be complicated. The result of statistical investigations will only rarely be to  give a stamp of truth on a specific effect or causal claim, but will generally show variation in effects and outcomes.
2. Whenever possible, the data, analytic approach, and methods should be made as transparent and replicable as possible, and should be fair game for anyone to examine, critique, or amend.
3. Social scientists should look to build upon a broad shared body of knowledge, not to “own” a particular intervention, theoretic framework, or technique. Such ownership creates incentive problems when the intervention, framework, or technique fail and the scientist is left trying to support a flawed structure.

Components

1. Measurement. How and what we measure is the first question, well before we decide on what the effects are or what is making that measurement change.
2. Sampling. Who we talk to or collect information from always matters, because we should always expect effects to depend on context.
3. Inference. While models should usually be complex, our inferential framework should be simple enough for anyone to follow along. And no p values.

He might disagree with all of this, or how it reflects his understanding of his own work. But I think it is a valuable guide to empirical work.
ratty  unaffiliated  summary  gelman  scitariat  philosophy  lens  stats  hypothesis-testing  science  meta:science  social-science  institutions  truth  is-ought  best-practices  data-science  info-dynamics  alt-inst  academia  empirical  evidence-based  checklists  strategy  epistemic 
november 2017 by nhaliday
Analysis of variance - Wikipedia
Analysis of variance (ANOVA) is a collection of statistical models used to analyze the differences among group means and their associated procedures (such as "variation" among and between groups), developed by statistician and evolutionary biologist Ronald Fisher. In the ANOVA setting, the observed variance in a particular variable is partitioned into components attributable to different sources of variation. In its simplest form, ANOVA provides a statistical test of whether or not the means of several groups are equal, and therefore generalizes the t-test to more than two groups. ANOVAs are useful for comparing (testing) three or more means (groups or variables) for statistical significance. It is conceptually similar to multiple two-sample t-tests, but is more conservative (results in less type I error) and is therefore suited to a wide range of practical problems.

good pic: https://en.wikipedia.org/wiki/Analysis_of_variance#Motivating_example

tutorial by Gelman: http://www.stat.columbia.edu/~gelman/research/published/econanova3.pdf

so one way to think of partitioning the variance:
y_ij = alpha_i + beta_j + eps_ij
Var(y_ij) = Var(alpha_i) + Var(beta_j) + Cov(alpha_i, beta_j) + Var(eps_ij)
and alpha_i, beta_j are independent, so Cov(alpha_i, beta_j) = 0

can you make this work w/ interaction effects?
data-science  stats  methodology  hypothesis-testing  variance-components  concept  conceptual-vocab  thinking  wiki  reference  nibble  multi  visualization  visual-understanding  pic  pdf  exposition  lecture-notes  gelman  scitariat  tutorial  acm  ground-up  yoga 
july 2017 by nhaliday
Say a little prior for me: more on climate change - Statistical Modeling, Causal Inference, and Social Science
http://www.fooledbyrandomness.com/climateletter.pdf
We have only one planet. This fact radically constrains the kinds of risks that are appropriate to take at a large scale. Even a risk with a very low probability becomes unacceptable when it affects all of us – there is no reversing mistakes of that magnitude.
gelman  scitariat  discussion  links  science  meta:science  epistemic  info-dynamics  climate-change  causation  models  thinking  priors-posteriors  atmosphere  environment  multi  pdf  rhetoric  uncertainty  risk  outcome-risk  moments 
april 2017 by nhaliday
The Association for Psychological Pseudoscience presents . . . - Statistical Modeling, Causal Inference, and Social Science
Hey! The organization that publishes all those Psychological Science-style papers has scheduled their featured presentations for their next meeting.

Included are:
– That person who slaps the label “terrorists” on people who have the nerve to question their statistical errors.
– One of the people who claimed that women were 20 percentage points were likely to vote for Barack Obama, during a certain time of the month.
– One of the people who claimed that women are three times as likely to wear red, during a certain time of the month.
– The editor of the notorious PPNAS papers on himmicanes, air rage, and ages ending in 9.
– One of the people who claimed, “That a person can, by assuming two simple 1-min poses, embody power and instantly become more powerful has real-world, actionable implications.”
– Yet another researcher who responded to a failed replication without even acknowledging the possibility that their original claims might have been in error.
– The person who claimed, “Barring intentional fraud, every finding is an accurate description of the sample on which it was run.”

The whole thing looks like a power play. The cargo-cult social psychologists have the power, and they’re going to use it. They’ll show everyone who’s boss. Nobody’s gonna use concerns such as failed replications, lack of face validity, and questionable research practices to push them around!

...

It’s a guild, man, nuthin but an ivy-covered Chamber of Commerce. Which is fine—restraint of trade is as American as baseball, hot dogs, apple pie, and Chevrolet.

The only trouble is that I’m guessing that the Association for Psychological Science has thousands of members who have no interest in protecting the interests of this particular club. I said it before and I’ll say it again: Psychology is not just a club of academics, and “psychological science” is not just the name of their treehouse.

Scientists are furious after a famous psychologist accused her peers of 'methodological terrorism': http://www.businessinsider.com/susan-fiske-methodological-terrorism-2016-9

When the Revolution Came for Amy Cuddy: https://www.nytimes.com/2017/10/18/magazine/when-the-revolution-came-for-amy-cuddy.html
As a young social psychologist, she played by the rules and won big: an influential study, a viral TED talk, a prestigious job at Harvard. Then, suddenly, the rules changed.

https://twitter.com/StuartJRitchie/status/920934644840849408
Silly me! I thought the rule "don't seek massive publicity for extremely flimsy results" had been around forever...

https://twitter.com/aleksjoksic/status/921507320978427904
Feeling victimized by criticism & the want to keep it quiet is related to a certain sex difference in doing science/intellectual discourse..
One mode is more masculine,the other is more feminine.@Steve_Sailer has great excerpts from Alastair Roberts on this http://www.unz.com/isteve/intellectual-discourse-taking/
gelman  scitariat  rant  critique  social-science  psychology  social-psych  replication  events  westminster  error  academia  commentary  science  drama  news  org:biz  org:lite  multi  realness  truth  org:mag  profile  longform  twitter  social  albion  gender  gender-diff  identity-politics  personality  emotion  short-circuit  meta:rhetoric  debate  gnon  isteveish  absolute-relative  egalitarianism-hierarchy  anomie  social-norms  quotes  responsibility 
april 2017 by nhaliday
‘How dare you work on whites’: Professors under fire for research on white mortality - The Washington Post
the paper: http://www.pnas.org/content/112/49/15078

https://www.theatlantic.com/business/archive/2017/03/angus-deaton-qa/518880/
The Nobel laureate Angus Deaton discusses extreme poverty, opioid addiction, Trump voters, robots, and rent-seeking.

co-authored the "dead white people paper" w/ wife

http://andrewgelman.com/2017/03/23/mortality-rate-trends-age-ethnicity-sex-state/
point about expansion of education seems important
https://spottedtoad.wordpress.com/2016/01/17/correlates-of-middle-aged-white-mortality/

https://www.washingtonpost.com/news/wonk/wp/2017/03/24/the-disease-killing-white-americans-goes-way-deeper-than-opioids/
https://www.wsj.com/articles/death-rates-rise-for-wide-swath-of-white-adults-1490240740
http://www.newyorker.com/news/benjamin-wallace-wells/the-despair-of-learning-that-experience-no-longer-matters
https://www.nytimes.com/interactive/2017/09/02/upshot/fentanyl-drug-overdose-deaths.html

Diverging Life Expectancies and Voting Patterns in the 2016 US Presidential Election.: https://www.ncbi.nlm.nih.gov/pubmed/28817322
Changes in county life expectancy from 1980 to 2014 were strongly negatively associated with Trump's vote share, with less support for Trump in counties experiencing greater survival gains. Counties in which life expectancy stagnated or declined saw a 10-percentage-point increase in the Republican vote share between 2008 and 2016.

DESPAIR AND DECADES-LONG DEINDUSTRIALIZATION: https://spottedtoad.wordpress.com/2018/01/10/despair-and-decades-long-deindustrialization/

WE’VE BEEN HERE BEFORE: https://spottedtoad.wordpress.com/2018/01/12/weve-been-here-before/
A concept that seems to me to be missing from the Ruhm vs. Case/Deaton debate on “deaths of despair” is that of social crisis.

This seems to me to be the case for American Indians, who began experiencing what looks like a similar social crisis to non-college educated whites about a decade beforehand: rapidly escalating rates of suicide, drug overdoses, exit from the workforce, and even alcohol-related deaths (which were already very high for American Indians well before 2000, of course):

...

The common thread here would seem to be replacement of workforce participation with transfer payments, particularly cash transfers (since, my own reservations about Medicaid aside, increases in in-kind payments and SNAP since the 80s haven’t seemed to exert the same disruptive effect.) As I’ve said before, it seems very likely to me that technology will push an ever larger segment of society out of the economy, sooner or later, but how to prevent this from tearing apart our social fabric I don’t know.

Once It Was Overdue Books. Now Librarians Fight Overdoses.: https://www.nytimes.com/2018/02/28/nyregion/librarians-opioid-heroin-overdoses.html
news  org:rec  current-events  sociology  academia  society  race  managerial-state  patho-altruism  class  clown-world  malaise  death  longevity  education  stagnation  wonkish  autor  opioids  public-health  coming-apart  dignity  org:mag  labor  usa  trends  age-generation  politics  social-capital  health  demographics  study  summary  anomie  economics  inequality  rent-seeking  automation  2016-election  drugs  interview  comparison  developing-world  midwest  northeast  trump  gelman  scitariat  gender  confounding  media  regularizer  class-warfare  medicine  polisci  disease  org:nat  epidemiology  data  spock  causation  analysis  ratty  unaffiliated  obesity  correlation  critique  debate  chart  🎩  visualization  time-series  multi  nihil  speculation  roots  impetus  trade  nationalism-globalism  china  asia  monetary-fiscal  money  debt  investing  heavy-industry  world  the-bones  rot  unintended-consequences  farmers-and-foragers  africa  reflection  welfare-state  futurism  books  urban-rural 
april 2017 by nhaliday
Placebo interventions for all clinical conditions. - PubMed - NCBI
We did not find that placebo interventions have important clinical effects in general. However, in certain settings placebo interventions can influence patient-reported outcomes, especially pain and nausea, though it is difficult to distinguish patient-reported effects of placebo from biased reporting. The effect on pain varied, even among trials with low risk of bias, from negligible to clinically important. Variations in the effect of placebo were partly explained by variations in how trials were conducted and how patients were informed.

How much of the placebo 'effect' is really statistical regression?: https://www.ncbi.nlm.nih.gov/pubmed/6369471
Statistical regression to the mean predicts that patients selected for abnormalcy will, on the average, tend to improve. We argue that most improvements attributed to the placebo effect are actually instances of statistical regression. First, whereas older clinical trials susceptible to regression resulted in a marked improvement in placebo-treated patients, in a modern series of clinical trials whose design tended to protect against regression, we found no significant improvement (median change 0.3 per cent, p greater than 0.05) in placebo-treated patients.

Placebo effects are weak: regression to the mean is the main reason ineffective treatments appear to work: http://www.dcscience.net/2015/12/11/placebo-effects-are-weak-regression-to-the-mean-is-the-main-reason-ineffective-treatments-appear-to-work/

A radical new hypothesis in medicine: give patients drugs they know don’t work: https://www.vox.com/science-and-health/2017/6/1/15711814/open-label-placebo-kaptchuk
People on no treatment got about 30 percent better. And people who were given an open-label placebo got 60 percent improvement in the adequate relief of their irritable bowel syndrome.

Surgery Is One Hell Of A Placebo: https://fivethirtyeight.com/features/surgery-is-one-hell-of-a-placebo/
study  psychology  social-psych  medicine  meta:medicine  contrarianism  evidence-based  embodied-cognition  intervention  illusion  realness  meta-analysis  multi  science  stats  replication  gelman  regularizer  thinking  regression-to-mean  methodology  insight  hmm  news  org:data  org:lite  interview  tricks  drugs  cost-benefit  health  ability-competence  chart 
march 2017 by nhaliday
Measurement error and the replication crisis | Science
In a low-noise setting, the theoretical results of Hausman and others correctly show that measurement error will attenuate coefficient estimates. But we can demonstrate with a simple exercise that the opposite occurs in the presence of high noise and selection on statistical significance.
study  org:nat  science  meta:science  stats  signal-noise  gelman  methodology  hypothesis-testing  replication  social-science  error  metabuch  unit  nibble  bounded-cognition  measurement  🔬  info-dynamics 
february 2017 by nhaliday
Selection against variants in the genome associated with educational attainment
first direct, genotypic, longitudinal evidence I think?
fulltext: https://www.dropbox.com/s/9vq5t6urtu930xe/2017-kong.pdf

Epidemiological and genetic association studies show that genetics play an important role in the attainment of education. Here, we investigate the effect of this genetic component on the reproductive history of 109,120 Icelanders and the consequent impact on the gene pool over time. We show that an educational attainment polygenic score, POLY_EDU, constructed from results of a recent study is associated with delayed reproduction (P < 10^−100) and fewer children overall. _The effect is stronger for women and remains highly significant after adjusting for educational attainment._ Based on 129,808 Icelanders born between 1910 and 1990, we find that the average POLY_EDU has been declining at a rate of ∼0.010 standard units per decade, which is substantial on an evolutionary timescale. Most importantly, because POLY_EDU only captures a fraction of the overall underlying genetic component the latter could be declining at a rate that is two to three times faster.

- POLY_EDU has negative effect on RS for men, while EDU itself (or just controlling for POLY_EDU?) has positive effect
- also has some trends for height (0) and schizophrenia (-)

Natural selection making 'education genes' rarer, says Icelandic study: https://www.reddit.com/r/slatestarcodex/comments/5opugw/natural_selection_making_education_genes_rarer/
Gwern pretty pessimistic
http://www.smithsonianmag.com/smart-news/study-shows-genes-associated-education-are-declining-180961836/

http://andrewgelman.com/2017/07/30/iceland-education-gene-trend-kangaroo/

The Marching Morons: https://westhunt.wordpress.com/2017/01/22/the-marching-morons/
There’s a new paper out on how the frequency of variants that affect educational achievement (which also affect IQ) have been changing over time in Iceland. Naturally, things are getting worse.

We don’t have all those variants identified yet, but from the fraction we do know and the rate of change, they estimate that genetic potential for IQ is dropping about 0.30 point per decade – 3 points per century, about a point a generation. In Iceland.

Sounds reasonable, in the same ballpark as demography-based estimates.

It would be interesting to look at moderately recent aDNA and see when this trend started – I doubt if has been going on very long. [ed.: I would guess since the demographic transition/industrial revolution, though, right?]

This is the most dangerous threat the human race faces.

Paper Review: Icelandic Dysgenics: http://www.unz.com/akarlin/paper-review-icelandic-dysgenics/
The main mechanism was greater age at first child, not total number of children (i.e. the clever are breeding more slowly).
study  gwern  psychology  cog-psych  iq  genetics  dysgenics  GWAS  🌞  longitudinal  europe  trends  education  multi  gender  genetic-correlation  nordic  recent-selection  org:nat  biodet  science-anxiety  biophysical-econ  embodied  psychiatry  disease  demographic-transition  deep-materialism  age-generation  gender-diff  behavioral-gen  rot  the-bones  modernity  microfoundations  dropbox  ratty  ssc  reddit  social  commentary  news  org:lite  enhancement  analysis  linearity  org:anglo  org:mag  org:sci  usa  gelman  scitariat  west-hunter  sapiens  discussion  gnon  demographics  new-religion  nihil  summary  fertility  rhythm  flynn  spearhead  direct-indirect  human-capital  hari-seldon 
january 2017 by nhaliday
Female Doctors May Be Better Than Male Doctors - The Atlantic
study is linked in article

We examined the association between physician sex and 30-day mortality and readmission rates, adjusted for patient and physician characteristics and hospital fixed effects (effectively comparing female and male physicians within the same hospital). As a sensitivity analysis, we examined only physicians focusing on hospital care (hospitalists), among whom patients are plausibly quasi-randomized to physicians based on the physician’s specific work schedules. We also investigated whether differences in patient outcomes varied by specific condition or by underlying severity of illness.

You'll have to figure this one out for yourselves: http://andrewgelman.com/2016/12/21/youll-figure-one/
news  org:mag  study  summary  medicine  gender  natural-experiment  longevity  healthcare  diversity  intervention  endogenous-exogenous  multi  gelman  replication  critique  scitariat 
december 2016 by nhaliday
Links 12/16: Site Makes Right | Slate Star Codex
Prescient Marginal Revolution post from last year on how celebrities and CEOs make better politicians than politicians.

Chinese scientists claim they can use machine learning to predict criminality from facial appearance. Still needs a lot of double-checking before accepted, but basically believable. Maybe related to mutational load: “The variation among criminal faces is significantly greater than that of the non-criminal faces. The two manifolds consisting of criminal and non-criminal faces appear to be concentric, with the non-criminal manifold lying in the kernel with a smaller span”.

Less Wrong is trying to regain its status as a good discussion hub and it’s actually going pretty well. Among the posts there worth checking out: A Return To Discussion, Double Crux: A Strategy For Resolving Disagreement, and Sample Means: How Do They Work?

Ozy at Thing of Things did a social justice Intellectual Turing Test.

Remember Trump’s claim that millions of non-citizens voted in the election? It comes from a journal article in Electoral Studies (article, popular summary) calculating that several hundred thousand non-citizens probably voted in the 2008 election. But further research has challenged that claim (study, popular article), and it now seems to be very much in doubt. [EDIT: National Review defends the study, and relevant SSC]

Andrew Gelman: How Can You Evaluate A Research Paper?
ratty  yvain  ssc  links  commentary  multi  politics  lesswrong  trump  2016-election  gelman  culture-war  replication  scitariat 
december 2016 by nhaliday
A Simple Explanation for the Replication Crisis in Science · Simply Statistics
The key point here is that in both astronomy and epidemiology expectations are low with respect to individual studies. It’s difficult to have a replication crisis when nobody believes the findings in the first place. Investigators have a culture of distrusting individual one-off findings until they have been replicated numerous times. In my own area of research, the idea that ambient air pollution causes health problems was difficult to believe for decades, until we started seeing the same associations appear in numerous studies conducted all around the world. It’s hard to imagine any single study “proving” that connection, no matter how well it was conducted.
replication  science  thinking  rhetoric  analysis  medicine  social-science  gelman  scitariat  meta:science  meta:medicine  space  epidemiology  info-dynamics 
september 2016 by nhaliday
A Variant on “Statistically Controlling for Confounding Constructs is Harder than you Think”
It’s taken me some time to master this formalism, but I now find it quite easy to reason about these kinds of issues thanks to the brevity of graphical models as a notational technique. I’d love to see this approach become more popular in psychology, given that it has already become quite widespread in other fields. Of course, Westfall and Yarkoni are already advocating for something very similar by advocating for the use of SEM’s, but the graphical approach is strictly more general than SEM’s and, in my personal opinion, strictly simpler to reason about.
bayesian  stats  thinking  visualization  study  science  gelman  hmm  methodology  causation  acmtariat  meta:science  graphs  commentary  techtariat  hypothesis-testing  org:bleg  nibble  scitariat  confounding  🔬  info-dynamics  direct-indirect  volo-avolo  endo-exo  endogenous-exogenous  control  graphical-models 
may 2016 by nhaliday
Academic urban legends
How an academic urban legend can spread because of the difficulty of clear citation: http://andrewgelman.com/2016/06/19/29425/
spinach and iron
science  academia  replication  critique  study  stories  meta:science  history  error  postmortem  info-dynamics  spreading  multi  cocktail  health  gelman  scitariat  myth  narrative 
august 2014 by nhaliday

bundles : peepssci

related tags

2016-election  ability-competence  absolute-relative  academia  acm  acmtariat  additive  advanced  adversarial  advice  africa  age-generation  aging  albion  alt-inst  analysis  anomie  antidemos  article  asia  atmosphere  automation  autor  bayesian  behavioral-gen  best-practices  biodet  biophysical-econ  bits  blog  blowhards  bonferroni  books  bounded-cognition  career  causation  chart  cheatsheet  checklists  china  civic  class  class-warfare  climate-change  clown-world  cocktail  cog-psych  coming-apart  commentary  comparison  compensation  complex-systems  concept  conceptual-vocab  confounding  contrarianism  control  correlation  cost-benefit  counterfactual  critique  culture  culture-war  current-events  data  data-science  dataviz  death  debate  debt  deep-materialism  democracy  demographic-transition  demographics  descriptive  developing-world  differential-privacy  dignity  direct-indirect  discussion  disease  diversity  draft  drama  dropbox  drugs  dysgenics  econometrics  economics  education  effect-size  egalitarianism-hierarchy  embodied  embodied-cognition  emotion  empirical  encyclopedic  endo-exo  endogenous-exogenous  enhancement  environment  epidemiology  epistemic  error  europe  events  evidence-based  expert  expert-experience  explanation  exposition  farmers-and-foragers  fertility  flynn  frequentist  futurism  gedanken  gelman  gender  gender-diff  generalization  genetic-correlation  genetics  gnon  gotchas  government  grad-school  gradient-descent  graphical-models  graphs  ground-up  group-level  growth  growth-mindset  guide  GWAS  gwern  hanson  happy-sad  hari-seldon  health  healthcare  heavy-industry  hidden-motives  higher-ed  history  hmm  howto  human-capital  human-ml  hypothesis-testing  identity-politics  illusion  impetus  incentives  inequality  info-dynamics  info-foraging  information-theory  insight  institutions  intervention  interview  intuition  investing  iq  is-ought  isteveish  iteration-recursion  judgement  labor  learning-theory  lecture-notes  len:long  len:short  lens  lesswrong  linearity  liner-notes  links  long-term  longevity  longform  longitudinal  lower-bounds  machine-learning  malaise  managerial-state  math  meaningness  measurement  media  medicine  meta-analysis  meta:medicine  meta:prediction  meta:rhetoric  meta:science  metabuch  metameta  methodology  metrics  microfoundations  midwest  models  modernity  moments  monetary-fiscal  money  monte-carlo  mrtz  multi  multiplicative  murray  myth  narrative  nationalism-globalism  natural-experiment  new-religion  news  nibble  nihil  no-go  noise-structure  nordic  northeast  numerics  obesity  objektbuch  oly  online-learning  open-closed  opioids  org:anglo  org:biz  org:bleg  org:data  org:edu  org:lite  org:mag  org:nat  org:rec  org:sci  outcome-risk  papers  patho-altruism  pdf  people  personality  perturbation  phd  philosophy  physics  pic  planning  plots  poast  polisci  politics  poll  postmortem  prediction  priors-posteriors  productivity  profile  progression  psychiatry  psychology  public-health  publishing  quotes  race  rant  ratty  realness  recent-selection  recommendations  reddit  reference  reflection  regional-scatter-plots  regression  regression-to-mean  regularizer  religion  rent-seeking  replication  research  research-program  responsibility  review  rhetoric  rhythm  risk  robust  roots  rot  sampling  sapiens  science  science-anxiety  scitariat  selection  sensitivity  short-circuit  signal-noise  signaling  social  social-capital  social-norms  social-psych  social-science  society  sociology  solid-study  space  spearhead  speculation  spock  spreading  ssc  stagnation  stats  status  stereotypes  stories  strategy  stream  street-fighting  stress  study  summary  survey  systems  tails  techtariat  tetlock  the-bones  the-south  thinking  time-series  trade  trends  tricks  trump  truth  tutorial  twitter  unaffiliated  uncertainty  unintended-consequences  unit  urban-rural  usa  values  vampire-squid  variance-components  video  visual-understanding  visualization  volo-avolo  welfare-state  west-hunter  westminster  wiki  wonkish  world  X-not-about-Y  yoga  yvain  🌞  🎩  🔬 

Copy this bookmark:



description:


tags: