statistics   198780

« earlier    

Psychologists surveyed hundreds of alt-right supporters. The results are unsettling. - Vox
A lot of the findings align with what we intuit about the alt-right: This group is supportive of social hierarchies that favor whites at the top. It’s distrustful of mainstream media and strongly opposed to Black Lives Matter. Respondents were highly supportive of statements like, “There are good reasons to have organizations that look out for the interests of white people.” And when they look at other groups — like black Americans, Muslims, feminists, and journalists — they’re willing to admit they see these people as “less evolved.”

But it’s the degree to which the alt-righters differed from the comparison sample that’s most striking — especially when it came to measures of dehumanization, support for collective white action, and admitting to harassing others online. That surprised even Forscher, the lead author and a professor at the University of Arkansas, who typically doesn’t find such large group difference in his work.
alt-right  psychology  racism  america  vox  statistics 
18 hours ago by laurenipsum
Think Stats: Probability and Statistics for Programmers
for Python:
Most introductory books don't cover Bayesian statistics, but Think Stats is based on the idea that Bayesian methods are too important to postpone. By taking advantage of the PMF and CDF libraries, it is possible for beginners to learn the concepts and solve challenging problems.
statistics  book  o'reilly  oreilly 
18 hours ago by bikesandbooks
why selection bias is the most powerful force in education
Imagine that you are a gubernatorial candidate who is making education and college preparedness a key facet of your campaign. Consider these two state average SAT scores. Quantitative Verbal Total Connecticut 450 480 930 Mississippi …
18 hours ago by michaelfox
In-depth review of Andrew Ng's courses (and comparison to Jeremy Howard's Fast.AI) - 2017
Andrew Ng’s new adventure is a bottom-up approach to teaching neural networks — powerful non-linearity learning algorithms, at a beginner-mid level. Jeremy’s FAST.AI course puts you in the drivers seat from the get-go. He teaches you to move the steering wheel, press the brake, accelerator etc.
AI  ML  NeuralNet  CompSci  programming  Math  statistics  MOOC  Learning  Tutorial 
18 hours ago by mfernando
Cost of Living in Shanghai. Aug 2017. Prices in Shanghai
Cost of Living > China > Shanghai
Cost of Living in Shanghai

Compare Shanghai with:
Summary about cost of living in Shanghai:
Four-person family monthly costs: 1,878.75£ (16,173.06¥) without rent (using our estimator).
A single person monthly costs: 507.59£ (4,369.56¥) without rent.
Cost of living index in Shanghai is 34.90% lower than in London.
Cost of living rank 325th out of 516 cities in the world.
Shanghai has a cost of living index of 57.71.
Do you live in Shanghai? Add data for Shanghai!
Currency: Sticky Currency      Switch to US measurement units
Restaurants [ Edit ] Range
Meal, Inexpensive Restaurant 3.48 £ 2.32-5.81
Meal for 2 People, Mid-range Restaurant, Three-course 23.23 £ 15.10-34.85
statistics  travel  china  world  wiki 
19 hours ago by ndf
Statlect, the digital textbook
Free online textbook on math, statistics and probability
books  free  math  statistics  probability 
19 hours ago by lena
Contra Grant On Exaggerated Differences

An article by Adam Grant called Differences Between Men And Women Are Vastly Exaggerated is going viral, thanks in part to a share by Facebook exec Sheryl Sandberg. It’s a response to an email by a Google employee saying that he thought Google’s low female representation wasn’t a result of sexism, but a result of men and women having different interests long before either gender thinks about joining Google. Grant says that gender differences are small and irrelevant to the current issue. I disagree.

Grant writes:

It’s always precarious to make claims about how one half of the population differs from the other half—especially on something as complicated as technical skills and interests. But I think it’s a travesty when discussions about data devolve into name-calling and threats. As a social scientist, I prefer to look at the evidence.

The gold standard is a meta-analysis: a study of studies, correcting for biases in particular samples and measures. Here’s what meta-analyses tell us about gender differences:

When it comes to abilities, attitudes, and actions, sex differences are few and small.

Across 128 domains of the mind and behavior, “78% of gender differences are small or close to zero.” A recent addition to that list is leadership, where men feel more confident but women are rated as more competent.

There are only a handful of areas with large sex differences: men are physically stronger and more physically aggressive, masturbate more, and are more positive on casual sex. So you can make a case for having more men than women… if you’re fielding a sports team or collecting semen.

The meta-analysis Grant cites is Hyde’s, available here. I’ve looked into it before, and I don’t think it shows what he wants it to show.

Suppose I wanted to convince you that men and women had physically identical bodies. I run studies on things like number of arms, number of kidneys, size of the pancreas, caliber of the aorta, whether the brain is in the head or the chest, et cetera. 90% of these come back identical – in fact, the only ones that don’t are a few outliers like “breast size” or “number of penises”. I conclude that men and women are mostly physically similar. I can even make a statistic like “men and women are physically the same in 78% of traits”.

Then I go back to the person who says women have larger breasts and men are more likely to have penises, and I say “Ha, actually studies prove men and women are mostly physically identical! I sure showed you, you sexist!”

I worry that Hyde’s analysis plays the same trick. She does a wonderful job finding that men and women have minimal differences in eg “likelihood of smiling when not being observed”, “interpersonal leadership style”, et cetera. But if you ask the man on the street “Are men and women different?”, he’s likely to say something like “Yeah, men are more aggressive and women are more sensitive”. And in fact, Hyde found that men were indeed definitely more aggressive, and women indeed definitely more sensitive. But throw in a hundred other effects nobody cares about like “likelihood of smiling when not observed”, and you can report that “78% of gender differences are small or zero”.

Hyde found moderate or large gender differences in (and here I’m paraphrasing very scientific-sounding constructs into more understandable terms) aggressiveness, horniness, language abilities, mechanical abilities, visuospatial skills, mechanical ability, tendermindness, assertiveness, comfort with body, various physical abilities, and computer skills.

Perhaps some peeople might think that finding moderate-to-large-differences in mechanical abilities, computer skills, etc supports the idea that gender differences might play a role in gender balance in the tech industry. But because Hyde’s meta-analysis drowns all of this out with stuff about smiling-when-not-observed, Grant is able to make it sound like Hyde proves his point.

It’s actually worse than this, because Grant misreports the study findings in various ways [EDIT: Or possibly not, see here]. For example, he states that the sex differences in physical aggression and physical strength are “large”. The study very specifically says the opposite of this. Its three different numbers for physical aggression (from three different studies) are 0.4, 0.59, and 0.6, and it sets a cutoff for “large” effects at 0.66 or more.

On the other hand, Grant fails to report an effect that actually is large: mechanical reasoning ability (in the paper as Feingold 1998 DAT mechanical reasoning). There is a large gender difference on this, d = 0.76.

And although Hyde doesn’t look into it in her meta-analysis, other meta-analyses like this one find a large effect size (d = 1.18) for thing-oriented vs. people-oriented interest, the very claim that the argument that Grant is trying to argue against centers around.

So Grant tries to argue against large thing-oriented vs. people-oriented differences by citing a meta-analysis that doesn’t look into them at all. He then misreports the findings of that meta-analysis, exaggerating effects that fit his thesis and failing to report the ones that don’t. Finally, he cites a “summary statistic” that averages away the variation we’re looking for out by combining it with a bunch of noise, and claims the noise proves his point even though the variation is as big as ever.


Next, Grant claims that there are no sex differences in mathematical ability, and also that the sex differences in mathematical ability are culturally determined. I’m not really sure what he means [EDIT: He means sex differences that exist in other countries] but I agree with his first argument – at the levels we’re looking at, there’s no gender difference in math ability.

Grant says that these foreign differences in math ability exist but are due to stereotypes, and so are less noticeable in more progressive, gender-equitable nations:

Girls do as well as boys—or slightly better—in math in elementary, but boys have an edge by high school. Male advantages are more likely to exist in countries that lack gender equity in school enrollment, women in research jobs, and women in parliament—and that have stereotypes associating science with males.

Again, my research suggests no average gender difference in ability, so I can’t speak to whether these differences are caused by stereotypes or not. But I want to go back to the original question: why is there a gender difference in tech-industry-representation [in the US]? Is this also due to stereotypes and the effect of an insufficiently gender-equitable society? Do we find that “countries that lack gender equity in school enrollment” and “stereotypes associating science with males” have fewer women in tech?

No. Galpin investigated the percent of women in computer classes all around the world. Her number of 26% for the US is slightly higher than I usually hear, probably because it’s older (the percent women in computing has actually gone down over time!). The least sexist countries I can think of – Sweden, New Zealand, Canada, etc – all have somewhere around the same number (30%, 20%, and 24%, respectively). The most sexist countries do extremely well on this metric! The highest numbers on the chart are all from non-Western, non-First-World countries that do middling-to-poor on the Gender Development Index: Thailand with 55%, Guyana with 54%, Malaysia with 51%, Iran with 41%, Zimbabwe with 41%, and Mexico with 39%. Needless to say, Zimbabwe is not exactly famous for its deep commitment to gender equality.

Why is this? It’s a very common and well-replicated finding that the more progressive and gender-equal a country, the larger gender differences in personality of the sort Hyde found become. I agree this is a very strange finding, but it’s definitely true. See eg Journal of Personality and Social Psychology, Sex Differences In Big Five Personality Traits Across 55 Cultures:

Previous research suggested that sex differences in personality traits are larger in prosperous, healthy, and egalitarian cultures in which women have more opportunities equal with those of men. In this article, the authors report cross-cultural findings in which this unintuitive result was replicated across samples from 55 nations (n = 17,637).

In case you’re wondering, the countries with the highest gender differences in personality are France, Netherlands, and the Czech Republic. The countries with the lowest sex differences are Indonesia, Fiji, and the Congo.

I conclude that whatever gender-equality-stereotype-related differences Grant has found in the nonexistent math ability difference between men and women, they are more than swamped by the large opposite effects in gender differences in personality. This meshes with what I’ve been saying all along: at the level we’re talking about here, it’s not about ability, it’s about interest.


We know that interests are highly malleable. Female students become significantly more interested in science careers after having a teacher who discusses the problem of underrepresentation. And at Harvey Mudd College, computer science majors were around 10% women a decade ago. Today they’re 55%.

I highly recommend Freddie deBoer’s Why Selection Bias Is The Most Powerful Force In Education. If an educational program shows amazing results, and there’s any possible way it’s selection bias – then it’s selection bias.

I looked into Harvey Mudd’s STEM admission numbers, and, sure enough, they admit women at 2.5x the rate as men. So, yeah, it’s selection bias.

I don’t blame them. All they have to do is cultivate a reputation as a place to go if you’re a woman interested in computer science, attract lots of female CS applicants, then make sure to admit all the CS-interested female applicants they get. In exchange, they get constant glowing … [more]
Feminism  Science  Statistics  Googlememo  db  ScottAlexander 
20 hours ago by walt74
why selection bias is the most powerful force in education – the ANOVA
As can be seen, there is a strong negative relationship between participation rate and average SAT score. Generally, the higher the percentage of students taking the test in a given state, the lower the average score. Why? Think about what it means for students in Mississippi, where the participation rate is 3%, to take the SAT. Those students are the ones who are most motivated to attend college and the ones who are most college-ready. In contrast, in Connecticut 88% of eligible juniors and seniors take the test. (Data.) This means that almost everyone of appropriate age takes the SAT in Connecticut, including many students who are not prepared for college or are only marginally prepared. Most Mississippi students self-select themselves out of the sample. The top performing quintile (20%) of Connecticut students handily outperform the top performing quintile of Mississippi students. Typically, the highest state average in the country is that of North Dakota—where only 2% of those eligible take the SAT at all.

In other words, what we might have perceived as a difference in education quality was really the product of systematic differences in how the considered populations were put together. The groups we considered had a hidden non-random distribution. This is selection bias.
statistics  interesting  success 
21 hours ago by ramitsethi
PostgreSQL, Aggregates and Histograms
Generating a histogram in the psql console using width_bucket.
postgresql  sql  statistics 
21 hours ago by micktwomey

« earlier    

related tags

2017  @subscribed  @toread  academia  academic  ai  alabama  alt-right  america  analytics  argument  article  awesomelists  barchart  bayes  bayesian  benefits  bestpractices  betting  bitcoin  blog  blogs  book  books  branding  business  causality  censusbureau  china  comics  compsci  correlation  course.use  courses  cryptocurrency  css  culture  data  datajournalism  datamanagement  datascience  db  debt  design  directory  discrimination  distribution  economics  economy  education  ethnicity  feminism  forum  france  free  funny  gallup  gaming  geog107  gini.coefficient  github  googlememo  government  graduates  grid  growth  highereducation  history  humor  humour  identity  inequality  inflation  infographic  interesting  javascript  jupyter  keras  language  law.enforcement  learning  leo  library  linguistics  list  livingstandards  machine-learning  machinelearning  map  marketing  math  mathematics  mediapart  metrics  ml  money  mooc  neuralnet  o'reilly  ocw  ons  open_source  opensource  oreilly  parser  pay  pdf  people  pocket  politics  poll  post  postgresql  power.laws  predictions  presentations  probability  product-management  programming  psychology  python  quanta  r  race  racism  reach  reading  recommendation  reference  regression-analysis  remote  research  reserach  salaries  science  scottalexander  seaborn  seo  sex  social.selection  splc  sports_analytics  sql  stats  story  study  success  syllabus  tax  teaching  textbook  timothynick  toolkit  travel  tutorial  tutorials  uk  unemployment  universities  usa  usability  ux  vanillaice  video  visualization  voterid  voterregistration  voting-rights  votingrights  vox  wages  washington  webdesign  webdevelopment  week8  wiki  work  worklaw  world  xkcd 

Copy this bookmark: