nhaliday + matrix-factorization   43

Pearson correlation coefficient - Wikipedia
https://en.wikipedia.org/wiki/Coefficient_of_determination
deleted but it was about the Pearson correlation distance: 1-r
I guess it's a metric

https://en.wikipedia.org/wiki/Explained_variation

http://infoproc.blogspot.com/2014/02/correlation-and-variance.html
A less misleading way to think about the correlation R is as follows: given X,Y from a standardized bivariate distribution with correlation R, an increase in X leads to an expected increase in Y: dY = R dX. In other words, students with +1 SD SAT score have, on average, roughly +0.4 SD college GPAs. Similarly, students with +1 SD college GPAs have on average +0.4 SAT.

this reminds me of the breeder's equation (but it uses r instead of h^2, so it can't actually be the same)

stats  science  hypothesis-testing  correlation  metrics  plots  regression  wiki  reference  nibble  methodology  multi  twitter  social  discussion  best-practices  econotariat  garett-jones  concept  conceptual-vocab  accuracy  causation  acm  matrix-factorization  todo  explanation  yoga  hsu  street-fighting  levers  🌞  2014  scitariat  variance-components  meta:prediction  biodet  s:**  mental-math  reddit  commentary  ssc  poast  gwern  data-science  metric-space  similarity  measure  dependence-independence
may 2017 by nhaliday
Growing Collectivism: Irrigation, Group Conformity and Technological Divergence
This paper examines the origins of collectivist cultures that emphasize group conformity over individual autonomy. In line with the hypothesis that collaboration within groups in pre-industrial agriculture favored the emergence of collectivism, I find that societies whose ancestors jointly practiced irrigation agriculture have stronger collectivist norms today. The positive effect of irrigation on contemporary collectivism holds across countries, subnational districts within countries, and migrants. For causal identification, I instrument the historical adoption of irrigation by its geographic suitability. Furthermore, this paper establishes that, by favoring conformity, irrigation agriculture has contributed to the global divergence of technology. I document (i) a negative effect of traditional irrigation agriculture on contemporary innovativeness of countries, cities, and migrants; (ii) a positive effect on selection into routine-intensive occupations; and (iii) that the initial technological advantage of irrigation societies was reversed after 1500.

http://marginalrevolution.com/marginalrevolution/2016/09/varying-rainfall-make-people-collectivists.html
This kind of investigation is always going to be fraught with uncertainty and also controversy, given imperfections of data and methods. Nonetheless I find this one of the more plausible macro-historical hypotheses, perhaps because of my own experience in central Mexico, where varying rainfall still is the most important economic event of the year, though it is rapidly being supplanted by the variability of tourist demand for arts and crafts. And yes, they are largely collectivist, at least at the clan level, with extensive systems of informal social insurance and very high implicit social marginal tax rates on accumulated wealth.

Have you noticed it rains a lot in England?

(lol)

http://marginalrevolution.com/marginalrevolution/2014/05/chinese-wheat-eaters-vs-rice-eaters-speculative.html
http://www.scmp.com/news/china/article/1508726/why-chinas-wheat-growing-north-produces-individualists-and-its-rice
in-depth reflection on agricultural ecologies, Europe vs China, and internal Chinese differences/ethnic identity/relations with barbarians/nomads, etc.: https://www.gnxp.com/blog/2008/08/wealth-of-communities.php

Irrigation and Autocracy: http://www.econ.ku.dk/bentzen/Irrigation_and_Autocracy.pdf
http://marginalrevolution.com/marginalrevolution/2012/08/in-defense-of-the-wittvogel-thesis.html

Emerging evidence of cultural differences linked to rice versus wheat agriculture: https://sci-hub.tw/https://www.sciencedirect.com/science/article/pii/S2352250X1930082X
- Historical rice farming linked to interdependent culture.
- Differences tested in China and Japan, as well as in worldwide comparison.
- There is evidence for differences among urbanites with no direct experience farming.
- Rice farming is also linked to holistic thought, fewer patents for inventions.
- Rice cultures are not ‘pro-social’ but rather tight ties, strong division of close versus distant ties.

The agricultural roots of Chinese innovation performance: https://sci-hub.tw/https://www.sciencedirect.com/science/article/pii/S0014292119300893
We provide robust evidence that counties with a legacy of rice cultivation generate fewer patent applications than other counties, and a legacy of wheat production tends to be associated with more patent applications. The results for rice are robust to, e.g., controlling for temperature, precipitation, irrigation, disease burden, religiosity, and corruption, as well as accounting for migration patterns.

Steve Hsu on this stuff:
Genetic variation in Han Chinese population: http://infoproc.blogspot.com/2017/07/genetic-variation-in-han-chinese.html
Largest component of genetic variation is a N-S cline (phenotypic N-S gradient discussed here). Variance accounted for by second (E-W) PC vector is much smaller and the Han population is fairly homogeneous in genetic terms: ...while we revealed East-to-West structure among the Han Chinese, the signal is relatively weak and very little structure is discernible beyond the second PC (p.24).

Neandertal ancestry does not vary significantly across provinces, consistent with admixture prior to the dispersal of modern Han Chinese.

http://infoproc.blogspot.com/2014/01/china-1793.html
My fellow officers informed me, that while the negotiation was going on, the ships were constantly crowded with all kinds of refreshments, and that when they were first boarded by the Chinese they received every attention from them that could be shown; and that the presents received by the different officers belonging to the embassy, were of immense value. That the natives of this part of China were of different complexions and manners from those in and near Canton; their colour being nearly white; and in their manners were much more free and candid; and that they were of a larger stature, and more athletic than the southern Chinese—they were much more sociable, and not so particular respecting their women being seen by the men. And were even fond of receiving the officers into their houses, when on shore, provided it could be done without the knowledge of the mandarins.

http://infoproc.blogspot.com/2014/06/large-scale-psychological-differences.html
The study below discusses a psychological/cognitive/personality gradient between N and S China, possibly driven by a history of wheat vs rice cultivation.

https://en.wikipedia.org/wiki/Northern_and_southern_China
http://shanghaiist.com/2015/07/01/average-heights-men-women.php
https://www.quora.com/Why-are-Northern-Chinese-people-generally-taller-than-Southern-Chinese

https://gnxp.nofe.me/2017/08/01/the-great-genetic-map-of-china/
pdf  study  economics  growth-econ  cliometrics  path-dependence  roots  wealth-of-nations  shift  homo-hetero  innovation  divergence  individualism-collectivism  broad-econ  values  stylized-facts  china  asia  sinosphere  agriculture  h2o  leviathan  institutions  group-level  social-structure  authoritarianism  scale  egalitarianism-hierarchy  europe  the-great-west-whale  madisonian  chart  prepping  cultural-dynamics  civilization  🎩  correlation  urban  transportation  frontier  regional-scatter-plots  rent-seeking  orient  anglosphere  great-powers  antidemos  n-factor  multi  econotariat  marginal-rev  commentary  within-group  gnxp  scitariat  gregory-clark  malthus  disease  parasites-microbiome  health  diet  modernity  political-econ  world  north-weingast-like  occident  microfoundations  open-closed  general-survey  fluid  branches  urban-rural  explanans  decentralized  domestication  anthropology  hari-seldon  straussian  britain  anglo  troll  responsibility  moments  outcome-risk  uncertainty  latin-america  pop-diff  recent-selection  flux-stasis  dist
may 2017 by nhaliday
Backwardness | West Hunter
Back around the time I was born, anthropologists sometimes talked about some cultures being more advanced than others. This was before they decided that all cultures are equal, except that some are more equal than others.

...

I’ve been trying to estimate the gap between Eurasian and Amerindian civilization. The Conquistadors were, in a sense, invaders from the future: but just how far in the future? What point in the history of the Middle East is most similar to the state of the Amerindian civilizations of 1500 AD ?

I would argue that the Amerindian civilizations were less advanced than the Akkadian Empire, circa 2300 BC. The Mayans had writing, but were latecomers in metallurgy. The Inca had tin and arsenical bronze, but didn’t have written records. The Akkadians had both – as well as draft animals and the wheel. You can maybe push the time as far back as 2600 BC, since Sumerian cuneiform was in pretty full swing by then. So the Amerindians were around four thousand years behind.

https://westhunt.wordpress.com/2012/02/10/backwardness/#comment-1520
Excepting the use of iron, sub-Saharan Africa, excepting Ethiopia, was well behind the most advanced Amerindian civilizations circa 1492. I am right now resisting the temptation to get into a hammer-and-tongs discussion of Isandlwana, Rorke’s Drift, Blood River, etc. – and we would all be better off if I continued to do so.

https://en.wikipedia.org/wiki/Battle_of_Blood_River
The Battle of Blood River (Afrikaans: Slag van Bloedrivier; Zulu: iMpi yaseNcome) is the name given for the battle fought between _470 Voortrekkers_ ("Pioneers"), led by Andries Pretorius, and _an estimated 80,000 Zulu attackers_ on the bank of the Ncome River on 16 December 1838, in what is today KwaZulu-Natal, South Africa. Casualties amounted to over 3,000 of king Dingane's soldiers dead, including two Zulu princes competing with Prince Mpande for the Zulu throne. _Three Pioneers commando members were lightly wounded_, including Pretorius himself.

https://en.wikipedia.org/wiki/Battle_of_Rorke%27s_Drift
https://en.wikipedia.org/wiki/Battle_of_Isandlwana

Déjà Vu all over again: America and Europe: https://westhunt.wordpress.com/2014/11/12/deja-vu-all-over-again-america-and-europe/
In terms of social organization and technology, it seems to me that Mesolithic Europeans (around 10,000 years ago) were like archaic Amerindians before agriculture. Many Amerindians on the west coast were still like that when Europeans arrived – foragers with bows and dugout canoes.

On the other hand, the farmers of Old Europe were in important ways a lot like English settlers: the pioneers planted wheat, raised pigs and cows and sheep, hunted deer, expanded and pushed aside the previous peoples, without much intermarriage. Sure, Anglo pioneers were literate, had guns and iron, were part of a state, all of which gave them a much bigger edge over the Amerindians than Old Europe ever had over the Mesolithic hunter-gatherers and made the replacement about ten times faster – but in some ways it was similar. Some of this similarity was the product of historical accidents: the local Amerindians were thin on the ground, like Europe’s Mesolithic hunters – but not so much because farming hadn’t arrived (it had in most of the United States), more because of an ongoing population crash from European diseases.

On the gripping hand, the Indo-Europeans seem to have been something like the Plains Indians: sure, they raised cattle rather than living off abundant wild buffalo, but they too were transformed into troublemakers by the advent of the horse. Both still did a bit of farming. They were also alike in that neither of them really knew what they were doing: neither were the perfected product of thousands of years of horse nomadry. The Indo-Europeans were the first raiders on horseback, and the Plains Indians had only been at it for a century, without any opportunity to learn state-of-the-art tricks from Eurasian horse nomads.

The biggest difference is that the Indo-Europeans won, while the Plains Indians were corralled into crappy reservations.

Quantitative historical analysis uncovers a single dimension of complexity that structures global variation in human social organization: http://www.pnas.org/content/early/2017/12/20/1708800115.full
Do human societies from around the world exhibit similarities in the way that they are structured, and show commonalities in the ways that they have evolved? These are long-standing questions that have proven difficult to answer. To test between competing hypotheses, we constructed a massive repository of historical and archaeological information known as “Seshat: Global History Databank.” We systematically coded data on 414 societies from 30 regions around the world spanning the last 10,000 years. We were able to capture information on 51 variables reflecting nine characteristics of human societies, such as social scale, economy, features of governance, and information systems. Our analyses revealed that these different characteristics show strong relationships with each other and that a single principal component captures around three-quarters of the observed variation. Furthermore, we found that different characteristics of social complexity are highly predictable across different world regions. These results suggest that key aspects of social organization are functionally related and do indeed coevolve in predictable ways. Our findings highlight the power of the sciences and humanities working together to rigorously test hypotheses about general rules that may have shaped human history.

Fig. 2.

The General Social Complexity Factor Is A Thing: https://www.gnxp.com/WordPress/2017/12/21/the-general-social-complexity-factor-is-a-thing/
west-hunter  scitariat  discussion  civilization  westminster  egalitarianism-hierarchy  history  early-modern  age-of-discovery  comparison  europe  usa  latin-america  farmers-and-foragers  technology  the-great-west-whale  divergence  conquest-empire  modernity  ranking  aphorism  rant  ideas  innovation  multi  africa  poast  war  track-record  death  nihil  nietzschean  lmao  wiki  attaq  data  twitter  social  commentary  gnon  unaffiliated  right-wing  inequality  quotes  big-peeps  old-anglo  aristos  literature  expansionism  world  genetics  genomics  gene-flow  gavisti  roots  analogy  absolute-relative  studying  sapiens  anthropology  archaeology  truth  primitivism  evolution  study  org:nat  turchin  broad-econ  deep-materialism  social-structure  sociology  cultural-dynamics  variance-components  exploratory  matrix-factorization  things  🌞  structure  scale  dimensionality  degrees-of-freedom  infrastructure  leviathan  polisci  religion  philosophy  government  institutions  money  monetary-fiscal  population  density  urban-rural  values  phalanges  cultu
may 2017 by nhaliday
Identifying the Determinants of Attitudes towards Immigrants: A Structural Cross-Country Analysis
not much of a difference by gender (seems female->more supportive in northern countries, less supportive in southern countries), biggest thing is urbanity+educational attainment

https://archive.is/nUJp2
Vote share for WEuro populist parties higher in regions: rural, ⇈foreigners, ⇈unemployment, AND ⇈education except in cities (surprising)
the urban educated hate the nationalist parties but the nonurban educated are prone to support them. Not all educated are alike

The success of radical right-wing parties in Western European regions – new challenging findings: http://www.tandfonline.com/doi/abs/10.1080/14782804.2016.1198691

Can Television Reduce Xenophobia? The Case of East Germany: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2932276
To examine this question, we exploit the fact that individuals in some areas of East Germany – due to their geographic location – could not receive West German television until 1989. We conjecture that individuals who received West German television were exposed more frequently to foreigners and thus have developed less xenophobia than people who were not exposed to those programs. Our results show that regions that could receive West German television were less likely to vote for right-wing parties during the national elections from 1998 to 2013. Only recently, the same regions were also more likely to vote for left-wing parties. Moreover, while counties that hosted more foreigners in 1989 were also more likely to vote for right-wing parties in most elections, we find counties that recently hosted more foreign visitors showed less xenophobia, which is in line with intergroup contact theory.

With an Open Mind: Openness to Experience Moderates the Effect of Interethnic Encounters on Support for Immigration: https://academic.oup.com/esr/article-abstract/33/5/721/4102111/With-an-Open-Mind-Openness-to-Experience-Moderates

Refugees Unwelcome? Changes in the Public Acceptance of Immigrants and Refugees in Germany in the Course of Europe’s ‘Immigration Crisis’: https://academic.oup.com/esr/article-abstract/doi/10.1093/esr/jcx071/4557564/Refugees-Unwelcome-Changes-in-the-Public
Derived from discussions making up the German immigration discourse during this time, we expect reduced acceptance especially of those immigrants who were explicitly connected to the salient events, like Muslims and the offenders of NYE. Most strikingly, we find that refugees were generally highly accepted and even more so in the second wave, whereas the acceptance of immigrants from Arab or African countries further decreased. Moreover, _female respondents’ initial preference for male immigrants_ disappeared. Contrary to our expectations, we find no changes in the acceptance of Muslims.
pdf  study  economics  sociology  demographics  gender  age-generation  urban  labor  education  poll  values  europe  nordic  germanic  mediterranean  correlation  regression  exploratory  migration  policy  coalitions  large-factor  ethnocentrism  org:ngo  us-them  phalanges  white-paper  matrix-factorization  wonkish  multi  scitariat  twitter  social  commentary  summary  backup  populism  data  intricacy  polisci  geography  econometrics  cliometrics  history  mostly-modern  cold-war  path-dependence  shift  tv  media  propaganda  diversity  natural-experiment  endo-exo  canada  politics  emotion  personality  creative  intervention  gender-diff  longitudinal  flux-stasis  islam  crime  sex  events  impetus  endogenous-exogenous  urban-rural  gnon  hari-seldon  sentiment
may 2017 by nhaliday
Trust, Trolleys and Social Dilemmas: A Replication Study
Overall, the present studies clearly confirmed the main finding of Everett et al., that deontologists are more trusted than consequentialists in social dilemma games. Study 1 replicates Everett et al.’s effect in the context of trust games. Study 2 generalizes the effect to public goods games, thus demonstrating that it is not specific to the type of social dilemma game used in Everett et al. Finally, both studies build on these results by demonstrating that the increased trust in deontologists may sometimes, but not always, be warranted: deontologists displayed increased cooperation rates but only in the public goods game and not in trust games.

The Adaptive Utility of Deontology: Deontological Moral Decision-Making Fosters Perceptions of Trust and Likeability: https://sci-hub.tw/http://link.springer.com/article/10.1007/s40806-016-0080-6
Consistent with previous research, participants liked and trusted targets whose decisions were consistent with deontological motives more than targets whose decisions were more consistent with utilitarian motives; this effect was stronger for perceptions of trust. Additionally, women reported greater dislike for targets whose decisions were consistent with utilitarianism than men. Results suggest that deontological moral reasoning evolved, in part, to facilitate positive relations among conspecifics and aid group living and that women may be particularly sensitive to the implications of the various motives underlying moral decision-making.

Inference of Trustworthiness From Intuitive Moral Judgments: https://sci-hub.tw/10.1037/xge0000165

Exposure to moral relativism compromises moral behavior: https://sci-hub.tw/http://www.sciencedirect.com/science/article/pii/S0022103113001339

Is utilitarian sacrifice becoming more morally permissible?: http://cushmanlab.fas.harvard.edu/docs/Hannikainanetal_2017.pdf

Disgust and Deontology: http://journals.sagepub.com/doi/abs/10.1177/1948550617732609
Trait Sensitivity to Contamination Promotes a Preference for Order, Hierarchy, and Rule-Based Moral Judgment

We suggest that a synthesis of these two literatures points to one specific emotion (disgust) that reliably predicts one specific type of moral judgment (deontological). In all three studies, we found that trait disgust sensitivity predicted more extreme deontological judgment.

The Influence of (Dis)belief in Free Will on Immoral Behavior: https://www.frontiersin.org/articles/10.3389/fpsyg.2017.00020/full

Beyond Sacrificial Harm: A Two-Dimensional Model of Utilitarian Psychology.: http://psycnet.apa.org/record/2017-57422-001
Recent research has relied on trolley-type sacrificial moral dilemmas to study utilitarian versus nonutilitarian modes of moral decision-making. This research has generated important insights into people’s attitudes toward instrumental harm—that is, the sacrifice of an individual to save a greater number. But this approach also has serious limitations. Most notably, it ignores the positive, altruistic core of utilitarianism, which is characterized by impartial concern for the well-being of everyone, whether near or far. Here, we develop, refine, and validate a new scale—the Oxford Utilitarianism Scale—to dissociate individual differences in the ‘negative’ (permissive attitude toward instrumental harm) and ‘positive’ (impartial concern for the greater good) dimensions of utilitarian thinking as manifested in the general population. We show that these are two independent dimensions of proto-utilitarian tendencies in the lay population, each exhibiting a distinct psychological profile. Empathic concern, identification with the whole of humanity, and concern for future generations were positively associated with impartial beneficence but negatively associated with instrumental harm; and although instrumental harm was associated with subclinical psychopathy, impartial beneficence was associated with higher religiosity. Importantly, although these two dimensions were independent in the lay population, they were closely associated in a sample of moral philosophers. Acknowledging this dissociation between the instrumental harm and impartial beneficence components of utilitarian thinking in ordinary people can clarify existing debates about the nature of moral psychology and its relation to moral philosophy as well as generate fruitful avenues for further research. (PsycINFO Database Record (c) 2017 APA, all rights reserved)

A breakthrough in moral psychology: https://nintil.com/2017/12/28/a-breakthrough-in-moral-psychology/

Gender Differences in Responses to Moral Dilemmas: A Process Dissociation Analysis: https://www.ncbi.nlm.nih.gov/pubmed/25840987
The principle of deontology states that the morality of an action depends on its consistency with moral norms; the principle of utilitarianism implies that the morality of an action depends on its consequences. Previous research suggests that deontological judgments are shaped by affective processes, whereas utilitarian judgments are guided by cognitive processes. The current research used process dissociation (PD) to independently assess deontological and utilitarian inclinations in women and men. A meta-analytic re-analysis of 40 studies with 6,100 participants indicated that men showed a stronger preference for utilitarian over deontological judgments than women when the two principles implied conflicting decisions (d = 0.52). PD further revealed that women exhibited stronger deontological inclinations than men (d = 0.57), while men exhibited only slightly stronger utilitarian inclinations than women (d = 0.10). The findings suggest that gender differences in moral dilemma judgments are due to differences in affective responses to harm rather than cognitive evaluations of outcomes.
study  psychology  social-psych  morality  ethics  things  trust  GT-101  coordination  hmm  adversarial  cohesion  replication  cooperate-defect  formal-values  public-goodish  multi  evopsych  gender  gender-diff  philosophy  values  decision-making  absolute-relative  universalism-particularism  intervention  pdf  piracy  deep-materialism  new-religion  stylized-facts  🌞  🎩  honor  trends  phalanges  age-generation  religion  theos  sanctity-degradation  correlation  order-disorder  egalitarianism-hierarchy  volo-avolo  organizing  impro  dimensionality  patho-altruism  altruism  exploratory  matrix-factorization  ratty  unaffiliated  commentary  summary  haidt  scitariat  reason  emotion  randy-ayndy  liner-notes  latent-variables  nature  autism  👽  focus  systematic-ad-hoc  analytical-holistic  expert-experience  economics  markets  civil-liberty  capitalism  personality  psych-architecture  cog-psych  psychometrics  tradition  left-wing  right-wing  ideology  politics  environment  big-peeps  old-anglo  good-evil  ends-means  nietzschean  effe
march 2017 by nhaliday
CS 731 Advanced Artificial Intelligence - Spring 2011
- statistical machine learning
- sparsity in regression
- graphical models
- exponential families
- variational methods
- MCMC
- dimensionality reduction, eg, PCA
- Bayesian nonparametrics
- compressive sensing, matrix completion, and Johnson-Lindenstrauss
course  lecture-notes  yoga  acm  stats  machine-learning  graphical-models  graphs  model-class  bayesian  learning-theory  sparsity  embeddings  markov  monte-carlo  norms  unit  nonparametric  compressed-sensing  matrix-factorization  features
january 2017 by nhaliday
Principal Component Analysis explained visually
The PCA transformation ensures that the horizontal axis PC1 has the most variation, the vertical axis PC2 the second-most, and a third axis PC3 the least.

I think this is equivalent to variational characterization of singular values?
data-science  explanation  visual-understanding  visualization  techtariat  stats  methodology  exploratory  large-factor  better-explained  let-me-see  matrix-factorization
january 2017 by nhaliday

bundles : acmmath

Copy this bookmark:

description:

tags: