nhaliday + libraries   133

Use and Interpretation of LD Score Regression
LD Score regression distinguishes confounding from polygenicity in genome-wide association studies: https://sci-hub.bz/10.1038/ng.3211
- Po-Ru Loh, Nick Patterson, et al.


Both polygenicity (i.e. many small genetic effects) and confounding biases, such as cryptic relatedness and population stratification, can yield inflated distributions of test statistics in genome-wide association studies (GWAS). However, current methods cannot distinguish between inflation from bias and true signal from polygenicity. We have developed an approach that quantifies the contributions of each by examining the relationship between test statistics and linkage disequilibrium (LD). We term this approach LD Score regression. LD Score regression provides an upper bound on the contribution of confounding bias to the observed inflation in test statistics and can be used to estimate a more powerful correction factor than genomic control. We find strong evidence that polygenicity accounts for the majority of test statistic inflation in many GWAS of large sample size.

Supplementary Note: https://images.nature.com/original/nature-assets/ng/journal/v47/n3/extref/ng.3211-S1.pdf

An atlas of genetic correlations across human diseases
and traits: https://sci-hub.bz/10.1038/ng.3406


Supplementary Note: https://images.nature.com/original/nature-assets/ng/journal/v47/n11/extref/ng.3406-S1.pdf

ldsc is a command line tool for estimating heritability and genetic correlation from GWAS summary statistics. ldsc also computes LD Scores.
nibble  pdf  slides  talks  bio  biodet  genetics  genomics  GWAS  genetic-correlation  correlation  methodology  bioinformatics  concept  levers  🌞  tutorial  explanation  pop-structure  gene-drift  ideas  multi  study  org:nat  article  repo  software  tools  libraries  stats  hypothesis-testing  biases  confounding  gotchas  QTL  simulation  survey  preprint  population-genetics 
november 2017 by nhaliday
Ancient Admixture in Human History
- Patterson, Reich et al., 2012
Population mixture is an important process in biology. We present a suite of methods for learning about population mixtures, implemented in a software package called ADMIXTOOLS, that support formal tests for whether mixture occurred and make it possible to infer proportions and dates of mixture. We also describe the development of a new single nucleotide polymorphism (SNP) array consisting of 629,433 sites with clearly documented ascertainment that was specifically designed for population genetic analyses and that we genotyped in 934 individuals from 53 diverse populations. To illustrate the methods, we give a number of examples that provide new insights about the history of human admixture. The most striking finding is a clear signal of admixture into northern Europe, with one ancestral population related to present-day Basques and Sardinians and the other related to present-day populations of northeast Asia and the Americas. This likely reflects a history of admixture between Neolithic migrants and the indigenous Mesolithic population of Europe, consistent with recent analyses of ancient bones from Sweden and the sequencing of the genome of the Tyrolean “Iceman.”
nibble  pdf  study  article  methodology  bio  sapiens  genetics  genomics  population-genetics  migration  gene-flow  software  trees  concept  history  antiquity  europe  roots  gavisti  🌞  bioinformatics  metrics  hypothesis-testing  levers  ideas  libraries  tools  pop-structure 
november 2017 by nhaliday
What is the best way to parse command-line arguments with Python? - Quora
- Anders Kaseorg

Use the standard optparse library.

It’s important to uphold your users’ expectation that your utility will parse arguments in the same way as every other UNIX utility. If you roll your own parsing code, you’ll almost certainly break that expectation in obvious or subtle ways.

Although the documentation claims that optparse has been deprecated in favor of argparse, which supports more features like optional option arguments and configurable prefix characters, I can’t recommend argparse until it’s been fixed to parse required option arguments in the standard UNIX way. Currently, argparse uses an unexpected heuristic which may lead to subtle bugs in other scripts that call your program.

consider also click (which uses the optparse behavior)
q-n-a  qra  oly  best-practices  programming  terminal  unix  python  libraries  protocol  gotchas  howto  pls  yak-shaving  integration-extension 
august 2017 by nhaliday
Broadcasting — NumPy v1.13 Manual
When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing dimensions, and works its way forward. Two dimensions are compatible when

they are equal, or
one of them is 1
If these conditions are not met, a ValueError: frames are not aligned exception is thrown, indicating that the arrays have incompatible shapes. The size of the resulting array is the maximum size along each dimension of the input arrays.

Arrays do not need to have the same number of dimensions. For example, if you have a 256x256x3 array of RGB values, and you want to scale each color in the image by a different value, you can multiply the image by a one-dimensional array with 3 values.
python  libraries  programming  howto  protocol  numerics  pls  linear-algebra 
august 2017 by nhaliday
Welcome to wbdata’s documentation! — wbdata 0.2.7 documentation
Wbdata is a simple python interface to find and request information from the World Bank’s various databases, either as a dictionary containing full metadata or as a pandas DataFrame. Currently, wbdata wraps most of the World Bank API, and also adds some convenience functions for searching and retrieving information.
data  metrics  econ-metrics  objektbuch  libraries  python  documentation  yak-shaving  api  hmm  world  developing-world  sleuthin  maps  programming  reference 
february 2017 by nhaliday
Decison Tree for Optimization Software
including convex programming

Mosek makes out pretty good but not pareto-optimal
benchmarks  optimization  software  libraries  comparison  data  performance  faq  frameworks  curvature  convexity-curvature 
november 2016 by nhaliday
Why xkcd-style graphs are important - Chris Stucchio
A: lowering expectations
and apparently matplotlib has this visualization built-in
rhetoric  dataviz  libraries  python  howto  techtariat 
november 2016 by nhaliday
I don't understand Python's Asyncio | Armin Ronacher's Thoughts and Writings
Man that thing is complex and it keeps getting more complex. I do not have the mental capacity to casually work with asyncio. It requires constantly updating the knowledge with all language changes and it has tremendously complicated the language. It's impressive that an ecosystem is evolving around it but I can't help but get the impression that it will take quite a few more years for it to become a particularly enjoyable and stable development experience.

What landed in 3.5 (the actual new coroutine objects) is great. In particular with the changes that will come up there is a sensible base that I wish would have been in earlier versions. The entire mess with overloading generators to be coroutines was a mistake in my mind. With regards to what's in asyncio I'm not sure of anything. It's an incredibly complex thing and super messy internally. It's hard to comprehend how it works in all details. When you can pass a generator, when it has to be a real coroutine, what futures are, what tasks are, how the loop works and that did not even come to the actual IO part.

The worst part is that asyncio is not even particularly fast. David Beazley's live demo hacked up asyncio replacement is twice as fast as it. There is an enormous amount of complexity that's hard to understand and reason about and then it fails on it's main promise. I'm not sure what to think about it but I know at least that I don't understand asyncio enough to feel confident about giving people advice about how to structure code for it.
python  libraries  review  concurrency  programming  pls  rant  🖥  techtariat  intricacy 
october 2016 by nhaliday
Spaceship Generator | Hacker News
some interesting discussion of the value of procedural generation in the comments
commentary  hn  graphics  games  programming  libraries  repo  oss  project  SIGGRAPH 
june 2016 by nhaliday
« earlier      
per page:    204080120160

bundles : meta

related tags

accuracy  acm  acmtariat  advice  aggregator  ai  algorithms  analysis  announcement  antiquity  api  app  applications  article  auto-learning  bangbang  bayesian  behavioral-gen  benchmarks  best-practices  better-explained  biases  bio  biodet  bioinformatics  biotech  blog  books  bots  brands  build-packaging  business  c(pp)  career  cheatsheet  checking  checklists  classification  cloud  cog-psych  commentary  community  comparison  compilers  computational-geometry  computer-vision  concept  concurrency  confounding  convexity-curvature  cool  coordination  correlation  critique  crypto  curvature  d3  data  data-science  database  dataset  dataviz  dbs  debate  deep-learning  design  desktop  developing-world  devtools  dirty-hands  discussion  distributed  distribution  documentation  duplication  econ-metrics  education  embedded  embeddings  empirical  engineering  ensembles  europe  exocortex  experiment  expert  expert-experience  explanation  exploratory  facebook  faq  features  ffi  finance  formal-methods  frameworks  free  frontend  functional  games  gavisti  gender  gene-drift  gene-flow  genetic-correlation  genetics  genomics  github  golang  google  gotchas  graphical-models  graphics  graphs  guide  GWAS  hardware  hashing  haskell  heuristic  history  hmm  hn  homo-hetero  howto  huge-data-the-biggest  human-ml  hypothesis-testing  ide  ideas  idk  infographic  init  integration-extension  intersection  intersection-connectedness  interview-prep  intricacy  javascript  jobs  julia  jvm  knowledge  language  latent-variables  latex  let-me-see  levers  libraries  linear-algebra  linear-models  liner-notes  links  linux  lisp  list  long-term  longitudinal  machine-learning  maps  markov  matching  math  matrix-factorization  measurement  mechanics  memory-management  meta:prediction  metabuch  metaprogramming  methodology  metrics  migration  mihai  mit  mobile  model-class  monte-carlo  multi  networking  nibble  nlp  numerics  objektbuch  ocaml-sml  oly  optimization  org:bleg  org:edu  org:junk  org:mat  org:nat  organization  os  oss  osx  papers  parametric  paste  pdf  pennsylvania  performance  philosophy  physics  pinboard  planning  plots  pls  plt  pop-structure  population-genetics  positivity  ppl  pragmatic  prediction  preprint  programming  progression  project  protocol  psychology  psychometrics  python  q-n-a  qra  QTL  quora  r-lang  ranking  rant  recommendations  recruiting  reference  reflection  repo  research  review  rhetoric  roadmap  roots  ruby  rust  s:*  saas  sampling  sapiens  scala  scaling-tech  scholar-pack  search  security  sequential  SIGGRAPH  signum  similarity  simulation  sleuthin  slides  software  stackex  stanford  startups  stats  stream  strings  study  studying  success  summary  survey  sv  systems  tactics  talks  tcs  teaching  tech  techtariat  terminal  thesis  threat-modeling  time-series  todo  tools  top-n  tracker  transitions  trees  tutorial  ui  unit  unix  ux  values  virginia-DC  virtualization  visual-understanding  visualization  volo-avolo  web  wiki  workflow  working-stiff  world  worrydream  yak-shaving  yc  🌞  🖥 

Copy this bookmark: