iidness   23

Section 10 Chi-squared goodness-of-fit test.
- pf that chi-squared statistic for Pearson's test (multinomial goodness-of-fit) actually has chi-squared distribution asymptotically
- the gotcha: terms Z_j in sum aren't independent
- solution:
- compute the covariance matrix of the terms to be E[Z_iZ_j] = -sqrt(p_ip_j)
- note that an equivalent way of sampling the Z_j is to take a random standard Gaussian and project onto the plane orthogonal to (sqrt(p_1), sqrt(p_2), ..., sqrt(p_r))
- that is equivalent to just sampling a Gaussian w/ 1 less dimension (hence df=r-1)
pdf  nibble  lecture-notes  mit  stats  hypothesis-testing  acm  probability  methodology  proofs  iidness  distribution  limits  identity  direction  lifts-projections 
october 2017 by nhaliday
Variance of product of multiple random variables - Cross Validated
prod_i (var[X_i] + (E[X_i])^2) - prod_i (E[X_i])^2

two variable case: var[X] var[Y] + var[X] (E[Y])^2 + (E[X])^2 var[Y]
nibble  q-n-a  overflow  stats  probability  math  identity  moments  arrows  multiplicative  iidness  dependence-independence 
october 2017 by nhaliday
Information Processing: The joy of Turkheimer
In the talk Turkheimer gives the following definition of social science, which emphasizes why it is hard:

Social science is the attempt to explain the causes of complex human behavior when:
- There are a large number of potential causes.
- The potential causes are non-independent.
- Randomized experimentation is not possible.
hsu  scitariat  genetics  genomics  causation  hypothesis-testing  social-science  nonlinearity  iidness  correlation  links  slides  presentation  audio  things  lens  metabuch  thinking  GxE  commentary 
february 2017 by nhaliday
Mixing (mathematics) - Wikipedia
One way to describe this is that strong mixing implies that for any two possible states of the system (realizations of the random variable), when given a sufficient amount of time between the two states, the occurrence of the states is independent.

Mixing coefficient is
α(n) = sup{|P(A∪B) - P(A)P(B)| : A in σ(X_0, ..., X_{t-1}), B in σ(X_{t+n}, ...), t >= 0}
for σ(...) the sigma algebra generated by those r.v.s.

So it's a notion of total variational distance between the true distribution and the product distribution.
concept  math  acm  physics  probability  stochastic-processes  definition  mixing  iidness  wiki  reference  nibble  limits  ergodic  math.DS  measure  dependence-independence 
february 2017 by nhaliday
bounds - What is the variance of the maximum of a sample? - Cross Validated
- sum of variances is always a bound
- can't do better even for iid Bernoulli
- looks like nice argument from well-known probabilist (using E[(X-Y)^2] = 2Var X), but not clear to me how he gets to sum_i instead of sum_{i,j} in the union bound?
edit: argument is that, for j = argmax_k Y_k, we have r < X_i - Y_j <= X_i - Y_i for all i, including i = argmax_k X_k
- different proof here (later pages): http://www.ism.ac.jp/editsec/aism/pdf/047_1_0185.pdf
Var(X_n:n) <= sum Var(X_k:n) + 2 sum_{i < j} Cov(X_i:n, X_j:n) = Var(sum X_k:n) = Var(sum X_k) = nσ^2
why are the covariances nonnegative? (are they?). intuitively seems true.
- for that, see https://pinboard.in/u:nhaliday/b:ed4466204bb1
- note that this proof shows more generally that sum Var(X_k:n) <= sum Var(X_k)
- apparently that holds for dependent X_k too? http://mathoverflow.net/a/96943/20644
q-n-a  overflow  stats  acm  distribution  tails  bias-variance  moments  estimate  magnitude  probability  iidness  tidbits  concentration-of-measure  multi  orders  levers  extrema  nibble  bonferroni  coarse-fine  expert  symmetry  s:*  expert-experience  proofs 
february 2017 by nhaliday
Bounds on the Expectation of the Maximum of Samples from a Gaussian
σ/sqrt(pi log 2) sqrt(log n) <= E[Y] <= σ sqrt(2) sqrt(log n)

upper bound pf: Jensen's inequality+mgf+union bound+choose optimal t (Chernoff bound basically)
lower bound pf: more ad-hoc (and difficult)
pdf  tidbits  math  probability  concentration-of-measure  estimate  acm  tails  distribution  calculation  iidness  orders  magnitude  extrema  tightness  outliers  expectancy  proofs 
october 2016 by nhaliday

related tags

2014  acm  acmtariat  adversarial  advice  algebra  analysis  aphorism  applicability-prereqs  arrows  article  audio  average-case  bandits  bare-hands  bayesian  behavioral-gen  benchmarks  best-practices  bias-variance  big-picture  biodet  bonferroni  boolean-analysis  calculation  causation  checklists  clarity  clever-rats  coarse-fine  commentary  commutativity  concentration-of-measure  concept  convergence  correlation  counterexample  critique  data-science  data  debate  definition  dependence-independence  direction  discussion  distribution  distributional  engineering  enhancement  entropy-like  ergodic  essay  estimate  evan-miller  expectancy  expert-experience  expert  explanation  exposition  extrema  faq  fourier  frequentist  genetics  genomics  gowers  gwern  gxe  hi-order-bits  hmm  homo-hetero  hsu  hypothesis-testing  ideas  identity  invariance  lecture-notes  left-wing  lens  lesswrong  levers  lifts-projections  limits  linear-models  linearity  links  magnitude  math.co  math.ds  math.gr  math  mathtariat  measure  meta:prediction  metabuch  methodology  mit  mixing  moments  monte-carlo  multi  multiplicative  nibble  nonlinearity  online-learning  orders  org:bleg  outliers  overflow  pareto  pdf  philosophy  physics  positivity  power-law  pragmatic  presentation  probability  proofs  pseudorandomness  q-n-a  random-matrices  random  ratty  reading  reference  regression  regularizer  rhetoric  rigidity  risk  s:**  s:*  scale  scaling-up  scitariat  selection  shalizi  signum  slides  social-science  spock  stats  stochastic-processes  street-fighting  subjective-objective  symmetry  synthesis  tails  tcs  tcstariat  tech  techtariat  things  thinking  tidbits  tightness  top-n  unaffiliated  unit  wiki  yoga  🤖 

Copy this bookmark: