**variance_estimation**24

[1506.02326] Estimation of the variance of partial sums of dependent processes

july 2015 by cshalizi

"We study subsampling estimators for the limit variance

σ2=Var(X1)+2∑k=2∞Cov(X1,Xk)

of partial sums of a stationary stochastic process (Xk)k≥1. We establish L2-consistency of a non-overlapping block resampling method. Our results apply to processes that can be represented as functionals of strongly mixing processes. Motivated by recent applications to rank tests, we also study estimators for the series Var(F(X1))+2∑∞k=2Cov(F(X1),F(Xk)), where F is the distribution function of X1. Simulations illustrate the usefulness of the proposed estimators and of a mean squared error optimal rule for the choice of the block length."

to:NB
statistics
time_series
variance_estimation
bootstrap
σ2=Var(X1)+2∑k=2∞Cov(X1,Xk)

of partial sums of a stationary stochastic process (Xk)k≥1. We establish L2-consistency of a non-overlapping block resampling method. Our results apply to processes that can be represented as functionals of strongly mixing processes. Motivated by recent applications to rank tests, we also study estimators for the series Var(F(X1))+2∑∞k=2Cov(F(X1),F(Xk)), where F is the distribution function of X1. Simulations illustrate the usefulness of the proposed estimators and of a mean squared error optimal rule for the choice of the block length."

july 2015 by cshalizi

Digital Locability and Interocular Trauma — Bull Market — Medium

april 2015 by cshalizi

Ah, the crushing intellectual superiority at the top of the financial & corporate heap...

have_read
bad_data_analysis
finance
utter_stupidity
dsquared
dimon.jamie
variance_estimation
april 2015 by cshalizi

Aronow , Green , Lee : Sharp bounds on the variance in randomized experiments

march 2015 by cshalizi

"We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773–785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465–472. Reprint of the original 1923 paper]. A practical implication of our results is that the upper bound estimator facilitates the asymptotically narrowest conservative Wald-type confidence intervals, with applications in randomized controlled and clinical trials."

to:NB
variance_estimation
experimental_design
causal_inference
statistics
march 2015 by cshalizi

[1403.6752] Confidence intervals for high-dimensional inverse covariance estimation

april 2014 by cshalizi

"In this paper, we establish confidence intervals for individual parameters of a sparse concentration matrix in a high-dimensional setting. We follow the idea of the projection approach proposed in \cite{vdgeer13}, which is applied to the graphical Lasso to obtain a de-sparsified estimator. Subsequently, we analyze the asymptotic properties of the novel estimator, establishing asymptotic normality and confidence intervals for the case of sub-Gaussian observations. Performance of the proposed method is illustrated in a simulation study."

in_NB
variance_estimation
high-dimensional_statistics
confidence_sets
van_de_geer.sara
statistics
april 2014 by cshalizi

[1206.3627] Posterior contraction in sparse Bayesian factor models for massive covariance matrices

january 2014 by cshalizi

"Sparse Bayesian factor models are routinely implemented for parsimonious dependence modeling and dimensionality reduction in high-dimensional applications. We provide theoretical understanding of such Bayesian procedures in terms of posterior convergence rates in inferring high-dimensional covariance matrices where the dimension can be larger than the sample size. Under relevant sparsity assumptions on the true covariance matrix, we show that commonly-used point mass mixture priors on the factor loadings lead to consistent estimation in the operator norm even when p≫n. One of our major contributions is to develop a new class of continuous shrinkage priors and provide insights into their concentration around sparse vectors. Using such priors for the factor loadings, we obtain similar rate of convergence as obtained with point mass mixture priors. To obtain the convergence rates, we construct test functions to separate points in the space of high-dimensional covariance matrices using insights from random matrix theory; the tools developed may be of independent interest. We also derive minimax rates and show that the Bayesian posterior rates of convergence coincide with the minimax rates upto a logn‾‾‾‾‾√ term."

in_NB
bayesian_consistency
factor_analysis
variance_estimation
statistics
january 2014 by cshalizi

[1106.2775] Covariance estimation for distributions with $2+varepsilon$ moments

october 2013 by cshalizi

"We study the minimal sample size N=N(n) that suffices to estimate the covariance matrix of an n-dimensional distribution by the sample covariance matrix in the operator norm, with an arbitrary fixed accuracy. We establish the optimal bound N=O(n) for every distribution whose k-dimensional marginals have uniformly bounded $2+\varepsilon$ moments outside the sphere of radius $O(\sqrt{k})$. In the specific case of log-concave distributions, this result provides an alternative approach to the Kannan-Lovasz-Simonovits problem, which was recently solved by Adamczak et al. [J. Amer. Math. Soc. 23 (2010) 535-561]. Moreover, a lower estimate on the covariance matrix holds under a weaker assumption - uniformly bounded $2+\varepsilon$ moments of one-dimensional marginals. Our argument consists of randomizing the spectral sparsifier, a deterministic tool developed recently by Batson, Spielman and Srivastava [SIAM J. Comput. 41 (2012) 1704-1721]. The new randomized method allows one to control the spectral edges of the sample covariance matrix via the Stieltjes transform evaluated at carefully chosen random points."

to:NB
probability
variance_estimation
statistics
october 2013 by cshalizi

[1309.6702] Statistical paleoclimate reconstructions via Markov random fields

september 2013 by cshalizi

"Understanding centennial scale climate variability requires datasets that are accurate, long, continuous, and of broad spatial coverage. Since instrumental measurements are generally only available after 1850, temperature fields must be reconstructed using paleoclimate archives, known as proxies. Various climate field reconstructions (CFR) methods have been proposed to relate past temperature and multiproxy networks, most notably the regularized EM algorithm (RegEM). In this work, we propose a new CFR method, called GraphEM, based on Gaussian Markov random fields (GMRF) embedded within RegEM. GMRFs provide a natural and flexible framework for modeling the inherent spatial heterogeneities of high-dimensional spatial fields, which would in general be more difficult with standard parametric covariance models. At the same time, they provide the parameter reduction necessary for obtaining precise and well-conditioned estimates of the covariance structure of the field, even when the sample size is much smaller than the number of variables (as is typically the case in paleoclimate applications). We demonstrate how the graphical structure of the field can be estimated from the data via l1-penalization methods, and how the GraphEM algorithm can be used to reconstruct past climate variations. The performance of GraphEM is then compared to a popular CFR method (RegEM TTLS) using synthetic data. Our results show that GraphEM can yield significant improvements over existing methods, with gains uniformly over space, and far better risk properties. We proceed to demonstrate that the increase in performance is directly related to recovering the underlying sparsity in the covariance of the spatial field. In particular, we show that spatial points with fewer neighbors in the recovered graph tend to be the ones where there are higher improvements in the reconstructions."

to:NB
climatology
variance_estimation
random_fields
sparsity
statistics
september 2013 by cshalizi

[1309.5109] Network Structure and Biased Variance Estimation in Respondent Driven Sampling

september 2013 by cshalizi

"This paper explores bias in the estimation of sampling variance in Respondent Driven Sampling (RDS). Prior work has demonstrated that RDS has the potential to exhibit high sampling variance in certain networks (Goel and Salganik 2010, Lu et al. 2011). However, because the sampling variance of RDS depends on the network structure of the population, it is unclear to what degree the results from specific networks apply to networks in general. In this paper, we show that RDS sampling variance relies on a critical assumption that the network is First Order Markov (FOM) with respect to the dependent variable of interest. We demonstrate, through intuitive examples, mathematical generalizations, and computational experiments that the sampling variance of RDS will always be underestimated in empirical networks that do not conform to the FOM assumption. Analysis of 215 observed university and school networks from Facebook and Add Health indicates that the FOM assumption is frequently violated in empirical networks (in over 90% of the networks we analyze) and that violating the FOM assumption leads to large biases in RDS estimates of sampling variance. We propose and test two alternative variance estimation strategies that show some promise for reducing biases, but which also illustrate the limits of estimating sampling variance with only partial information on the underlying population network."

to:NB
network_data_analysis
variance_estimation
statistics
mucha.peter_j.
respondent-driven_sampling
september 2013 by cshalizi

Efromovich : Nonparametric regression with the scale depending on auxiliary variable

september 2013 by cshalizi

"The paper is devoted to the problem of estimation of a univariate component in a heteroscedastic nonparametric multiple regression under the mean integrated squared error (MISE) criteria. The aim is to understand how the scale function should be used for estimation of the univariate component. It is known that the scale function does not affect the rate of the MISE convergence, and as a result sharp constants are explored. The paper begins with developing a sharp-minimax theory for a pivotal model Y=f(X)+σ(X,Z)ε, where ε is standard normal and independent of the predictor X and the auxiliary vector-covariate Z. It is shown that if the scale σ(x,z) depends on the auxiliary variable, then a special estimator, which uses the scale (or its estimate), is asymptotically sharp minimax and adaptive to unknown smoothness of f(x). This is an interesting conclusion because if the scale does not depend on the auxiliary covariate Z, then ignoring the heteroscedasticity can yield a sharp minimax estimation. The pivotal model serves as a natural benchmark for a general additive model Y=f(X)+g(Z)+σ(X,Z)ε, where ε may depend on (X,Z) and have only a finite fourth moment. It is shown that for this model a data-driven estimator can perform as well as for the benchmark. Furthermore, the estimator, suggested for continuous responses, can be also used for the case of discrete responses. Bernoulli and Poisson regressions, that are inherently heteroscedastic, are particular considered examples for which sharp minimax lower bounds are obtained as well. A numerical study shows that the asymptotic theory sheds light on small samples."

to:NB
regression
nonparametrics
variance_estimation
efromovich.sam
statistics
september 2013 by cshalizi

Crossett , Lee , Klei , Devlin , Roeder : Refining genetically inferred relationships using treelet covariance smoothing

june 2013 by cshalizi

"Recent technological advances coupled with large sample sets have uncovered many factors underlying the genetic basis of traits and the predisposition to complex disease, but much is left to discover. A common thread to most genetic investigations is familial relationships. Close relatives can be identified from family records, and more distant relatives can be inferred from large panels of genetic markers. Unfortunately these empirical estimates can be noisy, especially regarding distant relatives. We propose a new method for denoising genetically—inferred relationship matrices by exploiting the underlying structure due to hierarchical groupings of correlated individuals. The approach, which we call Treelet Covariance Smoothing, employs a multiscale decomposition of covariance matrices to improve estimates of pairwise relationships. On both simulated and real data, we show that smoothing leads to better estimates of the relatedness amongst distantly related individuals. We illustrate our method with a large genome-wide association study and estimate the “heritability” of body mass index quite accurately. Traditionally heritability, defined as the fraction of the total trait variance attributable to additive genetic effects, is estimated from samples of closely related individuals using random effects models. We show that by using smoothed relationship matrices we can estimate heritability using population-based samples. Finally, while our methods have been developed for refining genetic relationship matrices and improving estimates of heritability, they have much broader potential application in statistics. Most notably, for error-in-variables random effects models and settings that require regularization of matrices with block or hierarchical structure."

to:NB
heard_the_talk
have_read
heritability
smoothing
inference_to_latent_objects
genetics
statistics
variance_estimation
lee.ann_b.
roeder.kathryn
read_the_draft
to:blog
in_the_acknowledgments
june 2013 by cshalizi

Müller , Scealy , Welsh : Model Selection in Linear Mixed Models

may 2013 by cshalizi

"Linear mixed effects models are highly flexible in handling a broad range of data types and are therefore widely used in applications. A key part in the analysis of data is model selection, which often aims to choose a parsimonious model with other desirable properties from a possibly very large set of candidate statistical models. Over the last 5–10 years the literature on model selection in linear mixed models has grown extremely rapidly. The problem is much more complicated than in linear regression because selection on the covariance structure is not straightforward due to computational issues and boundary problems arising from positive semidefinite constraints on covariance matrices. To obtain a better understanding of the available methods, their properties and the relationships between them, we review a large body of literature on linear mixed model selection. We arrange, implement, discuss and compare model selection methods based on four major approaches: information criteria such as AIC or BIC, shrinkage methods based on penalized loss functions such as LASSO, the Fence procedure and Bayesian techniques."

to:NB
variance_estimation
hierarchical_statistical_models
model_selection
regression
linear_regression
may 2013 by cshalizi

[1305.3235] Optimal Estimation and Rank Detection for Sparse Spiked Covariance Matrices

may 2013 by cshalizi

"This paper considers sparse spiked covariance matrix models in the high-dimensional setting and studies the minimax estimation of the covariance matrix and the principal subspace as well as the minimax rank detection. The optimal rate of convergence for estimating the spiked covariance matrix under the spectral norm is established, which requires significantly different techniques from those for estimating other structured covariance matrices such as bandable or sparse covariance matrices. We also establish the minimax rate under the spectral norm for estimating the principal subspace, the primary object of interest in principal component analysis. In addition, the optimal rate for the rank detection boundary is obtained. This result also resolves the gap in a recent paper by Berthet and Rigollet [1] where the special case of rank one is considered."

in_NB
statistics
variance_estimation
factor_analysis
high-dimensional_statistics
estimation
sparsity
to_read
re:g_paper
to_teach:undergrad-ADA
low-rank_approximation
may 2013 by cshalizi

ScienceDirect.com - Journal of Econometrics - Inference with dependent data using cluster covariance estimators

may 2013 by cshalizi

"This paper presents an inference approach for dependent data in time series, spatial, and panel data applications. The method involves constructing t and Wald statistics using a cluster covariance matrix estimator (CCE). We use an approximation that takes the number of clusters/groups as fixed and the number of observations per group to be large. The resulting limiting distributions of the t and Wald statistics are standard t and F distributions where the number of groups plays the role of sample size. Using a small number of groups is analogous to ‘fixed-b’ asymptotics of and (KV) for heteroskedasticity and autocorrelation consistent inference. We provide simulation evidence that demonstrates that the procedure substantially outperforms conventional inference procedures."

to:NB
variance_estimation
time_series
spatial_statistics
spatio-temporal_statistics
econometrics
confidence_sets
network_data_analysis
via:ogburn
may 2013 by cshalizi

Saumard : Optimal model selection in heteroscedastic regression using piecewise polynomial functions

april 2013 by cshalizi

"We consider the estimation of a regression function with random design and heteroscedastic noise in a nonparametric setting. More precisely, we address the problem of characterizing the optimal penalty when the regression function is estimated by using a penalized least-squares model selection method. In this context, we show the existence of a minimal penalty, defined to be the maximum level of penalization under which the model selection procedure totally misbehaves. The optimal penalty is shown to be twice the minimal one and to satisfy a non-asymptotic pathwise oracle inequality with leading constant almost one. Finally, the ideal penalty being unknown in general, we propose a hold-out penalization procedure and show that the latter is asymptotically optimal."

to:NB
variance_estimation
regression
nonparametrics
model_selection
statistics
april 2013 by cshalizi

[1304.4549] Learning Heteroscedastic Models by Convex Programming under Group Sparsity

april 2013 by cshalizi

"Popular sparse estimation methods based on $\ell_1$-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutes a major obstacle in applying these methods in several frameworks---such as time series, random fields, inverse problems---for which the noise is rarely homoscedastic and its level is hard to know in advance. In this paper, we propose a new approach to the joint estimation of the conditional mean and the conditional variance in a high-dimensional (auto-) regression setting. An attractive feature of the proposed estimator is that it is efficiently computable even for very large scale problems by solving a second-order cone program (SOCP). We present theoretical analysis and numerical results assessing the performance of the proposed procedure."

to:NB
lasso
sparsity
variance_estimation
time_series
regression
statistics
april 2013 by cshalizi

Finding the sources of missing heritability in a yeast cross : Nature : Nature Publishing Group

february 2013 by cshalizi

"For many traits, including susceptibility to common diseases in humans, causal loci uncovered by genetic-mapping studies explain only a minority of the heritable contribution to trait variation. Multiple explanations for this ‘missing heritability’ have been proposed1. Here we use a large cross between two yeast strains to accurately estimate different sources of heritable variation for 46 quantitative traits, and to detect underlying loci with high statistical power. We find that the detected loci explain nearly the entire additive contribution to heritable variation for the traits studied. We also show that the contribution to heritability of gene–gene interactions varies among traits, from near zero to approximately 50 per cent. Detected two-locus interactions explain only a minority of this contribution. These results substantially advance our understanding of the missing heritability problem and have important implications for future studies of complex and quantitative traits."

to:NB
genetics
yeast
heritability
variance_estimation
via:arthegall
re:g_paper
february 2013 by cshalizi

[1208.5702] Positive Definite $ell_1$ Penalized Estimation of Large Covariance Matrices

august 2012 by cshalizi

"The thresholding covariance estimator has nice asymptotic properties for estimating sparse large covariance matrices, but it often has negative eigenvalues when used in real data analysis. To simultaneously achieve sparsity and positive definiteness, we develop a positive definite $ell_1$-penalized covariance estimator for estimating sparse large covariance matrices. An efficient alternating direction method is derived to solve the challenging optimization problem and its convergence properties are established. Under weak regularity conditions, non-asymptotic statistical theory is also established for the proposed estimator. The competitive finite-sample performance of our proposal is demonstrated by both simulation and real applications."

to:NB
sparsity
lasso
variance_estimation
statistics
august 2012 by cshalizi

[1207.5910] Groups related to Gaussian graphical models

august 2012 by cshalizi

"Gaussian graphical models have become a well-recognized tool for the analysis of conditional independencies within a set of continuous random variables. We describe the maximal group acting on the space of covariance matrices, which leaves this model invariant for any fixed graph. This group furnishes a representation of Gaussian graphical models as composite transformation families and enables to analyze properties of parameter estimators. We use these results in the robustness analysis to compute upper bounds on finite sample breakdown points of equivariant estimators of the covariance matrix. In addition we provide conditions on the sample size so that an equivariant estimator exists with probability 1."

to:NB
algebra
graphical_models
probability
statistics
variance_estimation
august 2012 by cshalizi

How Close is the Sample Covariance Matrix to the Actual Covariance Matrix?

august 2012 by cshalizi

"Given a probability distribution in ℝ n with general (nonwhite) covariance, a classical estimator of the covariance matrix is the sample covariance matrix obtained from a sample of N independent points. What is the optimal sample size N=N(n) that guarantees estimation with a fixed accuracy in the operator norm? Suppose that the distribution is supported in a centered Euclidean ball of radius O(n) . We conjecture that the optimal sample size is N=O(n) for all distributions with finite fourth moment, and we prove this up to an iterated logarithmic factor. This problem is motivated by the optimal theorem of Rudelson (J. Funct. Anal. 164:60–72, 1999), which states that N=O(nlog n) for distributions with finite second moment, and a recent result of Adamczak et al. (J. Am. Math. Soc. 234:535–561, 2010), which guarantees that N=O(n) for subexponential distributions."

to:NB
variance_estimation
estimation
statistics
probability
august 2012 by cshalizi

[1206.6382] High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains

july 2012 by cshalizi

"In this paper, we present a novel framework incorporating a combination of sparse models in different domains. We posit the observed data as generated from a linear combination of a sparse Gaussian Markov model (with a sparse precision matrix) and a sparse Gaussian independence model (with a sparse covariance matrix). We provide efficient methods for decomposition of the data into two domains, viz Markov and independence domains. We characterize a set of sufficient conditions for identifiability and model consistency. Our decomposition method is based on a simple modification of the popular $ell_1$-penalized maximum-likelihood estimator ($ell_1$-MLE). We establish that our estimator is consistent in both the domains, i.e., it successfully recovers the supports of both Markov and independence models, when the number of samples $n$ scales as $n = Omega(d^2 log p)$, where $p$ is the number of variables and $d$ is the maximum node degree in the Markov model. Our conditions for recovery are comparable to those of $ell_1$-MLE for consistent estimation of a sparse Markov model, and thus, we guarantee successful high-dimensional estimation of a richer class of models under comparable conditions. Our experiments validate these results and also demonstrate that our models have better inference accuracy under simple algorithms such as loopy belief propagation."

to:NB
variance_estimation
graphical_models
sparsity
statistics
july 2012 by cshalizi

**related tags**

Copy this bookmark: