variance_estimation   24

[1506.02326] Estimation of the variance of partial sums of dependent processes
"We study subsampling estimators for the limit variance
of partial sums of a stationary stochastic process (Xk)k≥1. We establish L2-consistency of a non-overlapping block resampling method. Our results apply to processes that can be represented as functionals of strongly mixing processes. Motivated by recent applications to rank tests, we also study estimators for the series Var(F(X1))+2∑∞k=2Cov(F(X1),F(Xk)), where F is the distribution function of X1. Simulations illustrate the usefulness of the proposed estimators and of a mean squared error optimal rule for the choice of the block length."
to:NB  statistics  time_series  variance_estimation  bootstrap 
july 2015 by cshalizi
Aronow , Green , Lee : Sharp bounds on the variance in randomized experiments
"We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773–785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465–472. Reprint of the original 1923 paper]. A practical implication of our results is that the upper bound estimator facilitates the asymptotically narrowest conservative Wald-type confidence intervals, with applications in randomized controlled and clinical trials."
to:NB  variance_estimation  experimental_design  causal_inference  statistics 
march 2015 by cshalizi
[1403.6752] Confidence intervals for high-dimensional inverse covariance estimation
"In this paper, we establish confidence intervals for individual parameters of a sparse concentration matrix in a high-dimensional setting. We follow the idea of the projection approach proposed in \cite{vdgeer13}, which is applied to the graphical Lasso to obtain a de-sparsified estimator. Subsequently, we analyze the asymptotic properties of the novel estimator, establishing asymptotic normality and confidence intervals for the case of sub-Gaussian observations. Performance of the proposed method is illustrated in a simulation study."
in_NB  variance_estimation  high-dimensional_statistics  confidence_sets  van_de_geer.sara  statistics 
april 2014 by cshalizi
[1206.3627] Posterior contraction in sparse Bayesian factor models for massive covariance matrices
"Sparse Bayesian factor models are routinely implemented for parsimonious dependence modeling and dimensionality reduction in high-dimensional applications. We provide theoretical understanding of such Bayesian procedures in terms of posterior convergence rates in inferring high-dimensional covariance matrices where the dimension can be larger than the sample size. Under relevant sparsity assumptions on the true covariance matrix, we show that commonly-used point mass mixture priors on the factor loadings lead to consistent estimation in the operator norm even when p≫n. One of our major contributions is to develop a new class of continuous shrinkage priors and provide insights into their concentration around sparse vectors. Using such priors for the factor loadings, we obtain similar rate of convergence as obtained with point mass mixture priors. To obtain the convergence rates, we construct test functions to separate points in the space of high-dimensional covariance matrices using insights from random matrix theory; the tools developed may be of independent interest. We also derive minimax rates and show that the Bayesian posterior rates of convergence coincide with the minimax rates upto a logn‾‾‾‾‾√ term."
in_NB  bayesian_consistency  factor_analysis  variance_estimation  statistics 
january 2014 by cshalizi
[1106.2775] Covariance estimation for distributions with $2+varepsilon$ moments
"We study the minimal sample size N=N(n) that suffices to estimate the covariance matrix of an n-dimensional distribution by the sample covariance matrix in the operator norm, with an arbitrary fixed accuracy. We establish the optimal bound N=O(n) for every distribution whose k-dimensional marginals have uniformly bounded $2+\varepsilon$ moments outside the sphere of radius $O(\sqrt{k})$. In the specific case of log-concave distributions, this result provides an alternative approach to the Kannan-Lovasz-Simonovits problem, which was recently solved by Adamczak et al. [J. Amer. Math. Soc. 23 (2010) 535-561]. Moreover, a lower estimate on the covariance matrix holds under a weaker assumption - uniformly bounded $2+\varepsilon$ moments of one-dimensional marginals. Our argument consists of randomizing the spectral sparsifier, a deterministic tool developed recently by Batson, Spielman and Srivastava [SIAM J. Comput. 41 (2012) 1704-1721]. The new randomized method allows one to control the spectral edges of the sample covariance matrix via the Stieltjes transform evaluated at carefully chosen random points."
to:NB  probability  variance_estimation  statistics 
october 2013 by cshalizi
[1309.6702] Statistical paleoclimate reconstructions via Markov random fields
"Understanding centennial scale climate variability requires datasets that are accurate, long, continuous, and of broad spatial coverage. Since instrumental measurements are generally only available after 1850, temperature fields must be reconstructed using paleoclimate archives, known as proxies. Various climate field reconstructions (CFR) methods have been proposed to relate past temperature and multiproxy networks, most notably the regularized EM algorithm (RegEM). In this work, we propose a new CFR method, called GraphEM, based on Gaussian Markov random fields (GMRF) embedded within RegEM. GMRFs provide a natural and flexible framework for modeling the inherent spatial heterogeneities of high-dimensional spatial fields, which would in general be more difficult with standard parametric covariance models. At the same time, they provide the parameter reduction necessary for obtaining precise and well-conditioned estimates of the covariance structure of the field, even when the sample size is much smaller than the number of variables (as is typically the case in paleoclimate applications). We demonstrate how the graphical structure of the field can be estimated from the data via l1-penalization methods, and how the GraphEM algorithm can be used to reconstruct past climate variations. The performance of GraphEM is then compared to a popular CFR method (RegEM TTLS) using synthetic data. Our results show that GraphEM can yield significant improvements over existing methods, with gains uniformly over space, and far better risk properties. We proceed to demonstrate that the increase in performance is directly related to recovering the underlying sparsity in the covariance of the spatial field. In particular, we show that spatial points with fewer neighbors in the recovered graph tend to be the ones where there are higher improvements in the reconstructions."
to:NB  climatology  variance_estimation  random_fields  sparsity  statistics 
september 2013 by cshalizi
[1309.5109] Network Structure and Biased Variance Estimation in Respondent Driven Sampling
"This paper explores bias in the estimation of sampling variance in Respondent Driven Sampling (RDS). Prior work has demonstrated that RDS has the potential to exhibit high sampling variance in certain networks (Goel and Salganik 2010, Lu et al. 2011). However, because the sampling variance of RDS depends on the network structure of the population, it is unclear to what degree the results from specific networks apply to networks in general. In this paper, we show that RDS sampling variance relies on a critical assumption that the network is First Order Markov (FOM) with respect to the dependent variable of interest. We demonstrate, through intuitive examples, mathematical generalizations, and computational experiments that the sampling variance of RDS will always be underestimated in empirical networks that do not conform to the FOM assumption. Analysis of 215 observed university and school networks from Facebook and Add Health indicates that the FOM assumption is frequently violated in empirical networks (in over 90% of the networks we analyze) and that violating the FOM assumption leads to large biases in RDS estimates of sampling variance. We propose and test two alternative variance estimation strategies that show some promise for reducing biases, but which also illustrate the limits of estimating sampling variance with only partial information on the underlying population network."
to:NB  network_data_analysis  variance_estimation  statistics  mucha.peter_j.  respondent-driven_sampling 
september 2013 by cshalizi
Efromovich : Nonparametric regression with the scale depending on auxiliary variable
"The paper is devoted to the problem of estimation of a univariate component in a heteroscedastic nonparametric multiple regression under the mean integrated squared error (MISE) criteria. The aim is to understand how the scale function should be used for estimation of the univariate component. It is known that the scale function does not affect the rate of the MISE convergence, and as a result sharp constants are explored. The paper begins with developing a sharp-minimax theory for a pivotal model Y=f(X)+σ(X,Z)ε, where ε is standard normal and independent of the predictor X and the auxiliary vector-covariate Z. It is shown that if the scale σ(x,z) depends on the auxiliary variable, then a special estimator, which uses the scale (or its estimate), is asymptotically sharp minimax and adaptive to unknown smoothness of f(x). This is an interesting conclusion because if the scale does not depend on the auxiliary covariate Z, then ignoring the heteroscedasticity can yield a sharp minimax estimation. The pivotal model serves as a natural benchmark for a general additive model Y=f(X)+g(Z)+σ(X,Z)ε, where ε may depend on (X,Z) and have only a finite fourth moment. It is shown that for this model a data-driven estimator can perform as well as for the benchmark. Furthermore, the estimator, suggested for continuous responses, can be also used for the case of discrete responses. Bernoulli and Poisson regressions, that are inherently heteroscedastic, are particular considered examples for which sharp minimax lower bounds are obtained as well. A numerical study shows that the asymptotic theory sheds light on small samples."
to:NB  regression  nonparametrics  variance_estimation  efromovich.sam  statistics 
september 2013 by cshalizi
Crossett , Lee , Klei , Devlin , Roeder : Refining genetically inferred relationships using treelet covariance smoothing
"Recent technological advances coupled with large sample sets have uncovered many factors underlying the genetic basis of traits and the predisposition to complex disease, but much is left to discover. A common thread to most genetic investigations is familial relationships. Close relatives can be identified from family records, and more distant relatives can be inferred from large panels of genetic markers. Unfortunately these empirical estimates can be noisy, especially regarding distant relatives. We propose a new method for denoising genetically—inferred relationship matrices by exploiting the underlying structure due to hierarchical groupings of correlated individuals. The approach, which we call Treelet Covariance Smoothing, employs a multiscale decomposition of covariance matrices to improve estimates of pairwise relationships. On both simulated and real data, we show that smoothing leads to better estimates of the relatedness amongst distantly related individuals. We illustrate our method with a large genome-wide association study and estimate the “heritability” of body mass index quite accurately. Traditionally heritability, defined as the fraction of the total trait variance attributable to additive genetic effects, is estimated from samples of closely related individuals using random effects models. We show that by using smoothed relationship matrices we can estimate heritability using population-based samples. Finally, while our methods have been developed for refining genetic relationship matrices and improving estimates of heritability, they have much broader potential application in statistics. Most notably, for error-in-variables random effects models and settings that require regularization of matrices with block or hierarchical structure."
to:NB  heard_the_talk  have_read  heritability  smoothing  inference_to_latent_objects  genetics  statistics  variance_estimation  lee.ann_b.  roeder.kathryn  read_the_draft  to:blog  in_the_acknowledgments 
june 2013 by cshalizi
Müller , Scealy , Welsh : Model Selection in Linear Mixed Models
"Linear mixed effects models are highly flexible in handling a broad range of data types and are therefore widely used in applications. A key part in the analysis of data is model selection, which often aims to choose a parsimonious model with other desirable properties from a possibly very large set of candidate statistical models. Over the last 5–10 years the literature on model selection in linear mixed models has grown extremely rapidly. The problem is much more complicated than in linear regression because selection on the covariance structure is not straightforward due to computational issues and boundary problems arising from positive semidefinite constraints on covariance matrices. To obtain a better understanding of the available methods, their properties and the relationships between them, we review a large body of literature on linear mixed model selection. We arrange, implement, discuss and compare model selection methods based on four major approaches: information criteria such as AIC or BIC, shrinkage methods based on penalized loss functions such as LASSO, the Fence procedure and Bayesian techniques."
to:NB  variance_estimation  hierarchical_statistical_models  model_selection  regression  linear_regression 
may 2013 by cshalizi
[1305.3235] Optimal Estimation and Rank Detection for Sparse Spiked Covariance Matrices
"This paper considers sparse spiked covariance matrix models in the high-dimensional setting and studies the minimax estimation of the covariance matrix and the principal subspace as well as the minimax rank detection. The optimal rate of convergence for estimating the spiked covariance matrix under the spectral norm is established, which requires significantly different techniques from those for estimating other structured covariance matrices such as bandable or sparse covariance matrices. We also establish the minimax rate under the spectral norm for estimating the principal subspace, the primary object of interest in principal component analysis. In addition, the optimal rate for the rank detection boundary is obtained. This result also resolves the gap in a recent paper by Berthet and Rigollet [1] where the special case of rank one is considered."
in_NB  statistics  variance_estimation  factor_analysis  high-dimensional_statistics  estimation  sparsity  to_read  re:g_paper  to_teach:undergrad-ADA  low-rank_approximation 
may 2013 by cshalizi - Journal of Econometrics - Inference with dependent data using cluster covariance estimators
"This paper presents an inference approach for dependent data in time series, spatial, and panel data applications. The method involves constructing t and Wald statistics using a cluster covariance matrix estimator (CCE). We use an approximation that takes the number of clusters/groups as fixed and the number of observations per group to be large. The resulting limiting distributions of the t and Wald statistics are standard t and F distributions where the number of groups plays the role of sample size. Using a small number of groups is analogous to ‘fixed-b’ asymptotics of  and  (KV) for heteroskedasticity and autocorrelation consistent inference. We provide simulation evidence that demonstrates that the procedure substantially outperforms conventional inference procedures."
to:NB  variance_estimation  time_series  spatial_statistics  spatio-temporal_statistics  econometrics  confidence_sets  network_data_analysis  via:ogburn 
may 2013 by cshalizi
Saumard : Optimal model selection in heteroscedastic regression using piecewise polynomial functions
"We consider the estimation of a regression function with random design and heteroscedastic noise in a nonparametric setting. More precisely, we address the problem of characterizing the optimal penalty when the regression function is estimated by using a penalized least-squares model selection method. In this context, we show the existence of a minimal penalty, defined to be the maximum level of penalization under which the model selection procedure totally misbehaves. The optimal penalty is shown to be twice the minimal one and to satisfy a non-asymptotic pathwise oracle inequality with leading constant almost one. Finally, the ideal penalty being unknown in general, we propose a hold-out penalization procedure and show that the latter is asymptotically optimal."
to:NB  variance_estimation  regression  nonparametrics  model_selection  statistics 
april 2013 by cshalizi
[1304.4549] Learning Heteroscedastic Models by Convex Programming under Group Sparsity
"Popular sparse estimation methods based on $\ell_1$-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutes a major obstacle in applying these methods in several frameworks---such as time series, random fields, inverse problems---for which the noise is rarely homoscedastic and its level is hard to know in advance. In this paper, we propose a new approach to the joint estimation of the conditional mean and the conditional variance in a high-dimensional (auto-) regression setting. An attractive feature of the proposed estimator is that it is efficiently computable even for very large scale problems by solving a second-order cone program (SOCP). We present theoretical analysis and numerical results assessing the performance of the proposed procedure."
to:NB  lasso  sparsity  variance_estimation  time_series  regression  statistics 
april 2013 by cshalizi
Finding the sources of missing heritability in a yeast cross : Nature : Nature Publishing Group
"For many traits, including susceptibility to common diseases in humans, causal loci uncovered by genetic-mapping studies explain only a minority of the heritable contribution to trait variation. Multiple explanations for this ‘missing heritability’ have been proposed1. Here we use a large cross between two yeast strains to accurately estimate different sources of heritable variation for 46 quantitative traits, and to detect underlying loci with high statistical power. We find that the detected loci explain nearly the entire additive contribution to heritable variation for the traits studied. We also show that the contribution to heritability of gene–gene interactions varies among traits, from near zero to approximately 50 per cent. Detected two-locus interactions explain only a minority of this contribution. These results substantially advance our understanding of the missing heritability problem and have important implications for future studies of complex and quantitative traits."
to:NB  genetics  yeast  heritability  variance_estimation  via:arthegall  re:g_paper 
february 2013 by cshalizi
[1208.5702] Positive Definite $ell_1$ Penalized Estimation of Large Covariance Matrices
"The thresholding covariance estimator has nice asymptotic properties for estimating sparse large covariance matrices, but it often has negative eigenvalues when used in real data analysis. To simultaneously achieve sparsity and positive definiteness, we develop a positive definite $ell_1$-penalized covariance estimator for estimating sparse large covariance matrices. An efficient alternating direction method is derived to solve the challenging optimization problem and its convergence properties are established. Under weak regularity conditions, non-asymptotic statistical theory is also established for the proposed estimator. The competitive finite-sample performance of our proposal is demonstrated by both simulation and real applications."
to:NB  sparsity  lasso  variance_estimation  statistics 
august 2012 by cshalizi
[1207.5910] Groups related to Gaussian graphical models
"Gaussian graphical models have become a well-recognized tool for the analysis of conditional independencies within a set of continuous random variables. We describe the maximal group acting on the space of covariance matrices, which leaves this model invariant for any fixed graph. This group furnishes a representation of Gaussian graphical models as composite transformation families and enables to analyze properties of parameter estimators. We use these results in the robustness analysis to compute upper bounds on finite sample breakdown points of equivariant estimators of the covariance matrix. In addition we provide conditions on the sample size so that an equivariant estimator exists with probability 1."
to:NB  algebra  graphical_models  probability  statistics  variance_estimation 
august 2012 by cshalizi
How Close is the Sample Covariance Matrix to the Actual Covariance Matrix?
"Given a probability distribution in ℝ n with general (nonwhite) covariance, a classical estimator of the covariance matrix is the sample covariance matrix obtained from a sample of N independent points. What is the optimal sample size N=N(n) that guarantees estimation with a fixed accuracy in the operator norm? Suppose that the distribution is supported in a centered Euclidean ball of radius O(n) . We conjecture that the optimal sample size is N=O(n) for all distributions with finite fourth moment, and we prove this up to an iterated logarithmic factor. This problem is motivated by the optimal theorem of Rudelson (J. Funct. Anal. 164:60–72, 1999), which states that N=O(nlog n) for distributions with finite second moment, and a recent result of Adamczak et al. (J. Am. Math. Soc. 234:535–561, 2010), which guarantees that N=O(n) for subexponential distributions."
to:NB  variance_estimation  estimation  statistics  probability 
august 2012 by cshalizi
[1206.6382] High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains
"In this paper, we present a novel framework incorporating a combination of sparse models in different domains. We posit the observed data as generated from a linear combination of a sparse Gaussian Markov model (with a sparse precision matrix) and a sparse Gaussian independence model (with a sparse covariance matrix). We provide efficient methods for decomposition of the data into two domains, viz Markov and independence domains. We characterize a set of sufficient conditions for identifiability and model consistency. Our decomposition method is based on a simple modification of the popular $ell_1$-penalized maximum-likelihood estimator ($ell_1$-MLE). We establish that our estimator is consistent in both the domains, i.e., it successfully recovers the supports of both Markov and independence models, when the number of samples $n$ scales as $n = Omega(d^2 log p)$, where $p$ is the number of variables and $d$ is the maximum node degree in the Markov model. Our conditions for recovery are comparable to those of $ell_1$-MLE for consistent estimation of a sparse Markov model, and thus, we guarantee successful high-dimensional estimation of a richer class of models under comparable conditions. Our experiments validate these results and also demonstrate that our models have better inference accuracy under simple algorithms such as loopy belief propagation."
to:NB  variance_estimation  graphical_models  sparsity  statistics 
july 2012 by cshalizi

related tags

algebra  bad_data_analysis  bayesian_consistency  bootstrap  causal_inference  climatology  confidence_sets  cross-validation  dimon.jamie  dsquared  econometrics  efromovich.sam  estimation  experimental_design  factor_analysis  fan.jianqing  finance  genetics  graphical_models  have_read  heard_the_talk  heritability  hierarchical_statistical_models  high-dimensional_statistics  in_nb  in_the_acknowledgments  inference_to_latent_objects  kernel_estimators  lasso  lee.ann_b.  linear_regression  low-rank_approximation  model_selection  mucha.peter_j.  network_data_analysis  nonparametrics  probability  random_fields  re:g_paper  read_the_draft  regression  respondent-driven_sampling  roeder.kathryn  smoothing  sparsity  spatial_statistics  spatio-temporal_statistics  statistics  time_series  to:blog  to:nb  to_read  to_teach:undergrad-ada  utter_stupidity  van_de_geer.sara  variable_selection  yeast 

Copy this bookmark: