**hoff.peter**6

Testing Sparsity-Inducing Penalties: Journal of Computational and Graphical Statistics: Vol 0, No 0

yesterday by cshalizi

"Many penalized maximum likelihood estimators correspond to posterior mode estimators under specific prior distributions. Appropriateness of a particular class of penalty functions can therefore be interpreted as the appropriateness of a prior for the parameters. For example, the appropriateness of a lasso penalty for regression coefficients depends on the extent to which the empirical distribution of the regression coefficients resembles a Laplace distribution. We give a testing procedure of whether or not a Laplace prior is appropriate and accordingly, whether or not using a lasso penalized estimate is appropriate. This testing procedure is designed to have power against exponential power priors which correspond to ℓqℓq penalties. Via simulations, we show that this testing procedure achieves the desired level and has enough power to detect violations of the Laplace assumption when the numbers of observations and unknown regression coefficients are large. We then introduce an adaptive procedure that chooses a more appropriate prior and corresponding penalty from the class of exponential power priors when the null hypothesis is rejected. We show that this can improve estimation of the regression coefficients both when they are drawn from an exponential power distribution and when they are drawn from a spike-and-slab distribution. Supplementary materials for this article are available online."

--- I feel like I fundamentally disagree with this approach. Those priors are merely (to quote Jamie Robins and Larry Wasserman) "frequentist pursuit", and have no bearing on whether (say) the Lasso will give a good sparse, linear approximation to the underlying regression function (see https://normaldeviate.wordpress.com/2013/09/11/consistency-sparsistency-and-presistency/). All of which said, Hoff is always worth listening to, so the last tag applies with special force.

to:NB
model_checking
sparsity
regression
hypothesis_testing
bayesianism
re:phil-of-bayes_paper
hoff.peter
to_besh
--- I feel like I fundamentally disagree with this approach. Those priors are merely (to quote Jamie Robins and Larry Wasserman) "frequentist pursuit", and have no bearing on whether (say) the Lasso will give a good sparse, linear approximation to the underlying regression function (see https://normaldeviate.wordpress.com/2013/09/11/consistency-sparsistency-and-presistency/). All of which said, Hoff is always worth listening to, so the last tag applies with special force.

yesterday by cshalizi

Modeling homophily and stochastic equivalence in symmetric relational data

may 2016 by cshalizi

"This article discusses a latent variable model for inference and prediction of symmetric relational data. The model, based on the idea of the eigenvalue decomposition, represents the relationship between two nodes as the weighted inner-product of node-specific vectors of latent characteristics. This eigenmodel'' generalizes other popular latent variable models, such as latent class and distance models: It is shown mathematically that any latent class or distance model has a representation as an eigenmodel, but not vice-versa. The practical implications of this are examined in the context of three real datasets, for which the eigenmodel has as good or better out-of-sample predictive performance than the other two models."

--- Why the EXPLETIVE hadn't I read this before?

in_NB
network_data_analysis
re:network_differences
statistics
hoff.peter
community_discovery
inference_to_latent_objects
cross-validation
re:XV_for_networks
have_read
--- Why the EXPLETIVE hadn't I read this before?

may 2016 by cshalizi

[1212.6234] Likelihoods for fixed rank nomination networks

december 2012 by cshalizi

"Many studies that gather social network data use survey methods that lead to censored, missing or otherwise incomplete information. For example, the popular fixed rank nomination (FRN) scheme, often used in studies of schools and businesses, asks study participants to nominate and rank at most a small number of contacts or friends, leaving the existence other relations uncertain. However, most statistical models are formulated in terms of completely observed binary networks. Statistical analyses of FRN data with such models ignore the censored and ranked nature of the data and could potentially result in misleading statistical inference. To investigate this possibility, we compare parameter estimates obtained from a likelihood for complete binary networks to those from a likelihood that is derived from the FRN scheme, and therefore recognizes the ranked and censored nature of the data. We show analytically and via simulation that the binary likelihood can provide misleading inference, at least for certain model parameters that relate network ties to characteristics of individuals and pairs of individuals. We also compare these different likelihoods in a data analysis of several adolescent social networks. For some of these networks, the parameter estimates from the binary and FRN likelihoods lead to different conclusions, indicating the importance of analyzing FRN data with a method that accounts for the FRN survey design."

--- This has long worried me, though I've never done anything about it.

to:NB
statistics
network_data_analysis
to_read
hoff.peter
stovel.katherine
--- This has long worried me, though I've never done anything about it.

december 2012 by cshalizi

Taylor & Francis Online :: Fast Inference for the Latent Space Network Model Using a Case-Control Approximate Likelihood - Journal of Computational and Graphical Statistics - Volume 21, Issue 4

december 2012 by cshalizi

"Network models are widely used in social sciences and genome sciences. The latent space model proposed by Hoff et al. (2002), and extended by Handcock et al. (2007) to incorporate clustering, provides a visually interpretable model-based spatial representation of relational data and takes account of several intrinsic network properties. Due to the structure of the likelihood function of the latent space model, the computational cost is of order O(N 2), where N is the number of nodes. This makes it infeasible for large networks. In this article, we propose an approximation of the log-likelihood function. We adapt the case-control idea from epidemiology and construct a case-control log-likelihood, which is an unbiased estimator of the log-full likelihood. Replacing the full likelihood by the case-control likelihood in the Markov chain Monte Carlo estimation of the latent space model reduces the computational time from O(N 2) to O(N), making it feasible for large networks. We evaluate its performance using simulated and real data. We fit the model to a large protein–protein interaction data using the case-control likelihood and use the model fitted link probabilities to identify false positive links. Supplemental materials are available online."

to:NB
statistics
network_data_analysis
approximation
likelihood
inference_to_latent_objects
hoff.peter
raftery.adrian
december 2012 by cshalizi

[1211.0087] Bayesian sandwich posteriors for pseudo-true parameters

november 2012 by cshalizi

"Under model misspecification, the MLE generally converges to the pseudo-true parameter, the parameter corresponding to the distribution within the model that is closest to the distribution from which the data are sampled. In many problems, the pseudo-true parameter corresponds to a population parameter of interest, and so a misspecified model can provide consistent estimation for this parameter. Furthermore, the well-known sandwich variance formula of Huber(1967) provides an asymptotically accurate sampling distribution for the MLE, even under model misspecification. However, confidence intervals based on a sandwich variance estimate may behave poorly for low sample sizes, partly due to the use of a plug-in estimate of the variance. From a Bayesian perspective, plug-in estimates of nuisance parameters generally underrepresent uncertainty in the unknown parameters, and averaging over such parameters is expected to give better performance. With this in mind, we present a Bayesian sandwich posterior distribution, whose likelihood is based on the sandwich sampling distribution of the MLE. This Bayesian approach allows for the incorporation of prior information about the parameter of interest, averages over uncertainty in the nuisance parameter and is asymptotically robust to model misspecification. In a small simulation study on estimating a regression parameter under heteroscedasticity, the addition of accurate prior information and the averaging over the nuisance parameter are both seen to improve the accuracy and calibration of confidence intervals for the parameter of interest."

- Cites Muller's preprint, which has an on-the-face-of-it extremely similar idea...

in_NB
bayesian_consistency
confidence_sets
statistics
misspecification
hoff.peter
- Cites Muller's preprint, which has an on-the-face-of-it extremely similar idea...

november 2012 by cshalizi

**related tags**

Copy this bookmark: