Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning

2 days ago

"Randomized neural networks are immortalized in this AI Koan: _In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. What are you doing?'' asked Minsky. I am training a randomly wired neural net to play tic-tac-toe,'' Sussman replied. Why is the net wired randomly?'' asked Minsky. Sussman replied, I do not want it to have any preconceptions of how to play.'' Minsky then shut his eyes. Why do you close your eyes?'' Sussman asked his teacher. So that the room will be empty,'' replied Minsky. At that moment, Sussman was enlightened._ We analyze shallow random networks with the help of concentration of measure inequalities. Specifically, we consider architectures that compute a weighted sum of their inputs after passing them through a bank of arbitrary randomized nonlinearities. We identify conditions under which these networks exhibit good classification performance, and bound their test error in terms of the size of the dataset and the number of random nonlinearities."

to:NB
have_read
random_projections
kernel_methods
prediction
computational_statistics
statistics
classifiers
2 days ago

Fastfood - Computing Hilbert Space Expansions in loglinear time | ICML 2013 | JMLR W&CP

2 days ago

"Fast nonlinear function classes are crucial for nonparametric estimation, such as in kernel methods. This paper proposes an improvement to random kitchen sinks that offers significantly faster computation in log-linear time without sacrificing accuracy. Furthermore, we show how one may adjust the regularization properties of the kernel simply by changing the spectral distribution of the projection matrix. We provide experimental results which show that even for for moderately small problems we already achieve two orders of magnitude faster computation and three orders of magnitude lower memory footprint."

to:NB
regression
nonparametrics
computational_statistics
hilbert_space
kernel_methods
smola.alex
le.quoc
prediction
statistics
random_projections
2 days ago

AWS Service Terms

2 days ago

"57.10 Acceptable Use; Safety-Critical Systems. Your use of the Lumberyard Materials must comply with the AWS Acceptable Use Policy. The Lumberyard Materials are not intended for use with life-critical or safety-critical systems, such as use in operation of medical equipment, automated transportation systems, autonomous vehicles, aircraft or air traffic control, nuclear facilities, manned spacecraft, or military use in connection with live combat. However, this restriction will not apply in the event of the occurrence (certified by the United States Centers for Disease Control or successor body) of a widespread viral infection transmitted via bites or contact with bodily fluids that causes human corpses to reanimate and seek to consume living human flesh, blood, brain or nerve tissue and is likely to result in the fall of organized civilization."

funny:geeky
well_slightly_funny
zombies
amazon
2 days ago

The Koch Effect: The Impact of a Cadre-Led Network on American Politics

3 days ago

"Presidential election years attract attention to the rhetoric, personalities, and agendas of contending White House aspirants, but these headlines do not reflect the ongoing political shifts that will confront whoever moves into the White House in 2017. Earthquakes and erosions have remade the U.S. political terrain, reconfiguring the ground on which politicians and social groups must maneuver, and it is important to make sure that narrow and short-term analyses do not blind us to this shifting terrain. In this paper, we draw from research on changes since 2000 in the organizational universes surrounding the Republican and Democratic parties to highlight a major emergent force in U.S. politics: the recently expanded “Koch network” that coordinates big money funders, idea producers, issue advocates, and innovative constituency-building efforts in an ongoing effort to pull the Republican Party and agendas of U.S. politics sharply to the right. We review the major components and evolution of the Koch network and explore how it has reshaped American politics and policy agendas, focusing especially on implications for right-tilted partisan polarization and rising economic inequality."

to:NB
to_read
political_science
us_politics
inequality
political_networks
vast_right-wing_conspiracy
skocpol.theda
class_struggles_in_america
3 days ago

[1601.00934] Confidence Intervals for Projections of Partially Identified Parameters

4 days ago

"This paper proposes a bootstrap-based procedure to build confidence intervals for single components of a partially identified parameter vector, and for smooth functions of such components, in moment (in)equality models. The extreme points of our confidence interval are obtained by maximizing/minimizing the value of the component (or function) of interest subject to the sample analog of the moment (in)equality conditions properly relaxed. The novelty is that the amount of relaxation, or critical level, is computed so that the component of θ, instead of θ itself, is uniformly asymptotically covered with prespecified probability. Calibration of the critical level is based on repeatedly checking feasibility of linear programming problems, rendering it computationally attractive. Computation of the extreme points of the confidence interval is based on a novel application of the response surface method for global optimization, which may prove of independent interest also for applications of other methods of inference in the moment (in)equalities literature. The critical level is by construction smaller (in finite sample) than the one used if projecting confidence regions designed to cover the entire parameter vector θ. Hence, our confidence interval is weakly shorter than the projection of established confidence sets (Andrews and Soares, 2010), if one holds the choice of tuning parameters constant. We provide simple conditions under which the comparison is strict. Our inference method controls asymptotic coverage uniformly over a large class of data generating processes. Our assumptions and those used in the leading alternative approach (a profiling based method) are not nested. We explain why we employ some restrictions that are not required by other methods and provide examples of models for which our method is uniformly valid but profiling based methods are not."

to:NB
statistics
confidence_sets
partial_identification
bootstrap
4 days ago

[1601.07460] Information-theoretic lower bounds on learning the structure of Bayesian networks

4 days ago

"In this paper, we study the information theoretic limits of learning the structure of Bayesian networks from data. We show that for Bayesian networks on continuous as well as discrete random variables, there exists a parameterization of the Bayesian network such that, the minimum number of samples required to learn the "true" Bayesian network grows as (m), where m is the number of variables in the network. Further, for sparse Bayesian networks, where the number of parents of any variable in the network is restricted to be at most l for l≪m, the minimum number of samples required grows as (llogm). We discuss conditions under which these limits are achieved. For Bayesian networks over continuous variables, we obtain results for Gaussian regression and Gumbel Bayesian networks. While for the discrete variables, we obtain results for Noisy-OR, Conditional Probability Table (CPT) based Bayesian networks and Logistic regression networks. Finally, as a byproduct, we also obtain lower bounds on the sample complexity of feature selection in logistic regression and show that the bounds are sharp."

to:NB
graphical_models
model_discovery
statistics
information_theory
4 days ago

[1602.01192] Regression with network cohesion

4 days ago

"Prediction problems typically assume the training data are independent samples, but in many modern applications samples come from individuals connected by a network. For example, in adolescent health studies of risk-taking behaviors, information on the subjects' social networks is often available and plays an important role through network cohesion, the empirically observed phenomenon of friends behaving similarly. Taking cohesion into account in prediction models should allow us to improve their performance. Here we propose a regression model with a network-based penalty on individual node effects to encourage network cohesion, and show that it performs better than traditional models both theoretically and empirically when network cohesion is present. The framework is easily extended to other models, such as the generalized linear model and Cox's proportional hazard model. Applications to predicting levels of recreational activity and marijuana usage among teenagers based on both demographic covariates and their friendship networks are discussed in detail and demonstrate the effectiveness of our approach."

to:NB
statistics
network_data_analysis
regression
levina.liza
re:ADAfaEPoV
4 days ago

[1601.00815] Semi-parametric efficiency bounds and efficient estimation for high-dimensional models

4 days ago

"Asymptotic lower bounds for estimation play a fundamental role in assessing the quality of statistical procedures. In this paper we consider the possibility of establishing semi-parametric efficiency bounds for high-dimensional models and construction of estimators reaching these bounds. We propose a local uniform asymptotic unbiasedness assumption for high-dimensional models and derive explicit lower bounds on the variance of any asymptotically unbiased estimator. We show that an estimator obtained by de-sparsifying (or de-biasing) an ℓ1-penalized M-estimator is asymptotically unbiased and achieves the lower bound on the variance: thus it is asymptotically efficient. In particular, we consider the linear regression model, Gaussian graphical models and Gaussian sequence models under mild conditions. Furthermore, motivated by the results of Le Cam on local asymptotic normality, we show that the de-sparsified estimator converges to the limiting normal distribution with zero mean and the smallest possible variance not only pointwise, but locally uniformly in the underlying parameter. This is achieved by deriving an extension of Le Cam's Lemma to the high-dimensional setting."

to:NB
statistics
high-dimensional_statistics
estimation
van_de_geer.sara
4 days ago

[1601.00670] Variational Inference: A Review for Statisticians

4 days ago

"One of the core problems of modern statistics is to approximate difficult-to-compute probability distributions. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation about the posterior. In this paper, we review variational inference (VI), a method from machine learning that approximates probability distributions through optimization. VI has been used in myriad applications and tends to be faster than classical methods, such as Markov chain Monte Carlo sampling. The idea behind VI is to first posit a family of distributions and then to find the member of that family which is close to the target. Closeness is measured by Kullback-Leibler divergence. We review the ideas behind mean-field variational inference, discuss the special case of VI applied to exponential family models, present a full example with a Bayesian mixture of Gaussians, and derive a variant that uses stochastic optimization to scale up to massive data. We discuss modern research in VI and highlight important open problems. VI is powerful, but it is not yet well understood. Our hope in writing this paper is to catalyze statistical research on this widely-used class of algorithms."

to:NB
variational_inference
statistics
computational_statistics
approximation
blei.david
4 days ago

[1601.00389] Interpreting Latent Variables in Factor Models via Convex Optimization

4 days ago

"Latent or unobserved phenomena pose a significant difficulty in data analysis as they induce complicated and confounding dependencies among a collection of observed variables. Factor analysis is a prominent multivariate statistical modeling approach that addresses this challenge by identifying the effects of (a small number of) latent variables on a set of observed variables. However, the latent variables in a factor model are purely mathematical objects that are derived from the observed phenomena, and they do not have any interpretation associated to them. A natural approach for attributing semantic information to the latent variables in a factor model is to obtain measurements of some additional plausibly useful covariates that may be related to the original set of observed variables, and to associate these auxiliary covariates to the latent variables. In this paper, we describe a systematic approach for identifying such associations. Our method is based on solving computationally tractable convex optimization problems, and it can be viewed as a generalization of the minimum-trace factor analysis procedure for fitting factor models via convex optimization. We analyze the theoretical consistency of our approach in a high-dimensional setting as well as its utility in practice via experimental demonstrations with real data."

to:NB
factor_analysis
inference_to_latent_objects
statistics
4 days ago

[1602.01107] Do Cascades Recur?

4 days ago

"Cascades of information-sharing are a primary mechanism by which content reaches its audience on social media, and an active line of research has studied how such cascades, which form as content is reshared from person to person, develop and subside. In this paper, we perform a large-scale analysis of cascades on Facebook over significantly longer time scales, and find that a more complex picture emerges, in which many large cascades recur, exhibiting multiple bursts of popularity with periods of quiescence in between. We characterize recurrence by measuring the time elapsed between bursts, their overlap and proximity in the social network, and the diversity in the demographics of individuals participating in each peak. We discover that content virality, as revealed by its initial popularity, is a main driver of recurrence, with the availability of multiple copies of that content helping to spark new bursts. Still, beyond a certain popularity of content, the rate of recurrence drops as cascades start exhausting the population of interested individuals. We reproduce these observed patterns in a simple model of content recurrence simulated on a real social network. Using only characteristics of a cascade's initial burst, we demonstrate strong performance in predicting whether it will recur in the future."

to:NB
social_networks
networked_life
social_media
adamic.lada
kleinberg.jon
leskovec.jure
4 days ago

[1601.05033] Optimal tracking for dynamical systems

4 days ago

"We study the limiting behavior of the average per-state cost when trajectories of a topological dynamical system are used to track a trajectory from an observed ergodic system. We establish a variational characterization of the limiting average cost in terms of dynamically invariant couplings, also known as joinings, of the two dynamical systems, and we show that the set of optimal joinings is convex and compact in the weak topology. Using these results, we establish a general convergence theorem for the limiting behavior of statistical inference procedures based on optimal tracking. The setting considered here is general enough to encompass traditional statistical problems with weakly dependent, real-valued observations. As applications of the general inference result, we consider the consistency of regression estimation under ergodic sampling and of system identification from quantized observations."

to:NB
dynamical_systems
stochastic_processes
statistical_inference_for_stochastic_processes
filtering
nobel.andrew
4 days ago

[1601.02522] Stationary signal processing on graphs

4 days ago

"Graphs are a central tool in machine learning and information processing as they allow to conveniently capture the structure of complex datasets. In this context, it is of high importance to develop flexible models of signals defined over graphs or networks. In this paper, we generalize the traditional concept of wide sense stationarity to signals defined over the vertices of arbitrary weighted undirected graphs. We show that stationarity is intimately linked to statistical invariance under a localization operator reminiscent of translation. We prove that stationary graph signals are characterized by a well-defined Power Spectral Density that can be efficiently estimated even for large graphs. We leverage this new concept to derive Wiener-type estimation procedures of noisy and partially observed signals and illustrate the performance of this new model for denoising and regression."

to:NB
network_data_analysis
stochastic_processes
to_read
4 days ago

[1601.01972] Cox process representation and inference for stochastic reaction-diffusion processes

4 days ago

"Complex behaviour in many systems arises from the stochastic interactions of spatially distributed particles or agents. Stochastic reaction-diffusion processes are widely used to model such behaviour in disciplines ranging from biology to the social sciences, yet they are notoriously difficult to simulate and calibrate to observational data. Here we use ideas from statistical physics and machine learning to provide a solution to the inverse problem of learning a stochastic reaction diffusion process to data. Our solution relies on a novel, non-trivial connection between stochastic reaction-diffusion processes and spatio-temporal Cox processes, a well-studied class of models from computational statistics. We develop an efficient and flexible algorithm which shows excellent accuracy on numeric and real data examples from systems biology and epidemiology. By using ideas from multiple disciplines, our approach provides both new and fundamental insights into spatio-temporal stochastic systems, and a practical solution to a long-standing problem in computational modelling."

to:NB
stochastic_processes
statistical_inference_for_stochastic_processes
reaction-diffusion
point_processes
statistics
re:stacs
4 days ago

[1601.02513] How to learn a graph from smooth signals

4 days ago

"We propose a framework that learns the graph structure underlying a set of smooth signals. Given X∈ℝm×n whose rows reside on the vertices of an unknown graph, we learn the edge weights w∈ℝm(m−1)/2+ under the smoothness assumption that trX⊤LX is small. We show that the problem is a weighted ℓ-1 minimization that leads to naturally sparse solutions. We point out how known graph learning or construction techniques fall within our framework and propose a new model that performs better than the state of the art in many settings. We present efficient, scalable primal-dual based algorithms for both our model and the previous state of the art, and evaluate their performance on artificial and real data."

to:NB
network_data_analysis
smoothing
statistics
4 days ago

[1601.00992] Models, Methods and Network Topology: Experimental Design for the Study of Interference

4 days ago

"How should a network experiment be designed to achieve high statistical power? Experimental treatments on networks may spread. Randomizing assignment of treatment to nodes enhances learning about the counterfactual causal effects of a social network experiment and also requires new methodology (Aronow and Samii, 2013; Bowers et al., 2013; Toulis and Kao, 2013, ex.). In this paper we show that the way in which a treatment propagates across a social network affects the statistical power of an experimental design. As such, prior information regarding treatment propagation should be incorporated into the experimental design. Our findings run against standard advice in circumstances where units are presumed to be independent: information about treatment effects is not maximized when we assign half the units to treatment and half to control. We also show that statistical power depends on the extent to which the network degree of nodes is correlated with treatment assignment probability. We recommend that researchers think carefully about the underlying treatment propagation model motivating their study in designing an experiment on a network."

to:NB
to_read
network_data_analysis
social_influence
experimental_design
statistics
4 days ago

[1601.01413] Local average causal effects and superefficiency

4 days ago

"Recent approaches in causal inference have proposed estimating average causal effects that are local to some subpopulation, often for reasons of efficiency. These inferential targets are sometimes data-adaptive, in that they are dependent on the empirical distribution of the data. In this short note, we show that if researchers are willing to adapt the inferential target on the basis of efficiency, then extraordinary gains in precision can be obtained. Specifically, when causal effects are heterogeneous, any asymptotically normal and root-n consistent estimator of the population average causal effect is superefficient for a data-adaptive local average causal effect. Our result illustrates the fundamental gain in statistical certainty afforded by indifference about the inferential target."

to:NB
causal_inference
estimation
statistics
aronow.peter
4 days ago

[1601.04736] A Consistent Direct Method for Estimating Parameters in Ordinary Differential Equations Models

4 days ago

"Ordinary differential equations provide an attractive framework for modeling temporal dynamics in a variety of scientific settings. We show how consistent estimation for parameters in ODE models can be obtained by modifying a direct (non-iterative) least squares method similar to the direct methods originally developed by Himmelbau, Jones and Bischoff. Our method is called the bias-corrected least squares (BCLS) method since it is a modification of least squares methods known to be biased. Consistency of the BCLS method is established and simulations are used to compare the BCLS method to other methods for parameter estimation in ODE models."

to:NB
statistics
dynamical_systems
estimation
re:stacs
4 days ago

[1601.06805] P-values: misunderstood and misused

4 days ago

"P-values are widely used in both the social and natural sciences to quantify the statistical significance of observed results. The recent surge of big data research has made p-value an even more popular tool to test the significance of a study. However, substantial literature has been produced critiquing how p-values are used and understood. In this paper we review this recent critical literature, much of which is routed in the life sciences, and consider its implications for social scientific research. We provide a coherent picture of what the main criticisms are, and draw together and disambiguate common themes. In particular, we explain how the False Discovery Rate is calculated, and how this differs from a p-value. We also make explicit the Bayesian nature of many recent criticisms, a dimension that is often underplayed or ignored. We also identify practical steps to help remediate some of the concerns identified, and argue that p-values need to be contextualised within (i) the specific study, and (ii) the broader field of inquiry."

to:NB
statistics
hypothesis_testing
goodness-of-fit
p-values
4 days ago

[1602.01522] Risk estimation for high-dimensional lasso regression

4 days ago

"In high-dimensional estimation, analysts are faced with more parameters p than available observations n, and asymptotic analysis of performance allows the ratio p/n→∞. This situation makes regularization both necessary and desirable in order for estimators to possess theoretical guarantees. However, the amount of regularization, often determined by one or more tuning parameters, is integral to achieving good performance. In practice, choosing the tuning parameter is done through resampling methods (e.g. cross-validation), generalized information criteria, or reformulating the optimization problem (e.g. square-root lasso or scaled sparse regression). Each of these techniques comes with varying levels of theoretical guarantee for the low- or high-dimensional regimes. However, there are some notable deficiencies in the literature. The theory, and sometimes practice, of many methods relies on either the knowledge or estimation of the variance parameter, which is difficult to estimate in high dimensions. In this paper, we provide theoretical intuition suggesting that some previously proposed approaches based on information criteria work poorly in high dimensions. We introduce a suite of new risk estimators leveraging the burgeoning literature on high-dimensional variance estimation. Finally, we compare our proposal to many existing methods for choosing the tuning parameters for lasso regression by providing an extensive simulation to examine their finite sample performance. We find that our new estimators perform quite well, often better than the existing approaches across a wide range of simulation conditions and evaluation criteria."

to:NB
cross-validation
lasso
statistics
high-dimensional_statistics
regression
information_criteria
kith_and_kin
mcdonald.daniel
homrighausen.darren
4 days ago

[1602.01130] GraphPrints: Towards a Graph Analytic Method for Network Anomaly Detection

4 days ago

"This paper introduces a novel graph-analytic approach for detecting anomalies in network flow data called GraphPrints. Building on foundational network-mining techniques, our method represents time slices of traffic as a graph, then counts graphlets -- small induced subgraphs that describe local topology. By performing outlier detection on the sequence of graphlet counts, anomalous intervals of traffic are identified, and furthermore, individual IPs experiencing abnormal behavior are singled-out. Initial testing of GraphPrints is performed on real network data with an implanted anomaly. Evaluation shows false positive rates bounded by 2.84% at the time-interval level, and 0.05% at the IP-level with 100% true positive rates at both."

--- And how do you know how much the subgraph counts should fluctuate?

to:NB
network_data_analysis
re:network_differences
statistics
data_mining
to_be_shot_after_a_fair_trial
--- And how do you know how much the subgraph counts should fluctuate?

4 days ago

Bottolo , Richardson : Evolutionary stochastic search for Bayesian model exploration

4 days ago

'Implementing Bayesian variable selection for linear Gaussian regression models for analysing high dimensional data sets is of current interest in many fields. In order to make such analysis operational, we propose a new sampling algorithm based upon Evolutionary Monte Carlo and designed to work under the "large p, small n" paradigm, thus making fully Bayesian multivariate analysis feasible, for example, in genetics/genomics experiments. Two real data examples in genomics are presented, demonstrating the performance of the algorithm in a space of up to 10,000 covariates. Finally the methodology is compared with a recently proposed search algorithms in an extensive simulation study."

to:NB
to_read
re:fitness_sampling
monte_carlo
statistics
4 days ago

[1602.00795] Gender, Productivity, and Prestige in Computer Science Faculty Hiring Networks

4 days ago

"Women are dramatically underrepresented in computer science at all levels in academia and account for just 15% of tenure-track faculty. Understanding the causes of this gender imbalance would inform both policies intended to rectify it and employment decisions by departments and individuals. Progress in this direction, however, is complicated by the complexity and decentralized nature of faculty hiring and the non-independence of hires. Using comprehensive data on both hiring outcomes and scholarly productivity for 2659 tenure-track faculty across 205 Ph.D.-granting departments in North America, we investigate the multi-dimensional nature of gender inequality in computer science faculty hiring through a network model of the hiring process. Overall, we find that hiring outcomes are most directly affected by (i) the relative prestige between hiring and placing institutions and (ii) the scholarly productivity of the candidates. After including these, and other features, the addition of gender did not significantly reduce modeling error. However, gender differences do exist, e.g., in scholarly productivity, postdoctoral training rates, and in career movements up the rankings of universities, suggesting that the effects of gender are indirectly incorporated into hiring decisions through gender's covariates. Furthermore, we find evidence that more highly ranked departments recruit female faculty at higher than expected rates, which appears to inhibit similar efforts by lower ranked departments. These findings illustrate the subtle nature of gender inequality in faculty hiring networks and provide new insights to the underrepresentation of women in computer science."

to:NB
sexism
science_as_a_social_process
inequality
academia
kith_and_kin
clauset.aaron
4 days ago

[1602.00531] Adaptive non-parametric estimation in the presence of dependence

4 days ago

"We consider non-parametric estimation problems in the presence of dependent data, notably non-parametric regression with random design and non-parametric density estimation. The proposed estimation procedure is based on a dimension reduction. The minimax optimal rate of convergence of the estimator is derived assuming a sufficiently weak dependence characterized by fast decreasing mixing coefficients. We illustrate these results by considering classical smoothness assumptions. However, the proposed estimator requires an optimal choice of a dimension parameter depending on certain characteristics of the function of interest, which are not known in practice. The main issue addressed in our work is an adaptive choice of this dimension parameter combining model selection and Lepski's method. It is inspired by the recent work of Goldenshluger and Lepski (2011). We show that this data-driven estimator can attain the lower risk bound up to a constant provided a fast decay of the mixing coefficients."

to:NB
statistics
regression
nonparametrics
learning_under_dependence
density_estimation
dimension_reduction
4 days ago

[1602.00359] Confidence intervals for means under constrained dependence

4 days ago

"We develop a general framework for conducting inference on the mean of dependent random variables given constraints on their dependency graph. We establish the consistency of an oracle variance estimator of the mean when the dependency graph is known, along with an associated central limit theorem. We derive an integer linear program for finding an upper bound for the estimated variance when the graph is unknown, but topological and degree-based constraints are available. We develop alternative bounds, including a closed-form bound, under an additional homoskedasticity assumption. We establish a basis for Wald-type confidence intervals for the mean that are guaranteed to have asymptotically conservative coverage. We apply the approach to inference from a social network link-tracing study and provide statistical software implementing the approach."

to:NB
network_data_analysis
graphical_models
estimation
statistics
confidence_sets
4 days ago

Large Sample Properties of Matching Estimators for Average Treatment Effects - Abadie - 2005 - Econometrica - Wiley Online Library

4 days ago

"Matching estimators for average treatment effects are widely used in evaluation research despite the fact that their large sample properties have not been established in many cases. The absence of formal results in this area may be partly due to the fact that standard asymptotic expansions do not apply to matching estimators with a fixed number of matches because such estimators are highly nonsmooth functionals of the data. In this article we develop new methods for analyzing the large sample properties of matching estimators and establish a number of new results. We focus on matching with replacement with a fixed number of matches. First, we show that matching estimators are not N1/2-consistent in general and describe conditions under which matching estimators do attain N1/2-consistency. Second, we show that even in settings where matching estimators are N1/2-consistent, simple matching estimators with a fixed number of matches do not attain the semiparametric efficiency bound. Third, we provide a consistent estimator for the large sample variance that does not require consistent nonparametric estimation of unknown functions. Software for implementing these methods is available in Matlab, Stata, and R."

--- An unkind version of this would be "matching is what happens when you do nearest-neighbor regression, and you forget that the bias-variance tradeoff is a _tradeoff_."

(Ungated version: http://www.ksg.harvard.edu/fs/aabadie/smep.pdf)

(ADA note: reference in the causal-estimation chapter, re connection between matching and nearest neighbors)

to:NB
statistics
estimation
causal_inference
regression
to_teach:undergrad-ADA
have_read
--- An unkind version of this would be "matching is what happens when you do nearest-neighbor regression, and you forget that the bias-variance tradeoff is a _tradeoff_."

(Ungated version: http://www.ksg.harvard.edu/fs/aabadie/smep.pdf)

(ADA note: reference in the causal-estimation chapter, re connection between matching and nearest neighbors)

4 days ago

On the Failure of the Bootstrap for Matching Estimators - Abadie - 2008 - Econometrica - Wiley Online Library

4 days ago

"Matching estimators are widely used in empirical economics for the evaluation of programs or treatments. Researchers using matching methods often apply the bootstrap to calculate the standard errors. However, no formal justification has been provided for the use of the bootstrap in this setting. In this article, we show that the standard bootstrap is, in general, not valid for matching estimators, even in the simple case with a single continuous covariate where the estimator is root-N consistent and asymptotically normally distributed with zero asymptotic bias. Valid inferential methods in this setting are the analytic asymptotic variance estimator of Abadie and Imbens (2006a) as well as certain modifications of the standard bootstrap, like the subsampling methods in Politis and Romano (1994)."

--- Need to re-read this carefully, to see exactly what bootstrap they're using and so how generic the result really is.

to:NB
bootstrap
causal_inference
imbens.guido
have_read
statistics
--- Need to re-read this carefully, to see exactly what bootstrap they're using and so how generic the result really is.

4 days ago

Hirschler , Cover : A Finite Memory Test of the Irrationality of the Parameter of a Coin

6 days ago

"Let X1,X2,... be a Bernoulli sequence with parameter p. An algorithm ... is found such that [a function of the data is 0] all but a finite number of times with probability one if p is rational, and [that same function is one] all but a finite number of times with probability one if p is irrational (and not in a given null set of irrationals). Thus, an 8-state memory with a time-varying algorithm makes only a finite number of mistakes with probability one on determining the rationality of the parameter of a coin. Thus, determining the rationality of the Bernoulli parameter p does not depend on infinite memory of the data."

--- I would not have thought this was possible.

to:NB
statistics
cover.thomas
hypothesis_testing
automata_theory
--- I would not have thought this was possible.

6 days ago

[1602.00721] Concentration of measure without independence: a unified approach via the martingale method

6 days ago

"The concentration of measure phenomenon may be summarized as follows: a function of many weakly dependent random variables that is not too sensitive to any of its individual arguments will tend to take values very close to its expectation. This phenomenon is most completely understood when the arguments are mutually independent random variables, and there exist several powerful complementary methods for proving concentration inequalities, such as the martingale method, the entropy method, and the method of transportation inequalities. The setting of dependent arguments is much less well understood. This chapter focuses on the martingale method for deriving concentration inequalities without independence assumptions. In particular, we use the machinery of so-called Wasserstein matrices to show that the Azuma-Hoeffding concentration inequality for martingales with almost surely bounded differences, when applied in a sufficiently abstract setting, is powerful enough to recover and sharpen several known concentration results for nonproduct measures. Wasserstein matrices provide a natural formalism for capturing the interplay between the metric and the probabilistic structures, which is fundamental to the concentration phenomenon."

to:NB
concentration_of_measure
martingales
stochastic_processes
kontorovich.aryeh
raginsky.maxim
kith_and_kin
6 days ago

Freedman : On Tail Probabilities for Martingales

6 days ago

"Watch a martingale with uniformly bounded increments until it first crosses the horizontal line of height $a$. The sum of the conditional variances of the increments given the past, up to the crossing, is an intrinsic measure of the crossing time. Simple and fairly sharp upper and lower bounds are given for the Laplace transform of this crossing time, which show that the distribution is virtually the same as that for the crossing time of Brownian motion, even in the tail. The argument can be adapted to extend inequalities of Bernstein and Kolmogorov to the dependent case, proving the law of the iterated logarithm for martingales. The argument can also be adapted to prove Levy's central limit theorem for martingales. The results can be extended to martingales whose increments satisfy a growth condition."

to:NB
deviation_inequalities
martingales
probability
re:AoS_project
6 days ago

[1602.00355] A Spectral Series Approach to High-Dimensional Nonparametric Regression

6 days ago

"A key question in modern statistics is how to make fast and reliable inferences for complex, high-dimensional data. While there has been much interest in sparse techniques, current methods do not generalize well to data with nonlinear structure. In this work, we present an orthogonal series estimator for predictors that are complex aggregate objects, such as natural images, galaxy spectra, trajectories, and movies. Our series approach ties together ideas from kernel machine learning, and Fourier methods. We expand the unknown regression on the data in terms of the eigenfunctions of a kernel-based operator, and we take advantage of orthogonality of the basis with respect to the underlying data distribution, P, to speed up computations and tuning of parameters. If the kernel is appropriately chosen, then the eigenfunctions adapt to the intrinsic geometry and dimension of the data. We provide theoretical guarantees for a radial kernel with varying bandwidth, and we relate smoothness of the regression function with respect to P to sparsity in the eigenbasis. Finally, using simulated and real-world data, we systematically compare the performance of the spectral series approach with classical kernel smoothing, k-nearest neighbors regression, kernel ridge regression, and state-of-the-art manifold and local regression methods."

to:NB
have_read
statistics
regression
nonparametrics
sparsity
kernel_methods
kith_and_kin
lee.ann
6 days ago

Bottlenecks, A New Theory of Equal Opportunity // Reviews // Notre Dame Philosophical Reviews // University of Notre Dame

6 days ago

This seems intriguing, but a lot of work would have to be done by way of distinguishing capacities which are worth developing and those which are not. E.g., some societies would offer many more opportunities to develop talents for theft, fraud, bloody vengeance and/or boot-licking ingratiation with bosses even than our own. Eliminating sanitation and vaccinations would give us all the opportunity to develop our capacities to deal with the early and random death of friends and family. Etc., etc.

to:NB
books:noted
book_reviews
inequality
equality_of_opportunity
political_philosophy
institutions
6 days ago

Negishi welfare weights in integrated assessment models: the mathematics of global inequality - Springer

7 days ago

"In a global climate policy debate fraught with differing understandings of right and wrong, the importance of making transparent the ethical assumptions used in climate-economics models cannot be overestimated. Negishi weighting is a key ethical assumption in climate-economics models, but it is virtually unknown to most model users. Negishi weights freeze the current distribution of income between world regions; without this constraint, IAMs that maximize global welfare would recommend an equalization of income across regions as part of their policy advice. With Negishi weights in place, these models instead recommend a course of action that would be optimal only in a world in which global income redistribution cannot and will not take place. This article describes the Negishi procedure and its origin in theoretical and applied welfare economics, and discusses the policy implications of the presentation and use of Negishi-weighted model results, as well as some alternatives to Negishi weighting in climate-economics models."

Ungated: http://sei-us.org/Publications_PDF/SEI-Stanton2010_ClimaticChange_Negishi.pdf

to:NB
economics
economic_policy
cost-benefit_analysis
moral_philosophy
inequality
climate_change
have_read
via:jbdelong
Ungated: http://sei-us.org/Publications_PDF/SEI-Stanton2010_ClimaticChange_Negishi.pdf

7 days ago

Credential Privilege or Cumulative Advantage? Prestige, Productivity, and Placement in the Academic Sociology Job Market by Spencer Headworth, Jeremy Freese :: SSRN

8 days ago

"Using data on the population of US sociology doctorates over a five-year period, we examine different predictors of placement in a research-oriented, tenure-track academic sociology jobs. More completely than prior studies, we document the enormous relationship between PhD institution and job placement that has, in part, prompted a popular metaphor that academic job allocation processes are like a caste system. Yet we also find comparable relationships between PhD program and both graduate student publishing and awards. Overall, we find results more consistent with PhD prestige operating indirectly through mediating achievements or as a quality signal than as a “pure prestige” effect. We suggest sociologists think of stratification in their profession as not requiring exceptionalist historical metaphors, but rather as involving the same ordinary but powerful processes of cumulative advantage that pervade contemporary life."

to:NB
sociology
sociology_of_science
inequality
cumulative_advantage
freese.jeremy
academia
science_as_a_social_process
8 days ago

AEAweb: JEP (30,1) p. 53 - The New Role for the World Bank

9 days ago

"The World Bank was founded to address what we would today call imperfections in international capital markets. Its founders thought that countries would borrow from the Bank temporarily until they grew enough to borrow commercially. Some critiques and analyses of the Bank are based on the assumption that this continues to be its role. For example, some argue that the growth of private capital flows to the developing world has rendered the Bank irrelevant. However, we will argue that modern analyses should proceed from the premise that the World Bank's central goal is and should be to reduce extreme poverty, and that addressing failures in global capital markets is now of subsidiary importance. In this paper, we discuss what the Bank does: how it spends money, how it influences policy, and how it presents its mission. We argue that the role of the Bank is now best understood as facilitating international agreements to reduce poverty, and we examine implications of this perspective."

to:NB
world_bank
development_economics
political_economy
9 days ago

AEAweb: JEP (30,1) p. 77 - The World Bank: Why It Is Still Needed and Why It Still Disappoints

9 days ago

"Does the World Bank still have an important role to play? How might it fulfill that role? The paper begins with a brief account of how the Bank works. It then argues that, while the Bank is no longer the primary conduit for capital from high-income to low-income countries, it still has an important role in supplying the public good of development knowledge—a role that is no less pressing today than ever. This argument is not a new one. In 1996, the Bank's President at the time, James D. Wolfensohn, laid out a vision for the "knowledge bank," an implicit counterpoint to what can be called the "lending bank." The paper argues that the past rhetoric of the "knowledge bank" has not matched the reality. An institution such as the World Bank—explicitly committed to global poverty reduction—should be more heavily invested in knowing what is needed in its client countries as well as in international coordination. It should be consistently arguing for well-informed pro-poor policies in its member countries, tailored to the needs of each country, even when such policies are unpopular with the powers-that-be. It should also be using its financial weight, combined with its analytic and convening powers, to support global public goods. In all this, there is a continuing role for lending, but it must be driven by knowledge—both in terms of what gets done and how it is geared to learning. The paper argues that the Bank disappoints in these tasks but that it could perform better."

to:NB
world_bank
development_economics
economics
political_economy
9 days ago

AEAweb: JEP (30,1) p. 185 - Power Laws in Economics: An Introduction

9 days ago

"Many of the insights of economics seem to be qualitative, with many fewer reliable quantitative laws. However a series of power laws in economics do count as true and nontrivial quantitative laws—and they are not only established empirically, but also understood theoretically. I will start by providing several illustrations of empirical power laws having to do with patterns involving cities, firms, and the stock market. I summarize some of the theoretical explanations that have been proposed. I suggest that power laws help us explain many economic phenomena, including aggregate economic fluctuations. I hope to clarify why power laws are so special, and to demonstrate their utility. In conclusion, I list some power-law-related economic enigmas that demand further exploration."

to:NB
heavy_tails
economics
to_be_shot_after_a_fair_trial
gabaix.xaiver
9 days ago

Needham, A.: Power Lines: Phoenix and the Making of the Modern Southwest. (eBook and Hardcover)

14 days ago

"In 1940, Phoenix was a small, agricultural city of sixty-five thousand, and the Navajo Reservation was an open landscape of scattered sheepherders. Forty years later, Phoenix had blossomed into a metropolis of 1.5 million people and the territory of the Navajo Nation was home to two of the largest strip mines in the world. Five coal-burning power plants surrounded the reservation, generating electricity for export to Phoenix, Los Angeles, and other cities. Exploring the postwar developments of these two very different landscapes, Power Lines tells the story of the far-reaching environmental and social inequalities of metropolitan growth, and the roots of the contemporary coal-fueled climate change crisis.

"Andrew Needham explains how inexpensive electricity became a requirement for modern life in Phoenix—driving assembly lines and cooling the oppressive heat. Navajo officials initially hoped energy development would improve their lands too, but as ash piles marked their landscape, air pollution filled the skies, and almost half of Navajo households remained without electricity, many Navajos came to view power lines as a sign of their subordination in the Southwest. Drawing together urban, environmental, and American Indian history, Needham demonstrates how power lines created unequal connections between distant landscapes and how environmental changes associated with suburbanization reached far beyond the metropolitan frontier. Needham also offers a new account of postwar inequality, arguing that residents of the metropolitan periphery suffered similar patterns of marginalization as those faced in America’s inner cities.

"Telling how coal from Indian lands became the fuel of modernity in the Southwest, Power Lines explores the dramatic effects that this energy system has had on the people and environment of the region."

to:NB
books:noted
american_history
american_southwest
native_american_history
electricity
20th_century_history
pollution
"Andrew Needham explains how inexpensive electricity became a requirement for modern life in Phoenix—driving assembly lines and cooling the oppressive heat. Navajo officials initially hoped energy development would improve their lands too, but as ash piles marked their landscape, air pollution filled the skies, and almost half of Navajo households remained without electricity, many Navajos came to view power lines as a sign of their subordination in the Southwest. Drawing together urban, environmental, and American Indian history, Needham demonstrates how power lines created unequal connections between distant landscapes and how environmental changes associated with suburbanization reached far beyond the metropolitan frontier. Needham also offers a new account of postwar inequality, arguing that residents of the metropolitan periphery suffered similar patterns of marginalization as those faced in America’s inner cities.

"Telling how coal from Indian lands became the fuel of modernity in the Southwest, Power Lines explores the dramatic effects that this energy system has had on the people and environment of the region."

14 days ago

The Economist's Tale: A Consultant Encounters Hunger and the World Bank | Zed Books

14 days ago

"What really happens when the World Bank imposes its policies on a country? This is an insider‘s view of one aid-made crisis. Peter Griffiths was at the interface between government and the Bank. In this ruthlessly honest, day by day account of a mission he undertook in Sierra Leone, he uses his diary to tell the story of how the World Bank, obsessed with the free market, imposed a secret agreement on the government, banning all government food imports or subsidies. The collapsing economy meant that the private sector would not import. Famine loomed. No ministry, no state marketing organization, no aid organization could reverse the agreement. It had to be a top-level government decision, whether Sierra Leone could afford to annoy minor World Bank officials. This is a rare and important portrait of the aid world which insiders will recognize, but of which the general public seldom get a glimpse."

in_NB
books:noted
economics
development_economics
political_economy
world_bank
via:crooked_timber
14 days ago

AEAweb: AER (95,3) p. 546 - The Rise of Europe: Atlantic Trade, Institutional Change, and Economic Growth

15 days ago

"The rise of Western Europe after 1500 is due largely to growth in countries with access to the Atlantic Ocean and with substantial trade with the New World, Africa, and Asia via the Atlantic. This trade and the associated colonialism affected Europe not only directly, but also indirectly by inducing institutional change. Where "initial" political institutions (those established before 1500) placed significant checks on the monarchy, the growth of Atlantic trade strengthened merchant groups by constraining the power of the monarchy, and helped merchants obtain changes in institutions to protect property rights. These changes were central to subsequent economic growth."

to:NB
economics
economic_history
institutions
economic_growth
to_teach:undergrad-ADA
via:jbdelong
have_read
15 days ago

Kutz, C.: On War and Democracy (eBook and Hardcover).

18 days ago

"On War and Democracy provides a richly nuanced examination of the moral justifications democracies often invoke to wage war. In this compelling and provocative book, Christopher Kutz argues that democratic principles can be both fertile and toxic ground for the project of limiting war’s violence. Only by learning to view war as limited by our democratic values—rather than as a tool for promoting them—can we hope to arrest the slide toward the borderless, seemingly endless democratic "holy wars" and campaigns of remote killings we are witnessing today, and to stop permanently the use of torture and secret law.

"Kutz shows how our democratic values, understood incautiously and incorrectly, can actually undermine the goal of limiting war. He helps us better understand why we are tempted to believe that collective violence in the name of politics can be legitimate when individual violence is not. In doing so, he offers a bold new account of democratic agency that acknowledges the need for national defense and the promotion of liberty abroad while limiting the temptations of military intervention. Kutz demonstrates why we must address concerns about the means of waging war—including remote war and surveillance—and why we must create institutions to safeguard some nondemocratic values, such as dignity and martial honor, from the threat of democratic politics."

to:NB
books:noted
war
democracy
political_philosophy
the_continuing_crises
to_be_shot_after_a_fair_trial
"Kutz shows how our democratic values, understood incautiously and incorrectly, can actually undermine the goal of limiting war. He helps us better understand why we are tempted to believe that collective violence in the name of politics can be legitimate when individual violence is not. In doing so, he offers a bold new account of democratic agency that acknowledges the need for national defense and the promotion of liberty abroad while limiting the temptations of military intervention. Kutz demonstrates why we must address concerns about the means of waging war—including remote war and surveillance—and why we must create institutions to safeguard some nondemocratic values, such as dignity and martial honor, from the threat of democratic politics."

18 days ago

Cities, Business, and the Politics of Urban Violence in Latin America | Eduardo Moncada

18 days ago

"This book analyzes and explains the ways in which major developing world cities respond to the challenge of urban violence. The study shows how the political projects that cities launch to confront urban violence are shaped by the interaction between urban political economies and patterns of armed territorial control. It introduces business as a pivotal actor in the politics of urban violence, and argues that how business is organized within cities and its linkages to local governments impacts whether or not business supports or subverts state efforts to stem and prevent urban violence. A focus on city mayors finds that the degree to which politicians rely upon clientelism to secure and maintain power influences whether they favor responses to violence that perpetuate or weaken local political exclusion. The book builds a new typology of patterns of armed territorial control within cities, and shows that each poses unique challenges and opportunities for confronting urban violence. The study develops sub-national comparative analyses of puzzling variation in the institutional outcomes of the politics of urban violence across Colombia's three principal cities—Medellin, Cali, and Bogota—and over time within each. The book's main findings contribute to research on violence, crime, citizen security, urban development, and comparative political economy. The analysis demonstrates that the politics of urban violence is a powerful new lens on the broader question of who governs in major developing world cities."

to:NB
books:noted
political_economy
cities
violence
crime
colombia
18 days ago

Convergence of One-parameter Operator Semigroups | Abstract Analysis | Cambridge University Press

18 days ago

"This book presents a detailed and contemporary account of the classical theory of convergence of semigroups. The author demonstrates the far-reaching applications of this theory using real examples from various branches of pure and applied mathematics, with a particular emphasis on mathematical biology. These examples also serve as short, non-technical introductions to biological concepts. The book may serve as a useful reference, containing a significant number of new results ranging from the analysis of fish populations to signalling pathways in living cells."

in_NB
books:noted
mathematics
analysis
stochastic_processes
markov_models
biology
re:almost_none
18 days ago

Mathematical Foundations of Infinite-Dimensional Statistical Models | Statistical Theory and Methods | Cambridge University Press

18 days ago

"In nonparametric and high-dimensional statistical models, the classical Gauss-Fisher-Le Cam theory of the optimality of maximum likelihood estimators and Bayesian posterior inference does not apply, and new foundations and ideas have been developed in the past several decades. This book gives a coherent account of the statistical theory in infinite-dimensional parameter spaces. The mathematical foundations include self-contained 'mini-courses' on the theory of Gaussian and empirical processes, on approximation and wavelet theory, and on the basic theory of function spaces. The theory of statistical inference in such models – hypothesis testing, estimation and confidence sets – is then presented within the minimax paradigm of decision theory. This includes the basic theory of convolution kernel and projection estimation, but also Bayesian nonparametrics and nonparametric maximum likelihood estimation. In a final chapter the theory of adaptive inference in nonparametric models is developed, including Lepski's method, wavelet thresholding, and adaptive inference for self-similar functions."

in_NB
books:noted
statistics
estimation
nonparametrics
18 days ago

Today in Feminist History: Carolyn Heilbrun, 1926-2003, and Amanda Cross – The Golden Notebooks

19 days ago

The Amanda Cross novels are great, and I should look up Heilbrun's academic work in my copious spare time.

feminism
literary_criticism
lives_of_the_scholars
lives_of_the_artists
have_read
19 days ago

On Algorithmic Communism - The Los Angeles Review of Books

24 days ago

I will be... very interested... in their analysis of the computational complexity of economic planning.

books:noted
book_reviews
progressive_forces
to_be_shot_after_a_fair_trial
24 days ago

When Algorithms Are Cheaper than Humans: The Hidden Curse of Automation - The Los Angeles Review of Books

24 days ago

Application to data analysis left as an exercise for the reader.

automation
cognitive_development
books:noted
book_reviews
have_read
cultural_criticism
24 days ago

A Natural History of Natural Theology: The Cognitive Science of Theology and Philosophy of Religion // Reviews // Notre Dame Philosophical Reviews // University of Notre Dame

in_NB books:noted book_reviews theology cognitive_science cognitive_development barely-comprehensible_metaphysics philosophy

27 days ago

in_NB books:noted book_reviews theology cognitive_science cognitive_development barely-comprehensible_metaphysics philosophy

27 days ago

Taking Text and Structure Really Seriously

4 weeks ago

An argument that, taking the text of the Constitution literally, no one who wasn't a citizen when it was adopted is eligible to become president.

have_read
law
ha_ha_only_serious
us_politics
balkin.jack_m.
via:kjhealy
4 weeks ago

Brownian Motion as a Limit to Physical Measuring Processes: A Chapter in the History of Noise from the Physicists’ Point of View

4 weeks ago

"In this paper, we examine the history of the idea that noise presents a fundamental limit to physical measuring processes. This idea had its origins in research aimed at improving the accuracy of instruments for electrical measurements. Out of these endeavors, the Swedish physicist Gustaf A. Ising formulated a general conclusion concerning the nature of physical measurements, namely that there is a definite limit to the ultimate sensitivity of measuring instruments beyond which we cannot advance, and that this limit is determined by Brownian motion. Ising’s conclusion agreed with experiments and received widespread recognition, but his way of modeling the system was contested by his contemporaries. With the more embracing notion of noise that developed during and after World War II, Ising’s conclusion was reinterpreted as showing that noise puts a limit on physical measurement processes. Hence, physicists in particular saw the work as an indication that noise is of practical relevance for their enterprise."

to:NB
stochastic_processes
physics_of_information
measurement
statistical_mechanics
history_of_physics
4 weeks ago

Two Mathematical Approaches to Random Fluctuations

4 weeks ago

"Physicists and mathematicians in the early twentieth century had established a research program on various random fluctuations. Historical reviews have portrayed this development as a linear progress toward a unified conceptual framework. In this paper, I argue that two approaches were at work. One operated in the “time domain,” as it aimed to formulate the diffusion-type equation for the probability density function and its solutions. The other operated in the “frequency domain,” as it focused on the spectral analysis of the fluctuation. The time-domain analysis was marshaled by statistical physicists, while the frequency-domain analysis was promoted by engineering researchers."

--- Of course, the two are equivalent...

to:NB
stochastic_processes
history_of_science
history_of_physics
fourier_analysis
--- Of course, the two are equivalent...

4 weeks ago

Radar, Modems, and Air Defense Systems: Noise as a Data Communication Problem in the 1950s

4 weeks ago

"The modem was created in the context of a US strategic automatic air defense network to transmit data from radar stations over large distances over the existing telephone system. A significant early challenge was how to minimize noise, which was conceptualized in terms of echo effects, impulse noise, and phase distortion. The approaches to solving the noise problem varied with the techniques of modulation and demodulation used by early modems. The problem of minimizing noise was crucial to the development of the modem, and the focus the military placed on automatic air defense spawned decades of work into the further refinement of modem technology."

to:NB
history_of_technology
computer_networks
signal_processing
cold_war
4 weeks ago

The Physics of Forgetting: Thermodynamics of Information at IBM 1959–1982

4 weeks ago

"The origin and history of Landauer’s principle is traced through the development of the thermodynamics of computation at IBM from 1959 to 1982. This development was characterized by multiple conceptual shifts: memory came to be seen not as information storage, but as delayed information transmission; information itself was seen not as a disembodied logical entity, but as participating in the physical world; and logical irreversibility was connected with physical, thermodynamic, irreversibility. These conceptual shifts were characterized by an ambivalence opposing strong metaphysical claims to practical considerations. Three sorts of practical considerations are discussed. First, these conceptual shifts engaged materials central to IBM’s business practice. Second, arguments for metaphysical certainties were made with reference to the practical functioning of typical computers. Third, arguments for metaphysical certainties were made in the context of establishing the thermodynamics of information as a sub-discipline of physics."

to:NB
physics_of_information
thermodynamics
information_theory
landauers_principle
history_of_physics
statistical_mechanics
4 weeks ago

Inference in finite state space non parametric Hidden Markov Models and applications - Springer

4 weeks ago

"Hidden Markov models (HMMs) are intensively used in various fields to model and classify data observed along a line (e.g. time). The fit of such models strongly relies on the choice of emission distributions that are most often chosen among some parametric family. In this paper, we prove that finite state space non parametric HMMs are identifiable as soon as the transition matrix of the latent Markov chain has full rank and the emission probability distributions are linearly independent. This general result allows the use of semi- or non-parametric emission distributions. Based on this result we present a series of classification problems that can be tackled out of the strict parametric framework. We derive the corresponding inference algorithms. We also illustrate their use on few biological examples, showing that they may improve the classification performances."

to:NB
markov_models
state-space_models
statistics
nonparametrics
time_series
4 weeks ago

Does data splitting improve prediction? - Springer

4 weeks ago

"Data splitting divides data into two parts. One part is reserved for model selection. In some applications, the second part is used for model validation but we use this part for estimating the parameters of the chosen model. We focus on the problem of constructing reliable predictive distributions for future observed values. We judge the predictive performance using log scoring. We compare the full data strategy with the data splitting strategy for prediction. We show how the full data score can be decomposed into model selection, parameter estimation and data reuse costs. Data splitting is preferred when data reuse costs are high. We investigate the relative performance of the strategies in four simulation scenarios. We introduce a hybrid estimator that uses one part for model selection but both parts for estimation. We argue that a split data analysis is prefered to a full data analysis for prediction with some exceptions."

--- Ungated: http://arxiv.org/abs/1301.2983

in_NB
statistics
regression
prediction
model_selection
faraway.j.j.
re:ADAfaEPoV
to_teach:undergrad-ADA
have_read
to_teach:mreg
--- Ungated: http://arxiv.org/abs/1301.2983

4 weeks ago

[1510.07389] The Human Kernel

5 weeks ago

"Bayesian nonparametric models, such as Gaussian processes, provide a compelling framework for automatic statistical modelling: these models have a high degree of flexibility, and automatically calibrated complexity. However, automating human expertise remains elusive; for example, Gaussian processes with standard kernels struggle on function extrapolation problems that are trivial for human learners. In this paper, we create function extrapolation problems and acquire human responses, and then design a kernel learning framework to reverse engineer the inductive biases of human learners across a set of behavioral experiments. We use the learned kernels to gain psychological insights and to extrapolate in human-like ways that go beyond traditional stationary and polynomial kernels. Finally, we investigate Occam's razor in human and Gaussian process based function learning."

to:NB
kernel_methods
nonparametrics
regression
statistics
via:arsyed
cognitive_science
5 weeks ago

Quantifying Life: A Symbiosis of Computation, Mathematics, and Biology, Kondrashov

5 weeks ago

"Since the time of Isaac Newton, physicists have used mathematics to describe the behavior of matter of all sizes, from subatomic particles to galaxies. In the past three decades, as advances in molecular biology have produced an avalanche of data, computational and mathematical techniques have also become necessary tools in the arsenal of biologists. But while quantitative approaches are now providing fundamental insights into biological systems, the college curriculum for biologists has not caught up, and most biology majors are never exposed to the computational and probabilistic mathematical approaches that dominate in biological research.

"With Quantifying Life, Dmitry A. Kondrashov offers an accessible introduction to the breadth of mathematical modeling used in biology today. Assuming only a foundation in high school mathematics, Quantifying Life takes an innovative computational approach to developing mathematical skills and intuition. Through lessons illustrated with copious examples, mathematical and programming exercises, literature discussion questions, and computational projects of various degrees of difficulty, students build and analyze models based on current research papers and learn to implement them in the R programming language. This interplay of mathematical ideas, systematically developed programming skills, and a broad selection of biological research topics makes Quantifying Life an invaluable guide for seasoned life scientists and the next generation of biologists alike."

--- Mineable for examples?

books:noted
biology
programming
modeling
to_teach:statcomp
to_teach:complexity-and-inference
"With Quantifying Life, Dmitry A. Kondrashov offers an accessible introduction to the breadth of mathematical modeling used in biology today. Assuming only a foundation in high school mathematics, Quantifying Life takes an innovative computational approach to developing mathematical skills and intuition. Through lessons illustrated with copious examples, mathematical and programming exercises, literature discussion questions, and computational projects of various degrees of difficulty, students build and analyze models based on current research papers and learn to implement them in the R programming language. This interplay of mathematical ideas, systematically developed programming skills, and a broad selection of biological research topics makes Quantifying Life an invaluable guide for seasoned life scientists and the next generation of biologists alike."

--- Mineable for examples?

5 weeks ago

Coevolution of Life on Hosts: Integrating Ecology and History, Clayton, Bush, Johnson

5 weeks ago

"For most, the mere mention of lice forces an immediate hand to the head and recollection of childhood experiences with nits, medicated shampoos, and traumatic haircuts. But for a certain breed of biologist, lice make for fascinating scientific fodder, especially enlightening in the study of coevolution. In this book, three leading experts on host-parasite relationships demonstrate how the stunning coevolution that occurs between such species in microevolutionary, or ecological, time generates clear footprints in macroevolutionary, or historical, time. By integrating these scales, Coevolution of Life on Hosts offers a comprehensive understanding of the influence of coevolution on the diversity of all life.

"Following an introduction to coevolutionary concepts, the authors combine experimental and comparative host-parasite approaches for testing coevolutionary hypotheses to explore the influence of ecological interactions and coadaptation on patterns of diversification and codiversification among interacting species. Ectoparasites—a diverse assemblage of organisms that ranges from herbivorous insects on plants, to monogenean flatworms on fish, and feather lice on birds—are powerful models for the study of coevolution because they are easy to observe, mark, and count. As lice on birds and mammals are permanent parasites that spend their entire lifecycles on the bodies of their hosts, they are ideally suited to generating a synthetic overview of coevolution—and, thereby, offer an exciting framework for integrating the concepts of coadaptation and codiversification."

in_NB
books:noted
evolutionary_biology
"Following an introduction to coevolutionary concepts, the authors combine experimental and comparative host-parasite approaches for testing coevolutionary hypotheses to explore the influence of ecological interactions and coadaptation on patterns of diversification and codiversification among interacting species. Ectoparasites—a diverse assemblage of organisms that ranges from herbivorous insects on plants, to monogenean flatworms on fish, and feather lice on birds—are powerful models for the study of coevolution because they are easy to observe, mark, and count. As lice on birds and mammals are permanent parasites that spend their entire lifecycles on the bodies of their hosts, they are ideally suited to generating a synthetic overview of coevolution—and, thereby, offer an exciting framework for integrating the concepts of coadaptation and codiversification."

5 weeks ago

High-Stakes Schooling: What We Can Learn from Japan's Experiences with Testing, Accountability, and Education Reform, Bjork

5 weeks ago

"If there is one thing that describes the trajectory of American education, it is this: more high-stakes testing. In the United States, the debates surrounding this trajectory can be so fierce that it feels like we are in uncharted waters. As Christopher Bjork reminds us in this study, however, we are not the first to make testing so central to education: Japan has been doing it for decades. Drawing on Japan’s experiences with testing, overtesting, and recent reforms to relax educational pressures, he sheds light on the best path forward for US schools.

"Bjork asks a variety of important questions related to testing and reform: Does testing overburden students? Does it impede innovation and encourage conformity? Can a system anchored by examination be reshaped to nurture creativity and curiosity? How should any reforms be implemented by teachers? Each chapter explores questions like these with careful attention to the actual effects policies have had on schools in Japan and other Asian settings, and each draws direct parallels to issues that US schools currently face. Offering a wake-up call for American education, Bjork ultimately cautions that the accountability-driven practice of standardized testing might very well exacerbate the precise problems it is trying to solve. "

in_NB
books:noted
education
standardized_testing
japan
"Bjork asks a variety of important questions related to testing and reform: Does testing overburden students? Does it impede innovation and encourage conformity? Can a system anchored by examination be reshaped to nurture creativity and curiosity? How should any reforms be implemented by teachers? Each chapter explores questions like these with careful attention to the actual effects policies have had on schools in Japan and other Asian settings, and each draws direct parallels to issues that US schools currently face. Offering a wake-up call for American education, Bjork ultimately cautions that the accountability-driven practice of standardized testing might very well exacerbate the precise problems it is trying to solve. "

5 weeks ago

Semantic Properties of Diagrams and Their Cognitive Potentials, Shimojima

5 weeks ago

"Why are diagrams sometimes so useful, facilitating our understanding and thinking, while at other times they can be unhelpful and even misleading? Drawing on a comprehensive survey of modern research in philosophy, logic, artificial intelligence, cognitive psychology, and graphic design, Semantic Properties of Diagrams and Their Cognitive Potentials reveals the systematic reasons for this dichotomy, showing that the cognitive functions of diagrams are rooted in the characteristic ways they carry information. In analyzing the logical mechanisms behind the relative efficacy of diagrammatic representation, Atsushi Shimojima provides deep insight into the crucial question: What makes a diagram a diagram?"

books:noted
visual_display_of_quantitative_information
cognition
diagrams
5 weeks ago

[1512.07942] Multi-Level Cause-Effect Systems

5 weeks ago

"We present a domain-general account of causation that applies to settings in which macro-level causal relations between two systems are of interest, but the relevant causal features are poorly understood and have to be aggregated from vast arrays of micro-measurements. Our approach generalizes that of Chalupka et al. (2015) to the setting in which the macro-level effect is not specified. We formalize the connection between micro- and macro-variables in such situations and provide a coherent framework describing causal relations at multiple levels of analysis. We present an algorithm that discovers macro-variable causes and effects from micro-level measurements obtained from an experiment. We further show how to design experiments to discover macro-variables from observational micro-variable data. Finally, we show that under specific conditions, one can identify multiple levels of causal structure. Throughout the article, we use a simulated neuroscience multi-unit recording experiment to illustrate the ideas and the algorithms."

to:NB
to_read
causality
causal_inference
macro_from_micro
eberhardt.frederick
kith_and_kin
re:what_is_a_macrostate
5 weeks ago

Ferreirós, J.: Mathematical Knowledge and the Interplay of Practices (eBook and Hardcover).

5 weeks ago

"This book presents a new approach to the epistemology of mathematics by viewing mathematics as a human activity whose knowledge is intimately linked with practice. Charting an exciting new direction in the philosophy of mathematics, José Ferreirós uses the crucial idea of a continuum to provide an account of the development of mathematical knowledge that reflects the actual experience of doing math and makes sense of the perceived objectivity of mathematical results.

"Describing a historically oriented, agent-based philosophy of mathematics, Ferreirós shows how the mathematical tradition evolved from Euclidean geometry to the real numbers and set-theoretic structures. He argues for the need to take into account a whole web of mathematical and other practices that are learned and linked by agents, and whose interplay acts as a constraint. Ferreirós demonstrates how advanced mathematics, far from being a priori, is based on hypotheses, in contrast to elementary math, which has strong cognitive and practical roots and therefore enjoys certainty.

"Offering a wealth of philosophical and historical insights, Mathematical Knowledge and the Interplay of Practices challenges us to rethink some of our most basic assumptions about mathematics, its objectivity, and its relationship to culture and science."

--- *ahem* P. Kitcher, _The Nature of Mathematical Knowledge_ 1983 *ahem*

to:NB
books:noted
mathematics
philosophy_of_science
"Describing a historically oriented, agent-based philosophy of mathematics, Ferreirós shows how the mathematical tradition evolved from Euclidean geometry to the real numbers and set-theoretic structures. He argues for the need to take into account a whole web of mathematical and other practices that are learned and linked by agents, and whose interplay acts as a constraint. Ferreirós demonstrates how advanced mathematics, far from being a priori, is based on hypotheses, in contrast to elementary math, which has strong cognitive and practical roots and therefore enjoys certainty.

"Offering a wealth of philosophical and historical insights, Mathematical Knowledge and the Interplay of Practices challenges us to rethink some of our most basic assumptions about mathematics, its objectivity, and its relationship to culture and science."

--- *ahem* P. Kitcher, _The Nature of Mathematical Knowledge_ 1983 *ahem*

5 weeks ago

Phys. Rev. Lett. 115, 268501 (2015) - Teleconnection Paths via Climate Network Direct Link Detection

5 weeks ago

"Teleconnections describe remote connections (typically thousands of kilometers) of the climate system. These are of great importance in climate dynamics as they reflect the transportation of energy and climate change on global scales (like the El Niño phenomenon). Yet, the path of influence propagation between such remote regions, and weighting associated with different paths, are only partially known. Here we propose a systematic climate network approach to find and quantify the optimal paths between remotely distant interacting locations. Specifically, we separate the correlations between two grid points into direct and indirect components, where the optimal path is found based on a minimal total cost function of the direct links. We demonstrate our method using near surface air temperature reanalysis data, on identifying cross-latitude teleconnections and their corresponding optimal paths. The proposed method may be used to quantify and improve our understanding regarding the emergence of climate patterns on global scales."

--- I am going to be pleasantly astonished if this amounts to more than the graph lasso, or even if there is any acknowledgment of prior art at all.

to:NB
statistics
time_series
graphical_models
time_series_connections
--- I am going to be pleasantly astonished if this amounts to more than the graph lasso, or even if there is any acknowledgment of prior art at all.

5 weeks ago

Phys. Rev. Lett. 115, 260602 (2015) - On-Chip Maxwell's Demon as an Information-Powered Refrigerator

5 weeks ago

"We present an experimental realization of an autonomous Maxwell’s demon, which extracts microscopic information from a system and reduces its entropy by applying feedback. It is based on two capacitively coupled single-electron devices, both integrated on the same electronic circuit. This setup allows a detailed analysis of the thermodynamics of both the demon and the system as well as their mutual information exchange. The operation of the demon is directly observed as a temperature drop in the system. We also observe a simultaneous temperature rise in the demon arising from the thermodynamic cost of generating the mutual information."

to:NB
physics_of_information
maxwells_demon
physics
statistical_mechanics
information_theory
thermodynamics
5 weeks ago

Herbst, E.P. and Schorfheide, F.: Bayesian Estimation of DSGE Models (eBook and Hardcover).

5 weeks ago

"Dynamic stochastic general equilibrium (DSGE) models have become one of the workhorses of modern macroeconomics and are extensively used for academic research as well as forecasting and policy analysis at central banks. This book introduces readers to state-of-the-art computational techniques used in the Bayesian analysis of DSGE models. The book covers Markov chain Monte Carlo techniques for linearized DSGE models, novel sequential Monte Carlo methods that can be used for parameter inference, and the estimation of nonlinear DSGE models based on particle filter approximations of the likelihood function. The theoretical foundations of the algorithms are discussed in depth, and detailed empirical applications and numerical illustrations are provided. The book also gives invaluable advice on how to tailor these algorithms to specific applications and assess the accuracy and reliability of the computations."

to:NB
books:noted
econometrics
macroeconomics
time_series
estimation
statistics
re:your_favorite_dsge_sucks
5 weeks ago

Card, D. and Krueger, A.B.: Myth and Measurement: The New Economics of the Minimum Wage. (Twentieth-Anniversary Edition) (eBook and Paperback)

5 weeks ago

"David Card and Alan B. Krueger have already made national news with their pathbreaking research on the minimum wage. Here they present a powerful new challenge to the conventional view that higher minimum wages reduce jobs for low-wage workers. In a work that has important implications for public policy as well as for the direction of economic research, the authors put standard economic theory to the test, using data from a series of recent episodes, including the 1992 increase in New Jersey's minimum wage, the 1988 rise in California's minimum wage, and the 1990-91 increases in the federal minimum wage. In each case they present a battery of evidence showing that increases in the minimum wage lead to increases in pay, but no loss in jobs.

"A distinctive feature of Card and Krueger's research is the use of empirical methods borrowed from the natural sciences, including comparisons between the "treatment" and "control" groups formed when the minimum wage rises for some workers but not for others. In addition, the authors critically reexamine the previous literature on the minimum wage and find that it, too, lacks support for the claim that a higher minimum wage cuts jobs. Finally, the effects of the minimum wage on family earnings, poverty outcomes, and the stock market valuation of low-wage employers are documented. Overall, this book calls into question the standard model of the labor market that has dominated economists' thinking on the minimum wage. In addition, it will shift the terms of the debate on the minimum wage in Washington and in state legislatures throughout the country.

"With a new preface discussing new data, Myth and Measurement continues to shift the terms of the debate on the minimum wage."

--- I can only hope the new preface also discusses the immortal "theory is evidence too".

to:NB
books:noted
economics
imperfect_competition
"A distinctive feature of Card and Krueger's research is the use of empirical methods borrowed from the natural sciences, including comparisons between the "treatment" and "control" groups formed when the minimum wage rises for some workers but not for others. In addition, the authors critically reexamine the previous literature on the minimum wage and find that it, too, lacks support for the claim that a higher minimum wage cuts jobs. Finally, the effects of the minimum wage on family earnings, poverty outcomes, and the stock market valuation of low-wage employers are documented. Overall, this book calls into question the standard model of the labor market that has dominated economists' thinking on the minimum wage. In addition, it will shift the terms of the debate on the minimum wage in Washington and in state legislatures throughout the country.

"With a new preface discussing new data, Myth and Measurement continues to shift the terms of the debate on the minimum wage."

--- I can only hope the new preface also discusses the immortal "theory is evidence too".

5 weeks ago

Pritchard, D.: Epistemic Angst: Radical Skepticism and the Groundlessness of Our Believing. (eBook and Hardcover)

5 weeks ago

"Epistemic Angst offers a completely new solution to the ancient philosophical problem of radical skepticism—the challenge of explaining how it is possible to have knowledge of a world external to us.

"Duncan Pritchard argues that the key to resolving this puzzle is to realize that it is composed of two logically distinct problems, each requiring its own solution. He then puts forward solutions to both problems. To that end, he offers a new reading of Wittgenstein’s account of the structure of rational evaluation and demonstrates how this provides an elegant solution to one aspect of the skeptical problem. Pritchard also revisits the epistemological disjunctivist proposal that he developed in previous work and shows how it can effectively handle the other aspect of the problem. Finally, he argues that these two antiskeptical positions, while superficially in tension with each other, are not only compatible but also mutually supporting."

--- I realize there is an irony to being skeptical about this largely on the grounds that all attempts to solve the problem for the last 2000+ years have failed. (Uncle Davey would've understood.)

to:NB
books:noted
epistemology
"Duncan Pritchard argues that the key to resolving this puzzle is to realize that it is composed of two logically distinct problems, each requiring its own solution. He then puts forward solutions to both problems. To that end, he offers a new reading of Wittgenstein’s account of the structure of rational evaluation and demonstrates how this provides an elegant solution to one aspect of the skeptical problem. Pritchard also revisits the epistemological disjunctivist proposal that he developed in previous work and shows how it can effectively handle the other aspect of the problem. Finally, he argues that these two antiskeptical positions, while superficially in tension with each other, are not only compatible but also mutually supporting."

--- I realize there is an irony to being skeptical about this largely on the grounds that all attempts to solve the problem for the last 2000+ years have failed. (Uncle Davey would've understood.)

5 weeks ago

academia
afghanistan
agent-based_models
american_history
ancient_history
archaeology
art
bad_data_analysis
bad_science_journalism
bayesian_consistency
bayesianism
biochemical_networks
blogged
book_reviews
books:noted
books:owned
books:recommended
bootstrap
causal_inference
causality
central_asia
central_limit_theorem
class_struggles_in_america
classifiers
climate_change
clustering
cognitive_science
collective_cognition
comics
community_discovery
complexity
computational_statistics
concentration_of_measure
confidence_sets
corruption
coveted
crime
cthulhiana
cultural_criticism
cultural_exchange
data_analysis
data_mining
debunking
decision-making
decision_theory
delong.brad
democracy
density_estimation
design
dimension_reduction
distributed_systems
dynamical_systems
econometrics
economic_history
economic_policy
economics
education
empirical_processes
ensemble_methods
entropy_estimation
epidemic_models
epistemology
ergodic_theory
estimation
evisceration
evolution_of_cooperation
evolutionary_biology
experimental_psychology
finance
financial_crisis_of_2007--
financial_speculation
fmri
food
fraud
functional_connectivity
funny
funny:academic
funny:geeky
funny:laughing_instead_of_screaming
funny:malicious
funny:pointed
genetics
graph_theory
graphical_models
have_read
heard_the_talk
heavy_tails
high-dimensional_statistics
hilbert_space
history_of_ideas
history_of_science
human_genetics
hypothesis_testing
ideology
imperialism
in_nb
inequality
inference_to_latent_objects
information_theory
institutions
kernel_methods
kith_and_kin
krugman.paul
large_deviations
lasso
learning_theory
liberman.mark
likelihood
linguistics
literary_criticism
machine_learning
macro_from_micro
macroeconomics
manifold_learning
market_failures_in_everything
markov_models
mathematics
mixing
model_selection
modeling
modern_ruins
monte_carlo
moral_psychology
moral_responsibility
mortgage_crisis
natural_history_of_truthiness
network_data_analysis
networked_life
networks
neural_data_analysis
neuroscience
non-equilibrium
nonparametrics
optimization
our_decrepit_institutions
philosophy
philosophy_of_science
photos
physics
pittsburgh
political_economy
political_science
practices_relating_to_the_transmission_of_genetic_information
prediction
pretty_pictures
principal_components
probability
programming
progressive_forces
psychology
r
racism
random_fields
re:almost_none
re:aos_project
re:democratic_cognition
re:do-institutions-evolve
re:g_paper
re:homophily_and_confounding
re:network_differences
re:smoothing_adjacency_matrices
re:social_networks_as_sensor_networks
re:stacs
re:your_favorite_dsge_sucks
recipes
regression
running_dogs_of_reaction
science_as_a_social_process
science_fiction
simulation
social_influence
social_life_of_the_mind
social_media
social_networks
social_science_methodology
sociology
something_about_america
sparsity
spatial_statistics
state-space_models
statistical_inference_for_stochastic_processes
statistical_mechanics
statistics
stochastic_processes
text_mining
the_american_dilemma
the_continuing_crises
time_series
to:blog
to:nb
to_be_shot_after_a_fair_trial
to_read
to_teach:complexity-and-inference
to_teach:data-mining
to_teach:statcomp
to_teach:undergrad-ada
track_down_references
us-iraq_war
us_politics
utter_stupidity
vast_right-wing_conspiracy
via:?
via:henry_farrell
via:jbdelong
via:klk
visual_display_of_quantitative_information
whats_gone_wrong_with_america
why_oh_why_cant_we_have_a_better_academic_publishing_system
why_oh_why_cant_we_have_a_better_press_corps