nhaliday + learning-theory   63

Applications of computational learning theory in the cognitive sciences - Psychology & Neuroscience Stack Exchange
1. Gold's theorem on the unlearnability in the limit of certain sets of languages, among them context-free ones.

2. Ronald de Wolf's master's thesis on the impossibility to PAC-learn context-free languages.

The first made quiet a stir in the poverty-of-the-stimulus debate, and the second has been unnoticed by cognitive science.
q-n-a  stackex  psychology  cog-psych  learning  learning-theory  machine-learning  PAC  lower-bounds  no-go  language  linguistics  models  fall-2015 
8 weeks ago by nhaliday
How to Escape Saddle Points Efficiently – Off the convex path
A core, emerging problem in nonconvex optimization involves the escape of saddle points. While recent research has shown that gradient descent (GD) generically escapes saddle points asymptotically (see Rong Ge’s and Ben Recht’s blog posts), the critical open problem is one of efficiency — is GD able to move past saddle points quickly, or can it be slowed down significantly? How does the rate of escape scale with the ambient dimensionality? In this post, we describe our recent work with Rong Ge, Praneeth Netrapalli and Sham Kakade, that provides the first provable positive answer to the efficiency question, showing that, rather surprisingly, GD augmented with suitable perturbations escapes saddle points efficiently; indeed, in terms of rate and dimension dependence it is almost as if the saddle points aren’t there!
acmtariat  org:bleg  nibble  liner-notes  machine-learning  acm  optimization  gradient-descent  local-global  off-convex  time-complexity  random  perturbation  michael-jordan  iterative-methods  research  learning-theory  math.DS  iteration-recursion 
july 2017 by nhaliday
CS 731 Advanced Artificial Intelligence - Spring 2011
- statistical machine learning
- sparsity in regression
- graphical models
- exponential families
- variational methods
- dimensionality reduction, eg, PCA
- Bayesian nonparametrics
- compressive sensing, matrix completion, and Johnson-Lindenstrauss
course  lecture-notes  yoga  acm  stats  machine-learning  graphical-models  graphs  model-class  bayesian  learning-theory  sparsity  embeddings  markov  monte-carlo  norms  unit  nonparametric  compressed-sensing  matrix-factorization  features 
january 2017 by nhaliday
CS229T/STATS231: Statistical Learning Theory
Course by Percy Liang covers a mix of statistics, computational learning theory, and some online learning. Also surveys the state-of-the-art in theoretical understanding of deep learning (not much to cover unfortunately).
yoga  stanford  course  machine-learning  stats  👳  lecture-notes  acm  kernels  learning-theory  deep-learning  frontier  init  ground-up  unit  dimensionality  vc-dimension  entropy-like  extrema  moments  online-learning  bandits  p:***  explore-exploit 
june 2016 by nhaliday

bundles : academeacmframe

related tags

aaronson  academia  accretion  acm  acmtariat  adversarial  ai  algorithms  ankur-moitra  applications  atoms  average-case  bandits  bayesian  ben-recht  berkeley  big-picture  big-surf  bio  bits  blog  bonferroni  books  boolean-analysis  characterization  circuits  clarity  classification  cmu  cog-psych  columbia  comparison  complexity  compressed-sensing  computation  concentration-of-measure  concept  conference  convergence  convexity-curvature  cool  cornell  course  crypto  curvature  data-science  database  deep-learning  definition  descriptive  differential-privacy  dimensionality  direction  discussion  draft  embeddings  encyclopedic  ensembles  entropy-like  estimate  events  evolution  examples  expert  expert-experience  explanans  explanation  exploratory  explore-exploit  exposition  extrema  fall-2015  features  frequentist  frontier  game-theory  gelman  generalization  geometry  georgia  giants  gradient-descent  graphical-models  graphs  ground-up  heterodox  high-dimension  hmm  homepage  human-ml  hypothesis-testing  ideas  inference  info-dynamics  information-theory  init  interdisciplinary  intersection  intersection-connectedness  isotropy  iteration-recursion  iterative-methods  kernels  language  latent-variables  learning  learning-theory  lecture-notes  lectures  levers  limits  linear-algebra  linear-models  linearity  liner-notes  linguistics  links  list  local-global  logic  lower-bounds  machine-learning  markov  math  math.CA  math.DS  math.MG  matrix-factorization  meta:science  metabuch  metameta  methodology  metrics  michael-jordan  miri-cfar  mit  ML-MAP-E  model-class  models  moments  monte-carlo  motivation  mrtz  multi  nibble  nlp  no-go  nonparametric  norms  occam  ocw  off-convex  oly  online-learning  open-problems  operational  optimization  org:bleg  org:edu  org:inst  org:junk  org:mat  overflow  p:*  p:**  p:***  p:someday  PAC  papers  pdf  people  perturbation  philosophy  polynomials  potential  preprint  presentation  princeton  priors-posteriors  probability  prof  proof-systems  pseudorandomness  psychology  publishing  q-n-a  quantum  quantum-info  questions  quixotic  rand-approx  rand-complexity  random  ratty  rec-math  reduction  reference  reflection  regularization  reinforcement  research  research-program  rhetoric  rigor  rigorous-crypto  robust  sample-complexity  sanjeev-arora  science  scitariat  sebastien-bubeck  sensitivity  signal-noise  slides  space-complexity  sparsity  spectral  stackex  stanford  state  stats  stochastic-processes  stream  structure  sublinear  summary  summer-2016  synthesis  talks  tcs  tcstariat  tensors  texas  things  time-complexity  toolkit  top-n  topics  turing  tutorial  uncertainty  unit  unsupervised  valiant  vc-dimension  video  volo-avolo  wiki  workshop  wormholes  yoga  👳  🔬 

Copy this bookmark: