nhaliday + ben-recht   12

Bottoming Out – arg min blog
Now, I’ve been hammering the point in my previous posts that saddle points are not what makes non-convex optimization difficult. Here, when specializing to deep learning, even local minima are not getting in my way. Deep neural nets are just very easy to minimize.
machine-learning  deep-learning  optimization  rhetoric  speculation  research  hmm  research-program  acmtariat  generalization  metabuch  local-global  off-convex  ben-recht  extrema  org:bleg  nibble  sparsity  curvature  ideas  aphorism  convexity-curvature  explanans  volo-avolo  hardness 
june 2016 by nhaliday
The News on Auto-tuning – arg min blog
bayesian optimization is not necessarily obviously better than randomized search on all fronts
critique  bayesian  optimization  machine-learning  expert  hmm  liner-notes  rhetoric  debate  acmtariat  ben-recht  mrtz  gwern  random  org:bleg  nibble  expert-experience 
june 2016 by nhaliday

bundles : academeacmpeeps

related tags

accretion  acm  acmtariat  aphorism  approximation  atoms  bare-hands  bayesian  ben-recht  benchmarks  books  characterization  coarse-fine  composition-decomposition  compressed-sensing  concept  confluence  convergence  convexity-curvature  cool  critique  curvature  debate  decision-making  decision-theory  deep-learning  descriptive  direction  empirical  estimate  examples  expert  expert-experience  explanans  explanation  exposition  extrema  frontier  generalization  geometry  gradient-descent  ground-up  guide  gwern  hardness  hi-order-bits  hmm  ideas  info-foraging  intricacy  isotropy  iteration-recursion  iterative-methods  kernels  learning-theory  levers  limits  linear-algebra  liner-notes  list  local-global  machine-learning  math  math.DS  math.MG  mathtariat  matrix-factorization  metabuch  michael-jordan  model-class  motivation  mrtz  music  nibble  norms  numerics  off-convex  openai  optimization  org:bleg  org:inst  p:someday  p:whenever  papers  parsimony  perturbation  philosophy  polynomials  presentation  quixotic  random  ranking  reading  realness  recommendations  reduction  reflection  reinforcement  replication  repo  research  research-program  review  rhetoric  rigor  roadmap  robust  rounding  science  search  sequential  signal-noise  sparsity  speculation  stories  success  summary  synthesis  systematic-ad-hoc  talks  tcs  the-trenches  thinking  tightness  toolkit  top-n  tricks  tutorial  unit  unsupervised  values  video  volo-avolo  yoga 

Copy this bookmark: