nhaliday + generative   20

Unsupervised learning, one notion or many? – Off the convex path
(Task A) Learning a distribution from samples. (Examples: gaussian mixtures, topic models, variational autoencoders,..)

(Task B) Understanding latent structure in the data. This is not the same as (a); for example principal component analysis, clustering, manifold learning etc. identify latent structure but don’t learn a distribution per se.

(Task C) Feature Learning. Learn a mapping from datapoint → feature vector such that classification tasks are easier to carry out on feature vectors rather than datapoints. For example, unsupervised feature learning could help lower the amount of labeled samples needed for learning a classifier, or be useful for domain adaptation.

Task B is often a subcase of Task C, as the intended user of “structure found in data” are humans (scientists) who pour over the representation of data to gain some intuition about its properties, and these “properties” can be often phrased as a classification task.

This post explains the relationship between Tasks A and C, and why they get mixed up in students’ mind. We hope there is also some food for thought here for experts, namely, our discussion about the fragility of the usual “perplexity” definition of unsupervised learning. It explains why Task A doesn’t in practice lead to good enough solution for Task C. For example, it has been believed for many years that for deep learning, unsupervised pretraining should help supervised training, but this has been hard to show in practice.
acmtariat  org:bleg  nibble  machine-learning  acm  thinking  clarity  unsupervised  conceptual-vocab  concept  explanation  features  bayesian  off-convex  deep-learning  latent-variables  generative  intricacy  distribution  sampling 
june 2017 by nhaliday
Spaceship Generator | Hacker News
some interesting discussion of the value of procedural generation in the comments
commentary  hn  graphics  games  programming  libraries  repo  oss  project  SIGGRAPH  random  generative 
june 2016 by nhaliday

bundles : abstractacm

related tags

academia  accuracy  acm  acmtariat  adversarial  ai  ai-control  algorithms  analogy  announcement  applications  approximation  arms  art  atoms  audio  automation  bayesian  best-practices  bioinformatics  biotech  boltzmann  causation  clarity  clever-rats  coarse-fine  columbia  commentary  comparison  computer-vision  concept  conceptual-vocab  confounding  confusion  convexity-curvature  correlation  cost-benefit  course  critique  crux  data-science  debate  deep-learning  deepgoog  developmental  dimensionality  direction  discrete  discussion  distribution  economics  embeddings  empirical  engineering  enhancement  error  evolution  examples  experiment  expert  expert-experience  explanans  explanation  exposition  extrema  features  frontier  futurism  game-theory  games  gaussian-processes  generalization  generative  genomics  gradient-descent  graphical-models  graphics  graphs  heterodox  heuristic  hn  hsu  humanity  ideas  info-dynamics  init  insight  interdisciplinary  interview  intricacy  isotropy  iteration-recursion  land  language  latent-variables  lecture-notes  lens  lesswrong  libraries  linear-algebra  linear-models  linearity  liner-notes  linguistics  links  list  local-global  machine-learning  markets  markov  matrix-factorization  meta:prediction  meta:science  ML-MAP-E  model-class  models  monte-carlo  motivation  nature  network-structure  neuro  neuro-nitgrit  nibble  nitty-gritty  nlp  nonlinearity  number  off-convex  offense-defense  openai  operational  optimization  org:bleg  org:mat  oss  overflow  p:*  papers  pdf  podcast  ppl  preprint  princeton  probability  programming  project  psychology  q-n-a  qra  random  ratty  reading  reduction  reference  regression  regularization  reinforcement  replication  repo  research  risk  robotics  roots  sample-complexity  sampling  sanjeev-arora  science  scifi-fantasy  scitariat  SIGGRAPH  signal-noise  similarity  singularity  slides  smoothness  social-psych  social-science  sparsity  speedometer  stanford  state-of-art  stats  stochastic-processes  summary  supply-demand  survey  talks  tcs  technology  techtariat  thinking  time  todo  track-record  trends  turing  tutorial  unit  unsupervised  VC-dimension  visualization  visuo  wiki  worrydream  yoga  🔬 

Copy this bookmark: