nhaliday + big-surf   38

Information Processing: Mathematical Theory of Deep Neural Networks (Princeton workshop)
"Recently, long-past-due theoretical results have begun to emerge. These results, and those that will follow in their wake, will begin to shed light on the properties of large, adaptive, distributed learning architectures, and stand to revolutionize how computer science and neuroscience understand these systems."
hsu  scitariat  commentary  links  research  research-program  workshop  events  princeton  sanjeev-arora  deep-learning  machine-learning  ai  generalization  explanans  off-convex  nibble  frontier  speedometer  state-of-art  big-surf  announcement 
january 2018 by nhaliday
New Theory Cracks Open the Black Box of Deep Learning | Quanta Magazine
A new idea called the “information bottleneck” is helping to explain the puzzling success of today’s artificial-intelligence algorithms — and might also explain how human brains learn.

sounds like he's just talking about autoencoders?
news  org:mag  org:sci  popsci  announcement  research  deep-learning  machine-learning  acm  information-theory  bits  neuro  model-class  big-surf  frontier  nibble  hmm  signal-noise  deepgoog  expert  ideas  wild-ideas  summary  talks  video  israel  roots  physics  interdisciplinary  ai  intelligence  shannon  giants  arrows  preimage  lifts-projections  composition-decomposition  characterization  markov  gradient-descent  papers  liner-notes  experiment  hi-order-bits  generalization  expert-experience  explanans  org:inst  speedometer 
september 2017 by nhaliday
Correlated Equilibria in Game Theory | Azimuth
Given this, it’s not surprising that Nash equilibria can be hard to find. Last September a paper came out making this precise, in a strong way:

• Yakov Babichenko and Aviad Rubinstein, Communication complexity of approximate Nash equilibria.

The authors show there’s no guaranteed method for players to find even an approximate Nash equilibrium unless they tell each other almost everything about their preferences. This makes finding the Nash equilibrium prohibitively difficult to find when there are lots of players… in general. There are particular games where it’s not difficult, and that makes these games important: for example, if you’re trying to run a government well. (A laughable notion these days, but still one can hope.)

Klarreich’s article in Quanta gives a nice readable account of this work and also a more practical alternative to the concept of Nash equilibrium. It’s called a ‘correlated equilibrium’, and it was invented by the mathematician Robert Aumann in 1974. You can see an attempt to define it here:
baez  org:bleg  nibble  mathtariat  commentary  summary  news  org:mag  org:sci  popsci  equilibrium  GT-101  game-theory  acm  conceptual-vocab  concept  definition  thinking  signaling  coordination  tcs  complexity  communication-complexity  lower-bounds  no-go  liner-notes  big-surf  papers  research  algorithmic-econ  volo-avolo 
july 2017 by nhaliday
Alzheimers | West Hunter
Some disease syndromes almost have to be caused by pathogens – for example, any with a fitness impact (prevalence x fitness reduction) > 2% or so, too big to be caused by mutational pressure. I don’t think that this is the case for AD: it hits so late in life that the fitness impact is minimal. However, that hardly means that it can’t be caused by a pathogen or pathogens – a big fraction of all disease syndromes are, including many that strike in old age. That possibility is always worth checking out, not least because infectious diseases are generally easier to prevent and/or treat.

There is new work that strongly suggests that pathogens are the root cause. It appears that the amyloid is an antimicrobial peptide. amyloid-beta binds to invading microbes and then surrounds and entraps them. ‘When researchers injected Salmonella into mice’s hippocampi, a brain area damaged in Alzheimer’s, A-beta quickly sprang into action. It swarmed the bugs and formed aggregates called fibrils and plaques. “Overnight you see the plaques throughout the hippocampus where the bugs were, and then in each single plaque is a single bacterium,” Tanzi says. ‘

obesity and pathogens: https://westhunt.wordpress.com/2016/05/29/alzheimers/#comment-79757
not sure about this guy, but interesting: https://westhunt.wordpress.com/2016/05/29/alzheimers/#comment-79748

All too often we see large, long-lasting research efforts that never produce, never achieve their goal.

For example, the amyloid hypothesis [accumulation of amyloid-beta oligomers is the cause of Alzheimers] has been dominant for more than 20 years, and has driven development of something like 15 drugs. None of them have worked. At the same time the well-known increased risk from APOe4 has been almost entirely ignored, even though it ought to be a clue to the cause.

In general, when a research effort has been spinning its wheels for a generation or more, shouldn’t we try something different? We could at least try putting a fraction of those research dollars into alternative approaches that have not yet failed repeatedly.

Mostly this applies to research efforts that at least wish they were science. ‘educational research’ is in a special class, and I hardly know what to recommend. Most of the remedial actions that occur to me violate one or more of the Geneva conventions.

APOe4 related to lymphatic system: https://en.wikipedia.org/wiki/Apolipoprotein_E

Look,if I could find out the sort of places that I usually misplace my keys – if I did, which I don’t – I could find the keys more easily the next time I lose them. If you find out that practitioners of a given field are not very competent, it marks that field as a likely place to look for relatively easy discovery. Thus medicine is a promising field, because on the whole doctors are not terribly good investigators. For example, none of the drugs developed for Alzheimers have worked at all, which suggests that our ideas on the causation of Alzheimers are likely wrong. Which suggests that it may (repeat may) be possible to make good progress on Alzheimers, either by an entirely empirical approach, which is way underrated nowadays, or by dumping the current explanation, finding a better one, and applying it.

You could start by looking at basic notions of field X and asking yourself: How do we really know that? Is there serious statistical evidence? Does that notion even accord with basic theory? This sort of checking is entirely possible. In most of the social sciences, we don’t, there isn’t, and it doesn’t.

Hygiene and the world distribution of Alzheimer’s disease: Epidemiological evidence for a relationship between microbial environment and age-adjusted disease burden: https://academic.oup.com/emph/article/2013/1/173/1861845/Hygiene-and-the-world-distribution-of-Alzheimer-s

Amyloid-β peptide protects against microbial infection in mouse and worm models of Alzheimer’s disease: http://stm.sciencemag.org/content/8/340/340ra72

Fungus, the bogeyman: http://www.economist.com/news/science-and-technology/21676754-curious-result-hints-possibility-dementia-caused-fungal
Fungus and dementia
paper: http://www.nature.com/articles/srep15015
west-hunter  scitariat  disease  parasites-microbiome  medicine  dementia  neuro  speculation  ideas  low-hanging  todo  immune  roots  the-bones  big-surf  red-queen  multi  🌞  poast  obesity  strategy  info-foraging  info-dynamics  institutions  meta:medicine  social-science  curiosity  🔬  science  meta:science  meta:research  wiki  epidemiology  public-health  study  arbitrage  alt-inst  correlation  cliometrics  path-dependence  street-fighting  methodology  nibble  population-genetics  org:nat  health  embodied  longevity  aging  org:rec  org:biz  org:anglo  news  neuro-nitgrit  candidate-gene  nutrition  diet  org:health  explanans  fashun  empirical  theory-practice  ability-competence  dirty-hands  education  aphorism  truth  westminster  innovation  evidence-based  religion  prudence  track-record  problem-solving 
july 2017 by nhaliday
Information Processing: Learn to solve every problem that has been solved
While it may be impossible to achieve Feynman's goal, I'm surprised that more people don't attempt the importance threshold-modified version. Suppose we set the importance bar really, really high: what are the most important results that everyone should try to understand? Here's a very biased partial list: basic physics and mathematics (e.g., to the level of the Feynman Lectures); quantitative theory of genetics and evolution; information, entropy and probability; basic ideas about logic and computation (Godel and Turing?); ... What else? Dynamics of markets? Complex Systems? Psychometrics? Descriptive biology? Organic chemistry?
hsu  scitariat  feynman  giants  stories  aphorism  curiosity  interdisciplinary  frontier  signal-noise  top-n  discussion  caltech  problem-solving  big-picture  vitality  🎓  virtu  big-surf  courage  🔬  allodium  nietzschean  ideas  quixotic  accretion  learning  hi-order-bits 
february 2017 by nhaliday
Shtetl-Optimized » Blog Archive » Logicians on safari
So what are they then? Maybe it’s helpful to think of them as “quantitative epistemology”: discoveries about the capacities of finite beings like ourselves to learn mathematical truths. On this view, the theoretical computer scientist is basically a mathematical logician on a safari to the physical world: someone who tries to understand the universe by asking what sorts of mathematical questions can and can’t be answered within it. Not whether the universe is a computer, but what kind of computer it is! Naturally, this approach to understanding the world tends to appeal most to people for whom math (and especially discrete math) is reasonably clear, whereas physics is extremely mysterious.

the sequel: http://www.scottaaronson.com/blog/?p=153
tcstariat  aaronson  tcs  computation  complexity  aphorism  examples  list  reflection  philosophy  multi  summary  synthesis  hi-order-bits  interdisciplinary  lens  big-picture  survey  nibble  org:bleg  applications  big-surf  s:*  p:whenever  ideas 
january 2017 by nhaliday
ho.history overview - Proofs that require fundamentally new ways of thinking - MathOverflow
my favorite:
Although this has already been said elsewhere on MathOverflow, I think it's worth repeating that Gromov is someone who has arguably introduced more radical thoughts into mathematics than anyone else. Examples involving groups with polynomial growth and holomorphic curves have already been cited in other answers to this question. I have two other obvious ones but there are many more.

I don't remember where I first learned about convergence of Riemannian manifolds, but I had to laugh because there's no way I would have ever conceived of a notion. To be fair, all of the groundwork for this was laid out in Cheeger's thesis, but it was Gromov who reformulated everything as a convergence theorem and recognized its power.

Another time Gromov made me laugh was when I was reading what little I could understand of his book Partial Differential Relations. This book is probably full of radical ideas that I don't understand. The one I did was his approach to solving the linearized isometric embedding equation. His radical, absurd, but elementary idea was that if the system is sufficiently underdetermined, then the linear partial differential operator could be inverted by another linear partial differential operator. Both the statement and proof are for me the funniest in mathematics. Most of us view solving PDE's as something that requires hard work, involving analysis and estimates, and Gromov manages to do it using only elementary linear algebra. This then allows him to establish the existence of isometric embedding of Riemannian manifolds in a wide variety of settings.
q-n-a  overflow  soft-question  big-list  math  meta:math  history  insight  synthesis  gowers  mathtariat  hi-order-bits  frontier  proofs  magnitude  giants  differential  geometry  limits  flexibility  nibble  degrees-of-freedom  big-picture  novelty  zooming  big-surf  wild-ideas  metameta  courage  convergence  ideas  innovation  the-trenches  discovery  creative 
january 2017 by nhaliday
Reflections on the recent solution of the cap-set problem I | Gowers's Weblog
As regular readers of this blog will know, I have a strong interest in the question of where mathematical ideas come from, and a strong conviction that they always result from a fairly systematic process — and that the opposite impression, that some ideas are incredible bolts from the blue that require “genius” or “sudden inspiration” to find, is an illusion that results from the way mathematicians present their proofs after they have discovered them.
math  research  academia  gowers  hmm  mathtariat  org:bleg  nibble  big-surf  algebraic-complexity  math.CO  questions 
may 2016 by nhaliday

bundles : academegood-vibesmathstarsvague

related tags

aaronson  ability-competence  absolute-relative  abstraction  academia  accretion  acm  additive  additive-combo  advice  aging  ai  alg-combo  algebraic-complexity  algorithmic-econ  algorithms  allodium  alt-inst  announcement  aphorism  applications  approximation  arbitrage  arrows  baez  biases  big-list  big-peeps  big-picture  big-surf  bio  bits  borel-cantelli  caltech  candidate-gene  career  cartoons  characterization  checklists  chicago  circuits  cliometrics  cog-psych  commentary  communication  communication-complexity  complex-systems  complexity  composition-decomposition  computation  concept  conceptual-vocab  convergence  coordination  correlation  counting  courage  creative  crypto  cs  curiosity  cybernetics  decision-theory  deep-learning  deepgoog  definition  degrees-of-freedom  dementia  dennett  descriptive  detail-architecture  diet  differential  dimensionality  direction  dirty-hands  discovery  discussion  disease  education  embodied  empirical  epidemiology  epistemic  equilibrium  essay  events  evidence  evidence-based  examples  experiment  expert  expert-experience  explanans  fashun  feynman  finiteness  flexibility  flux-stasis  frontier  game-theory  generalization  geometry  giants  gowers  grad-school  gradient-descent  graph-theory  graphs  growth  GT-101  gtd  habit  hardness  health  heuristic  hi-order-bits  hierarchy  history  hmm  hsu  humanity  ideas  immune  impetus  info-dynamics  info-foraging  information-theory  innovation  insight  institutions  intelligence  interdisciplinary  interview  intricacy  intuition  israel  large-factor  learning  learning-theory  len:long  lens  lifts-projections  limits  linear-algebra  liner-notes  links  list  logic  long-term  longevity  low-hanging  lower-bounds  luca-trevisan  machine-learning  magnitude  markov  matching  math  math.CA  math.CO  math.GR  math.NT  mathtariat  measure  measurement  mechanics  medicine  meta:math  meta:medicine  meta:research  meta:science  metabuch  metameta  methodology  michael-nielsen  model-class  models  mostly-modern  motivation  multi  multiplicative  neuro  neuro-nitgrit  news  nibble  nietzschean  nitty-gritty  no-go  novelty  nutrition  obesity  objektbuch  occam  off-convex  open-problems  optimization  order-disorder  org:anglo  org:biz  org:bleg  org:health  org:inst  org:mag  org:mat  org:nat  org:rec  org:sci  overflow  p:someday  p:whenever  papers  parasites-microbiome  parsimony  path-dependence  pdf  phd  philosophy  physics  plots  poast  polynomials  popsci  population-genetics  pragmatic  preimage  preprint  princeton  probability  problem-solving  productivity  profile  proof-systems  proofs  properties  prudence  pseudorandomness  psychology  psychometrics  public-health  q-n-a  quantum  quantum-info  questions  quixotic  quotes  rand-approx  rand-complexity  red-queen  reference  reflection  reinforcement  relativization  religion  research  research-program  rhetoric  rigor  rigorous-crypto  roots  s:*  s:***  sampling  sanjeev-arora  scholar  science  scitariat  SDP  search  shannon  signal-noise  signaling  skeleton  slides  social-science  soft-question  space  spatial  speculation  speed  speedometer  spock  state-of-art  stories  strategy  street-fighting  structure  study  success  summary  survey  synthesis  tactics  talks  tcs  tcstariat  telos-atelos  the-bones  the-self  the-trenches  theory-of-mind  theory-practice  thick-thin  things  thinking  tidbits  time-complexity  todo  top-n  track-record  tricks  truth  turing  UGC  unit  universalism-particularism  valiant  values  video  virtu  vitality  volo-avolo  west-hunter  westminster  wigderson  wiki  wild-ideas  wire-guided  within-without  workshop  yoga  zooming  🌞  🎓  👳  🔬 

Copy this bookmark: