nhaliday + noise-structure   16

Could you explain the character of Fat Tony in Antifragile by Taleb? - Quora
Dr. John can make gigantic errors that affect other people by ignoring reality in favor of assumptions. Fat Tony makes smaller errors that affect only himself, but more seriously (they kill him).
q-n-a  qra  aphorism  jargon  analogy  narrative  blowhards  outcome-risk  noise-structure 
may 2017 by nhaliday
Deming regression - Wikipedia
Deming regression. The red lines show the error in both x and y. This is different from the traditional least squares method which measures error parallel to the y axis. The case shown, with deviations measured perpendicularly, arises when errors in x and y have equal variances.

stats  data-science  regression  methodology  direction  noise-structure  wiki  reference  nibble  multi 
may 2017 by nhaliday
Improving Economic Research | askblog
To make a long story short:

1. Economic phenomena are rife with causal density. Theories make predictions assuming “other things equal,” but other things are never equal.

2. When I was a student, the solution was thought to be multiple regression analysis. You entered a bunch of variables into an estimated equation, and in doing so you “controlled for” those variables and thereby created conditions of “other things equal.” However, in 1978, Edward Leamer pointed out that actual practice diverges from theory. The researcher typically undertakes a lot of exploratory data analysis before reporting a final result. This process of exploratory analysis creates a bias toward finding the result desired by the researcher, rather than achieving a scientific ideal of objectivity.

3. In recent decades, the approach has shifted toward “natural experiments” and laboratory experiments. These suffer from other problems. The experimental population may not be representative. Even if this problem is not present, studies that offer definitive results are more likely to be published but consequently less likely to be replicated.
econotariat  cracker-econ  study  summary  methodology  economics  causation  social-science  best-practices  academia  hypothesis-testing  thick-thin  density  replication  complex-systems  roots  noise-structure  endo-exo  info-dynamics  natural-experiment  endogenous-exogenous 
january 2017 by nhaliday
Not Final! | West Hunter
In mathematics we often prove that some proposition is true by showing that  the alternative is false.  The principle can sometimes work in other disciplines, but it’s tricky.  You have to have a very good understanding  to know that some things are impossible (or close enough to impossible).   You can do it fairly often in physics, less often in biology.
west-hunter  science  history  reflection  epistemic  occam  contradiction  parsimony  noise-structure  scitariat  info-dynamics  hetero-advantage  sapiens  evolution  disease  sexuality  ideas  genetics  s:*  thinking  the-trenches  no-go  thick-thin  theory-practice  inference  apollonian-dionysian 
november 2016 by nhaliday
Thick and thin | West Hunter
There is a spectrum of problem-solving, ranging from, at one extreme, simplicity and clear chains of logical reasoning (sometimes long chains) and, at the other, building a picture by sifting through a vast mass of evidence of varying quality. I will give some examples. Just the other day, when I was conferring, conversing and otherwise hobnobbing with my fellow physicists, I mentioned high-altitude lighting, sprites and elves and blue jets. I said that you could think of a thundercloud as a vertical dipole, with an electric field that decreased as the cube of altitude, while the breakdown voltage varied with air pressure, which declines exponentially with altitude. At which point the prof I was talking to said ” and so the curves must cross!”. That’s how physicists think, and it can be very effective. The amount of information required to solve the problem is not very large. I call this a ‘thin’ problem’.


In another example at the messy end of the spectrum, Joe Rochefort, running Hypo in the spring of 1942, needed to figure out Japanese plans. He had an an ever-growing mass of Japanese radio intercepts, some of which were partially decrypted – say, one word of five, with luck. He had data from radio direction-finding; his people were beginning to be able to recognize particular Japanese radio operators by their ‘fist’. He’d studied in Japan, knew the Japanese well. He had plenty of Navy experience – knew what was possible. I would call this a classic ‘thick’ problem, one in which an analyst needs to deal with an enormous amount of data of varying quality. Being smart is necessary but not sufficient: you also need to know lots of stuff.


Nimitz believed Rochefort – who was correct. Because of that, we managed to prevail at Midway, losing one carrier and one destroyer while the the Japanese lost four carriers and a heavy cruiser*. As so often happens, OP-20-G won the bureaucratic war: Rochefort embarrassed them by proving them wrong, and they kicked him out of Hawaii, assigning him to a floating drydock.

The usual explanation of Joe Rochefort’s fall argues that John Redman’s ( head of OP-20-G, the Navy’s main signals intelligence and cryptanalysis group) geographical proximity to Navy headquarters was a key factor in winning the bureaucratic struggle, along with his brother’s influence (Rear Admiral Joseph Redman). That and being a shameless liar.

Personally, I wonder if part of the problem is the great difficulty of explaining the analysis of a thick problem to someone without a similar depth of knowledge. At best, they believe you because you’ve been right in the past. Or, sometimes, once you have developed the answer, there is a ‘thin’ way of confirming your answer – as when Rochefort took Jasper Holmes’s suggestion and had Midway broadcast an uncoded complaint about the failure of their distillation system – soon followed by a Japanese report that ‘AF’ was short of water.

Most problems in the social sciences are ‘thick’, and unfortunately, almost all of the researchers are as well. There are a lot more Redmans than Rocheforts.
west-hunter  thinking  things  science  social-science  rant  problem-solving  innovation  pre-2013  metabuch  frontier  thick-thin  stories  intel  mostly-modern  history  flexibility  rigidity  complex-systems  metameta  s:*  noise-structure  discovery  applications  scitariat  info-dynamics  world-war  analytical-holistic  the-trenches  creative  theory-practice  being-right  management  track-record  alien-character  darwinian  old-anglo  giants  magnitude  intersection-connectedness  knowledge  alt-inst  sky  physics  electromag  oceans  military  statesmen  big-peeps  organizing  communication  fire  inference  apollonian-dionysian  consilience  bio  evolution 
november 2016 by nhaliday
Noise: dinosaurs, syphilis, and all that | West Hunter
Generally speaking, I thought the paleontologists were a waste of space: innumerate, ignorant about evolution, and simply not very smart.

None of them seemed to understand that a sharp, short unpleasant event is better at causing a mass extinction, since it doesn’t give flora and fauna time to adapt.

Most seemed to think that gradual change caused by slow geological and erosion forces was ‘natural’, while extraterrestrial impact was not. But if you look at the Moon, or Mars, or the Kirkwood gaps in the asteroids, or think about the KAM theorem, it is apparent that Newtonian dynamics implies that orbits will be perturbed, and that sometimes there will be catastrophic cosmic collisions. Newtonian dynamics is as ‘natural’ as it gets: paleontologists not studying it in school and not having much math hardly makes it ‘unnatural’.

One of the more interesting general errors was not understanding how to to deal with noise – incorrect observations. There’s a lot of noise in the paleontological record. Dinosaur bones can be eroded and redeposited well after their life times – well after the extinction of all dinosaurs. The fossil record is patchy: if a species is rare, it can easily look as if it went extinct well before it actually did. This means that the data we have is never going to agree with a perfectly correct hypothesis – because some of the data is always wrong. Particularly true if the hypothesis is specific and falsifiable. If your hypothesis is vague and imprecise – not even wrong – it isn’t nearly as susceptible to noise. As far as I can tell, a lot of paleontologists [ along with everyone in the social sciences] think of of unfalsifiability as a strength.

Done Quickly: https://westhunt.wordpress.com/2011/12/03/done-quickly/
I’ve never seen anyone talk about it much, but when you think about mass extinctions, you also have to think about rates of change

You can think of a species occupying a point in a many-dimensional space, where each dimension represents some parameter that influences survival and/or reproduction: temperature, insolation, nutrient concentrations, oxygen partial pressure, toxin levels, yada yada yada. That point lies within a zone of habitability – the set of environmental conditions that the species can survive. Mass extinction occurs when environmental changes are so large that many species are outside their comfort zone.

The key point is that, with gradual change, species adapt. In just a few generations, you can see significant heritable responses to a new environment. Frogs have evolved much greater tolerance of acidification in 40 years (about 15 generations). Some plants in California have evolved much greater tolerance of copper in just 70 years.

As this happens, the boundaries of the comfort zone move. Extinctions occur when the rate of environmental change is greater than the rate of adaptation, or when the amount of environmental change exceeds the limit of feasible adaptation. There are such limits: bar-headed geese fly over Mt. Everest, where the oxygen partial pressure is about a third of that at sea level, but I’m pretty sure that no bird could survive on the Moon.


Paleontologists prefer gradualist explanations for mass extinctions, but they must be wrong, for the most part.
disease  science  critique  rant  history  thinking  regularizer  len:long  west-hunter  thick-thin  occam  social-science  robust  parasites-microbiome  early-modern  parsimony  the-trenches  bounded-cognition  noise-structure  signal-noise  scitariat  age-of-discovery  sex  sexuality  info-dynamics  alt-inst  map-territory  no-go  contradiction  dynamical  math.DS  space  physics  mechanics  archaeology  multi  speed  flux-stasis  smoothness  evolution  environment  time  shift  death  nihil  inference  apollonian-dionysian  error  explanation  spatial  discrete  visual-understanding  consilience  traces  evidence 
september 2016 by nhaliday
natural language processing blog: Debugging machine learning
I've been thinking, mostly in the context of teaching, about how to specifically teach debugging of machine learning. Personally I find it very helpful to break things down in terms of the usual error terms: Bayes error (how much error is there in the best possible classifier), approximation error (how much do you pay for restricting to some hypothesis class), estimation error (how much do you pay because you only have finite samples), optimization error (how much do you pay because you didn't find a global optimum to your optimization problem). I've generally found that trying to isolate errors to one of these pieces, and then debugging that piece in particular (eg., pick a better optimizer versus pick a better hypothesis class) has been useful.
machine-learning  debugging  checklists  best-practices  pragmatic  expert  init  system-design  data-science  acmtariat  error  engineering  clarity  intricacy  model-selection  org:bleg  nibble  noise-structure  signal-noise  knowledge  accuracy  expert-experience  checking 
september 2016 by nhaliday

bundles : abstractpredictionthinking

related tags

academia  accuracy  acm  acmtariat  age-of-discovery  alien-character  alt-inst  analogy  analytical-holistic  aphorism  apollonian-dionysian  applications  archaeology  bangbang  being-right  berkeley  best-practices  bias-variance  big-peeps  bio  blowhards  books  bounded-cognition  causation  chapman  chart  checking  checklists  clarity  commentary  communication  comparison  complex-systems  concept  conceptual-vocab  consilience  contradiction  convexity-curvature  core-rats  cracker-econ  creative  critique  curvature  darwinian  data-science  death  debugging  decision-making  decision-theory  density  direction  discovery  discrete  disease  dynamical  early-modern  economics  econotariat  electromag  endo-exo  endogenous-exogenous  engineering  environment  epistemic  error  essay  evidence  evolution  examples  expert  expert-experience  explanation  finance  fire  flexibility  flux-stasis  frontier  gedanken  gelman  generalization  genetics  giants  hetero-advantage  history  hmm  hn  hypothesis-testing  ideas  idk  incentives  inference  info-dynamics  info-econ  init  innovation  intel  intelligence  interdisciplinary  intersection-connectedness  intricacy  jargon  knowledge  len:long  lens  lesswrong  machine-learning  magnitude  management  map-territory  marginal  markets  math.DS  measurement  mechanics  meta:prediction  metabuch  metameta  methodology  military  model-selection  models  mostly-modern  multi  narrative  natural-experiment  neurons  nibble  nihil  no-go  noise-structure  occam  oceans  old-anglo  ORFE  org:bleg  org:edge  org:edu  organizing  oscillation  outcome-risk  parasites-microbiome  parsimony  physics  postrat  pragmatic  pre-2013  prediction-markets  problem-solving  q-n-a  qra  rant  rationality  ratty  reference  reflection  regression  regularizer  replication  review  rigidity  risk  robust  roots  s:*  sapiens  science  scitariat  sex  sexuality  shift  signal-noise  sky  smoothness  social-science  space  spatial  speed  spock  statesmen  stats  stories  street-fighting  study  summary  system-design  systematic-ad-hoc  tails  techtariat  tetlock  the-trenches  theory-practice  thick-thin  things  thinking  time  traces  track-record  tradeoffs  tutorial  unaffiliated  uncertainty  vague  visual-understanding  water  west-hunter  wiki  world-war  🎩 

Copy this bookmark: