nhaliday + stock-flow   52

performance - What is the difference between latency, bandwidth and throughput? - Stack Overflow
Latency is the amount of time it takes to travel through the tube.
Bandwidth is how wide the tube is.
The amount of water flow will be your throughput

Vehicle Analogy:

Container travel time from source to destination is latency.
Container size is bandwidth.
Container load is throughput.

--

Note, bandwidth in particular has other common meanings, I've assumed networking because this is stackoverflow but if it was a maths or amateur radio forum I might be talking about something else entirely.
q-n-a  stackex  programming  IEEE  nitty-gritty  definition  jargon  network-structure  metrics  speedometer  time  stock-flow  performance  latency-throughput  amortization-potential  thinking 
may 2019 by nhaliday
A genetic map of the world – Gene Expression
The above map is from a new preprint on the patterns of genetic variation as a function of geography for humans, Genetic landscapes reveal how human genetic diversity aligns with geography. The authors assemble an incredibly large dataset to generate these figures. The orange zones are “troughs” of gene flow. Basically barriers to gene flow.  It is no great surprise that so many of the barriers correlate with rivers, mountains, and deserts. But the aim of this sort of work seems to be to make precise and quantitative intuitions which are normally expressed verbally.
gnxp  scitariat  commentary  bio  preprint  study  summary  genetics  genomics  pic  data  maps  visualization  stock-flow  gene-flow  walls  world  africa  europe  MENA  india  asia  china  japan  korea  anglo  developing-world  russia  sapiens  gavisti  population-genetics  geography  🌞  mediterranean  britain  oceans  race  tribalism 
december 2017 by nhaliday
Autoignition temperature - Wikipedia
The autoignition temperature or kindling point of a substance is the lowest temperature at which it spontaneously ignites in normal atmosphere without an external source of ignition, such as a flame or spark. This temperature is required to supply the activation energy needed for combustion. The temperature at which a chemical ignites decreases as the pressure or oxygen concentration increases. It is usually applied to a combustible fuel mixture.

The time {\displaystyle t_{\text{ig}}} {\displaystyle t_{\text{ig}}} it takes for a material to reach its autoignition temperature {\displaystyle T_{\text{ig}}} {\displaystyle T_{\text{ig}}} when exposed to a heat flux {\displaystyle q''} {\displaystyle q''} is given by the following equation:
nibble  wiki  reference  concept  metrics  identity  physics  thermo  temperature  time  stock-flow  phys-energy  chemistry  article  street-fighting  fire  magnitude  data  list 
november 2017 by nhaliday
Tax Evasion and Inequality
This paper attempts to estimate the size and distribution of tax evasion in rich countries. We combine stratified random audits—the key source used to study tax evasion so far—with new micro-data leaked from two large offshore financial institutions, HSBC Switzerland (“Swiss leaks”) and Mossack Fonseca (“Panama Papers”). We match these data to population-wide wealth records in Norway, Sweden, and Denmark. We find that tax evasion rises sharply with wealth, a phenomenon that random audits fail to capture. On average about 3% of personal taxes are evaded in Scandinavia, but this figure rises to about 30% in the top 0.01% of the wealth distribution, a group that includes households with more than $40 million in net wealth. A simple model of the supply of tax evasion services can explain why evasion rises steeply with wealth. Taking tax evasion into account increases the rise in inequality seen in tax data since the 1970s markedly, highlighting the need to move beyond tax data to capture income and wealth at the top, even in countries where tax compliance is generally high. We also find that after reducing tax evasion—by using tax amnesties—tax evaders do not legally avoid taxes more. This result suggests that fighting tax evasion can be an effective way to collect more tax revenue from the ultra-wealthy.

Figure 1

America’s unreported economy: measuring the size, growth and determinants of income tax evasion in the U.S.: https://link.springer.com/article/10.1007/s10611-011-9346-x
This study empirically investigates the extent of noncompliance with the tax code and examines the determinants of federal income tax evasion in the U.S. Employing a refined version of Feige’s (Staff Papers, International Monetary Fund 33(4):768–881, 1986, 1989) General Currency Ratio (GCR) model to estimate a time series of unreported income as our measure of tax evasion, we find that 18–23% of total reportable income may not properly be reported to the IRS. This gives rise to a 2009 “tax gap” in the range of $390–$540 billion. As regards the determinants of tax noncompliance, we find that federal income tax evasion is an increasing function of the average effective federal income tax rate, the unemployment rate, the nominal interest rate, and per capita real GDP, and a decreasing function of the IRS audit rate. Despite important refinements of the traditional currency ratio approach for estimating the aggregate size and growth of unreported economies, we conclude that the sensitivity of the results to different benchmarks, imperfect data sources and alternative specifying assumptions precludes obtaining results of sufficient accuracy and reliability to serve as effective policy guides.
pdf  study  economics  micro  evidence-based  data  europe  nordic  scale  class  compensation  money  monetary-fiscal  political-econ  redistribution  taxes  madisonian  inequality  history  mostly-modern  natural-experiment  empirical  🎩  cocktail  correlation  models  supply-demand  GT-101  crooked  elite  vampire-squid  nationalism-globalism  multi  pro-rata  usa  time-series  trends  world-war  cold-war  government  todo  planning  long-term  trivia  law  crime  criminology  estimate  speculation  measurement  labor  macro  econ-metrics  wealth  stock-flow  time  density  criminal-justice  frequency  dark-arts  traces  evidence 
october 2017 by nhaliday
German Election: The AfD Profits from Non-Voters and Merkel Defectors | ZEIT ONLINE
The conservatives and center-left suffered huge losses while the right-wing populist AfD became Saxony's most powerful party. We analyzed the data so you don’t have to.

The anti-Muslim AfD just scored big in Germany’s election. What does this mean for German Muslims?: https://www.washingtonpost.com/news/monkey-cage/wp/2017/09/25/the-anti-muslim-afd-just-scored-big-in-germanys-election-what-does-this-mean-for-german-muslims/

The Germans Turn Right: http://www.weeklystandard.com/the-germans-turn-right/article/2009880
- Christopher Caldwell

The German Election—A Conservative Analysis: http://quillette.com/2017/09/28/german-election-conservative-analysis/
As traditional European conservatives were pushed Left-ward by media and academic communities on immigration and social and religious rights, the vacuum was filled by the rise of blut-und-boden far right parties all across Europe. In a Sophoclean twist of fate, the European political center and center-Left is now dying, thanks to the very person that liberals like to portray as the “new leader” of the free world.

later:
The End of German Stability: http://www.slate.com/articles/news_and_politics/the_good_fight/2017/11/germany_s_coalition_talks_collapse_threatening_stability.html
The collapse of coalition talks bodes badly for Angela Merkel, and for democratic governments everywhere.

Austrian legislative election, 2017: https://en.wikipedia.org/wiki/Austrian_legislative_election,_2017
Austria heads for right-leaning coalition: http://www.politico.eu/article/austria-heads-for-right-leaning-coalition-early-projections/
https://twitter.com/almodozo/status/919662426966118403
https://archive.is/4vkG6
Finally an election where the far right actually did best (barely) among youngest voters: FPÖ got 30% of the <30 vote. SPÖ voters old #nrw17

European Populism Is Here to Stay: https://www.nytimes.com/2017/10/20/opinion/european-populism-is-here-to-stay.html

Czech Republic:
http://www.nydailynews.com/newswires/news/world/populist-billionaire-party-wins-big-czech-republic-article-1.3578312
https://www.reuters.com/article/us-czech-election-farright/far-right-scores-surprise-success-in-czech-election-idUSKBN1CQ0T3

Dealing with the Dignity Deficit: https://www.the-american-interest.com/2018/04/05/dealing-dignity-deficit/
Are the grievances tearing our societies apart at this hyper-polarized moment insurmountable? A week with some AfD voters and politicians gave me a measure of hope.

More importantly, a generous welfare state appears at best to be an inadequate solution to the problems and pathologies of checker-boarded economic dislocation. The pesky question of dignity remains: Despite what boosters of a universal basic income try to tell us, our modern sense of purpose and identity remains closely tied to what we “do” for a living. Making disempowered people more comfortable does not necessarily make them less frustrated.

...

But since when are openness, cosmopolitanism, and diversity intrinsic to a complete definition of liberalism? They make no significant appearance in any of the foundational texts of liberal political thought. They are merely the product of the demands that globalization and urbanization places upon us “anywheres”, which we in turn try to wedge into the liberal canon in order to erase any discomfort we may feel about change pulling the ideological rug out from under our feet. Historical accounts of earlier periods of urbanization are rightly dispassionate in describing people flocking to cities out of economic want. And they correctly identify the emergence of the middle class and its attendant values as the product of complex factors interacting in these new circumstances. We instead now disfigure analysis into a form or moral self-congratulation.
news  org:euro  data  analysis  visualization  europe  germanic  politics  elections  postmortem  demographics  class  compensation  urban  education  correlation  gender  gender-diff  populism  stock-flow  flux-stasis  nationalism-globalism  phalanges  multi  org:rec  islam  migration  migrant-crisis  shift  org:mag  right-wing  journos-pundits  douthatish  org:popup  strategy  coalitions  current-events  wiki  tracker  twitter  social  commentary  eastern-europe  backup  age-generation  org:lite  japan  asia  prediction  long-short-run  org:local  urban-rural  EU  dignity  open-closed  redistribution  welfare-state  labor  economics  roots  ideology 
september 2017 by nhaliday
tcjfs on Twitter: "Yearly legal permanent residencies 1996-2015 with a bit more disaggregated and common-sensical designations than DHS https://t.co/167ms5Xr0s"
https://archive.is/70nNG
https://twitter.com/tcjfs/status/900052649147543552
https://archive.is/5U3Mi
Asian origin according to Department of Homeland Security
not sure tbh. i was just trying to disaggregate "Asian immigration" and I was like holy shit some of these places I would never include

U.S. Lawful Permanent Residents: 2014: https://www.dhs.gov/sites/default/files/publications/Lawful_Permanent_Residents_2014.pdf
Yearbook of Immigration Statistics: https://www.dhs.gov/immigration-statistics/yearbook

https://twitter.com/tcjfs/status/933066198161469440
https://archive.is/pRTqS
Foreign born population by Chinese, Indian, Mexican birth whose residence one year ago was abroad, 2000-2013
The above chart, extended to 2000-2016, with Mexico but also all of Latin/Central/South America:
our latin american immigrants are probably getting less "huwhite"
gnon  unaffiliated  right-wing  twitter  social  discussion  data  analysis  visualization  migration  usa  history  mostly-modern  time-series  scale  distribution  world  developing-world  latin-america  india  asia  china  backup  government  intricacy  gotchas  demographics  population  multi  race  stock-flow  org:gov  white-paper  pdf  the-west  california  northeast  nyc  list  top-n  database  age-generation  gender  pop-structure  genetics  genomics  maps  pro-rata 
september 2017 by nhaliday
Lecture 3: Global Energy Cycle
solar flux, albedo, greenhouse effect, energy balance, vertical distribution of energy, tilt and seasons
pdf  slides  nibble  physics  electromag  space  earth  sky  atmosphere  environment  temperature  stock-flow  data  magnitude  scale  phys-energy  distribution  oscillation  cycles  lectures  geography 
august 2017 by nhaliday
IMMIGRATION DRIVES U.S. POPULATION GROWTH
pdf  white-paper  data  analysis  demographics  migration  variance-components  population  trends  usa  org:ngo  multi  news  org:data  distribution  org:lite  right-wing  commentary  current-events  trump  nascent-state  politics  policy  wonkish  government  twitter  social  gnon  labor  human-capital  unaffiliated  fertility  chart  🎩  pic  visualization  org:rec  age-generation  race  the-bones  zeitgeist  law  scale  stock-flow  backup 
july 2017 by nhaliday
Levels or changes?: Ethnic context, immigration and the UK Independence party vote
It argues that high levels of established ethnic minorities reduce opposition to immigration and support for UKIP among White Britons. Conversely, more rapid ethnic changes increase opposition to immigration and support for UKIP. Longitudinal data demonstrates that these effects are not produced by self-selection.
study  sociology  politics  polisci  government  elections  britain  eric-kaufmann  causation  endo-exo  migration  diversity  putnam-like  populism  longitudinal  stock-flow  marginal  us-them  identity-politics  flux-stasis  wonkish  phalanges  endogenous-exogenous 
may 2017 by nhaliday
Why isn't ethanol used more widely as an automobile fuel? - Quora
Some of these are straightforward engineering challenges, but others are fundamental and insurmountable flaws.

Simply put, corn-based ethanol is bad energy policy. We're already at the maximum blending level in the US (10%) where any further increases will likely harm consumers. There's little environmental OR energy security benefit to ethanol, because it takes about a gallon worth of fossil fuels to produce a gallon of ethanol. (Studies vary on the exact number, ranging from an EROI of 0.8 to 1.3, but the average is zero net gain.)

Basically US ethanol blending is an enormous make-work scheme for big corporate farm interests and Iowa co-ops. It is an enormous sop to the key farm lobby in early Presidential Primary states. That's the only real reason we make vast quantities of corn-based ethanol and mandate its use in the US. Nobody would want to buy the stuff if it weren't for legal blending requirements imposed on refiners and fuel distributors.

Again, some countries can do ethanol better. Climate is important to biofuels. But it is not a good fuel choice for most nations.

- Ryan Carlyle
q-n-a  qra  expert  explanation  ethanol  energy-resources  stock-flow  transportation  heavy-industry  biophysical-econ  expert-experience 
may 2017 by nhaliday
What is the likelihood we run out of fossil fuels before we can switch to renewable energy sources? - Quora
1) Can we de-carbon our primary energy production before global warming severely damages human civilization? In the short term this means switching from coal to natural gas, and in the long term replacing both coal and gas generation with carbon-neutral sources such as renewables or nuclear. The developed world cannot accomplish this alone -- it requires worldwide action, and most of the pain will be felt by large developing nations such as India and China. Ultimately this is a political and economic problem. The technology to eliminate most carbon from electricity generation exists today at fairly reasonable cost.

2) Can we develop a better transportation energy storage technology than oil, before market forces drive prices to levels that severely damage the global economy? Fossil fuels are a source of energy, but primarily we use oil in vehicles because it is an exceptional energy TRANSPORT medium. Renewables cannot meet this need because battery technology is completely uncompetitive for most fuel consumers -- prices are an order of magnitude too high and energy density is an order of magnitude too low for adoption of all-electric vehicles outside developed-world urban centers. (Heavy trucking, cargo ships, airplanes, etc will never be all-electric with chemical batteries. There are hard physical limits to the energy density of electrochemical reactions. I'm not convinced passenger vehicles will go all-electric in our lifetimes either.) There are many important technologies in existence that will gain increasing traction in the next 50 years such as natural gas automobiles and improved gas/electric hybrids, but ultimately we need a better way to store power than fossil fuels. _This is a deep technological problem that will not be solved by incremental improvements in battery chemistry or any process currently in the R&D pipeline_.

Based on these two unresolved issues, _I place the odds of us avoiding fossil-fuel-related energy issues (major climate or economic damage) at less than 10%_. The impetus for the major changes required will not be sufficiently urgent until the world is seeing severe and undeniable impacts. Civilization will certainly survive -- but there will be no small amount of human suffering during the transition to whatever comes next.

- Ryan Carlyle
q-n-a  qra  expert  energy-resources  climate-change  environment  risk  civilization  nihil  prediction  threat-modeling  world  futurism  biophysical-econ  stock-flow  transportation  technology  economics  long-short-run  no-go  speedometer  modernity  expert-experience 
may 2017 by nhaliday
[1502.05274] How predictable is technological progress?
Recently it has become clear that many technologies follow a generalized version of Moore's law, i.e. costs tend to drop exponentially, at different rates that depend on the technology. Here we formulate Moore's law as a correlated geometric random walk with drift, and apply it to historical data on 53 technologies. We derive a closed form expression approximating the distribution of forecast errors as a function of time. Based on hind-casting experiments we show that this works well, making it possible to collapse the forecast errors for many different technologies at different time horizons onto the same universal distribution. This is valuable because it allows us to make forecasts for any given technology with a clear understanding of the quality of the forecasts. As a practical demonstration we make distributional forecasts at different time horizons for solar photovoltaic modules, and show how our method can be used to estimate the probability that a given technology will outperform another technology at a given point in the future.

model:
- p_t = unit price of tech
- log(p_t) = y_0 - μt + ∑_{i <= t} n_i
- n_t iid noise process
preprint  study  economics  growth-econ  innovation  discovery  technology  frontier  tetlock  meta:prediction  models  time  definite-planning  stylized-facts  regression  econometrics  magnitude  energy-resources  phys-energy  money  cost-benefit  stats  data-science  🔬  ideas  speedometer  multiplicative  methodology  stochastic-processes  time-series  stock-flow  iteration-recursion  org:mat  street-fighting  the-bones 
april 2017 by nhaliday
Charles Joseph Minard - Wikipedia
Charles Minard's map of Napoleon's disastrous Russian campaign of 1812. The graphic is notable for its representation in two dimensions of six types of data: the number of Napoleon's troops; distance; temperature; the latitude and longitude; direction of travel; and location relative to specific dates.[2]
people  history  early-modern  europe  gallic  war  russia  dataviz  visualization  wiki  maps  stock-flow  military  classic 
april 2017 by nhaliday
O Canada! | West Hunter
Imagine a country with an average IQ of 100, some average amount of education (with some distribution), some average amount of capital per head (with some distribution of ownership of capital). Now add immigrants – 10% of the population – that are the same in every way. Same average IQ, same distribution of IQ, same average amount of capital and same distribution. They speak the same language. They have similar political traditions. In other words, it is as if the US had just peacefully annexed an imaginary country that’s a lot like Canada.

Would the original inhabitants gain economically from this merger? Strikes me that this could only happen from economies of scale – since nothing has changed other than a 10% increase in overall size. There might be some diseconomies of scale as well. I wouldn’t expect a big payoff. Except for Nawapa, of course.

Contrast this with a situation in which the extra 10% is fairly different – lower average IQ, much less education on average, don’t speak English. They don’t bring along a lot of capital. They have and bring along their native political traditions, like everyone, but theirs stink. I can easily see how those immigrants might have improved their economic lot but it’s kindof hard to see how bringing in people with low human capital benefits the original citizens more than bringing in people with considerably higher human capital. Yet it must, because adding more of the same clearly has a small effect, while adding in lower-skilled must have a big positive effect. Practically all the economists say so.

https://westhunt.wordpress.com/2017/04/18/o-canada/#comment-90631
place of birth for the foreign-born population of the US, 2013:
all of Latin America, ~25 million China, ~2.5 million

https://westhunt.wordpress.com/2017/04/18/o-canada/#comment-90632
Caplan’s full of shit. Prosperity through favelas? Hasn’t worked anywhere else.

https://westhunt.wordpress.com/2017/04/18/o-canada/#comment-90800
The countries that look somewhat like our likely demographic destination ( considering recent trends) do worse economically than the United States, including the subgroups with high human capital. Brazil, say.

On the other hand, if you’re talking positional wealth, bringing in people with low human capital definitely works. Servants.

Sponsor An Immigrant Yourself: https://www.politico.com/magazine/story/2018/02/13/immigration-visas-economics-216968
No, really: A new kind of visa would let individual Americans—instead of corporations—reap the economic benefits of migration.

https://twitter.com/NoTrueScotist/status/963566542049832960
https://archive.is/FGQrp
I’ve always wanted my own sla—immigrant.......
I feel like people are neglecting the fact that this was written by Eric Posner....
west-hunter  scitariat  discussion  rant  gedanken  thinking  being-right  realness  propaganda  economics  migration  assimilation  iq  hive-mind  canada  usa  latin-america  china  asia  poll  growth-econ  critique  policy  econotariat  cracker-econ  commentary  human-capital  islam  pop-diff  multi  poast  aphorism  clown-world  vampire-squid  biophysical-econ  microfoundations  attaq  wealth-of-nations  impetus  demographics  elite  population  stock-flow  class  homo-hetero  garett-jones  spearhead  pseudoE  courage  fluid  geoengineering  environment  news  org:mag  crooked  class-warfare  proposal  alt-inst  markets  randy-ayndy  corporation  twitter  social  backup  gnon  🐸 
april 2017 by nhaliday
Overview of current development in electrical energy storage technologies and the application potential in power system operation
- An overview of the state-of-the-art in Electrical Energy Storage (EES) is provided.
- A comprehensive analysis of various EES technologies is carried out.
- An application potential analysis of the reviewed EES technologies is presented.
- The presented synthesis to EES technologies can be used to support future R&D and deployment.

Prospects and Limits of Energy Storage in Batteries: http://pubs.acs.org/doi/abs/10.1021/jz5026273
study  survey  state-of-art  energy-resources  heavy-industry  chemistry  applications  electromag  stock-flow  wonkish  frontier  technology  biophysical-econ  the-world-is-just-atoms  🔬  phys-energy  ideas  speedometer  dirty-hands  multi 
april 2017 by nhaliday
Evangelical Protestants Are The Biggest Winners When People Change Faiths | FiveThirtyEight
https://twitter.com/lymanstoneky/status/1187666230389305344
https://archive.is/eWF0w
Without immigration, the Catholicism would be declining even faster than the mainlines. That's the unavoidable conclusion from RCC's own data on baptisms as well.
And, to be clear, it's not because of unusually high rates of people leaving the church. It's because of peculiarly low rates of conversion *into* the church.
...
Nor is Catholic fertility particularly high. Here's completed fertility by religious tradition compared to what they would have needed to have in order to grow.
news  org:data  usa  data  demographics  religion  culture  fertility  equilibrium  analysis  sociology  other-xtian  stock-flow  ratty  distribution  population  counter-revolution  nascent-state  tradition  christianity  spreading  multi  twitter  social  commentary  discussion  backup  journos-pundits  douthatish  econotariat  wonkish  visualization  protestant-catholic  trends  phalanges  migration  latin-america  increase-decrease 
february 2017 by nhaliday
Give me your caffeinated, your well-off, your above-average yearning to get ahead – spottedtoad
Even though the mean has shifted only 0.2 standard deviations to the left, the % of the population that exceeds certain benchmarks has dropped considerably: there are only 58% as many people who exceed 1 standard deviation above the starting mean, 38% as many who exceed 2 standard deviations, and less than 25% as many who exceed 3 standard deviations as before the 30 years. This obviously influences the percentage of the population who can become competent government workers, or doctors or teachers or college professors or engineers.

this kinda argument always seems like concern trolling to me tho

see https://pinboard.in/u:nhaliday/b:2fc26306583e for some relevant calculations (need to take into account heritability too)

Brain Drain: Socio-Economic Impact on Indian Society: http://www.ijhssi.org/papers/v2(5)/version-3/C251217.pdf

https://spottedtoad.wordpress.com/2018/01/12/digging-your-way-out-of-a-hole/
ratty  unaffiliated  migration  world  developing-world  distribution  models  iq  unintended-consequences  selection  elite  gene-flow  inequality  dynamical  nationalism-globalism  chart  stock-flow  human-capital  multi  pdf  study  economics  india  asia  europe  EU  labor  diaspora  trump  current-events  africa 
february 2017 by nhaliday
Lecture 11
In which we prove that the Edmonds-Karp algorithm for maximum flow is a strongly polynomial time algorithm, and we begin to talk about the push-relabel approach.
pdf  lecture-notes  exposition  optimization  algorithms  linear-programming  graphs  stanford  luca-trevisan  nibble  direction  stock-flow  tcs  constraint-satisfaction  tcstariat 
january 2017 by nhaliday
Lecture 16
In which we define a multi-commodity flow problem, and we see that its dual is the relaxation of a useful graph partitioning problem. The relaxation can be rounded to yield an approximate graph partitioning algorithm.
pdf  lecture-notes  exposition  optimization  linear-programming  graphs  graph-theory  algorithms  duality  rounding  stanford  approximation  rand-approx  luca-trevisan  relaxation  nibble  stock-flow  constraint-satisfaction  tcs  tcstariat 
january 2017 by nhaliday
Dgsh – Directed graph shell | Hacker News
I've worked with and looked at a lot of data processing helpers. Tools, that try to help you build data pipelines, for the sake of performance, reproducibility or simply code uniformity.
What I found so far: Most tools, that invent a new language or try to cram complex processes into lesser suited syntactical environments are not loved too much.

...

I'll give dgsh a try. The tool reuse approach and the UNIX spirit seems nice. But my initial impression of the "C code metrics" example from the site is mixed: It reminds me of awk, about which one of the authors said, that it's a beautiful language, but if your programs getting longer than hundred lines, you might want to switch to something else.

Two libraries which have a great grip at the plumbing aspect of data processing systems are airflow and luigi. They are python libraries and with it you have a concise syntax and basically all python libraries plus non-python tools with a command line interface at you fingertips.

I am curious, what kind of process orchestration tools people use and can recommend?

--

Exactly our experience too, from complex machine learning workflows in various aspects of drug discovery.
We basically did not really find any of the popular DSL-based bioinformatics pipeline tools (snakemake, bpipe etc) to fit the bill. Nextflow came close, but in fact allows quite some custom code too.

What worked for us was to use Spotify's Luigi, which is a python library rather than DSL.

The only thing was that we had to develop a flow-based inspired API on top of Luigi's more functional programming based one, in order to make defining dependencies fluent and easy enough to specify for our complex workflows.

Our flow-based inspired Luigi API (SciLuigi) for complex workflows, is available at:

https://github.com/pharmbio/sciluigi

--

We have measured many of the examples against the use of temporary files and the web report one against (single-threaded) implementations in Perl and Java. In almost all cases dgsh takes less wall clock time, but often consumes more CPU resources.
commentary  project  programming  terminal  worrydream  pls  plt  unix  hn  graphs  tools  devtools  let-me-see  composition-decomposition  yak-shaving  workflow  exocortex  hmm  cool  software  desktop  sci-comp  stock-flow  performance  comparison  links  libraries  python 
january 2017 by nhaliday
The Impact of Immigrants on Public Finances: A Forecast Analysis for Denmark
All over Europe, ageing populations threaten nations’ financial sustainability. In this paper we examine the potential of immigration to strengthen financial sustainability. We look at a particularly challenging case, namely that of Denmark, which has extensive tax-financed welfare programmes that provide a high social safety net. The analysis is based on a forecast for the entire Danish economy made using a dynamic computable general equilibrium model with overlapping generations. Net contributions to the public purse are presented both as cross-sectional figures for a long time horizon and as average individual life-cycle contributions. The main conclusion is that immigrants from richer countries have a positive fiscal impact, while immigrants from poorer countries have a large negative one. The negative effect is caused by both a weak labour market performance and early retirement in combination with the universal Danish welfare schemes

In Denmark, 84 Per Cent Of Welfare Recipients Are ‘Non-Western Immigrants: http://www.breitbart.com/london/2016/03/17/in-denmark-84-per-cent-of-welfare-recipients-are-non-western-immigrants/

Time favors them not: Some migrant groups have low employment rates even after 25+ years of residence: https://medium.com/@afn/time-favors-them-not-some-migrant-groups-have-low-employment-rates-even-after-25-years-of-3c3e36094108
In Britain, for example, social problems that may point to religious and ethnic divides are treated as mostly taboo, and the Office of National Statistics has decided on an categorization of ethnicity which admirably combines almost every category error one could come up with: it juxtaposes races, cultural groups, single national origins, and a continent as supposedly mutually exclusive categories. In France, ethnicity is largely absent from national statistics as everyone with a passport becomes a citoyen and hence a Frenchman by pure principle. Once again, Danish data becomes helpful. The Danish state has no qualms about analyzing the connections between national origins and other aspects of its citizens’ lives. It does so in population-wide registry databases that links everything from tax records over medical journals, criminal records, school records to civil status (the data contain no information on subjective measures such as sexuality, religion, or politics). I analyzed some of these aggregated labor market data to show how specific national-origin groups do on the Danish labor market. They show considerable differences in outcomes.

THE FISCAL EFFECTS OF IMMIGRATION TO THE UK*: http://www.cream-migration.org/files/FiscalEJ.pdf
Our findings indicate that, when considering the resident immigrant population in each year from 1995 to 2011, immigrants from the European Economic Area (EEA) have made a positive fiscal contribution, even during periods when the UK was running budget deficits, while Non-EEA immigrants, not dissimilar to natives, have made a negative contribution. For immigrants that arrived since 2000, contributions have been positive throughout, and particularly so for immigrants from EEA countries. Notable is the strong positive contribution made by immigrants from countries that joined the EU in 2004.

Table 1 has population numbers, Table 6 has fiscal impact

Only 13 percent of recent refugees in Germany have found work: survey: http://www.reuters.com/article/us-europe-migrants-germany-survey-idUSKBN13A22F
Most refugees to be jobless for years, German minister warns: https://www.ft.com/content/022de0a4-54f4-11e7-9fed-c19e2700005f
Up to three quarters of Germany’s refugees will still be unemployed in five years’ time, according to a government minister, in a stark admission of the challenges the country faces in integrating its huge migrant population.

The Welfare Magnet Hypothesis: Evidence From an Immigrant Welfare Scheme in Denmark∗: https://www.henrikkleven.com/uploads/3/7/3/1/37310663/agersnap-jensen-kleven_welfaremagnets_oct2019.pdf
We study the effects of welfare generosity on international migration using a series of large
changes in welfare benefits for immigrants in Denmark.
https://twitter.com/DurRobert/status/1190976044238540801
https://archive.is/FNNFD
study  pdf  economics  prediction  europe  migrant-crisis  equilibrium  polisci  nordic  macro  econometrics  diversity  redistribution  migration  wonkish  monetary-fiscal  intervention  free-riding  human-capital  stylized-facts  patho-altruism  org:ngo  assimilation  welfare-state  white-paper  multi  news  org:lite  lurid  clown-world  government  data  current-events  culture-war  org:rec  org:anglo  org:biz  labor  germanic  chart  org:med  sociology  attaq  measurement  britain  money  cost-benefit  EU  regularizer  🎩  demographics  population  right-wing  policy  garett-jones  natural-experiment  endogenous-exogenous  stock-flow  effect-size  twitter  social  commentary  backup  summary 
december 2016 by nhaliday
Psychological comments: Does Age make us sage or sag?
Khan on Twitter: "figure on right from @tuckerdrob lab is depressing (the knowledge plateau). do i read in vain??? https://t.co/DZzBD8onEv": https://twitter.com/razibkhan/status/809439911627493377
- reasoning rises then declines after age ~20
- knowledge plateaus by age 35-40
- different interpretation provided by study authors w/ another graph (renewal)
- study (can't find the exact graph anywhere): http://www.iapsych.com/wj3ewok/LinkedDocuments/McArdle2002.pdf

School’s out: https://westhunt.wordpress.com/2016/12/29/schools-out/
I saw a note by Razib Khan, in which he mentioned that psychometric research suggests that people plateau in their knowledge base as adults. I could believe it. But I’m not sure it’s true in my case. One might estimate total adult knowledge in terms of BS equivalents…

Age-related IQ decline is reduced markedly after adjustment for the Flynn effect: https://www.ncbi.nlm.nih.gov/m/pubmed/20349385/
Twenty-year-olds outperform 70-year-olds by as much as 2.3 standard deviations (35 IQ points) on subtests of the Wechsler Adult Intelligence Scale (WAIS). We show that most of the difference can be attributed to an intergenerational rise in IQ known as the Flynn effect.

...

For these verbal subtests, the Flynn effect masked a modest increase in ability as individuals grow older.

Predictors of ageing-related decline across multiple cognitive functions: http://www.sciencedirect.com/science/article/pii/S0160289616302707
Cognitive ageing is likely a process with few large-effect predictors

A strong link between speed of visual discrimination and cognitive ageing: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4123160/
Results showed a moderate correlation (r = 0.460) between inspection time performance and intelligence, and a strong correlation between change in inspection time and change in intelligence from 70 to 76 (r = 0.779). These results support the processing speed theory of cognitive ageing. They go beyond cross-sectional correlation to show that cognitive change is accompanied by changes in basic visual information processing as we age.
albion  psychology  cog-psych  psychometrics  aging  iq  objektbuch  long-term  longitudinal  study  summary  variance-components  scitariat  multi  gnxp  learning  metabuch  twitter  social  discussion  pic  data  planning  tradeoffs  flux-stasis  volo-avolo  west-hunter  studying  knowledge  age-generation  flexibility  rigidity  plots  manifolds  universalism-particularism  being-becoming  essence-existence  intelligence  stock-flow  large-factor  psych-architecture  visuo  correlation  time  speed  short-circuit  roots  flynn  trends  dysgenics  language  explanans  direction  chart 
december 2016 by nhaliday
The intelligent inheriting the earth | EVOLVING ECONOMICS
Will the intelligent inherit the earth? IQ and time preference in the global economy: http://mason.gmu.edu/~gjonesb/IITE.pdf
IQ in the Utility Function: Cognitive skills, time preference, and cross-country differences in savings rates: http://mason.gmu.edu/~gjonesb/IQsavings.pdf

"Thus, all countries except the most patient have negative net worth and negative holdings of net foreign assets: the most patient country holds title to all capital flows from the less patient. Consumption per unit of effective labor approaches zero (kept from zero consumption only by the Inada condition) because income flows are devoted to debt repayment. Indeed, in steady state all but the most patient country have savings rates near 100%—but these savings are mere debt repayments to the most patient country. In this steady state, the most patient country (or more realistically, countries) would continue to consume a non-negligible amount and would have the lowest savings rate(s) in the world."

China’s Creditor Imperialism: https://www.project-syndicate.org/commentary/china-sri-lanka-hambantota-port-debt-by-brahma-chellaney-2017-12
study  summary  economics  human-capital  hive-mind  🌞  🎩  econotariat  multi  spearhead  cracker-econ  time-preference  garett-jones  biophysical-econ  monetary-fiscal  science-anxiety  sinosphere  asia  china  models  network-structure  long-short-run  sociology  patience  winner-take-all  debt  money  psychology  cog-psych  big-peeps  capital  equilibrium  stylized-facts  s:*  nationalism-globalism  temperance  broad-econ  wealth  group-level  world  rindermann-thompson  slippery-slope  government  pop-diff  chart  stock-flow  commentary  life-history  wealth-of-nations  behavioral-econ  kumbaya-kult  microfoundations  alien-character  natural-experiment  supply-demand  anthropology  cultural-dynamics  expansionism  news  org:mag  org:davos  hari-seldon 
november 2016 by nhaliday

bundles : abstract

related tags

acm  advanced  adversarial  advice  africa  age-generation  aging  agriculture  ai  ai-control  albion  algorithms  alien-character  alignment  allodium  alt-inst  amazon  amortization-potential  analogy  analysis  analytical-holistic  anglo  anglosphere  announcement  anthropology  antidemos  antiquity  aphorism  apollonian-dionysian  apple  applications  approximation  archaeology  architecture  aristos  arrows  art  article  asia  assimilation  atmosphere  attaq  authoritarianism  automation  backup  barons  behavioral-econ  being-becoming  being-right  benevolence  best-practices  big-peeps  big-picture  bio  biodet  bioinformatics  biophysical-econ  biotech  blowhards  brands  britain  broad-econ  build-packaging  business  business-models  c(pp)  california  canada  cancer  canon  capital  capitalism  cartoons  causation  characterization  charity  chart  checklists  chemistry  china  christianity  civic  civil-liberty  civilization  cjones-like  class  class-warfare  classic  classification  climate-change  clown-world  coalitions  coarse-fine  cocktail  code-organizing  cog-psych  cold-war  collaboration  combo-optimization  commentary  communication  community  comparison  compensation  competition  complement-substitute  composition-decomposition  computation  computer-vision  concept  concrete  concurrency  constraint-satisfaction  contrarianism  convexity-curvature  cool  cooperate-defect  corporation  correlation  cost-benefit  cost-disease  counter-revolution  courage  course  cracker-econ  cracker-prog  creative  crime  criminal-justice  criminology  critique  crooked  cs  cultural-dynamics  culture  culture-war  curiosity  current-events  curvature  cycles  cynicism-idealism  dark-arts  darwinian  data  data-science  data-structures  database  dataviz  dbs  death  debate  debt  decision-making  deep-learning  deep-materialism  defense  definite-planning  definition  degrees-of-freedom  democracy  demographics  density  design  desktop  detail-architecture  developing-world  devops  devtools  diaspora  dignity  dimensionality  diogenes  direction  dirty-hands  discovery  discussion  distributed  distribution  diversity  diy  douthatish  drugs  duality  duplication  dynamic  dynamical  dysgenics  early-modern  earth  eastern-europe  econ-metrics  econ-productivity  econometrics  economics  econotariat  ecosystem  education  effect-size  effective-altruism  efficiency  egalitarianism-hierarchy  einstein  elections  electromag  elite  email  empirical  ems  endo-exo  endogenous-exogenous  energy-resources  engineering  enhancement  entrepreneurialism  environment  envy  equilibrium  eric-kaufmann  error  essay  essence-existence  estimate  ethanol  ethics  EU  europe  evidence  evidence-based  evolution  examples  exocortex  expansionism  expert  expert-experience  explanans  explanation  exploratory  exposition  extra-introversion  facebook  fashun  FDA  fertility  feudal  fiction  film  finance  fire  flexibility  fluid  flux-stasis  flynn  focus  food  frameworks  free-riding  frequency  frontier  futurism  gallic  games  garett-jones  gavisti  gedanken  gender  gender-diff  gene-flow  genetics  genomics  geoengineering  geography  geometry  germanic  giants  github  gnon  gnosis-logos  gnxp  god-man-beast-victim  golang  google  gotchas  government  gradient-descent  graph-theory  graphs  greedy  grokkability  grokkability-clarity  ground-up  group-level  growth-econ  GT-101  h2o  hanson  hard-tech  hari-seldon  harvard  hashing  healthcare  heavy-industry  heterodox  hidden-motives  high-variance  higher-ed  history  hive-mind  hmm  hn  homo-hetero  honor  human-capital  human-ml  hypocrisy  ideas  identity  identity-politics  ideology  IEEE  impetus  incentives  increase-decrease  india  individualism-collectivism  inequality  info-dynamics  info-foraging  infographic  init  innovation  insight  institutions  intel  intelligence  interdisciplinary  interests  interface-compatibility  internet  intervention  intricacy  investing  iq  iron-age  islam  iteration-recursion  iterative-methods  janus  japan  jargon  journos-pundits  justice  jvm  knowledge  korea  kumbaya-kult  labor  language  large-factor  latency-throughput  latin-america  law  leadership  learning  lecture-notes  lectures  lens  let-me-see  leviathan  libraries  life-history  limits  linear-programming  links  list  literature  local-global  long-short-run  long-term  longevity  longitudinal  love-hate  lower-bounds  luca-trevisan  lurid  machine-learning  macro  madisonian  magnitude  malaise  management  managerial-state  manifolds  maps  marginal  market-power  markets  matching  math  math.CA  math.CO  math.MG  measure  measurement  mechanics  media  medicine  mediterranean  MENA  meta:prediction  meta:war  metabuch  metameta  methodology  metrics  micro  microfoundations  microsoft  migrant-crisis  migration  mihai  military  minimalism  mit  mobile  mobility  models  modernity  mokyr-allen-mccloskey  moments  monetary-fiscal  money  morality  mostly-modern  move-fast-(and-break-things)  multi  multiplicative  musk  myth  n-factor  narrative  nascent-state  nationalism-globalism  natural-experiment  nature  network-structure  neuro  new-religion  news  nibble  nietzschean  nihil  nitty-gritty  nl-and-so-can-you  no-go  noble-lie  nonlinearity  nordic  northeast  nostalgia  nuclear  numerics  nutrition  nyc  objektbuch  occident  oceans  old-anglo  oly  oly-programming  open-closed  optimate  optimism  optimization  order-disorder  org:anglo  org:biz  org:data  org:davos  org:edu  org:euro  org:gov  org:junk  org:lite  org:local  org:mag  org:mat  org:med  org:nat  org:ngo  org:popup  org:rec  org:sci  organizing  orient  oscillation  other-xtian  outcome-risk  outliers  overflow  p:***  paradox  parallax  parsimony  patho-altruism  patience  paul-romer  pdf  peace-violence  people  performance  personality  pessimism  phalanges  pharma  philosophy  phys-energy  physics  pic  piketty  planning  plots  pls  plt  poast  polanyi-marx  polarization  policy  polisci  political-econ  politics  poll  pop-diff  pop-structure  population  population-genetics  populism  postmortem  power  power-law  pre-ww2  prediction  prepping  preprint  presentation  primitivism  princeton  pro-rata  probability  procrastination  productivity  programming  project  propaganda  properties  proposal  protestant-catholic  pseudoE  psych-architecture  psychology  psychometrics  public-health  putnam-like  python  q-n-a  qra  quantitative-qualitative  quantum  questions  quixotic  quotes  race  rand-approx  random  randy-ayndy  ranking  rant  ratty  realness  reason  recruiting  redistribution  reference  reflection  regression  regularizer  regulation  relaxation  religion  rent-seeking  revolution  rhetoric  rhythm  right-wing  rigidity  rindermann-thompson  risk  ritual  robotics  roots  rounding  rsc  russia  s:*  saas  safety  sapiens  scale  sci-comp  science  science-anxiety  scifi-fantasy  scitariat  search  securities  selection  sequential  shakespeare  shift  short-circuit  signal-noise  signaling  sinosphere  skeleton  skunkworks  sky  slides  slippery-slope  social  social-choice  social-norms  sociology  socs-and-mops  software  space  speaking  spearhead  spectral  speculation  speed  speedometer  spreading  stackex  stagnation  stanford  startups  state  state-of-art  statesmen  stats  status  stereotypes  stochastic-processes  stock-flow  stories  strategy  street-fighting  structure  study  studying  stylized-facts  sub-super  subculture  success  summary  summer-2014  supply-demand  survey  sv  synchrony  system-design  systems  tactics  tails  taxes  tcs  tcstariat  tech  tech-infrastructure  technocracy  technology  techtariat  telos-atelos  temperance  temperature  terminal  tetlock  the-bones  the-classics  the-devil  the-founding  the-great-west-whale  the-monster  the-trenches  the-watchers  the-west  the-world-is-just-atoms  theory-of-mind  theos  thermo  thick-thin  thiel  things  thinking  threat-modeling  time  time-preference  time-series  todo  toolkit  tools  top-n  traces  track-record  tracker  trade  tradeoffs  tradition  transportation  trees  trends  tribalism  trivia  trump  trust  truth  twitter  types  ubiquity  unaffiliated  uncertainty  unintended-consequences  unit  universalism-particularism  unix  urban  urban-rural  us-them  usa  vaclav-smil  vampire-squid  variance-components  vcs  venture  video  visual-understanding  visualization  visuo  vitality  volo-avolo  walls  war  wealth  wealth-of-nations  welfare-state  west-hunter  white-paper  wiki  winner-take-all  wisdom  within-without  wonkish  workflow  working-stiff  world  world-war  worrydream  worse-is-better/the-right-thing  X-not-about-Y  yak-shaving  yoga  zeitgeist  zero-positive-sum  zooming  🌞  🎩  🐸  👳  🔬 

Copy this bookmark:



description:


tags: