nhaliday + list   1013

REST is the new SOAP | Hacker News
hn  commentary  techtariat  org:ngo  programming  engineering  web  client-server  networking  rant  rhetoric  contrarianism  idk  org:med  best-practices  working-stiff  api  models  protocol-metadata  internet  state  structure  chart  multi  q-n-a  discussion  expert-experience  track-record  reflection  cost-benefit  design  system-design  comparison  code-organizing  flux-stasis  interface-compatibility  trends  gotchas  stackex  state-of-art  distributed  concurrency  abstraction  concept  conceptual-vocab  python  ubiquity  list  top-n  duplication  synchrony  performance  caching 
3 days ago by nhaliday
Ask HN: What's a promising area to work on? | Hacker News
hn  discussion  q-n-a  ideas  impact  trends  the-bones  speedometer  technology  applications  tech  cs  programming  list  top-n  recommendations  lens  machine-learning  deep-learning  security  privacy  crypto  software  hardware  cloud  biotech  CRISPR  bioinformatics  biohacking  blockchain  cryptocurrency  crypto-anarchy  healthcare  graphics  SIGGRAPH  vr  automation  universalism-particularism  expert-experience  reddit  social  arbitrage  supply-demand  ubiquity  cost-benefit  compensation  chart  career  planning  strategy  long-term  advice  sub-super  commentary  rhetoric  org:com  techtariat  human-capital  prioritizing  tech-infrastructure  working-stiff  data-science 
3 days ago by nhaliday
How I Choose What To Read — David Perell
READING HEURISTICS
1. TRUST RECOMMENDATIONS — BUT NOT TOO MUCH
2. TAME THE THRILLERS
3. BLEND A BIZARRE BOWL
4. TRUST THE LINDY EFFECT
5. FAVOR BIOGRAPHIES OVER SELF-HELP
unaffiliated  advice  reflection  checklists  metabuch  learning  studying  info-foraging  skeleton  books  heuristic  contrarianism  ubiquity  time  track-record  thinking  blowhards  bret-victor  worrydream  list  top-n  recommendations  arbitrage  trust  aphorism 
3 days ago by nhaliday
Japanese sound symbolism - Wikipedia
Japanese has a large inventory of sound symbolic or mimetic words, known in linguistics as ideophones.[1][2] Sound symbolic words are found in written as well as spoken Japanese.[3] Known popularly as onomatopoeia, these words are not just imitative of sounds but cover a much wider range of meanings;[1] indeed, many sound-symbolic words in Japanese are for things that don't make any noise originally, most clearly demonstrated by shiinto (しいんと), meaning "silently".
language  foreign-lang  trivia  wiki  reference  audio  hmm  alien-character  culture  list  objektbuch  japan  asia  writing 
5 weeks ago by nhaliday
Software Testing Anti-patterns | Hacker News
I haven't read this but both the article and commentary/discussion look interesting from a glance

hmm: https://news.ycombinator.com/item?id=16896390
In small companies where there is no time to "waste" on tests, my view is that 80% of the problems can be caught with 20% of the work by writing integration tests that cover large areas of the application. Writing unit tests would be ideal, but time-consuming. For a web project, that would involve testing all pages for HTTP 200 (< 1 hour bash script that will catch most major bugs), automatically testing most interfaces to see if filling data and clicking "save" works. Of course, for very important/dangerous/complex algorithms in the code, unit tests are useful, but generally, that represents a very low fraction of a web application's code.
hn  commentary  techtariat  discussion  programming  engineering  methodology  best-practices  checklists  thinking  correctness  api  interface-compatibility  jargon  list  metabuch  objektbuch  workflow  documentation  debugging  span-cover  checking  metrics  abstraction  within-without  characterization  error  move-fast-(and-break-things)  minimum-viable  efficiency  multi  poast  pareto  coarse-fine 
5 weeks ago by nhaliday
Choose the best - Slant
I've noticed I fairly often agree w/ the rankings from this (at least when they show up in my search results). more accurate than I would've expected
organization  community  aggregator  data  database  search  review  software  tools  devtools  app  recommendations  ranking  list  top-n  workflow  track-record  saas  tech-infrastructure  consumerism  hardware  sleuthin  judgement 
5 weeks ago by nhaliday
Measures of cultural distance - Marginal REVOLUTION
A new paper with many authors — most prominently Joseph Henrich — tries to measure the cultural gaps between different countries.  I am reproducing a few of their results (see pp.36-37 for more), noting that higher numbers represent higher gaps:

...

Overall the numbers show much greater cultural distance of other nations from China than from the United States, a significant and under-discussed problem for China. For instance, the United States is about as culturally close to Hong Kong as China is.

[ed.: Japan is closer to the US than China. Interesting. I'd like to see some data based on something other than self-reported values though.]

the study:
Beyond WEIRD Psychology: Measuring and Mapping Scales of Cultural and Psychological Distance: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3259613
We present a new tool that provides a means to measure the psychological and cultural distance between two societies and create a distance scale with any population as the point of comparison. Since psychological data is dominated by samples drawn from the United States or other WEIRD nations, this tool provides a “WEIRD scale” to assist researchers in systematically extending the existing database of psychological phenomena to more diverse and globally representative samples. As the extreme WEIRDness of the literature begins to dissolve, the tool will become more useful for designing, planning, and justifying a wide range of comparative psychological projects. We have made our code available and developed an online application for creating other scales (including the “Sino scale” also presented in this paper). We discuss regional diversity within nations showing the relative homogeneity of the United States. Finally, we use these scales to predict various psychological outcomes.
econotariat  marginal-rev  henrich  commentary  study  summary  list  data  measure  metrics  similarity  culture  cultural-dynamics  sociology  things  world  usa  anglo  anglosphere  china  asia  japan  sinosphere  russia  developing-world  canada  latin-america  MENA  europe  eastern-europe  germanic  comparison  great-powers  thucydides  foreign-policy  the-great-west-whale  generalization  anthropology  within-group  homo-hetero  moments  exploratory  phalanges  the-bones  🎩  🌞  broad-econ  cocktail  n-factor  measurement  expectancy  distribution  self-report  values  expression-survival  uniqueness 
8 weeks ago by nhaliday
Unix philosophy - Wikipedia
1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".
2. Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
3. Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them.
4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them.
wiki  concept  philosophy  lens  ideas  design  system-design  programming  engineering  systems  unix  subculture  composition-decomposition  coupling-cohesion  metabuch  skeleton  hi-order-bits  summary  list  top-n  quotes  aphorism  minimalism  minimum-viable  best-practices  intricacy  parsimony  protocol-metadata 
12 weeks ago by nhaliday
Learn to speak vim — verbs, nouns, and modifiers! - Yan Pritzker
learn some verbs: v (visual), c (change), d (delete), y (yank/copy). these are the most important. there are others
learn some modifiers: i (inside), a (around), t (till..finds a character), f (find..like till except including the char), / (search..find a string/regex)
learn some text objects: w (word), s (sentence) p (paragraph) b (block/parentheses), t (tag, works for html/xml) there are others
techtariat  howto  objektbuch  top-n  list  editors  composition-decomposition  DSL  lexical  atoms  yak-shaving  language 
august 2019 by nhaliday
syntax highlighting - List known filetypes - Vi and Vim Stack Exchange
Type :setfiletype (with a space afterwards), then press Ctrl-d.
q-n-a  stackex  editors  howto  list  pls  config 
august 2019 by nhaliday
Call graph - Wikipedia
I've found both static and dynamic versions useful (former mostly when I don't want to go thru pain of compiling something)

best options AFAICT:

C/C++ and maybe Go: https://github.com/gperftools/gperftools
https://gperftools.github.io/gperftools/cpuprofile.html

static: https://github.com/Vermeille/clang-callgraph
https://stackoverflow.com/questions/5373714/how-to-generate-a-calling-graph-for-c-code
I had to go through some extra pain to get this to work:
- if you use Homebrew LLVM (that's slightly incompatible w/ macOS c++filt, make sure to pass -n flag)
- similarly macOS sed needs two extra backslashes for each escape of the angle brackets

another option: doxygen

Go: https://stackoverflow.com/questions/31362332/creating-call-graph-in-golang
both static and dynamic in one tool

Java: https://github.com/gousiosg/java-callgraph
both static and dynamic in one tool

Python:
https://github.com/gak/pycallgraph
more up-to-date forks: https://github.com/daneads/pycallgraph2 and https://github.com/YannLuo/pycallgraph
old docs: https://pycallgraph.readthedocs.io/en/master/

static: https://github.com/davidfraser/pyan

various: https://github.com/jrfonseca/gprof2dot

I believe all the dynamic tools listed here support weighting nodes and edges by CPU time/samples (inclusive and exclusive of descendants) and discrete calls. In the case of the gperftools and the Java option you probably have to parse the output to get the latter, tho.

IIRC Dtrace has probes for function entry/exit. So that's an option as well.
concept  wiki  reference  tools  devtools  graphs  trees  programming  code-dive  let-me-see  big-picture  libraries  software  recommendations  list  top-n  links  c(pp)  golang  python  javascript  jvm  stackex  q-n-a  howto  yak-shaving  visualization  dataviz  performance  structure  oss  osx  unix  linux  static-dynamic 
july 2019 by nhaliday
Errors in Math Functions (The GNU C Library)
https://stackoverflow.com/questions/22259537/guaranteed-precision-of-sqrt-function-in-c-c
For C99, there are no specific requirements. But most implementations try to support Annex F: IEC 60559 floating-point arithmetic as good as possible. It says:

An implementation that defines __STDC_IEC_559__ shall conform to the specifications in this annex.

And:

The sqrt functions in <math.h> provide the IEC 60559 square root operation.

IEC 60559 (equivalent to IEEE 754) says about basic operations like sqrt:

Except for binary <-> decimal conversion, each of the operations shall be performed as if it first produced an intermediate result correct to infinite precision and with unbounded range, and then coerced this intermediate result to fit in the destination's format.

The final step consists of rounding according to several rounding modes but the result must always be the closest representable value in the target precision.

[ed.: The list of other such correctly rounded functions is included in the IEEE-754 standard (which I've put w/ the C1x and C++2x standard drafts) under section 9.2, and it mainly consists of stuff that can be expressed in terms of exponentials (exp, log, trig functions, powers) along w/ sqrt/hypot functions.

Fun fact: this question was asked by Yeputons who has a codeforces profile.]
https://stackoverflow.com/questions/20945815/math-precision-requirements-of-c-and-c-standard
oss  libraries  systems  c(pp)  numerics  documentation  objektbuch  list  linux  unix  multi  q-n-a  stackex  programming  nitty-gritty  sci-comp  accuracy  types  approximation  IEEE  protocol-metadata  gnu 
july 2019 by nhaliday
Mutation testing - Wikipedia
Mutation testing involves modifying a program in small ways.[1] Each mutated version is called a mutant and tests detect and reject mutants by causing the behavior of the original version to differ from the mutant. This is called killing the mutant. Test suites are measured by the percentage of mutants that they kill. New tests can be designed to kill additional mutants.
wiki  reference  concept  mutation  selection  analogy  programming  checking  formal-methods  debugging  random  list  libraries  links  functional  haskell  javascript  jvm  c(pp)  python  dotnet  oop  perturbation  static-dynamic 
july 2019 by nhaliday
How to work with GIT/SVN — good practices - Jakub Kułak - Medium
best part of this is the links to other guides
Commit Often, Perfect Later, Publish Once: https://sethrobertson.github.io/GitBestPractices/

My Favourite Git Commit: https://news.ycombinator.com/item?id=21289827
I use the following convention to start the subject of commit(posted by someone in a similar HN thread):
...
org:med  techtariat  tutorial  faq  guide  howto  workflow  devtools  best-practices  vcs  git  engineering  programming  multi  reference  org:junk  writing  technical-writing  hn  commentary  jargon  list  objektbuch  examples  analysis 
june 2019 by nhaliday
PythonSpeed/PerformanceTips - Python Wiki
some are obsolete, but I think, eg, the tip about using local vars over globals is still applicable
wiki  reference  cheatsheet  objektbuch  list  programming  python  performance  pls  local-global 
june 2019 by nhaliday
The End of the Editor Wars » Linux Magazine
Moreover, even if you assume a broad margin of error, the pollings aren't even close. With all the various text editors available today, Vi and Vim continue to be the choice of over a third of users, while Emacs well back in the pack, no longer a competitor for the most popular text editor.

https://www.quora.com/Are-there-more-Emacs-or-Vim-users
I believe Vim is actually more popular, but it's hard to find any real data on it. The best source I've seen is the annual StackOverflow developer survey where 15.2% of developers used Vim compared to a mere 3.2% for Emacs.

Oddly enough, the report noted that "Data scientists and machine learning developers are about 3 times more likely to use Emacs than any other type of developer," which is not necessarily what I would have expected.

[ed. NB: Vim still dominates overall.]

https://pinboard.in/u:nhaliday/b:6adc1b1ef4dc

Time To End The vi/Emacs Debate: https://cacm.acm.org/blogs/blog-cacm/226034-time-to-end-the-vi-emacs-debate/fulltext

Vim, Emacs and their forever war. Does it even matter any more?: https://blog.sourcerer.io/vim-emacs-and-their-forever-war-does-it-even-matter-any-more-697b1322d510
Like an episode of “Silicon Valley”, a discussion of Emacs vs. Vim used to have a polarizing effect that would guarantee a stimulating conversation, regardless of an engineer’s actual alignment. But nowadays, diehard Emacs and Vim users are getting much harder to find. Maybe I’m in the wrong orbit, but looking around today, I see that engineers are equally or even more likely to choose any one of a number of great (for any given definition of ‘great’) modern editors or IDEs such as Sublime Text, Visual Studio Code, Atom, IntelliJ (… or one of its siblings), Brackets, Visual Studio or Xcode, to name a few. It’s not surprising really — many top engineers weren’t even born when these editors were at version 1.0, and GUIs (for better or worse) hadn’t been invented.

...

… both forums have high traffic and up-to-the-minute comment and discussion threads. Some of the available statistics paint a reasonably healthy picture — Stackoverflow’s 2016 developer survey ranks Vim 4th out of 24 with 26.1% of respondents in the development environments category claiming to use it. Emacs came 15th with 5.2%. In combination, over 30% is, actually, quite impressive considering they’ve been around for several decades.

What’s odd, however, is that if you ask someone — say a random developer — to express a preference, the likelihood is that they will favor for one or the other even if they have used neither in anger. Maybe the meme has spread so widely that all responses are now predominantly ritualistic, and represent something more fundamental than peoples’ mere preference for an editor? There’s a rather obvious political hypothesis waiting to be made — that Emacs is the leftist, socialist, centralized state, while Vim represents the right and the free market, specialization and capitalism red in tooth and claw.

How is Emacs/Vim used in companies like Google, Facebook, or Quora? Are there any libraries or tools they share in public?: https://www.quora.com/How-is-Emacs-Vim-used-in-companies-like-Google-Facebook-or-Quora-Are-there-any-libraries-or-tools-they-share-in-public
In Google there's a fair amount of vim and emacs. I would say at least every other engineer uses one or another.

Among Software Engineers, emacs seems to be more popular, about 2:1. Among Site Reliability Engineers, vim is more popular, about 9:1.
--
People use both at Facebook, with (in my opinion) slightly better tooling for Emacs than Vim. We share a master.emacs and master.vimrc file, which contains the bare essentials (like syntactic highlighting for the Hack language). We also share a Ctags file that's updated nightly with a cron script.

Beyond the essentials, there's a group for Emacs users at Facebook that provides tips, tricks, and major-modes created by people at Facebook. That's where Adam Hupp first developed his excellent mural-mode (ahupp/mural), which does for Ctags what iDo did for file finding and buffer switching.
--
For emacs, it was very informal at Google. There wasn't a huge community of Emacs users at Google, so there wasn't much more than a wiki and a couple language styles matching Google's style guides.

https://trends.google.com/trends/explore?date=all&geo=US&q=%2Fm%2F07zh7,%2Fm%2F01yp0m

https://www.quora.com/Why-is-interest-in-Emacs-dropping
And it is still that. It’s just that emacs is no longer unique, and neither is Lisp.

Dynamically typed scripting languages with garbage collection are a dime a dozen now. Anybody in their right mind developing an extensible text editor today would just use python, ruby, lua, or JavaScript as the extension language and get all the power of Lisp combined with vibrant user communities and millions of lines of ready-made libraries that Stallman and Steele could only dream of in the 70s.

In fact, in many ways emacs and elisp have fallen behind: 40 years after Lambda, the Ultimate Imperative, elisp is still dynamically scoped, and it still doesn’t support multithreading — when I try to use dired to list the files on a slow NFS mount, the entire editor hangs just as thoroughly as it might have in the 1980s. And when I say “doesn’t support multithreading,” I don’t mean there is some other clever trick for continuing to do work while waiting on a system call, like asynchronous callbacks or something. There’s start-process which forks a whole new process, and that’s about it. It’s a concurrency model straight out of 1980s UNIX land.

But being essentially just a decent text editor has robbed emacs of much of its competitive advantage. In a world where every developer tool is scriptable with languages and libraries an order of magnitude more powerful than cranky old elisp, the reason to use emacs is not that it lets a programmer hit a button and evaluate the current expression interactively (which must have been absolutely amazing at one point in the past).

https://www.reddit.com/r/emacs/comments/bh5kk7/why_do_many_new_users_still_prefer_vim_over_emacs/

more general comparison, not just popularity:
Differences between Emacs and Vim: https://stackoverflow.com/questions/1430164/differences-between-Emacs-and-vim

https://www.reddit.com/r/emacs/comments/9hen7z/what_are_the_benefits_of_emacs_over_vim/

https://unix.stackexchange.com/questions/986/what-are-the-pros-and-cons-of-vim-and-emacs

https://www.quora.com/Why-is-Vim-the-programmers-favorite-editor
- Adrien Lucas Ecoffet,

Because it is hard to use. Really.

However, the second part of this sentence applies to just about every good editor out there: if you really learn Sublime Text, you will become super productive. If you really learn Emacs, you will become super productive. If you really learn Visual Studio… you get the idea.

Here’s the thing though, you never actually need to really learn your text editor… Unless you use vim.

...

For many people new to programming, this is the first time they have been a power user of… well, anything! And because they’ve been told how great Vim is, many of them will keep at it and actually become productive, not because Vim is particularly more productive than any other editor, but because it didn’t provide them with a way to not be productive.

They then go on to tell their friends how great Vim is, and their friends go on to become power users and tell their friends in turn, and so forth. All these people believe they became productive because they changed their text editor. Little do they realize that they became productive because their text editor changed them[1].

This is in no way a criticism of Vim. I myself was a beneficiary of such a phenomenon when I learned to type using the Dvorak layout: at that time, I believed that Dvorak would help you type faster. Now I realize the evidence is mixed and that Dvorak might not be much better than Qwerty. However, learning Dvorak forced me to develop good typing habits because I could no longer rely on looking at my keyboard (since I was still using a Qwerty physical keyboard), and this has made me a much more productive typist.

Technical Interview Performance by Editor/OS/Language: https://triplebyte.com/blog/technical-interview-performance-by-editor-os-language
[ed.: I'm guessing this is confounded to all hell.]

The #1 most common editor we see used in interviews is Sublime Text, with Vim close behind.

Emacs represents a fairly small market share today at just about a quarter the userbase of Vim in our interviews. This nicely matches the 4:1 ratio of Google Search Trends for the two editors.

...

Vim takes the prize here, but PyCharm and Emacs are close behind. We’ve found that users of these editors tend to pass our interview at an above-average rate.

On the other end of the spectrum is Eclipse: it appears that someone using either Vim or Emacs is more than twice as likely to pass our technical interview as an Eclipse user.

...

In this case, we find that the average Ruby, Swift, and C# users tend to be stronger, with Python and Javascript in the middle of the pack.

...

Here’s what happens after we select engineers to work with and send them to onsites:

[Python does best.]

There are no wild outliers here, but let’s look at the C++ segment. While C++ programmers have the most challenging time passing Triplebyte’s technical interview on average, the ones we choose to work with tend to have a relatively easier time getting offers at each onsite.

The Rise of Microsoft Visual Studio Code: https://triplebyte.com/blog/editor-report-the-rise-of-visual-studio-code
This chart shows the rates at which each editor's users pass our interview compared to the mean pass rate for all candidates. First, notice the preeminence of Emacs and Vim! Engineers who use these editors pass our interview at significantly higher rates than other engineers. And the effect size is not small. Emacs users pass our interview at a rate 50… [more]
news  linux  oss  tech  editors  devtools  tools  comparison  ranking  flux-stasis  trends  ubiquity  unix  increase-decrease  multi  q-n-a  qra  data  poll  stackex  sv  facebook  google  integration-extension  org:med  politics  stereotypes  coalitions  decentralized  left-wing  right-wing  chart  scale  time-series  distribution  top-n  list  discussion  ide  parsimony  intricacy  cost-benefit  tradeoffs  confounding  analysis  crosstab  pls  python  c(pp)  jvm  microsoft  golang  hmm  correlation  debate  critique  quora  contrarianism  ecosystem  DSL 
june 2019 by nhaliday
algorithm, algorithmic, algorithmicx, algorithm2e, algpseudocode = confused - TeX - LaTeX Stack Exchange
algorithm2e is only one currently maintained, but answerer prefers style of algorithmicx, and after perusing the docs, so do I
q-n-a  stackex  libraries  list  recommendations  comparison  publishing  cs  programming  algorithms  tools 
june 2019 by nhaliday
package writing - Where do I start LaTeX programming? - TeX - LaTeX Stack Exchange
I think there are three categories which need to be mastered (perhaps not all in the same degree) in order to become comfortable around TeX programming:

1. TeX programming. That's very basic, it deals with expansion control, counters, scopes, basic looping constructs and so on.

2. TeX typesetting. That's on a higher level, it includes control over boxes, lines, glues, modes, and perhaps about 1000 parameters.

3. Macro packages like LaTeX.
q-n-a  stackex  programming  latex  howto  nitty-gritty  yak-shaving  links  list  recommendations  books  guide  DSL 
may 2019 by nhaliday
documentation - Materials for learning TikZ - TeX - LaTeX Stack Exchange
The way I learned all three was basically demand-driven --- "learning by doing". Whenever I needed something "new", I'd dig into the manual and try stuff until either it worked (not always most elegantly), or in desperation go to the examples website, or moan here on TeX-'n-Friends. Occasionally supplemented by trying to answer "challenging" questions here.

yeah I kinda figured that was the right approach. just not worth the time to be proactive.
q-n-a  stackex  latex  list  links  tutorial  guide  learning  yak-shaving  recommendations  programming  visuo  dataviz  prioritizing  technical-writing 
may 2019 by nhaliday
Solution concept - Wikipedia
In game theory, a solution concept is a formal rule for predicting how a game will be played. These predictions are called "solutions", and describe which strategies will be adopted by players and, therefore, the result of the game. The most commonly used solution concepts are equilibrium concepts, most famously Nash equilibrium.

Many solution concepts, for many games, will result in more than one solution. This puts any one of the solutions in doubt, so a game theorist may apply a refinement to narrow down the solutions. Each successive solution concept presented in the following improves on its predecessor by eliminating implausible equilibria in richer games.

nice diagram
concept  conceptual-vocab  list  wiki  reference  acm  game-theory  inference  equilibrium  extrema  reduction  sub-super 
may 2019 by nhaliday
Basic Error Rates
This page describes human error rates in a variety of contexts.

Most of the error rates are for mechanical errors. A good general figure for mechanical error rates appears to be about 0.5%.

Of course the denominator differs across studies. However only fairly simple actions are used in the denominator.

The Klemmer and Snyder study shows that much lower error rates are possible--in this case for people whose job consisted almost entirely of data entry.

The error rate for more complex logic errors is about 5%, based primarily on data on other pages, especially the program development page.
org:junk  list  links  objektbuch  data  database  error  accuracy  human-ml  machine-learning  ai  pro-rata  metrics  automation  benchmarks  marginal  nlp  language  density  writing  dataviz  meta:reading  speedometer 
may 2019 by nhaliday
One week of bugs
If I had to guess, I'd say I probably work around hundreds of bugs in an average week, and thousands in a bad week. It's not unusual for me to run into a hundred new bugs in a single week. But I often get skepticism when I mention that I run into multiple new (to me) bugs per day, and that this is inevitable if we don't change how we write tests. Well, here's a log of one week of bugs, limited to bugs that were new to me that week. After a brief description of the bugs, I'll talk about what we can do to improve the situation. The obvious answer to spend more effort on testing, but everyone already knows we should do that and no one does it. That doesn't mean it's hopeless, though.

...

Here's where I'm supposed to write an appeal to take testing more seriously and put real effort into it. But we all know that's not going to work. It would take 90k LOC of tests to get Julia to be as well tested as a poorly tested prototype (falsely assuming linear complexity in size). That's two person-years of work, not even including time to debug and fix bugs (which probably brings it closer to four of five years). Who's going to do that? No one. Writing tests is like writing documentation. Everyone already knows you should do it. Telling people they should do it adds zero information1.

Given that people aren't going to put any effort into testing, what's the best way to do it?

Property-based testing. Generative testing. Random testing. Concolic Testing (which was done long before the term was coined). Static analysis. Fuzzing. Statistical bug finding. There are lots of options. Some of them are actually the same thing because the terminology we use is inconsistent and buggy. I'm going to arbitrarily pick one to talk about, but they're all worth looking into.

...

There are a lot of great resources out there, but if you're just getting started, I found this description of types of fuzzers to be one of those most helpful (and simplest) things I've read.

John Regehr has a udacity course on software testing. I haven't worked through it yet (Pablo Torres just pointed to it), but given the quality of Dr. Regehr's writing, I expect the course to be good.

For more on my perspective on testing, there's this.

Everything's broken and nobody's upset: https://www.hanselman.com/blog/EverythingsBrokenAndNobodysUpset.aspx
https://news.ycombinator.com/item?id=4531549

https://hypothesis.works/articles/the-purpose-of-hypothesis/
From the perspective of a user, the purpose of Hypothesis is to make it easier for you to write better tests.

From my perspective as the primary author, that is of course also a purpose of Hypothesis. I write a lot of code, it needs testing, and the idea of trying to do that without Hypothesis has become nearly unthinkable.

But, on a large scale, the true purpose of Hypothesis is to drag the world kicking and screaming into a new and terrifying age of high quality software.

Software is everywhere. We have built a civilization on it, and it’s only getting more prevalent as more services move online and embedded and “internet of things” devices become cheaper and more common.

Software is also terrible. It’s buggy, it’s insecure, and it’s rarely well thought out.

This combination is clearly a recipe for disaster.

The state of software testing is even worse. It’s uncontroversial at this point that you should be testing your code, but it’s a rare codebase whose authors could honestly claim that they feel its testing is sufficient.

Much of the problem here is that it’s too hard to write good tests. Tests take up a vast quantity of development time, but they mostly just laboriously encode exactly the same assumptions and fallacies that the authors had when they wrote the code, so they miss exactly the same bugs that you missed when they wrote the code.

Preventing the Collapse of Civilization [video]: https://news.ycombinator.com/item?id=19945452
- Jonathan Blow

NB: DevGAMM is a game industry conference

- loss of technological knowledge (Antikythera mechanism, aqueducts, etc.)
- hardware driving most gains, not software
- software's actually less robust, often poorly designed and overengineered these days
- *list of bugs he's encountered recently*:
https://youtu.be/pW-SOdj4Kkk?t=1387
- knowledge of trivia becomes more than general, deep knowledge
- does at least acknowledge value of DRY, reusing code, abstraction saving dev time
techtariat  dan-luu  tech  software  error  list  debugging  linux  github  robust  checking  oss  troll  lol  aphorism  webapp  email  google  facebook  games  julia  pls  compilers  communication  mooc  browser  rust  programming  engineering  random  jargon  formal-methods  expert-experience  prof  c(pp)  course  correctness  hn  commentary  video  presentation  carmack  pragmatic  contrarianism  pessimism  sv  unix  rhetoric  critique  worrydream  hardware  performance  trends  multiplicative  roots  impact  comparison  history  iron-age  the-classics  mediterranean  conquest-empire  gibbon  technology  the-world-is-just-atoms  flux-stasis  increase-decrease  graphics  hmm  idk  systems  os  abstraction  intricacy  worse-is-better/the-right-thing  build-packaging  microsoft  osx  apple  reflection  assembly  things  knowledge  detail-architecture  thick-thin  trivia  info-dynamics  caching  frameworks  generalization  systematic-ad-hoc  universalism-particularism  analytical-holistic  structure  tainter  libraries  tradeoffs  prepping  threat-modeling  network-structure  writing  risk  local-glob 
may 2019 by nhaliday
« earlier      
per page:    204080120160

bundles : meta

related tags

-_-  2016-election  80000-hours  :)  :/  aaronson  ability-competence  abortion-contraception-embryo  absolute-relative  abstraction  academia  accelerationism  accretion  accuracy  acemoglu  acm  acmtariat  additive  additive-combo  aDNA  advanced  adversarial  advertising  advice  aesthetics  africa  afterlife  age-generation  age-of-discovery  aggregator  aging  agri-mindset  agriculture  ai  ai-control  akrasia  albion  alesina  alg-combo  algebra  algebraic-complexity  algorithmic-econ  algorithms  alien-character  alignment  allodium  alt-inst  altruism  ama  amazon  american-nations  AMT  analogy  analysis  analytical-holistic  anarcho-tyranny  anglo  anglosphere  announcement  anomie  anonymity  anthropic  anthropology  antidemos  antiquity  aphorism  api  apollonian-dionysian  app  apple  applicability-prereqs  applications  approximation  arbitrage  archaeology  archaics  architecture  aristos  arms  arrows  art  article  ascetic  asia  assembly  assimilation  assortative-mating  atmosphere  atoms  attaq  attention  audio  authoritarianism  autism  auto-learning  automata-languages  automation  autor  average-case  aversion  axelrod  axioms  backup  baez  bandits  bangbang  bare-hands  barons  bayesian  beauty  beeminder  begin-middle-end  behavioral-econ  behavioral-gen  being-becoming  being-right  ben-recht  benchmarks  benevolence  berkeley  best-practices  better-explained  biases  bible  bifl  big-list  big-peeps  big-picture  big-surf  big-yud  binomial  bio  biodet  biohacking  bioinformatics  biomechanics  biophysical-econ  biotech  bitcoin  bits  blockchain  blog  blowhards  boaz-barak  books  boolean-analysis  bootstraps  borel-cantelli  borjas  bostrom  bots  bounded-cognition  branches  brands  bret-victor  brexit  britain  broad-econ  brooks  browser  buddhism  build-packaging  business  business-models  c(pp)  c:*  c:***  caching  calculation  calculator  california  caltech  canada  cancer  canon  capital  capitalism  carcinisation  cardio  career  carmack  cartoons  CAS  causation  censorship  certificates-recognition  chan  chapman  characterization  charity  chart  cheatsheet  checking  checklists  chemistry  chicago  china  christianity  christopher-lasch  circuits  civic  civil-liberty  civilization  cjones-like  clarity  class  class-warfare  classic  classical  classification  clever-rats  client-server  climate-change  clinton  cliometrics  clojure  closure  cloud  clown-world  coalitions  coarse-fine  cocktail  code-dive  code-organizing  coding-theory  cog-psych  cohesion  cold-war  collaboration  comedy  coming-apart  commentary  communication  communication-complexity  communism  community  comparison  compensation  competition  compilers  complement-substitute  complex-systems  complexity  composition-decomposition  compressed-sensing  compression  computation  computational-geometry  computer-memory  computer-vision  concentration-of-measure  concept  conceptual-vocab  concrete  concurrency  conference  confidence  config  confluence  confounding  confucian  confusion  conquest-empire  constraint-satisfaction  consumerism  contest  context  contracts  contradiction  contrarianism  control  convergence  convexity-curvature  cooking  cool  cooperate-defect  coordination  core-rats  corporation  correctness  correlation  corruption  cost-benefit  cost-disease  counter-revolution  counterexample  counterfactual  coupling-cohesion  courage  course  cracker-econ  cracker-prog  creative  crime  criminal-justice  criminology  CRISPR  critique  crooked  crosstab  crux  crypto  crypto-anarchy  cryptocurrency  cs  cultural-dynamics  culture  culture-war  curiosity  current-events  curvature  cybernetics  cycles  cynicism-idealism  d-lang  d3  dan-luu  dark-arts  darwinian  data  data-science  data-structures  database  dataset  dataviz  dbs  death  debate  debt  debugging  decentralized  decision-making  decision-theory  deep-learning  deep-materialism  deepgoog  defense  definite-planning  definition  degrees-of-freedom  democracy  demographic-transition  demographics  dennett  density  dependence-independence  descriptive  design  desktop  detail-architecture  deterrence  developing-world  developmental  devops  devtools  diaspora  diet  differential  dignity  dimensionality  diogenes  direct-indirect  direction  dirty-hands  discipline  discovery  discrimination  discussion  disease  distributed  distribution  divergence  diversity  divide-and-conquer  documentation  domestication  dominant-minority  dotnet  douthatish  DP  draft  drama  dropbox  drugs  DSL  duality  dumb-ML  duplication  duty  dynamic  dynamical  dysgenics  early-modern  easterly  eastern-europe  ecology  econ-metrics  econ-productivity  econometrics  economics  econotariat  ecosystem  eden  eden-heaven  editors  education  EEA  effect-size  effective-altruism  efficiency  egalitarianism-hierarchy  ego-depletion  EGT  eh  einstein  elections  electromag  elegance  elite  email  embedded-cognition  embeddings  embodied  embodied-pack  embodied-street-fighting  emergent  emotion  empirical  ems  encyclopedic  endo-exo  endocrine  endogenous-exogenous  ends-means  endurance  energy-resources  engineering  enhancement  enlightenment-renaissance-restoration-reformation  ensembles  entertainment  entrepreneurialism  entropy-like  environment  environmental-effects  envy  epidemiology  epigenetics  epistemic  equilibrium  ergo  ergodic  eric-kaufmann  erik-demaine  erlang  error  error-handling  essay  estimate  ethanol  ethical-algorithms  ethics  ethnocentrism  ethnography  EU  europe  evan-miller  events  evidence-based  evolution  evopsych  examples  exegesis-hermeneutics  existence  exit-voice  exocortex  expansionism  expectancy  experiment  expert  expert-experience  explanans  explanation  exploration-exploitation  exploratory  explore-exploit  exposition  expression-survival  externalities  extra-introversion  extratricky  extrema  facebook  fall-2016  faq  farmers-and-foragers  fashun  FDA  features  fedja  fermi  fertility  feudal  feynman  fiction  fields  fighting  film  finance  finiteness  fire  fisher  fitness  fitsci  fixed-point  flexibility  fluid  flux-stasis  focs  focus  food  foreign-lang  foreign-policy  form-design  formal-methods  formal-values  forms-instances  forum  fourier  frameworks  free  free-riding  frequency  frequentist  frisson  frontend  frontier  functional  futurism  gallic  galor-like  galton  game-theory  games  garett-jones  gaussian-processes  gavisti  gbooks  GCTA  gedanken  gender  gender-diff  gene-drift  gene-flow  general-survey  generalization  generative  genetic-correlation  genetics  genomics  geoengineering  geography  geometry  geopolitics  germanic  giants  gibbon  gif  gilens-page  git  github  gnon  gnosis-logos  gnu  gnxp  god-man-beast-victim  golang  good-evil  google  gotchas  government  gowers  grad-school  gradient-descent  graph-theory  graphical-models  graphics  graphs  gravity  gray-econ  great-powers  greedy  gregory-clark  grokkability  grokkability-clarity  ground-up  group-level  group-selection  growth  growth-econ  grugq  GT-101  gtd  guessing  guide  guilt-shame  GWAS  gwern  GxE  h2o  habit  hacker  haidt  hanson  hanushek  happy-sad  hard-tech  hardness  hardware  hari-seldon  harvard  haskell  hate  hci  health  healthcare  heavy-industry  heavyweights  henrich  heterodox  heuristic  hg  hi-order-bits  hidden-motives  hierarchy  high-dimension  high-variance  higher-ed  hiit  hip-hop  history  hive-mind  hmm  hn  homepage  homo-hetero  homogeneity  honor  horror  houellebecq  housing  howto  hsu  huge-data-the-biggest  human-bean  human-capital  human-ml  human-study  humanity  humility  huntington  hypochondria  hypocrisy  hypothesis-testing  icml  ide  ideas  identification-equivalence  identity  identity-politics  ideology  idk  IEEE  iidness  illusion  immune  impact  impetus  impro  incentives  increase-decrease  india  individualism-collectivism  induction  industrial-org  industrial-revolution  inequality  inference  info-dynamics  info-econ  info-foraging  infographic  information-theory  infrastructure  inhibition  init  inner-product  innovation  input-output  insight  instinct  institutions  integral  integration-extension  integrity  intel  intellectual-property  intelligence  interdisciplinary  interests  interface  interface-compatibility  internet  interpretability  intervention  interview  interview-prep  intricacy  intuition  invariance  investigative-journo  investing  iq  iran  iraq-syria  iron-age  is-ought  islam  israel  isteveish  iteration-recursion  janus  japan  jargon  javascript  jazz  jobs  journos-pundits  judaism  judgement  julia  justice  jvm  kaggle  kernels  keyboard  keyboards  kinship  kissinger  knowledge  korea  krugman  kumbaya-kult  labor  land  language  large-factor  larry-summers  latency-throughput  latent-variables  latex  latin-america  law  leadership  leaks  learning  learning-theory  lecture-notes  lectures  lee-kuan-yew  left-wing  legacy  legibility  len:long  len:short  lens  lesswrong  let-me-see  letters  levers  leviathan  lexical  libraries  life-history  lifehack  lifestyle  lifts-projections  limits  linear-algebra  linear-models  linear-programming  linearity  liner-notes  linguistics  links  linux  lisp  list  literature  lived-experience  llvm  lmao  local-global  logic  logistics  logos  lol  long-short-run  long-term  longevity  longform  longitudinal  love-hate  low-hanging  lower-bounds  lurid  machiavelli  machine-learning  macro  madhu-sudan  madisonian  magnitude  malaise  malthus  management  managerial-state  manifolds  map-territory  maps  marginal  marginal-rev  market-failure  market-power  marketing  markets  markov  martial  martingale  matching  math  math.AC  math.AG  math.AT  math.CA  math.CO  math.CT  math.CV  math.DS  math.FA  math.GN  math.GR  math.NT  math.RT  mathtariat  matrix-factorization  maxim-gun  meaningness  measure  measurement  mechanics  mechanism-design  media  medicine  medieval  mediterranean  memes(ew)  memory-management  MENA  mena4  mendel-randomization  mental-math  meta-analysis  meta:math  meta:medicine  meta:prediction  meta:reading  meta:research  meta:rhetoric  meta:science  meta:war  metabolic  metabuch  metal-to-virtual  metameta  methodology  metric-space  metrics  michael-nielsen  micro  microbiz  microfoundations  microsoft  midwest  migrant-crisis  migration  mihai  military  minimalism  minimum-viable  miri-cfar  missing-heritability  mit  mixing  ML-MAP-E  mobile  mobility  model-class  model-organism  models  modernity  mokyr-allen-mccloskey  moloch  moments  monetary-fiscal  money  money-for-time  monte-carlo  mooc  mood-affiliation  morality  mostly-modern  motivation  move-fast-(and-break-things)  multi  multiplicative  murray  music  music-theory  musk  mutation  mystic  myth  n-factor  narrative  nascent-state  nationalism-globalism  natural-experiment  nature  navigation  near-far  negotiation  neocons  network-structure  networking  neuro  neuro-nitgrit  neurons  new-religion  news  nibble  nietzschean  nihil  nips  nitty-gritty  nl-and-so-can-you  nlp  no-go  noahpinion  noble-lie  noblesse-oblige  nonlinearity  nootropics  nordic  norms  north-weingast-like  northeast  nostalgia  notation  notetaking  novelty  nuclear  null-result  number  numerics  nutrition  nyc  obama  objective-measure  objektbuch  ocaml-sml  occam  occident  oceans  ocw  off-convex  offense-defense  old-anglo  oly  oly-programming  online-learning  oop  open-closed  open-problems  open-things  openai  operational  opioids  opsec  optimate  optimism  optimization  order-disorder  orders  ORFE  org:anglo  org:biz  org:bleg  org:bv  org:com  org:data  org:davos  org:econlib  org:edge  org:edu  org:euro  org:fin  org:foreign  org:gov  org:health  org:inst  org:junk  org:lite  org:local  org:mag  org:mat  org:med  org:nat  org:ngo  org:popup  org:rec  org:sci  org:theos  organization  organizing  orient  orwellian  os  oscillation  oss  osx  other-xtian  outcome-risk  outdoors  overflow  oxbridge  p2p  p:*  p:**  p:***  p:null  p:someday  p:whenever  paganism  paleocon  papers  parable  paradox  parallax  parasites-microbiome  parenting  pareto  parsimony  paste  path-dependence  patho-altruism  patience  paul-romer  paying-rent  pdf  peace-violence  people  performance  personal-finance  personality  persuasion  perturbation  pessimism  peter-singer  phalanges  pharma  phase-transition  phd  philosophy  photography  phys-energy  physics  pic  pigeonhole-markov  piketty  pinboard  pinker  piracy  plan9  planning  play  plots  pls  plt  poast  podcast  poetry  polanyi-marx  polarization  policy  polis  polisci  political-econ  politics  poll  polynomials  pop-diff  pop-structure  popsci  population  population-genetics  populism  positivity  postmortem  postrat  power  power-law  practice  pragmatic  pre-2013  pre-ww2  prediction  prediction-markets  predictive-processing  preference-falsification  prejudice  prepping  preprint  presentation  primitivism  princeton  prioritizing  priors-posteriors  privacy  pro-rata  probabilistic-method  probability  problem-solving  procrastination  productivity  prof  profile  programming  progression  project  proof-systems  proofs  propaganda  properties  property-rights  proposal  protestant-catholic  protocol-metadata  prudence  pseudoE  pseudorandomness  psych-architecture  psychiatry  psycho-atoms  psychology  psychometrics  public-goodish  public-health  publishing  putnam-like  puzzles  python  q-n-a  qra  QTL  quality  quantified-self  quantifiers-sums  quantitative-qualitative  quantum  quantum-info  questions  quixotic  quiz  quora  quotes  r-lang  race  rand-approx  random  random-matrices  randy-ayndy  ranking  rant  rat-pack  rationality  ratty  reading  real-nominal  realness  realpolitik  reason  rec-math  recent-selection  recommendations  recording  recruiting  red-queen  reddit  redistribution  reduction  reference  reflection  regression  regression-to-mean  regularization  regularizer  regulation  reinforcement  relativity  religion  rent-seeking  replication  repo  reputation  research  research-program  responsibility  retention  retrofit  revealed-preference  review  revolution  rhetoric  rhythm  right-wing  rigidity  rigor  rigorous-crypto  risk  ritual  roadmap  robotics  robust  rock  roots  rot  rounding  rsc  running  russia  rust  ryan-odonnell  s-factor  s:*  s:**  s:***  s:null  saas  safety  sample-complexity  sampling  sampling-bias  sanctity-degradation  sanjeev-arora  sapiens  scala  scale  scaling-tech  scaling-up  scaruffi  schelling  scholar  scholar-pack  schools  sci-comp  science  science-anxiety  scifi-fantasy  scitariat  scott-sumner  search  sebastien-bubeck  securities  security  selection  self-control  self-interest  self-report  seminar  sentiment  sequential  series  sex  sexuality  shakespeare  shalizi  shannon  shift  shipping  short-circuit  SIGGRAPH  signal-noise  signaling  signum  similarity  simler  simplex  simulation  singularity  sinosphere  skeleton  skunkworks  sky  sleep  sleuthin  slides  slippery-slope  smart-contracts  smoothness  social  social-capital  social-choice  social-norms  social-psych  social-science  social-structure  sociality  society  sociology  socs-and-mops  soft-question  software  solid-study  space  space-complexity  span-cover  sparsity  spatial  speaking  spearhead  spectral  speculation  speed  speedometer  spengler  spock  sports  spotify  spreading  spring-2017  spring-2019  ssc  stackex  stagnation  stanford  startups  stat-mech  stat-power  state  state-of-art  statesmen  static-dynamic  stats  status  steel-man  stereotypes  stirling  stoc  stochastic-processes  stock-flow  stoic  store  stories  strategy  straussian  stream  street-fighting  stress  strings  stripe  structure  study  studying  stylized-facts  sub-super  subculture  subjective-objective  sublinear  submodular  success  sulla  sum-of-squares  summary  summer-2014  summer-2016  supply-demand  survey  survival  sv  symbols  symmetry  synchrony  syntax  synthesis  system-design  systematic-ad-hoc  systems  szabo  tactics  tails  tainter  talks  tapes  taubes-guyenet  taxes  tcs  tcstariat  teaching  tech  tech-infrastructure  technical-writing  technocracy  technology  techtariat  telos-atelos  temperance  temperature  tensors  terminal  terrorism  tetlock  the-basilisk  the-bones  the-classics  the-devil  the-founding  the-great-west-whale  the-monster  the-self  the-south  the-trenches  the-watchers  the-west  the-world-is-just-atoms  theory-of-mind  theory-practice  theos  thermo  thick-thin  thiel  things  thinking  threat-modeling  thucydides  thurston  tidbits  tim-roughgarden  time  time-complexity  time-preference  time-series  time-use  tip-of-tongue  tocqueville  todo  tolkienesque  toolkit  tools  top-n  topics  topology  toxoplasmosis  traces  track-record  tracker  trade  tradecraft  tradeoffs  tradition  transitions  transportation  travel  trees  trends  tribalism  tricki  tricks  trivia  troll  trump  trust  truth  tumblr  turchin  turing  tutorial  tutoring  tv  twin-study  twitter  types  ubiquity  ui  unaffiliated  uncertainty  unintended-consequences  uniqueness  unit  universalism-particularism  unix  unsupervised  urban  urban-rural  us-them  usa  usaco-ioi  utopia-dystopia  ux  vaclav-smil  vague  valiant  values  vampire-squid  variance-components  vcs  venture  vgr  video  virginia-DC  virtu  virtualization  visual-understanding  visualization  visuo  vitality  vocab  volo-avolo  von-neumann  vr  vulgar  walls  walter-scheidel  war  war-nerd  washington  water  waves  wealth  wealth-of-nations  web  webapp  weightlifting  weird  weird-sun  welfare-state  west-hunter  westminster  whiggish-hegelian  white-paper  whole-partial-many  wigderson  wiki  wild-ideas  winner-take-all  winter-2017  wire-guided  wisdom  within-group  within-without  woah  wonkish  wordlessness  workflow  working-stiff  workshop  world  world-war  wormholes  worrydream  worse-is-better/the-right-thing  writing  wtf  X-not-about-Y  x-sports  xenobio  yak-shaving  yarvin  yc  yoga  yvain  zeitgeist  zero-positive-sum  zooming  🌞  🎓  🎩  🐝  🐸  👳  👽  🔬  🖥  🤖  🦀  🦉 

Copy this bookmark:



description:


tags: