jm + toread   18

'STELLA Report from the SNAFUcatchers Workshop on Coping With Complexity', March 14-16 2017
'A consortium workshop of high end techs reviewed postmortems to better understand how engineers cope with the complexity of anomalies (SNAFU and SNAFU catching episodes) and how to support them. These cases reveal common themes regarding factors that produce resilient performances. The themes that emerge also highlight opportunities to move forward.'

The 'Dark debt' concept is interesting here.
complexity  postmortems  dark-debt  technical-debt  resilience  reliability  systems  snafu  reports  toread  stella  john-allspaw 
4 days ago by jm
Wired on the new O'Reilly SRE book
"Site Reliability Engineering: How Google Runs Production Systems", by Chris Jones, Betsy Beyer, Niall Richard Murphy, Jennifer Petoff. Go Niall!
google  sre  niall-murphy  ops  devops  oreilly  books  toread  reviews 
april 2016 by jm
Evidence-Based Software Engineering

Objective: Our objective is to describe how software engineering might benefit from an evidence-based approach and to identify the potential difficulties associated with the approach.
Method: We compared the organisation and technical infrastructure supporting evidence-based medicine (EBM) with the situation in software engineering. We considered the impact that factors peculiar to software engineering (i.e. the skill factor and the lifecycle factor) would have on our ability to practice evidence-based software engineering (EBSE).
Results: EBSE promises a number of benefits by encouraging integration of research results with a view to supporting the needs of many different stakeholder groups. However, we do not currently have the infrastructure needed for widespread adoption of EBSE. The skill factor means software engineering experiments are vulnerable to subject and experimenter bias. The lifecycle factor means it is difficult to determine how technologies will behave once deployed.
Conclusions: Software engineering would benefit from adopting what it can of the evidence approach provided that it deals with the specific problems that arise from the nature of software engineering.


(via Mark Dennehy)
papers  toread  via:markdennehy  software  coding  ebse  evidence-based-medicine  medicine  research 
june 2015 by jm
'Addressing the rebalancing problem in bike-sharing systems' [paper]
Many of the bike-sharing systems introduced around the world in the past 15 years have the same problem: Riders tend to take some routes and not others. As a result, the bikes tend to collect in a few places, which is a drag for users and a costly problem for the operators, who "rebalance" the system using trucks that take bikes from full stations to empty ones. Now, scientists are coming up with special algorithms to improve this process. One of them, developed by scientists at the Vienna University of Technology and the Austrian Institute of Technology, is now being tested in Vienna's bike-sharing system; another, developed at Cornell University, is already in use in New York City.


Timely -- here's what Dublin Bikes looked like this morning: https://twitter.com/jmason/status/503828246086295552

(via Andrew Caines)
cycling  bike-sharing  borisbikes  dublinbikes  rebalancing  fleet  availability  optimization  maths  papers  toread  algorithms 
august 2014 by jm
Google's Influential Papers for 2013
Googlers across the company actively engage with the scientific community by publishing technical papers, contributing open-source packages, working on standards, introducing new APIs and tools, giving talks and presentations, participating in ongoing technical debates, and much more. Our publications offer technical and algorithmic advances, feature aspects we learn as we develop novel products and services, and shed light on some of the technical challenges we face at Google. Below are some of the especially influential papers co-authored by Googlers in 2013.
google  papers  toread  reading  2013  scalability  machine-learning  algorithms 
july 2014 by jm
Questioning the Lambda Architecture
Jay Kreps (Kafka, Samza) with a thought-provoking post on the batch/stream-processing dichotomy
jay-kreps  toread  architecture  data  stream-processing  batch  hadoop  storm  lambda-architecture 
july 2014 by jm
Comics For Children…. a visual list…. | The Forbidden Planet International Blog
some great recommendations here. Hildafolk has been popular with my 5-year-old, must pick up a few more
comics  kids  children  books  reading  library  toget  toread 
july 2013 by jm
_Dynamic Histograms: Capturing Evolving Data Sets_ [pdf]

Currently, histograms are static structures: they are created from scratch periodically and their creation is based on looking at the entire data distribution as it exists each time. This creates problems, however, as data stored in DBMSs usually varies with time. If new data arrives at a high rate and old data is likewise deleted, a histogram’s accuracy may deteriorate fast as the histogram becomes older, and the optimizer’s effectiveness may be lost. Hence, how often a histogram is reconstructed becomes very critical, but choosing the right period is a hard problem, as the following trade-off exists: If the period is too long, histograms may become outdated. If the period is too short, updates of the histogram may incur a high overhead.

In this paper, we propose what we believe is the most elegant solution to the problem, i.e., maintaining dynamic histograms within given limits of memory space. Dynamic histograms are continuously updateable, closely tracking changes to the actual data. We consider two of the best static histograms proposed in the literature [9], namely V-Optimal and Compressed, and modify them. The new histograms are naturally called Dynamic V-Optimal (DVO) and Dynamic Compressed (DC). In addition, we modified V-Optimal’s partition constraint to create the Static Average-Deviation Optimal (SADO) and Dynamic Average-Deviation Optimal (DADO) histograms.


(via d2fn)
via:d2fn  histograms  streaming  big-data  data  dvo  dc  sado  dado  dynamic-histograms  papers  toread 
may 2013 by jm
The Bw-Tree: A B-tree for New Hardware - Microsoft Research
The emergence of new hardware and platforms has led to reconsideration of how data management systems are designed. However, certain basic functions such as key indexed access to records remain essential. While we exploit the common architectural layering of prior systems, we make radically new design decisions about each layer. Our new form of B tree, called the Bw-tree achieves its very high performance via a latch-free approach that effectively exploits the processor caches of modern multi-core chips. Our storage manager uses a unique form of log structuring that blurs the distinction between a page and a record store and works well with flash storage. This paper describes the architecture and algorithms for the Bw-tree, focusing on the main memory aspects. The paper includes results of our experiments that demonstrate that this fresh approach produces outstanding performance.
bw-trees  database  paper  toread  research  algorithms  microsoft  sql  sql-server  b-trees  data-structures  storage  cache-friendly  mechanical-sympathy 
april 2013 by jm
Spanner: Google's Globally-Distributed Database [PDF]

Abstract: Spanner is Google's scalable, multi-version, globally-distributed, and synchronously-replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed transactions. This paper describes how Spanner is structured, its feature set, the rationale underlying various design decisions, and a novel time API that exposes clock uncertainty. This API and its implementation are critical to supporting external consistency and a variety of powerful features: non-blocking reads in the past, lock-free read-only transactions, and atomic schema changes, across all of Spanner.

To appear in:
OSDI'12: Tenth Symposium on Operating System Design and Implementation, Hollywood, CA, October, 2012.
database  distributed  google  papers  toread  pdf  scalability  distcomp  transactions  cap  consistency 
september 2012 by jm
Michael "Liar's Poker" Lewis on Ireland's economic collapse
PDF of the 15-page Vanity Fair article -- from interviews I've read in advance, this seems pretty good
michael-lewis  vanity-fair  articles  pdf  toread  economy  ireland  disaster  collapse  from delicious
february 2011 by jm
Brick, A Literary Journal: Issue 85: The Lizard, the Catacombs, and the Clock
'The Story of Paris’s Most Secret Underground Society': among the Parisian catacomb-dwellers and subterranean explorers. fascinating
france  paris  underground  toread  ux  catacombs  marquis  writing  from delicious
july 2010 by jm
Comparing genomes to computer operating systems in terms of the topology and evolution of their regulatory control networks — PNAS
'we present a comparison between the transcriptional regulatory network of a well-studied bacterium (E. coli) and the call graph of a canonical OS (Linux) in terms of topology and evolution. ... both networks have a fundamentally hierarchical layout, but there is a key difference: The transcriptional regulatory network possesses a few global regulators at the top and many targets at the bottom; conversely, the call graph has many regulators controlling a small set of generic functions. This top-heavy organization leads to highly overlapping functional modules in the call graph, in contrast to the relatively independent modules in the regulatory network. ... These findings stem from the design principles of the two systems: robustness for biological systems and cost effectiveness (reuse) for software systems.' (via adulau)
via:adulau  papers  toread  genetics  genome  call-graph  linux  kernel  e-coli  operating-systems  transcriptional-regulatory-network  from delicious
may 2010 by jm
_Botnet Judo: Fighting Spam with Itself_
reverse-engineering the output of spam templates. paper isn't published yet, but sounds very interesting, particularly since it overlaps with the SpamAssassin SOUGHT ruleset's methodology, a little, it sounds like. looking forward to reading it
spam  anti-spam  botnets  templates  toread  papers  from delicious
january 2010 by jm
GameFAQs: Assassin's Creed II (X360) Puzzle/Codex FAQ
linked by Nelson; will return to this once i've gotten into the game
assassins-creed  games  via:nelson  toread  xbox  from delicious
december 2009 by jm
Science fiction: The stories of now - 16 September 2009 - New Scientist
(via Pierce) Kim Stanley Robinson on today's British SF "golden age". I have a lot of reading to catch up on
sf  science-fiction  uk  scifi  kim-stanley-robinson  new-scientist  culture  toread  books  from delicious
september 2009 by jm

related tags

algorithms  anti-spam  architecture  articles  assassins-creed  availability  b-trees  batch  best-practices  big-data  bike-sharing  book  books  borisbikes  botnets  bw-trees  cache-friendly  call-graph  cap  catacombs  children  coding  collapse  comics  complexity  consistency  culture  cycling  dado  dark-debt  data  data-structures  database  dc  devops  disaster  distcomp  distributed  download  dublinbikes  dvo  dynamic-histograms  e-coli  ebse  economy  edward-snowden  evidence-based-medicine  fleet  france  free  games  genetics  genome  google  hadoop  histograms  interviews  ireland  james-bamford  jay-kreps  john-allspaw  kernel  kids  kim-stanley-robinson  lambda-architecture  leaks  library  linux  machine-learning  marquis  maths  mechanical-sympathy  medicine  michael-lewis  microsoft  new-scientist  niall-murphy  nsa  operating-systems  ops  optimization  oreilly  paper  papers  paris  pdf  postmortems  reading  rebalancing  reference  reliability  reports  research  resilience  reviews  ruby  sado  scalability  science-fiction  scifi  sf  snafu  software  spam  sql  sql-server  sre  stella  storage  storm  stream-processing  streaming  systems  technical-debt  templates  toget  toread  transactions  transcriptional-regulatory-network  uk  underground  us-politics  ux  vanity-fair  via:adulau  via:d2fn  via:markdennehy  via:nelson  whistleblowers  wired  writing  xbox 

Copy this bookmark:



description:


tags: