jm + backups   11

GitLab.com Database Incident - 2017/01/31
Horrible, horrible postmortem doc. This is the kicker:
So in other words, out of 5 backup/replication techniques deployed none are working reliably or set up in the first place.


Reddit comments: https://www.reddit.com/r/linux/comments/5rd9em/gitlab_is_down_notes_on_the_incident_and_why_you/
devops  backups  cloud  outage  incidents  postmortem  gitlab 
12 weeks ago by jm
Tahoe LAFS accidentally lose Bitcoin wallet with loads of donations in it, get it back
But ECDSA private keys don't trigger the same protective instincts that
we'd apply to, say, a bar of gold. One sequence of 256 random bits looks
just as worthless as any other. And the cold hard unforgeability of
these keys means we can't rely upon other humans to get our money back
when we lose them. Plus, we have no experience at all with things that grow in value by
four orders of magnitude, without any attention, in just three years.

So we have a cryptocurrency-tool UX task in front of us: to avoid
mistakes like the one we made, we must to either move these digital
assets into solid-feeling physical containers, or retrain our
perceptions to attach value to the key strings themselves.
backups  cryptography  bitcoin  cryptocurrency  ecdsa  private-keys  ux  money 
march 2016 by jm
Nik Cubrilovic - Notes on the Celebrity Data Theft
tl;dr: a lot of people are spending a lot of time stealing nudie pics from celebrities. See also http://www.zdziarski.com/blog/?p=3783 for more details on the probable approaches used. Grim.
apple  privacy  security  celebrities  pics  hacking  iphone  ipad  ios  exploits  brute-force  passwords  2fa  mfa  find-my-iphone  icloud  backups 
september 2014 by jm
Dead Man's Snitch
a cron job monitoring tool that keeps an eye on your periodic processes and notifies you when something doesn't happen. Daily backups, monthly emails, or cron jobs you need to monitor? Dead Man's Snitch has you covered. Know immediately when one of these processes doesn't work.


via Marc.
alerts  cron  monitoring  sysadmin  ops  backups  alarms 
april 2014 by jm
"Toy Story 2" was almost entirely deleted by accident at one point
A stray "rm -rf" on the main network share managed to wipe out 90% of the movie's assets, and the backups were corrupt. Horrific backups war story
movies  ops  backups  pixar  recovery  accidents  rm-rf  delete 
october 2013 by jm
How the NSA Spies on Smartphones
One of the US agents' tools is the use of backup files established by smartphones. According to one NSA document, these files contain the kind of information that is of particular interest to analysts, such as lists of contacts, call logs and drafts of text messages. To sort out such data, the analysts don't even require access to the iPhone itself, the document indicates. The department merely needs to infiltrate the target's computer, with which the smartphone is synchronized, in advance. Under the heading "iPhone capability," the NSA specialists list the kinds of data they can analyze in these cases. The document notes that there are small NSA programs, known as "scripts," that can perform surveillance on 38 different features of the iPhone 3 and 4 operating systems. They include the mapping feature, voicemail and photos, as well as the Google Earth, Facebook and Yahoo Messenger applications.


and, of course, the alternative means of backup is iCloud.... wonder how secure those backups are.
nsa  surveillance  gchq  iphone  smartphones  backups  icloud  security 
september 2013 by jm
KDE's brush with git repository corruption: post-mortem
a barely-averted disaster... phew.

while we planned for the case of the server losing a disk or entirely biting the dust, or the total loss of the VM’s filesystem, we didn’t plan for the case of filesystem corruption, and the way the corruption affected our mirroring system triggered some very unforeseen and pathological conditions. [...] the corruption was perfectly mirrored... or rather, due to its nature, imperfectly mirrored. And all data on the anongit [mirrors] was lost.

One risk demonstrated: by trusting in mirroring, rather than a schedule of snapshot backups covering a wide time range, they nearly had a major outage. Silent data corruption, and code bugs, happen -- backups protect against this, but RAID, replication, and mirrors do not.

Another risk: they didn't have a rate limit on project-deletion, which resulted in the "anongit" mirrors deleting their (safe) data copies in response to the upstream corruption. Rate limiting to sanity-check automated changes is vital. What they should have had in place was described by the fix: 'If a new projects file is generated and is more than 1% different than the previous file, the previous file is kept intact (at 1500 repositories, that means 15 repositories would have to be created or deleted in the span of three minutes, which is extremely unlikely).'
rate-limiting  case-studies  post-mortems  kde  git  data-corruption  risks  mirroring  replication  raid  bugs  backups  snapshots  sanity-checks  automation  ops 
march 2013 by jm
Improving Linux performance by preserving Buffer Cache State
handy -- a patch to rsync(1) which will not disturb the buffer cache, so that large file transfers and backups will not interfere with what's been cached previously
performance  linux  caching  buffer-cache  rsync  io  cache  patches  backups  from delicious
march 2011 by jm

Copy this bookmark:



description:


tags: