Files are hard


140 bookmarks. First posted by FTS december 2015.


haven't used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine, Eudora, and outlook have all corrupted my inbox, forcing me to restore from backup. How is it that desktop mail clients are less reliable than gmail, even though my gmail account not only handles more email than I ever had on desktop clients, but also allows simultaneous access from multiple locations across the globe? Distributed systems have an unfair advantage, in that they can be robust against total disk failure in a way that desktop clients can't, but none of the file corruption issues I've had have been from total disk failure. Why has my experience with desktop applications been so bad?
3 days ago by gustavoluz
"Well, what sort of failures can occur? Crash consistency (maintaining consistent state even if there's a crash) is probably the easiest property to consider, since we can assume that everything, from the filesystem to the disk, works correctly; let's consider that first."
programming  filesystem  unix  linux 
april 2018 by ddribin
It's difficult to update files in a crash-safe manner.
linux  programming  unix 
march 2018 by aristidb
Underneath your distributed, redundant, highly-available database is a filesystem just waiting to corrupt your data
from twitter_favs
october 2016 by tjweir
I haven’t used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine, eudora, and outlook have all…
from readability
january 2016 by mrchrisadams
yeah, wow, my head hurts
file  linux  software  hardware  design  failure  programming  system 
january 2016 by teffalump
great summary of research on crash protections in various filesystem and much other wisdom. include discussion of fsync on directories being needed
filesystem  linux  unix  file 
january 2016 by squiddisco
the ext3 code had lots of comments like “I really hope a write error doesn’t happen here” in places where errors weren’t handled.
linux  unix  posix  filesystems  programming 
january 2016 by ajbw
I haven’t used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine, eudora, and outlook have all corrupted my inbox, forcing me to restore from backup. via Pocket
Pocket 
december 2015 by driptray
Info about filesystems and error checking. Not embedded, specifically but relevant.
embedded 
december 2015 by pleclerc
long survey about Crash Consistency, Filesystem semantics, Filesystem correctness and Error recovery and frequency

based on Pillai et al. OSDI ’14 and introduce previous work and ways to find bug related to file system

sqlite is a good choice for multi-platform reliable file system
operation-system  file-system  linux 
december 2015 by w5s5j
Today in computing news: everything is still broken. Video at 11.
IFTTT  Facebook 
december 2015 by mpasternacki
I haven’t used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my …
unix  fs  linux  arch  code  files 
december 2015 by cjitlal
I haven’t used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine,…
from instapaper
december 2015 by wjbr
This is true. Having been away for a while, working on #eurolobster reminded me that the filesystem is surprisingly heavy compared to, say, storing stuff in an SQLite DB. More code per idea than you'd expect.
filesystem  linux  ext4  programming  failure 
december 2015 by yorksranter
“it’s hard to do ... because grad students don’t want to repeat work that’s been done before,”
dev/thoughts 
december 2015 by bluegol
Files are hard. Butler Lampson has remarked that when they came up with threads, locks, and conditional variables at PARC, they thought that they were creating a programming model that anyone could use, but that there’s now decades of evidence that they were wrong. We’ve accumulated a lot of evidence that humans are very bad at reasoning at these kinds of problems, which are very similar to the problems you have when writing correct code to interact with current filesystems. Lampson suggests that the best known general purpose solution is to package up all of your parallelism into as small a box as possible and then have a wizard write the code in the box.
december 2015 by keeganpoppen
I haven’t used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine,…
from instapaper
december 2015 by mccutchen
I haven’t used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine,…
from instapaper
december 2015 by kevinold
RT : Want to write to your filesystem safely? No worries, the manpage has all the latest rumors
from twitter
december 2015 by yitznewton
I haven’t used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my …
filesystem 
december 2015 by synergyfactor
Once again makes me feel like Wile E. Coyote suddenly realising there's no ground beneath his feet:
from twitter
december 2015 by pozorvlak
"most…email client[s] have chosen to…destroy all of your email if corruption happens" saw this and thought of
from twitter
december 2015 by jmtd