jm + unix   75

explainshell.com
This is pretty excellent work -- paste a UNIX command line and it'll contextually inline manual page snippets to match, highlighting the matching part of the command line.
cli  unix  documentation  explainshell  shell  scripting  syntax  manual-pages 
22 days ago by jm
[RFE] add a way to run in a new systemd scope automatically · Issue #428 · tmux/tmux
omgwtfbbq. 1: User reports that their gnome session leaks processes; 2: systemd modifies default session behaviour to kill all processes, including screen/tmux; 3: _everyone_ complains because they break 30 years of UNIX process semantics, then 4: they request that tmux/screen hack their shit to workaround their brokenness. Get fucked, systemd. This is the kind of shit that would finally drive me to BSDland
systemd  horror  linux  fail  unix  gnome  tmux  bugs  omgwtfbbq 
may 2016 by jm
plainas/tq
command line utility that performs an HTML element selection on HTML content passed to the stdin. Using css selectors that everybody knows. Since input comes from stdin and output is sent to stdout, it can easily be used inside traditional UNIX pipelines to extract content from webpages and html files. tq provides extra formating options such as json-encoding or newlines squashing, so it can play nicely with everyones favourite command line tooling.
tq  linux  unix  cli  command-line  html  parsing  css  tools 
may 2016 by jm
Detecting the use of "curl | bash" server side
tl;dr:
The better solution is never to pipe untrusted data streams into bash. If you still want to run untrusted bash scripts a better approach is to pipe the contents of URL into a file, review the contents on disk and only then execute it.
bash  security  shell  unix  curl  tcp  buffers 
april 2016 by jm
'Devastating' bug pops secure doors at airports, hospitals
"A command injection vulnerability exists in this function due to a lack of any sanitisation on the user-supplied input that is fed to the system() call," Lawshae says.


:facepalm:
security  iot  funny  fail  linux  unix  backticks  system  udp  hid  vertx  edge 
april 2016 by jm
percol
interactive menu selection for the UNIX command line
cli  linux  unix  grep  menus  selection  ui  interactive  terminal 
february 2016 by jm
Files Are Hard
This is basically terrifying. A catalog of race conditions and reliability horrors around the POSIX filesystem abstraction in Linux -- it's a wonder anything works.

'Where’s this documented? Oh, in some mailing list post 6-8 years ago (which makes it 12-14 years from today). The fs devs whose posts I’ve read are quite polite compared to LKML’s reputation, and they generously spend a lot of time responding to basic questions, but it’s hard for outsiders to troll [sic] through a decade and a half of mailing list postings to figure out which ones are still valid and which ones have been obsoleted! I don’t mean to pick on filesystem devs. In their OSDI 2014 talk, the authors of the paper we’re discussing noted that when they reported bugs they’d found, developers would often respond “POSIX doesn’t let filesystems do that”, without being able to point to any specific POSIX documentation to support their statement. If you’ve followed Kyle Kingsbury’s Jepsen work, this may sound familiar, except devs respond with “filesystems don’t do that” instead of “networks don’t do that”.I think this is understandable, given how much misinformation is out there. Not being a filesystem dev myself, I’d be a bit surprised if I don’t have at least one bug in this post.'
filesystems  linux  unix  files  operating-systems  posix  fsync  osdi  papers  reliability 
december 2015 by jm
Structural and semantic deficiencies in the systemd architecture for real-world service management, a technical treatise
Despite its overarching abstractions, it is semantically non-uniform and its complicated transaction and job scheduling heuristics ordered around a dependently networked object system create pathological failure cases with little debugging context that would otherwise not necessarily occur on systems with less layers of indirection. The use of bus APIs complicate communication with the service manager and lead to duplication of the object model for little gain. Further, the unit file options often carry implicit state or are not sufficiently expressive. There is an imbalance with regards to features of an eager service manager and that of a lazy loading service manager, having rusty edge cases of both with non-generic, manager-specific facilities. The approach to logging and the circularly dependent architecture seem to imply that lots of prior art has been ignored or understudied.
analysis  systemd  linux  unix  ops  init  critiques  software  logging 
november 2015 by jm
S3QL
a file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack. S3QL effectively provides a hard disk of dynamic, infinite capacity that can be accessed from any computer with internet access running Linux, FreeBSD or OS-X.
S3QL is a standard conforming, full featured UNIX file system that is conceptually indistinguishable from any local file system. Furthermore, S3QL has additional features like compression, encryption, data de-duplication, immutable trees and snapshotting which make it especially suitable for online backup and archival.
S3QL is designed to favor simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection and error handling have been included from the very first line, and S3QL comes with extensive automated test cases for all its components.
filesystems  aws  s3  storage  unix  google-storage  openstack 
september 2015 by jm
GoTTY
'a simple command line tool that turns your CLI tools into web applications'
cli  terminal  web  tools  unix 
september 2015 by jm
Miller
'like sed, awk, cut, join, and sort for name-indexed data such as CSV'


Written in "modern C" with zero runtime dependencies. Looks great
cli  csv  unix  miller  tsv  data  tools 
august 2015 by jm
peco
'Simplistic interactive filtering tool' -- live incremental-search filtering in a terminal window
cli  shell  terminal  tools  go  peco  interactive  incremental-search  search  ui  unix 
june 2015 by jm
Why I dislike systemd
Good post, and hard to disagree.
One of the "features" of systemd is that it allows you to boot a system without needing a shell at all. This seems like such a senseless manoeuvre that I can't help but think of it as a knee-jerk reaction to the perception of Too Much Shell in sysv init scripts.
In exactly which universe is it reasonable to assume that you have a running D-Bus service (or kdbus) and a filesystem containing unit files, all the binaries they refer to, all the libraries they link against, and all the configuration files any of them reference, but that you lack that most ubiquitous of UNIX binaries, /bin/sh?
history  linux  unix  systemd  bsd  system-v  init  ops  dbus 
june 2015 by jm
How to change Gradle cache location
$GRADLE_USER_HOME, basically -- it may also be possible to set from the Gradle script itself too
gradle  build  caching  environment  unix  cache 
may 2015 by jm
SCADA systems online, and a horror story about a non-airgapped Boeing 747 engine management system
747's are big flying Unix hosts. At the time, the engine management system on this particular airline was Solaris based. The patching was well behind and they used telnet as SSH broke the menus and the budget did not extend to fixing this. The engineers could actually access the engine management system of a 747 in route. If issues are noted, they can re-tune the engine in air.

The issue here is that all that separated the engine control systems and the open network was NAT based filters. There were (and as far as I know this is true today), no extrusion controls. They filter incoming traffic, but all outgoing traffic is allowed.


(via Paddy Benson)
air-gap  planes  boeing  security  747  solaris  unix 
april 2015 by jm
Ag: faster than Ack
Some nice performance tricks; I particularly like the use of sljit:
Ag uses Pthreads to take advantage of multiple CPU cores and search files in parallel.
Files are mmap()ed instead of read into a buffer.
Literal string searching uses Boyer-Moore strstr.
Regex searching uses PCRE's JIT compiler (if Ag is built with PCRE >=8.21).
Ag calls pcre_study() before executing the same regex on every file.
Instead of calling fnmatch() on every pattern in your ignore files, non-regex patterns are loaded into arrays and binary searched.
jit  cli  grep  search  ack  ag  unix  pcre  sljit  boyer-moore  tools 
march 2015 by jm
Using Named Pipes and Process Substitution in Bioinformatics
Wow. I've been using bash for nigh on 14 years and I didn't know about process substitution. Nifty trick
bash  linux  pipes  shell  unix  via:igrigorik  cli  named-pipes  process-substitution 
march 2015 by jm
uselessd
A project to reduce systemd to a base initd, process supervisor and transactional dependency system, while minimizing intrusiveness and isolationism. Basically, it’s systemd with the superfluous stuff cut out, a (relatively) coherent idea of what it wants to be, support for non-glibc platforms and an approach that aims to minimize complicated design. uselessd is still in its early stages and it is not recommended for regular use or system integration.


This may be the best option to evade the horrors of systemd.
init  linux  systemd  unix  ops  uselessd 
march 2015 by jm
Ubuntu To Officially Switch To systemd Next Monday - Slashdot
Jesus. This is going to be the biggest shitfest in the history of Linux...
linux  slashdot  ubuntu  systemd  init  unix  ops 
march 2015 by jm
Two recent systemd crashes
Hey look, PID 1 segfaulting! I haven't seen that happen since we managed to corrupt /bin/sh on Ultrix in 1992. Nice work Fedora
fedora  reliability  unix  linux  systemd  ops  bugs 
december 2014 by jm
testing latency measurements using CTRL-Z
An excellent tip from Gil "HDRHistogram" Tene:
Good example of why I always "calibrate" latency tools with ^Z tests. If ^Z results don't make sense, don't use [the] tool. ^Z test math examples: If you ^Z for half the time, Max is obvious. [90th percentile] should be 80% of the ^Z stall time.
control-z  suspend  unix  testing  latencies  latency  measurement  percentiles  tips 
november 2014 by jm
sferik/t
"A command-line power tool for Twitter." It really is -- much better timeline searchability than the "real" Twitter UI, for example
twitter  ruby  github  cli  tools  unix  search 
october 2014 by jm
The End of Linux
'Linux is becoming the thing that we adopted Linux to get away from.'

Great post on the horrible complexity of systemd. It reminds me of nothing more than mid-90s AIX, which I had the displeasure of opsing for a while -- the Linux distros have taken a very wrong turn here.
linux  unix  complexity  compatibility  ops  rant  systemd  bloat  aix 
september 2014 by jm
oss-sec: Re: CVE-2014-6271: remote code execution through bash
this is truly heinous. Given that any CGI which invokes popen()/system() on a Linux system where /bin/sh is a link to bash is vulnerable, there will be a lot of vulnerable services out there (via Elliot)
via:elliottucker  cgi  security  bash  sh  exploits  linux  popen  unix 
september 2014 by jm
tinystat - GoDoc
tinystat is used to compare two or more sets of measurements (e.g., runs of a multiple runs of benchmarks of two possible implementations) and determine if they are statistically different, using Student's t-test. It's inspired largely by FreeBSD's ministat (written by Poul-Henning Kamp).
t-test  student  statistics  go  coda-hale  tinystat  stats  tools  command-line  unix 
september 2014 by jm
Russell91/sshrc
'bring your .bashrc, .vimrc, etc. with you when you ssh'. A really nice implementation of this idea (much nicer than my own version!)
hacks  productivity  ssh  remote  shell  sh  bash  via:johnke  home-directory  unix 
september 2014 by jm
Axel
A nice curl/wget replacement which supports multi-TCP-connection downloads of HTTP/FTP resources. packaged for most Linux variants and OSX via brew
axel  curl  wget  via:johnke  downloading  tcp  http  ftp  ubuntu  debian  unix  linux 
september 2014 by jm
Nix: The Purely Functional Package Manager
'a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. It provides atomic upgrades and rollbacks, side-by-side installation of multiple versions of a package, multi-user package management and easy setup of build environments. '

Basically, this is a third-party open source reimplementation of Amazon's (excellent) internal packaging system, using symlinks to versioned package directories to ensure atomicity and the ability to roll back. This is definitely the *right* way to build packages -- I know what tool I'll be pushing for, next time this question comes up.

See also nixos.org for a Linux distro built on Nix.
ops  linux  devops  unix  packaging  distros  nix  nixos  atomic  upgrades  rollback  versioning 
september 2014 by jm
Revisiting How We Put Together Linux Systems
Building a running OS out of layered btrfs filesystems. This sounds awesome.
Instantiating a new system or OS container (which is exactly the same in this scheme) just consists of creating a new appropriately named root sub-volume. Completely naturally you can share one vendor OS copy in one specific version with a multitude of container instances.

Everything is double-buffered (or actually, n-fold-buffered), because usr, runtime, framework, app sub-volumes can exist in multiple versions. Of course, by default the execution logic should always pick the newest release of each sub-volume, but it is up to the user keep multiple versions around, and possibly execute older versions, if he desires to do so. In fact, like on ChromeOS this could even be handled automatically: if a system fails to boot with a newer snapshot, the boot loader can automatically revert back to an older version of the OS.


(via Tony Finch)
via:fanf  linux  docker  btrfs  filesystems  unionfs  copy-on-write  os  hacking  unix 
september 2014 by jm
Systemd: Harbinger of the Linux apocalypse
While there are many defensible aspects of Systemd, other aspects boggle the mind. Not the least of these was that, as of a few months ago, trying to debug the kernel from the boot line would cause the system to crash. This was because of Systemd's voracious logging and the fact that Systemd responds to the "debug" flag on the kernel boot line -- a flag meant for the kernel, not anything else. That, straight up, is a bug.

However, the Systemd developers didn't see it that way and actively fought with those experiencing the problem. Add the fact that one of the Systemd developers was banned by Linus Torvalds for poor attitude and bad design and another was responsible for causing significant issues with Linux audio support, but blamed the problem on everything else but his software, and you have a bad situation on your hands.

There's no shortage of egos in the open source development world. There's no shortage of new ideas and veteran developers and administrators pooh-poohing something new simply because it's new. But there are also 45 years of history behind Unix and extremely good reasons it's still flourishing. Tools designed like Systemd do not fit the Linux mold, to their own detriment. Systemd's design has more in common with Windows than with Unix -- down to the binary logging.


The link re systemd consuming the "debug" kernel boot arg is a canonical example of inflexible coders refusing to fix their own bugs. (via Jason Dixon)
systemd  linux  red-hat  egos  linus-torvalds  unix  init  booting  debugging  logging  design  software  via:obfuscurity 
august 2014 by jm
Comment #28 : Bug #255161 : Bugs : “cupsys” package : Ubuntu
file(1) bug causes the input Postscript file to be misidentified as an Erlang JAM file if it contains the string 'Tue' starting at byte 4.
via:hackernews  file  unix  cups  printing  funny  bugs  fail  ubuntu  linux 
august 2014 by jm
inotify one-liner hack
install inotify-tools, then: 'while true do inotifywait -r -e modify -e create -e close . ./run.sh done' #opscookie
inotify  al-tobey  one-liners  unix  hacks  opscookie  twitter 
august 2014 by jm
Collection Pipeline
a nice summarisation of the state of pipe/stream-oriented collection operations in various languages, from Martin Fowler
martin-fowler  patterns  coding  ruby  clojure  streams  pipelines  pipes  unix  lambda  fp  java  languages 
july 2014 by jm
Xfennec/cv
'This tool can be described as a Tiny Dirty Linux Only C command that looks for coreutils basic commands (cp, mv, dd, tar, gzip/gunzip, cat, ...) currently running on your system and displays the percentage of copied data. It can now also display an estimated throughput (using -w flag).'
coreutils  via:pixelbeat  linux  ops  hacks  procfs  dataviz  unix 
july 2014 by jm
Friends don't let friends use mmap(2)
Rather horrific update from the trenches of Mozilla
mozilla  mmap  performance  linux  io  files  memory  unix  windows 
may 2014 by jm
fcron
Fcron is a scheduler. It aims at replacing Vixie Cron, so it implements most of its functionalities. But contrary to Vixie Cron, fcron does not need your system to be up 7 days a week, 24 hours a day : it also works well with systems which are running only occasionnally (contrary to anacrontab). In other words, fcron does both the job of Vixie Cron and anacron, but does even more and better :)) ...


Thanks Craig!
via:chughes  cron  fcron  unix  linux  ops  scheduler  automation  scripts 
april 2014 by jm
websocketd
'like inetd, but for WebSockets' -- 'a small command line tool that will wrap an existing command line interface program, and allow it to be accessed via a WebSocket. It provides a quick mechanism for allowing web-applications to interact with existing command line tools.'

Awesome idea. BSD-licensed. (Via Mike Loukides)
websockets  cli  server  tools  unix  inetd  web  http  open-source 
december 2013 by jm
Flock for Cron jobs
good blog post writing up the 'flock -n -c' trick to ensure single-concurrent-process locking for cron jobs
cron  concurrency  unix  linux  flock  locking  ops 
december 2013 by jm
Shared Libraries are Obsolete
pretty strong argument. However, I think shlibs still have an advantage in that their pages are easier to share...
shared-libraries  unix  linux  linker  deployment 
november 2013 by jm
pt-summary
from the Percona toolkit. 'Conveniently summarizes the status and configuration of a server. It is not a tuning tool or diagnosis tool. It produces a report that is easy to diff and can be pasted into emails without losing the formatting. This tool works well on many types of Unix systems.' --- summarises OOM history, top, netstat connection table, interface stats, network config, RAID, LVM, disks, inodes, disk scheduling, mounts, memory, processors, and CPU.
percona  tools  cli  unix  ops  linux  diagnosis  raid  netstat  oom 
october 2013 by jm
How to configure ntpd so it will not move time backwards
The "-x" switch will expand the step/slew boundary from 128ms to 600 seconds, ensuring the time is slewed (drifted slowly towards the correct time at a max of 5ms per second) rather than "stepped" (a sudden jump, potentially backwards). Since slewing has a max of 5ms per second, time can never "jump backwards", which is important to avoid some major application bugs (particularly in Java timers).
ntpd  time  ntp  ops  sysadmin  slew  stepping  time-synchronization  linux  unix  java  bugs 
august 2013 by jm
gnuplot's dumb terminal
Turns out gnuplot has a pretty readable ASCII terminal rendering mode; combined with 'watch' it makes for a nifty graphing one-liner
gnuplot  plotting  charts  graphs  cli  command-line  unix  gnu  hacks  dataviz  visualization  ascii 
june 2013 by jm
graphite-metrics
metric collectors for various stuff not (or poorly) handled by other monitoring daemons

Core of the project is a simple daemon (harvestd), which collects metric values and sends them to graphite carbon daemon (and/or other configured destinations) once per interval. Includes separate data collection components ("collectors") for processing of:

/proc/slabinfo for useful-to-watch values, not everything (configurable).
/proc/vmstat and /proc/meminfo in a consistent way.
/proc/stat for irq, softirq, forks.
/proc/buddyinfo and /proc/pagetypeinfo (memory fragmentation).
/proc/interrupts and /proc/softirqs.
Cron log to produce start/finish events and duration for each job into a separate metrics, adapts jobs to metric names with regexes.
Per-system-service accounting using systemd and it's cgroups.
sysstat data from sadc logs (use something like sadc -F -L -S DISK -S XDISK -S POWER 60 to have more stuff logged there) via sadf binary and it's json export (sadf -j, supported since sysstat-10.0.something, iirc).
iptables rule "hits" packet and byte counters, taken from ip{,6}tables-save, mapped via separate "table chain_name rule_no metric_name" file, which should be generated along with firewall rules (I use this script to do that).


Pretty exhaustive list of system metrics -- could have some interesting ideas for Linux OS-level metrics to monitor in future.
graphite  monitoring  metrics  unix  linux  ops  vm  iptables  sysadmin 
june 2013 by jm
HyperLevelDB: A High-Performance LevelDB Fork
'HyperLevelDB improves on LevelDB in two key ways:
Improved parallelism: HyperLevelDB uses more fine-grained locking internally to provide higher throughput for multiple writer threads.
Improved compaction: HyperLevelDB uses a different method of compaction that achieves higher throughput for write-heavy workloads, even as the database grows.'
leveldb  storage  key-value-stores  persistence  unix  libraries  open-source 
june 2013 by jm
jq
like sed for JSON data – you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text. [it] is written in portable C, and it has zero runtime dependencies. You can download a single binary, scp it to a far away machine, and expect it to work.


Nice tool. Needs to get into the Debian/Ubuntu apt repos pronto ;)
jq  tools  cli  via:peakscale  json  coding  data  sed  unix 
april 2013 by jm
moreutils
Some really cool-looking UNIX command line utils, packaged in Debian (and therefore in Ubuntu too). A few of these I've reimplemented separately, but it's always good to replace a hack with a more widely available "official" tool. Thanks, Joey Hess!
sponge: accept input, wait til EOF, then rewrite a file;
chronic: runs a command quietly unless it fails;
combine: combine the lines in two files using boolean operations;
ifdata: get network interface info without parsing ifconfig output;
ifne: run a program if the standard input is not empty;
isutf8: check if a file or standard input is utf-8;
lckdo: execute a program with a lock held;
mispipe: pipe two commands, returning the exit status of the first;
parallel: run multiple jobs at once;
pee: tee standard input to pipes;
sponge: soak up standard input and write to a file;
ts: timestamp standard input;
vidir: edit a directory in your text editor;
vipe: insert a text editor into a pipe;
zrun: automatically uncompress arguments to command
bash  shell  cli  unix  scripting  via:peakscale  joey-hess  debian  ubuntu  tools  command-line  commands 
march 2013 by jm
runit
'a UNIX init scheme with service supervision' - philosophically similar to daemontools, widely packaged, LSB init.d-script-compliant, BSD-licensed
daemon  supervision  services  unix  lsb  server  ops 
january 2013 by jm
EFF Wins Protection for Time Zone Database
'The Electronic Frontier Foundation (EFF) is pleased to announce that a copyright lawsuit threatening an important database of time zone information has been dismissed. The astrology software company that filed the lawsuit, Astrolabe, has also apologized and agreed to a 'covenant not to sue' going forward, which will help protect the database from future baseless legal actions and disruptions.

Software engineers around the world depend on the time zone database to make sure that time-stamps for email and other files work correctly no matter where you are. However, last September, Astrolabe filed a lawsuit against Arthur David Olson and Paul Eggert – the researchers who coordinated the database's development for decades – because the database includes information from an atlas in which Astrolabe claimed to own copyright. But facts – like what time the sun rises – are not copyrightable. EFF, along with co-counsel Adam Kessel and Olivia Nguyen at the Boston office of Fish & Richardson P.C, promptly signed on to defend Olson and Eggert and protect this essential tool. In January, EFF advised Astrolabe that Olson and Eggert would move for sanctions if Astrolabe did not withdraw its complaint. Today's dismissal followed.'
copyright  eff  timezones  via:fanf  time  unix  olson 
february 2012 by jm
vmtouch - the Virtual Memory Toucher
'vmtouch is a tool for learning about and controlling the file system cache of unix and unix-like systems. It is BSD licensed.'
vmtouch  vm  linux  unix  fs  filesystems  instagram 
december 2011 by jm
Determining response times with tcprstat
'Tcprstat is a free, open-source TCP analysis tool that watches network traffic and computes the delay between requests and responses. From this it derives response-time statistics and prints them out.' Computes percentiles, too
tcp  tcprstat  tcp-ip  networking  measurement  statistics  performance  instrumentation  linux  unix  tools  cli 
november 2011 by jm
DTrace and Erlang
from Basho, via istvan. DTrace is becoming more compelling as a deep instrumentation/monitoring API -- I didn't realise disabled DTrace probes were virtually 0-overhead (a "2 NOOP instruction placeholder", apparently), that's nifty. Wonder if they've fixed the licensing mess, though
dtrace  monitoring  instrumentation  debugging  tracing  unix  erlang  via:istvan 
november 2011 by jm
A few git tips you didn't know about
'git checkout -t' alone is worth the bookmark
git  tips  coding  unix  reference  tricks  via:proggit 
september 2011 by jm
VT220 terminal hooked up to a 2010-era Mac Pro
with picture. WAAAAAANT despite the sheer massive impracticality of it. I still love VT*20 terminals....
unix  vt220  mac  terminal  dec  digital  retro 
august 2011 by jm
Telehack: May the command line live forever
'Connected to TELEHACK port 13 / It is 8:16 am on Saturday, April 30, 2011 in Mountain View, California, USA. There are 10 local users. There are 24139 hosts on the network.' via Waxy
via:waxy  simulation  history  telnet  arpanet  networking  unix  bbs 
june 2011 by jm
Cool, but obscure unix tools
these are great - some new ones on me!
cli  linux  terminal  unix  tools  command-line 
may 2011 by jm
gist: 782263 - How to redirect a running process' output to a file and logout
a nifty gdb hack; essentially dup()s a couple of files in /tmp in place of fd 1 and 2, then uses the bashism "detach" to nohup the running process
gdb  hacks  linux  process  shell  unix  via:hn  nifty  dup  detach  bash  from delicious
january 2011 by jm
Project Middleman
another concurrency shell command; interesting approach to dashboarding the results, with the "mdm.screen" utility provided
mdm  unix  concurrency  shell  linux  forking  background  xargs  parallelism  from delicious
october 2010 by jm
ioprofile
wraps strace(1) to summarise and aggregate I/O ops performed by a Linux process. looks pretty nifty (via Jeremy Zawodny)
via:jzawodny  io  strace  linux  monitoring  debugging  performance  profiling  sysadmin  ioprofile  unix  tools  from delicious
october 2010 by jm
How I boosted my Vim
some interesting tips; even handy for a seasoned vimmer like myself. I like the idea of remapping ";" to ":" to save 2 keystrokes
remapping  vim  keyboard  control  ui  unix  vi  ed_man  from delicious
september 2010 by jm
moreutils
'a growing collection of the Unix tools that nobody thought to write long ago, when Unix was young.' these are really, really nifty (via popey)
via:popey  unix  cli  command-line  linux  moreutils  sponge  pee  ts  vipe  zrun  perl  from delicious
july 2010 by jm
XZ Utils
15% smaller than bzip, 30% smaller than gzip, and now shipped with Fedora and Ubuntu. uses LZMA2
xz  xzdec  gzip  bzip  compression  lzma  via:wmf  unix  compress  from delicious
october 2009 by jm
Unicorn at GitHub
new Ruby HTTP server, using a preforked process pool based on select(). Github like it because of failure-recovery problems with Ruby threading bugs in Mongrel. The preforking algo used is extremely rudimentary -- the kind of thing we used in SpamAssassin before I implemented Apache-style preforking in 3.0
web  ruby  rails  github  nginx  httpd  server  mongrel  unicorn  rubyonrails  preforking  unix  fork  select  process-pool  from delicious
october 2009 by jm
Colm argues against the 'sleep rand % 3600' hack
it's not sufficiently evenly-distributed, apparently. Also: got linked from Hack The Planet!
scheduling  probability  sleep  unix  updating  cron  random  from delicious
september 2009 by jm
Postfix - (almost) a satellite system
how to keep a small number of user accounts (ie. root) delivering locally while the rest are delivered to a smarthost
postfix  sysadmin  unix  mail  mta  smtp  from delicious
september 2009 by jm
Ben Collins-Sussman: The True Path
Ben C-S rejigs the legendary "ed, man! !man ed" net.humor for a new age of DVCSes
dvcs  ed  git  funny  humour  zealotry  unix  from delicious
september 2009 by jm
filemap
'File-based, rather than tuple-based processing'; based around UNIX command-line toolset; good UNIXish UI; lots of caching of intermediate results; low setup overhead -- although it does require a shared POSIX filesystem, e.g. NFS, for synchronization
networking  python  opensource  grid  map-reduce  filemap  files  unix  command-line  parallel  distcomp 
july 2009 by jm
Infrastructures.Org: Best Practices in Automated Systems Administration and Infrastructure Architecture: Gold Server
well-written, and it's good to see version control listed right at the top of the list. But quite dead; interesting for historical reasons only at this stage
via:fanf  deployment  sysadmin  unix  rsync  ssh  cvs  infrastructure  cfengine 
july 2009 by jm

related tags

ack  ag  air-gap  airflow  aix  al-tobey  analysis  architecture  arpanet  ascii  atomic  automation  aws  axel  background  backticks  backups  bash  bbs  bloat  boeing  booting  boyer-moore  bsd  btrfs  buffers  bugs  build  bzip  cache  caching  cfengine  cgi  charts  circus  cli  clojure  coda-hale  coding  command-line  commands  compatibility  complexity  compress  compression  concurrency  containers  control  control-z  copy-on-write  copyright  coreutils  crash-only-software  critiques  cron  css  csv  cups  curl  cvs  daemon  daemons  data  dataviz  dbus  debian  debugging  dec  deployment  design  detach  devops  diagnosis  digital  distcomp  distros  dns  docker  documentation  downloading  dtrace  dumb-init  dup  duplicity  duply  dvcs  ed  edge  ed_man  eff  egos  environment  erlang  explainshell  exploits  fail  fcron  fedora  file  filemap  files  filesystems  flock  fork  forking  fp  fs  fsync  ftp  funny  gdb  git  github  gnome  gnu  gnuplot  go  god  google-storage  gradle  graphite  graphs  grep  grid  gzip  hacking  hacks  hid  history  home-directory  horror  html  http  httpd  humour  incremental-search  inetd  infrastructure  init  inotify  instagram  instrumentation  interactive  io  ioprofile  iot  iptables  java  javascript  jit  job  joey-hess  jq  json  key-value-stores  keyboard  lambda  languages  latencies  latency  leveldb  libraries  linker  linus-torvalds  linux  locking  logging  lsb  lzma  mac  mail  manual-pages  map-reduce  martin-fowler  mdm  measurement  memory  menus  metrics  miller  mmap  mongrel  monitoring  moreutils  mozilla  mta  named-pipes  nannies  netstat  networking  nginx  nifty  nix  nixos  node.js  ntp  ntpd  olson  omgwtfbbq  one-liners  oom  open-source  opensource  openstack  operating-systems  ops  opscookie  os  osdi  packaging  papers  parallel  parallelism  parsing  patterns  pcre  peco  pee  percentiles  percona  performance  perl  persistence  pipelines  pipes  planes  plotting  popen  posix  postfix  preforking  printing  probability  process  process-pool  process-substitution  processes  procfs  productivity  profiling  python  raid  rails  random  rant  red-hat  reference  reliability  remapping  remote  resolvers  retro  rollback  rsync  ruby  rubyonrails  runit  s3  scheduler  scheduling  scripting  scripts  search  security  sed  select  selection  server  servers  services  sh  shared-libraries  shell  signals  simulation  slashdot  sleep  slew  sljit  smtp  software  solaris  sponge  ssh  statistics  stats  stepping  storage  strace  streams  student  supervision  supervisord  suspend  syntax  sysadmin  system  system-v  systemd  t-test  tcp  tcp-ip  tcprstat  telnet  terminal  testing  time  time-synchronization  timezones  tinystat  tips  tmux  tools  tq  tracing  tricks  ts  tsv  twitter  ubuntu  udp  ui  unicorn  unionfs  unix  updating  upgrades  upstart  uselessd  versioning  vertx  vi  via:chughes  via:elliottucker  via:fanf  via:hackernews  via:hn  via:igrigorik  via:istvan  via:johnke  via:jzawodny  via:obfuscurity  via:peakscale  via:pixelbeat  via:popey  via:proggit  via:waxy  via:wmf  vim  vipe  visualization  vm  vmtouch  vt220  web  websockets  wget  windows  xargs  xz  xzdec  yelp  zealotry  zrun 

Copy this bookmark:



description:


tags: