jm + cli   51

aws-vault
'A vault for securely storing and accessing AWS credentials in development environments'.

Scott Piper says: 'You should not use the AWS CLI with MFA without aws-vault, and probably should not use the CLI at all without aws-vault, because of it's benefit of storing your keys outside of ~/.aws/credentials (since every once in a while a developer will decide to upload all their dot-files in their home directory to github so they can use the same .vimrc and .bashrc aliases everywhere, and will end up uploading their AWS creds).'
aws  vault  security  cli  development  coding  dotfiles  credentials  mfa 
15 days ago by jm
GitHub - jorgebastida/awslogs: AWS CloudWatch logs for Humans™
This feature alone is a bit of a killer app:
$ awslogs get /var/log/syslog ip-10-1.* --start='2h ago' | grep ERROR


Nice.
cli  logging  aws  cloudwatch  logs  awslogs  ec2 
august 2017 by jm
explainshell.com
This is pretty excellent work -- paste a UNIX command line and it'll contextually inline manual page snippets to match, highlighting the matching part of the command line.
cli  unix  documentation  explainshell  shell  scripting  syntax  manual-pages 
may 2017 by jm
saitoha/libsixel: The new standard of SIXEL development
SIXEL is one of image formats for printer and terminal imaging introduced by Digital Equipment Corp. (DEC). Its data scheme is represented as a terminal-friendly escape sequence. So if you want to view a SIXEL image file, all you have to do is "cat" it to your terminal.


Wow. gif320 lives on!
cli  graphics  terminal  dec  vt320  vt300  sixel  images 
april 2017 by jm
USE Method: Linux Performance Checklist
Really late in bookmarking this, but has some up-to-date sample commandlines for sar, mpstat and iostat on linux
linux  sar  iostat  mpstat  cli  ops  sysadmin  performance  tuning  use  metrics 
june 2016 by jm
#825394 - systemd kill background processes after user logs out - Debian Bug report logs
Systemd breaks UNIX behaviour which has been standard practice for 30 years:
It is now indeed the case that any background processes that were still
running are killed automatically when the user logs out of a session,
whether it was a desktop session, a VT session, or when you SSHed into a
machine. Now you can no longer expect a long running background processes to
continue after logging out. I believe this breaks the expectations of
many users. For example, you can no longer start a screen or tmux
session, log out, and expect to come back to it.
systemd  ops  debian  linux  fail  background  cli  commandline 
may 2016 by jm
plainas/tq
command line utility that performs an HTML element selection on HTML content passed to the stdin. Using css selectors that everybody knows. Since input comes from stdin and output is sent to stdout, it can easily be used inside traditional UNIX pipelines to extract content from webpages and html files. tq provides extra formating options such as json-encoding or newlines squashing, so it can play nicely with everyones favourite command line tooling.
tq  linux  unix  cli  command-line  html  parsing  css  tools 
may 2016 by jm
raboof/nethogs: Linux 'net top' tool
NetHogs is a small 'net top' tool. Instead of breaking the traffic down per protocol or per subnet, like most tools do, it groups bandwidth by process.
nethogs  cli  networking  performance  measurement  ops  linux  top 
may 2016 by jm
lbzip2
a free, multi-threaded compression utility with support for bzip2 compressed file format. lbzip2 can process standard bz2 files in parallel. It uses POSIX threading model (pthreads), which allows it to take full advantage of symmetric multiprocessing (SMP) systems. It has been proven to scale linearly, even to over one hundred processor cores.

lbzip2 is fully compatible with bzip2 – both at file format and command line level. Files created by lbzip2 can be decompressed by all versions of bzip2 and other software supporting bz2 format. lbzip2 can decompress any bz2 files in parallel. All bzip2 command-line options are also accepted by lbzip2. This makes lbzip2 a drop-in replacement for bzip2.
bzip2  gzip  compression  lbzip2  parallel  cli  tools 
march 2016 by jm
percol
interactive menu selection for the UNIX command line
cli  linux  unix  grep  menus  selection  ui  interactive  terminal 
february 2016 by jm
Amazon ECS CLI Tutorial - Amazon EC2 Container Service
super-basic ECS tutorial, using a docker-compose.yml to create a new ECS-managed service fleet
ecs  cli  linux  aws  ec2  hosting  docker  tutorials 
october 2015 by jm
Hologram
Hologram exposes an imitation of the EC2 instance metadata service on developer workstations that supports the [IAM Roles] temporary credentials workflow. It is accessible via the same HTTP endpoint to calling SDKs, so your code can use the same process in both development and production. The keys that Hologram provisions are temporary, so EC2 access can be centrally controlled without direct administrative access to developer workstations.
iam  roles  ec2  authorization  aws  adroll  open-source  cli  osx  coding  dev 
october 2015 by jm
Google Cloud Shell
your command line environment in the [Google] Cloud. This feature enables you to connect to a shell environment on a virtual machine, pre-loaded with the tools you need to easily run commands to develop, deploy and manage your projects. Currently, Cloud Shell is an f1-micro Google Compute Engine machine that exposes a Debian-based development environment. You are also assigned 5 GB of standard persistent disk space as the home disk so you can store files between sessions.


It's also free. This is a great idea -- handy both for beginners getting to grips with GoogCloud and for experts looking for a quite dev env to hack with. I wish AWS had something similar.
google  cloud  shell  google-cloud  gcs  gce  cli  tools 
october 2015 by jm
GoTTY
'a simple command line tool that turns your CLI tools into web applications'
cli  terminal  web  tools  unix 
september 2015 by jm
Miller
'like sed, awk, cut, join, and sort for name-indexed data such as CSV'


Written in "modern C" with zero runtime dependencies. Looks great
cli  csv  unix  miller  tsv  data  tools 
august 2015 by jm
peco
'Simplistic interactive filtering tool' -- live incremental-search filtering in a terminal window
cli  shell  terminal  tools  go  peco  interactive  incremental-search  search  ui  unix 
june 2015 by jm
Kappa
'a command line tool that (hopefully) makes it easier to deploy, update, and test functions for AWS Lambda.' much needed IMO -- Lambda is too closed
aws  lambda  mitch-garnaat  coding  testing  cli  kappa 
april 2015 by jm
ShellCheck
Static code analysis for shell scripts (via Tony Finch)
bash  cli  sh  linux  shell  coding  static-analysis  lint 
april 2015 by jm
Subscribing AWS Lambda Function To SNS Topic With aws-cli
how to use the AWS command line tools to do this
aws  aws-cli  cli  lambda  sns  hacks 
april 2015 by jm
Ag: faster than Ack
Some nice performance tricks; I particularly like the use of sljit:
Ag uses Pthreads to take advantage of multiple CPU cores and search files in parallel.
Files are mmap()ed instead of read into a buffer.
Literal string searching uses Boyer-Moore strstr.
Regex searching uses PCRE's JIT compiler (if Ag is built with PCRE >=8.21).
Ag calls pcre_study() before executing the same regex on every file.
Instead of calling fnmatch() on every pattern in your ignore files, non-regex patterns are loaded into arrays and binary searched.
jit  cli  grep  search  ack  ag  unix  pcre  sljit  boyer-moore  tools 
march 2015 by jm
Using Named Pipes and Process Substitution in Bioinformatics
Wow. I've been using bash for nigh on 14 years and I didn't know about process substitution. Nifty trick
bash  linux  pipes  shell  unix  via:igrigorik  cli  named-pipes  process-substitution 
march 2015 by jm
mermaid
Generate graphs/flowcharts from text a la Markdown. Pretty much identical to graphviz surely?
mermaid  graphviz  markdown  cli  open-source 
december 2014 by jm
wrk2
'A constant throughput, correct latency-recording variant of wrk. This is a must-have when measuring network service latency -- corrects for Coordinated Omission error:
wrk's model, which is similar to the model found in many current load generators, computes the latency for a given request as the time from the sending of the first byte of the request to the time the complete response was received. While this model correctly measures the actual completion time of individual requests, it exhibits a strong Coordinated Omission effect, through which most of the high latency artifacts exhibited by the measured server will be ignored. Since each connection will only begin to send a request after receiving a response, high latency responses result in the load generator coordinating with the server to avoid measurement during high latency periods.
wrk  latency  measurement  tools  cli  http  load-testing  testing  load-generation  coordinated-omission  gil-tene 
november 2014 by jm
sferik/t
"A command-line power tool for Twitter." It really is -- much better timeline searchability than the "real" Twitter UI, for example
twitter  ruby  github  cli  tools  unix  search 
october 2014 by jm
sysdig
open source, system-level exploration: capture system state and activity from a running Linux instance, then save, filter and analyze.
Think of it as strace + tcpdump + lsof + awesome sauce.
With a little Lua cherry on top.


This sounds excellent. Linux-based, GPLv2.
debugging  tools  linux  ops  tracing  strace  open-source  sysdig  cli  tcpdump  lsof 
april 2014 by jm
htcat/htcat
a utility to perform parallel, pipelined execution of a single HTTP GET. htcat is intended for the purpose of incantations like: htcat https://host.net/file.tar.gz | tar -zx

It is tuned (and only really useful) for faster interconnects: [....] 109MB/s on a gigabit network, between an AWS EC2 instance and S3. This represents 91% use of the theoretical maximum of gigabit (119.2 MiB/s).
go  cli  http  file-transfer  ops  tools 
march 2014 by jm
Harry - A Tool for Measuring String Similarity
a small tool for comparing strings and measuring their similarity. The tool supports several common distance and kernel functions for strings as well as some exotic similarity measures. The focus of Harry lies on implicit similarity measures, that is, comparison functions that do not give rise to an explicit vector space. Examples of such similarity measures are the Levenshtein distance and the Jaro-Winkler distance.
For comparison Harry loads a set of strings from input, computes the specified similarity measure and writes a matrix of similarity values to output. The similarity measure can be computed based on the granularity of characters as well as words contained in the strings. The configuration of this process, such as the input format, the similarity measure and the output format, are specified in a configuration file and can be additionally refined using command-line options.
Harry is implemented using OpenMP, such that the computation time for a set of strings scales linear with the number of available CPU cores. Moreover, efficient implementations of several similarity measures, effective caching of similarity values and low-overhead locking further speedup the computation.


via kragen.
via:kragen  strings  similarity  levenshtein-distance  algorithms  openmp  jaro-winkler  edit-distance  cli  commandline  hamming-distance  compression 
january 2014 by jm
websocketd
'like inetd, but for WebSockets' -- 'a small command line tool that will wrap an existing command line interface program, and allow it to be accessed via a WebSocket. It provides a quick mechanism for allowing web-applications to interact with existing command line tools.'

Awesome idea. BSD-licensed. (Via Mike Loukides)
websockets  cli  server  tools  unix  inetd  web  http  open-source 
december 2013 by jm
pt-summary
from the Percona toolkit. 'Conveniently summarizes the status and configuration of a server. It is not a tuning tool or diagnosis tool. It produces a report that is easy to diff and can be pasted into emails without losing the formatting. This tool works well on many types of Unix systems.' --- summarises OOM history, top, netstat connection table, interface stats, network config, RAID, LVM, disks, inodes, disk scheduling, mounts, memory, processors, and CPU.
percona  tools  cli  unix  ops  linux  diagnosis  raid  netstat  oom 
october 2013 by jm
awscli

The future of the AWS command line tools is awscli, a single, unified, consistent command line tool that works with almost all of the AWS services. Here is a quick list of the services that awscli currently supports: Auto Scaling, CloudFormation, CloudSearch, CloudWatch, Data Pipeline, Direct Connect, DynamoDB, EC2, ElastiCache, Elastic Beanstalk, Elastic Transcoder, ELB, EMR, Identity and Access Management, Import/Export, OpsWorks, RDS, Redshift, Route 53, S3, SES, SNS, SQS, Storage Gateway, Security Token Service, Support API, SWF, VPC. Support for the following appears to be planned: CloudFront, Glacier, SimpleDB.

The awscli software is being actively developed as an open source project on Github, with a lot of support from Amazon. You’ll note that the biggest contributors to awscli are Amazon employees with Mitch Garnaat leading. Mitch is also the author of boto, the amazing Python library for AWS.
aws  awscli  cli  tools  command-line  ec2  s3  amazon  api 
august 2013 by jm
Juniper Adds Puppet support
This is super-cool.

'Network engineering no longer should be mundane tasks like conf, set interfaces fe-0/0/0 unit o family inet address 10.1.1.1/24. How does mindless CLI work translate to efficiently spent time ? What if you need to change 300 devices? What if you are writing it by hand? An error-prone waste of time. Juniper today announced Puppet support for their 12.2R3,5 JUNOS code. This is compatible with EX4200, EX4550, and QFX3500 switches. These are top end switches, but this start is directly aimed at their DC and enterprise devices. Initially, the manifest interactions offered are interface, layer 2 interface, vlan, port aggregation groups, and device names.'

Based on what I saw in the Network Automation team in Amazon, this is an amazing leap forward; it'd instantly render obsolete a bunch of horrific SSH-CLI automation cruft.
ssh  cli  automation  networking  networks  puppet  ops  juniper  cisco 
august 2013 by jm
gnuplot's dumb terminal
Turns out gnuplot has a pretty readable ASCII terminal rendering mode; combined with 'watch' it makes for a nifty graphing one-liner
gnuplot  plotting  charts  graphs  cli  command-line  unix  gnu  hacks  dataviz  visualization  ascii 
june 2013 by jm
shades
A command-line utility in Ruby to perform (a) OLAP cubing and (b) histogramming, given whitespace-delimited line data
ruby  olap  number-crunching  data  histograms  cli 
june 2013 by jm
jq
like sed for JSON data – you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text. [it] is written in portable C, and it has zero runtime dependencies. You can download a single binary, scp it to a far away machine, and expect it to work.


Nice tool. Needs to get into the Debian/Ubuntu apt repos pronto ;)
jq  tools  cli  via:peakscale  json  coding  data  sed  unix 
april 2013 by jm
OS X Lion tweaks
A bunch of magic command lines to set useful OS X prefs without pointy-clicky. at least some also seem to work on Mountain Lion
tweaks  lion  mac  reference  cli  mountain-lion  macosx 
march 2013 by jm
Using DiffMerge as your Git visual merge and diff tool
A decent 3-way-diff GUI merge tool which works with git on OSX. "git config" command-lines included in this blog post
git  merge  osx  mac  macosx  diff  mergetool  merging  cli  diffmerge 
march 2013 by jm
moreutils
Some really cool-looking UNIX command line utils, packaged in Debian (and therefore in Ubuntu too). A few of these I've reimplemented separately, but it's always good to replace a hack with a more widely available "official" tool. Thanks, Joey Hess!
sponge: accept input, wait til EOF, then rewrite a file;
chronic: runs a command quietly unless it fails;
combine: combine the lines in two files using boolean operations;
ifdata: get network interface info without parsing ifconfig output;
ifne: run a program if the standard input is not empty;
isutf8: check if a file or standard input is utf-8;
lckdo: execute a program with a lock held;
mispipe: pipe two commands, returning the exit status of the first;
parallel: run multiple jobs at once;
pee: tee standard input to pipes;
sponge: soak up standard input and write to a file;
ts: timestamp standard input;
vidir: edit a directory in your text editor;
vipe: insert a text editor into a pipe;
zrun: automatically uncompress arguments to command
bash  shell  cli  unix  scripting  via:peakscale  joey-hess  debian  ubuntu  tools  command-line  commands 
march 2013 by jm
Making Really Executable Jars
Who knew? you can make a runnable JAR file!
There has long been a hack known in some circles, but not widely known, to make jars really executable, in the chmod +x sense. The hack takes advantage of the fact that jar files are zip files, and zip files allow arbitrary cruft to be prepended to the zip file itself (this is how self-extracting zip files work).
jars  via:netflix  shell  java  executable  chmod  zip  hacks  command-line  cli 
march 2013 by jm
airlift/airline · GitHub
Annotations-based git-like CLI helper for Java
git  cli  java 
january 2013 by jm
spark
sparklines in your terminal window. Simply give it a comma or space-separated list of data values, and it'll generate an ANSI-graphics sparkline chart. Brilliant! (via mjd)
via:mjdominus  sparklines  charts  graphs  bash  shell  terminal  cli  ansi 
december 2011 by jm
Determining response times with tcprstat
'Tcprstat is a free, open-source TCP analysis tool that watches network traffic and computes the delay between requests and responses. From this it derives response-time statistics and prints them out.' Computes percentiles, too
tcp  tcprstat  tcp-ip  networking  measurement  statistics  performance  instrumentation  linux  unix  tools  cli 
november 2011 by jm
Linux SS Utility To Investigate Sockets / Network Connections
'When amount of sockets is enough large, netstat or even plain cat /proc/net/tcp/ cause nothing but pains and curses. In linux-2.4 the desease [sic] became worse: even if amount of sockets is small reading /proc/net/tcp/ is slow enough. This utility presents a new approach, which is supposed to scale well.' via scanlan
via:scanlan  ss  linux  sockets  networking  tools  cli 
october 2011 by jm
Cool, but obscure unix tools
these are great - some new ones on me!
cli  linux  terminal  unix  tools  command-line 
may 2011 by jm
RecordStream
'A set of programs for creating, manipulating, and outputing a stream of Records, or hashes. Inspired by Monad.' looks very powerful
monad  recordstream  open-source  recs  cli  grep  from delicious
september 2010 by jm
autojump
interesting idea; extend "cd" to track which directories you cd to most frequently, then add a command to "jump" to the most-frequently used one which matches a substring you specify
autojump  cli  bash  command-line  navigation  terminal  shell  directory  cd  from delicious
july 2010 by jm
Mac OS X command-line tricks
not quite up to par with modern Ubuntu, but still a few interesting ones here for when I'm stuck using the missus' laptop ;)
apple  bash  cli  osx  mac  sysadmin  shell  tricks  command-line  from delicious
july 2010 by jm
moreutils
'a growing collection of the Unix tools that nobody thought to write long ago, when Unix was young.' these are really, really nifty (via popey)
via:popey  unix  cli  command-line  linux  moreutils  sponge  pee  ts  vipe  zrun  perl  from delicious
july 2010 by jm
practical Linux commands quick-ref sheet
from Padraig Brady. lots of nice one-liners I wasn't familiar with
padraig-brady  bash  cli  linux  reference  sysadmin  tips  commands  from delicious
june 2010 by jm
Top Ten One-Liners from CommandLineFu Explained
worth it for #10: 'Capture video of a linux desktop': '$ ffmpeg -f x11grab -s wxga -r 25 -i :0.0 -sameq /tmp/out.mpg'
video  capture  x11  ffmpeg  cli  bash  linux  from delicious
march 2010 by jm

related tags

ack  adroll  ag  algorithms  amazon  ansi  api  apple  ascii  authorization  autojump  automation  aws  aws-cli  awscli  awslogs  background  bash  boyer-moore  bzip2  capture  cd  charts  chmod  cisco  cli  cloud  cloudflare  cloudwatch  coding  command-line  commandline  commands  compression  coordinated-omission  copying  credentials  css  csv  data  dataviz  debian  debugging  dec  dev  development  diagnosis  diff  diffmerge  directory  distribution  docker  documentation  dotfiles  ec2  ecs  edit-distance  executable  explainshell  fail  ffmpeg  file-transfer  gce  gcs  gil-tene  git  git-bundle  github  gnu  gnuplot  go  google  google-cloud  graphics  graphs  graphviz  grep  gzip  hacks  hamming-distance  histograms  hosting  html  http  http2  iam  images  incremental-search  inetd  instrumentation  interactive  iostat  jaro-winkler  jars  java  jit  joey-hess  jq  json  juniper  kappa  lambda  latency  lbzip2  levenshtein-distance  lint  linux  lion  load-generation  load-testing  logging  logs  lsof  mac  macosx  manual-pages  markdown  measurement  menus  merge  mergetool  merging  mermaid  metrics  mfa  miller  mitch-garnaat  monad  moreutils  mountain-lion  mpstat  named-pipes  navigation  nethogs  netstat  networking  networks  number-crunching  olap  oom  open-source  openmp  ops  osx  padraig-brady  parallel  parsing  pcre  peco  pee  percona  performance  perl  pipes  plotting  process-substitution  puppet  raid  recordstream  recs  reference  roles  ruby  s3  sar  scripting  search  security  sed  selection  server  sh  shell  similarity  sixel  sljit  sns  sockets  sparklines  spdy  sponge  ss  ssh  static-analysis  statistics  strace  strings  syntax  sysadmin  sysdig  systemd  tcp  tcp-ip  tcpdump  tcprstat  terminal  testing  tips  tools  top  tq  tracing  tricks  ts  tsv  tuning  tutorials  tweaks  twitter  ubuntu  ui  unix  use  vault  via:igrigorik  via:kragen  via:mjdominus  via:netflix  via:peakscale  via:popey  via:scanlan  video  vipe  visualization  vt300  vt320  web  websockets  wrk  x11  zip  zrun 

Copy this bookmark:



description:


tags: