jm + configuration   16

mozilla/sops: Secrets management stinks, use some sops!
sops is an editor of encrypted files that supports YAML, JSON and BINARY formats and encrypts with AWS KMS and PGP.
secrets  encryption  security  kms  pgp  gpg  editors  configuration 
15 days ago by jm
'a small library to manage encrypted secrets using asymmetric encryption.'
The main benefits provided by ejson are:

Secrets can be safely stored in a git repo.
Changes to secrets are auditable on a line-by-line basis with git blame.
Anyone with git commit access has access to write new secrets.
Decryption access can easily be locked down to production servers only.
Secrets change synchronously with application source (as opposed to secrets provisioned by Configuration Management).
Simple, well-tested, easily-auditable source.
crypto  security  credentials  encryption  ejson  json  configuration  config 
july 2016 by jm
The Challenges of Container Configuration // Speaker Deck
Some good advice on Docker metadata/config from Gareth Rushgrove
docker  metadata  configuration  build  devops  dev  containers  slidfes 
may 2016 by jm
How Facebook avoids failures
Great paper from Ben Maurer of Facebook in ACM Queue.
A "move-fast" mentality does not have to be at odds with reliability. To make these philosophies compatible, Facebook's infrastructure provides safety valves.

This is full of interesting techniques.

* Rapidly deployed configuration changes: Make everybody use a common configuration system; Statically validate configuration changes; Run a canary; Hold on to good configurations; Make it easy to revert.

* Hard dependencies on core services: Cache data from core services. Provide hardened APIs. Run fire drills.

* Increased latency and resource exhaustion: Controlled Delay (based on the anti-bufferbloat CoDel algorithm -- this is really cool); Adaptive LIFO (last-in, first-out) for queue busting; Concurrency Control (essentially a form of circuit breaker).

* Tools that Help Diagnose Failures: High-Density Dashboards with Cubism (horizon charts); What just changed?

* Learning from Failure: the DERP (!) methodology,
ben-maurer  facebook  reliability  algorithms  codel  circuit-breakers  derp  failure  ops  cubism  horizon-charts  charts  dependencies  soa  microservices  uptime  deployment  configuration  change-management 
november 2015 by jm
Holistic Configuration Management at Facebook
How FB push config changes from Git (where it is code reviewed, version controlled, and history tracked with strong auth) to Zeus (their Zookeeper fork) and from there to live production servers.
facebook  configuration  zookeeper  git  ops  architecture 
october 2015 by jm
Consul case study from Hootsuite
Hootsuite used Consul for distributed configuration, specifically dark-launch feature flags, with great results:

'Trying out bleeding edge software can be a risky proposition, but in the case of Consul, we’ve found it to be a solid system that works basically as described and was easy to get up and running. We managed to go from initial investigations to production within a month. The value was immediately obvious after looking into the key-value store combined with the events system and it’s DNS features and each of these has worked how we expected. Overall it has been fun to work with and has worked well and based on the initial work we have done with the Dark Launching system we’re feeling confident in Consul’s operation and are looking forward to expanding the scope of it’s use.'
consul  dark-launches  feature-flags  configuration  distributed  hootsuite  notification 
november 2014 by jm
on using JSON as a config file format
Ben Hughes on twitter:

"JSON is fine for config files, if you don't want to comment your config file. Which is a way of saying, it isn't fine for config files."
ben-hughes  funny  json  file-formats  config-files  configuration  software  coding 
september 2014 by jm
BPF - the forgotten bytecode
'In essence Tcpdump asks the kernel to execute a BPF program within the kernel context. This might sound risky, but actually isn't. Before executing the BPF bytecode kernel ensures that it's safe:

* All the jumps are only forward, which guarantees that there aren't any loops in the BPF program. Therefore it must terminate.
* All instructions, especially memory reads are valid and within range.
* The single BPF program has less than 4096 instructions.

All this guarantees that the BPF programs executed within kernel context will run fast and will never infinitely loop. That means the BPF programs are not Turing complete, but in practice they are expressive enough for the job and deal with packet filtering very well.'

Good example of a carefully-designed DSL allowing safe "programs" to be written and executed in a privileged context without security risk, or risk of running out of control.
coding  dsl  security  via:oisin  linux  tcpdump  bpf  bsd  kernel  turing-complete  configuration  languages 
may 2014 by jm
Nice-looking new tool from Hashicorp; service discovery and configuration service, built on Raft for leader election, Serf for gossip-based messaging, and Go. Some features:

* Gossip is performed over both TCP and UDP;

* gossip messages are encrypted symmetrically and therefore secure from eavesdropping, tampering, spoofing and packet corruption (like the incident which brought down S3 for days: );

* exposes both a HTTP interface and (even better) DNS;

* includes explicit support for long-distance WAN operation as well as on LANs.

It all looks very practical and usable. MPL-licensed.

The only potential risk I can see is that expecting to receive config updates from a blocking poll of the HTTP interface needs some good "best practice" docs, to ensure that people don't mishandle the scenario where there is a network partition between your calling code and the Consul server/agent. Without any heartbeating protocol behind the scenes, HTTP is vulnerable to "hung connections" which would result in a config change being silently missed by the client until the connection eventually is timed out, either by the calling code or the client-side kernel. This could potentially take minutes to occur, which in some usage scenarios could be a big, unforeseen problem.
configuration  service-discovery  distcomp  raft  consensus-algorithms  go  mpl  open-source  dns  http  gossip-protocol  hashicorp 
april 2014 by jm
A highly-available key value store for shared configuration and service discovery. etcd is inspired by zookeeper and doozer, with a focus on:

Simple: curl'able user facing API (HTTP+JSON);
Secure: optional SSL client cert authentication;
Fast: benchmarked 1000s of writes/s per instance;
Reliable: Properly distributed using Raft;

Etcd is written in go and uses the raft consensus algorithm to manage a highly availably replicated log.

One of the core components of CoreOS -- .
configuration  distributed  raft  ha  doozer  zookeeper  go  replication  consensus-algorithm  etcd  coreos 
august 2013 by jm
Announcing Zuul: Edge Service in the Cloud
Netflix' library to implement "edge services" -- ie. a front end to their API, web servers, and streaming servers. Some interesting features: dynamic filtering using Groovy scripts; Hystrix for software load balancing, fault tolerance, and error handling for originated HTTP requests; fine-grained service metrics; Archaius for configuration; and canary requests to detect overload risks. Pretty complex though
edge-services  api  netflix  zuul  archaius  canary-requests  http  groovy  hystrix  load-balancing  fault-tolerance  error-handling  configuration 
june 2013 by jm
Thoughts on configuration file complexity
some interesting thoughts on the old "Turing complete configuration language" question
configuration  turing-complete  programming  ops  testing 
march 2013 by jm
'SSH-Based Configuration Management & Deployment'. deploy via SSH; no target-side daemons required. GPLv3 licensed, unfortunately :(
ansible  devops  configuration  deployment  sysadmin  python  ssh 
july 2012 by jm
The things make got right (and how to make it better)
jgc provides a good demonstration of how a general-purpose programming language tends to make a crappy DSL -- specifically Rakefiles
dsl  build  make  coding  jgc  languages  configuration  makefiles  rake  ruby  from delicious
january 2011 by jm
Turing-incomplete Lua?
discussion thread on the cons of using Turing-complete general-purpose programming languages in places where it's not necessary, such as configuration files
configuration  turing-complete  safety  coding  software  lua  from delicious
december 2009 by jm

related tags

algorithms  ansible  api  archaius  architecture  ben-hughes  ben-maurer  bpf  bsd  build  canary-requests  cassandra  change-management  charts  circuit-breakers  codel  coding  config  config-files  configuration  consensus-algorithm  consensus-algorithms  consul  containers  coreos  credentials  crypto  cubism  dark-launches  dependencies  deployment  derp  dev  devops  distcomp  distributed  dns  docker  doozer  dsl  edge-services  editors  ejson  encryption  error-handling  etcd  facebook  failure  fault-tolerance  feature-flags  file-formats  funny  git  go  gossip-protocol  gpg  groovy  ha  hashicorp  hootsuite  horizon-charts  http  hystrix  jgc  json  kernel  kms  languages  linux  load-balancing  lua  make  makefiles  metadata  microservices  mpl  netflix  notification  open-source  ops  pgp  programming  python  raft  rake  reliability  replication  ruby  safety  scale  secrets  security  service-discovery  slidfes  soa  software  ssh  sysadmin  tcpdump  testing  tuning  turing-complete  uptime  via:oisin  zookeeper  zuul 

Copy this bookmark: