jm + deployment   44

Is Docker ready for production? Feedbacks of a 2 weeks hands on
I have to agree with this assessment -- there are a lot of loose ends still for production use of Docker in a SOA stack environment:
From my point of view, Docker is probably the best thing I’ve seen in ages to automate a build. It allows to pre build and reuse shared dependencies, ensuring they’re up to date and reducing your build time. It avoids you to either pollute your Jenkins environment or boot a costly and slow Virtualbox virtual machine using Vagrant. But I don’t feel like it’s production ready in a complex environment, because it adds too much complexity. And I’m not even sure that’s what it was designed for.
docker  complexity  devops  ops  production  deployment  soa  web-services  provisioning  networking  logging 
10 days ago by jm
Netflix release new code to production before completing tests
Interesting -- I hadn't heard of this being an official practise anywhere before (although we actually did it ourselves this week)...
If a build has made it [past the 'integration test' phase], it is ready to be deployed to one or more internal environments for user-acceptance testing. Users could be UI developers implementing a new feature using the API, UI Testers performing end-to-end testing or automated UI regression tests. As far as possible, we strive to not have user-acceptance tests be a gating factor for our deployments. We do this by wrapping functionality in Feature Flags so that it is turned off in Production while testing is happening in other environments. 
devops  deployment  feature-flags  release  testing  integration-tests  uat  qa  production  ops  gating  netflix 
20 days ago by jm
Avoiding Chef-Suck with Auto Scaling Groups - forty9ten
Some common problems which arise using Chef with ASGs in EC2, and how these guys avoided it -- they stopped using Chef for service provisioning, and instead baked AMIs when a new version was released. ASGs using pre-baked AMIs definitely works well so this makes good sense IMO.
infrastructure  chef  ops  asg  auto-scaling  ec2  provisioning  deployment 
5 weeks ago by jm
Richard Clayton - Failing at Microservices
Solid warts-and-all confessional blogpost about a team failing to implement a microservices architecture. I'd put most of the blame on insufficient infrastructure to support them (at a code level), inter-personal team problems, and inexperience with large-scale complex multi-service production deployment and the work it was going to require
microservices  devops  collaboration  architecture  fail  team  deployment  soa 
10 weeks ago by jm
Building a Smarter Application Stack - DevOps Ireland
This sounds like a very interesting Dublin meetup -- Engine Yard on thursday night:
This month, we'll have Tomas Doran from Yelp talking about Docker, service discovery, and deployments. 'There are many advantages to a container based, microservices architecture - however, as always, there is no silver bullet. Any serious deployment will involve multiple host machines, and will have a pressing need to migrate containers between hosts at some point. In such a dynamic world hard coding IP addresses, or even host names is not a viable solution. This talk will take a journey through how Yelp has solved the discovery problems using Airbnb’s SmartStack to dynamically discover service dependencies, and how this is helping unify our architecture, from traditional metal to EC2 ‘immutable’ SOA images, to Docker containers.'
meetups  talks  dublin  deployment  smartstack  ec2  docker  yelp  service-discovery 
june 2014 by jm
Database Migrations Done Right
The rule is simple. You should never tie database migrations to application deploys or vice versa. By minimising dependencies you enable faster, easier and cleaner deployments.


A solid description of why this is a good idea, from an ex-Guardian dev.
migrations  database  sql  mysql  postgres  deployment  ops  dependencies  loose-coupling 
may 2014 by jm
Go: Best Practices for Production Environments
how Soundcloud deploy their Go services, after 2.5 years of Go in production
go  tips  deployment  best-practices  soundcloud  ops 
april 2014 by jm
Continuous Delivery with ETL Systems [video]
Lonely Planet and Dr Foster Intelligence both make heavy use of ETL in their products, and both organisations have applied the principles of Continuous Delivery to their delivery process.
Some of the Continuous Delivery norms need to be adapted in the context of ETL, and some interesting patterns emerge, such as running Continuous Integration against data, as well as code.
etl  video  presentations  lonely-planet  dr-foster-intelligence  continuous-delivery  deployment  pipelines 
march 2014 by jm
The Microservice Declaration of Independence
"Microservices" seems to be yet another term for SOA; small, decoupled, independently-deployed services, with well-defined public HTTP APIs. Pretty much all the services I've worked on over the past few years have been built in this style. Still, let's keep an eye on this concept anyway.

Another definition seems to be a more FP-style one: http://www.slideshare.net/michaelneale/microservices-and-functional-programming -- where the "microservice" does one narrowly-defined thing, and that alone.
microservices  soa  architecture  handwaving  http  services  web  deployment 
march 2014 by jm
Trousseau
'an interesting approach to a common problem, that of securely passing secrets around an infrastructure. It uses GPG signed files under the hood and nicely integrates with both version control systems and S3.'

I like this as an approach to securely distributing secrets across a stack of services during deployment. Check in the file of keys, gpg keygen on the server, and add it to the keyfile's ACL during deployment. To simplify, shared or pre-generated GPG keys could also be used.

(via the Devops Weekly newsletter)
gpg  encryption  crypto  secrets  key-distribution  pki  devops  deployment 
february 2014 by jm
deploy_to_runit
A nice node.js app to perform continuous deployment from a GitHub repo via its webhook support, from Matt Sergeant
github  node.js  runit  deployment  git  continuous-deployment  devops  ops 
january 2014 by jm
Don’t get stuck
Good description of Etsy's take on continuous deployment, committing directly to trunk, hidden with feature-flags, from Rafe Colburn
continuous-deployment  coding  agile  deployment  devops  etsy  rafe-colburn 
january 2014 by jm
Moving to Multiple Deployments Per Week at thetrainline.com
continuous delivery using Thoughtworks Go. The UI is quite similar to the internal Amazon C-D system, notably
thoughtworks  continuous-delivery  continuous-deployment  devops  deployment 
december 2013 by jm
Shared Libraries are Obsolete
pretty strong argument. However, I think shlibs still have an advantage in that their pages are easier to share...
shared-libraries  unix  linux  linker  deployment 
november 2013 by jm
Why Every Company Needs A DevOps Team Now - Feld Thoughts
Bookmarking particularly for the 3 "favourite DevOps patterns":

"Make sure we have environments available early in the Development process"; enforce a policy that the code and environment are tested together, even at the earliest stages of the project; “Wake up developers up at 2 a.m. when they break things"; and "Create reusable deployment procedures".
devops  work  ops  deployment  testing  pager-duty 
november 2013 by jm
Mesosphere · Docker on Mesos
This is cool. Deploy Docker container images onto a Mesos cluster: key point, in the description of the Redis example: 'there’s no need to install Redis or its supporting libraries on your Mesos hosts.'
mesos  docker  deployment  ops  images  virtualization  containers  linux 
september 2013 by jm
Docker: Git for deployment
Docker is to deployment as Git is to development.

Developers are able to leverage Git's performance and flexibility when building applications. Git encourages experiments and doesn't punish you when things go wrong: start your experiments in a branch, if things fall down, just git rebase or git reset. It's easy to start a branch and fast to push it.

Docker encourages experimentation for operations. Containers start quickly. Building images is a snap. Using another images as a base image is easy. Deploying whole images is fast, and last but not least, it's not painful to rollback.

Fast + flexible = deployments are about to become a lot more enjoyable.
docker  deployment  sysadmin  ops  devops  vms  vagrant  virtualization  containers  linux  git 
august 2013 by jm
Next Generation Continuous Integration & Deployment with dotCloud’s Docker and Strider
Since Docker treats it’s images as a tree of derivations from a source image, you have the ability to store an image at each stage of a build. This means we can provide full binary images of the environment in which the tests failed. This allows you to run locally bit-for-bit the same container as the CI server ran. Due to the magic of Docker and AUFS Copy-On-Write filesystems, we can store this cheaply.

Often tests pass when built in a CI environment, but when built in another (e.g. production) environment break due to subtle differences. Docker makes it trivial to take exactly the binary environment in which the tests pass, and ship that to production to run it.
docker  strider  continuous-integration  continuous-deployment  deployment  devops  ops  dotcloud  lxc  virtualisation  copy-on-write  images 
july 2013 by jm
Docker
'the Linux container engine'. I totally misunderstood what Docker was -- this is cool.
Heterogeneous payloads: Any combination of binaries, libraries, configuration files, scripts, virtualenvs, jars, gems, tarballs, you name it. No more juggling between domain-specific tools. Docker can deploy and run them all.

Any server: Docker can run on any x64 machine with a modern linux kernel - whether it's a laptop, a bare metal server or a VM. This makes it perfect for multi-cloud deployments.

Isolation: Docker isolates processes from each other and from the underlying host, using lightweight containers.

Repeatability: Because each container is isolated in its own filesystem, they behave the same regardless of where, when, and alongside what they run.
lxc  containers  virtualization  cloud  ops  linux  docker  deployment 
july 2013 by jm
Videos from the Continuous Delivery track at QCon SF 2012
Think we'll be watching some of these in work soon -- Jez Humble's talk (the last one) in particular looks good:

Amazon, Etsy, Google and Facebook are all primarily software development shops which command enormous amounts of resources. They are, to use Christopher Little’s metaphor, unicorns. How can the rest of us adopt continuous delivery? That’s the subject of my talk, which describes four case studies of organizations that adopted continuous delivery, with varying degrees of success.

One of my favourites – partly because it’s embedded software, not a website – is the story of HP’s LaserJet Firmware team, who re-architected their software around the principles of continuous delivery. People always want to know the business case for continuous delivery: the FutureSmart team provide one in the book they wrote that discusses how they did it.
continuous-integration  continuous-delivery  build  release  process  dev  deployment  videos  qcon  towatch  hp 
may 2013 by jm
Testing Your Automation [slides]
Test-driven infrastructure, using Chef -- slides from Big Ruby 2013. Tools used: foodcritic (lol), Chefspec, minitest-chef-handler, fauxhai, cucumber chef. This is really good to see -- TDD applied to ops. Video at: http://confreaks.com/videos/2309-bigruby2013-testing-your-automation-ttd-for-chef-cookbooks
devops  ops  chef  automation  testing  tdd  infrastructure  provisioning  deployment 
april 2013 by jm
The first pillar of agile sysadmin: We alert on what we draw
'One of [the] purposes of monitoring systems was to provide data to allow us, as engineers, to detect patterns, and predict issues before they become production impacting. In order to do this, we need to be capturing data and storing it somewhere which allows us to analyse it. If we care about it - if the data could provide the kind of engineering insight which helps us to understand our systems and give early warning - we should be capturing it. ' .... 'There are a couple of weaknesses in [Nagios' design]. Assuming we’ve agreed that if we care about a metric enough to want to alert on it then we should be gathering that data for analysis, and graphing it, then we already have the data upon which to base our check. Furthermore, this data is not on the machine we’re monitoring, so our checks don’t in any way add further stress to that machine.' I would add that if we are alerting on a different set of data from what we collect for graphing, then using the graphs to investigate an alarm may run into problems if they don't sync up.
devops  monitoring  deployment  production  sysadmin  ops  alerting  metrics 
march 2013 by jm
SoloWizard
'bootstrap an OSX development machine with a one-liner'.
Many teams use chef to manage their production machines, but developers often build their development boxes by hand. SoloWizard makes it painless to create a configurable chef solo script to get your development machine humming: mysql, sublime text, .bash_profile tweaks to OS-X settings - it's all there!
osx  chef  mac  build-out  ops  macosx  deployment  developers  desktops  laptops  mysql  rabbitmq  activemq  nginx 
march 2013 by jm
A Continuous Packaging Pipeline
presentation describing some nice automation tools for packaging vendor code for deployment
deployment  fosdem  presentations  slides  debian  deb  fpm  apt-get 
february 2013 by jm
Ironfan
'an expressive toolset for constructing scalable, resilient [service] architectures. It works in the cloud, in the data center, and on your laptop, and it makes your system diagram visible and inevitable. Inevitable systems coordinate automatically to interconnect, removing the hassle of manual configuration of connection points (and the associated danger of human error).' Looks like a pretty neat cluster deployment tool; driven from a single configuration file, using Chef, integrating closely with AWS and providing many useful additional features
chef  deployment  clusters  knife  services  aws  ec2  ops  ironfan  demo 
january 2013 by jm
OmniTI's Experiences Adopting Chef
A good, in-depth writeup of OmniTI's best practices with respect to build-out of multiple customer deployments, using multi-tenant Chef from a version-controlled repo. Good suggestions, and I am really looking forward to this bit:

'Chef tries to turn your system configuration into code. That means you now inherit all the woes of software engineering: making changes in a coordinated manner and ensuring that changes integrate well are now an even greater concern. In part three of this series, we’ll look at applying software quality assurance and release management practices to Chef cookbooks and roles.'
chef  deployment  ops  omniti  systems  vagrant  automation 
january 2013 by jm
How We Vagrant
the enStratus “solo installer”; what they use for one-box testing, staging, and customer stack deployment, using chef-solo and Vagrant
chef  virtualization  vagrant  chef-solo  deployment  enstratus  cluster  stack 
december 2012 by jm
Shell Scripts Are Like Gremlins
Shell Scripts are like Gremlins. You start out with one adorably cute shell script. You commented it and it does one thing really well. It’s easy to read, everyone can use it. It’s awesome! Then you accidentally spill some water on it, or feed it late one night and omgwtf is happening!?


+1. I have to wean myself off the habit of automating with shell scripts where a clean, well-unit-tested piece of code would work better.
shell-scripts  scripting  coding  automation  sysadmin  devops  chef  deployment 
december 2012 by jm
Ansible
'SSH-Based Configuration Management & Deployment'. deploy via SSH; no target-side daemons required. GPLv3 licensed, unfortunately :(
ansible  devops  configuration  deployment  sysadmin  python  ssh 
july 2012 by jm
Exclusive: a behind-the-scenes look at Facebook release engineering
'Facebook gave me an exclusive behind-the-scenes look at the process it uses to deploy new functionality. I watched first-hand as the company's release engineers rolled out the new "timeline" feature for brand pages'. Hiphop, BitTorrent, 1.5GB binaries, and IRC!
facebook  deployment  engineering  releases  via:bos 
april 2012 by jm
Building with Legos
Netflix tech blog on how they deploy their services. Notably, they avoid the Puppet/Chef approach, citing these reasons: 'One is that it eliminates a number of dependencies in the production environment: a master control server, package repository and client scripts on the servers, network permissions to talk to all of these. Another is that it guarantees that what we test in the test environment is the EXACT same thing that is deployed in production; there is very little chance of configuration or other creep/bit rot. Finally, it means that there is no way for people to change or install things in the production environment (this may seem like a really harsh restriction, but if you can build a new AMI fast enough it doesn't really make a difference).'
devops  cloud  aws  netflix  puppet  chef  deployment 
august 2011 by jm
Silver Lining
'an application packaging format, a server configuration library, a cloud server management tool, a persistence management tool, and a tool to manage the application with respect to all these services over time.'  interesting, possibly too Pythonic
python  programming  dist  deployment  packaging  from delicious
april 2011 by jm
Using Git to manage a web site
simple, basic demo of a git post-receive hook to auto-check-out every rev committed to a git repository
git  deployment  howto  via:hackernews  from delicious
february 2011 by jm
Etsy's metrics infrastructure
I never really understood how useful a good metrics infrastructure could be for operational visibility until I joined Amazon.  Here's a good demo of Etsy's metrics system (via Netlson)
via:nelson  metrics  deployment  change-monitoring  etsy  software  monitoring  ops  from delicious
december 2010 by jm
GitHub scheduled maintainance due to Redis upgrade
good comments on the processes useful for large-scale Redis upgrades
upgrades  redis  spof  nosql  databases  github  deployment  from delicious
may 2010 by jm
What Second Life can teach your datacenter about scaling Web apps
good scaling advice from Linden Labs' Ian Wilkes (who doesn't seem to have a blog, sadly)
linden  ian-wilkes  scaling  datacenters  scalability  deployment  ops  services  from delicious
february 2010 by jm
A new way to deploy web applications
interesting Django/Pythonic approach, based on concepts from AppEngine
django  python  virtualenv  deployment  web-apps  linux  appengine  from delicious
january 2010 by jm
Deployment is just a part of dev/ops cooperation, not the whole thing
metrics, monitoring, instrumentation, fault tolerance, load mitigation called out as other factors by Allspaw
ops  deployment  operations  engineering  metrics  devops  monitoring  fault-tolerance  load  from delicious
december 2009 by jm
Infrastructures.Org: Best Practices in Automated Systems Administration and Infrastructure Architecture: Gold Server
well-written, and it's good to see version control listed right at the top of the list. But quite dead; interesting for historical reasons only at this stage
via:fanf  deployment  sysadmin  unix  rsync  ssh  cvs  infrastructure  cfengine 
july 2009 by jm

related tags

activemq  agile  alerting  ansible  appengine  apt-get  architecture  asg  auto-scaling  automation  aws  best-practices  branch  branching  build  build-out  cfengine  change-monitoring  chef  chef-solo  cloud  cluster  clusters  coding  collaboration  complexity  configuration  containers  continuous-delivery  continuous-deployment  continuous-integration  continuousintegration  copy-on-write  crypto  cvs  database  databases  datacenters  deb  debian  demo  dependencies  deployinator  deployment  desktops  dev  developers  devops  dist  django  docker  dotcloud  dr-foster-intelligence  dublin  ec2  encryption  engineering  enstratus  etl  etsy  facebook  fail  fault-tolerance  feature-flags  flickr  fosdem  fpm  gating  git  github  go  gpg  handwaving  howto  hp  http  ian-wilkes  images  infrastructure  integration  integration-tests  ios  iphone  ironfan  jenkins  key-distribution  knife  laptops  linden  linker  linux  load  logging  lonely-planet  loose-coupling  lxc  mac  macosx  meetups  mesos  metrics  microservices  migrations  monitoring  mysql  netflix  networking  nginx  node.js  nosql  omniti  operations  ops  osx  packaging  pager-duty  pipelines  pki  postgres  presentations  process  production  programming  provisioning  puppet  python  qa  qcon  rabbitmq  rafe-colburn  redis  release  releases  rsync  runit  scalability  scaling  scripting  secrets  service-discovery  services  shared-libraries  shell-scripts  slides  smartstack  soa  software  soundcloud  spof  sql  ssh  stack  strider  sysadmin  systems  talks  tdd  team  testing  thoughtworks  tips  towatch  uat  unix  upgrades  vagrant  version-control  via:bos  via:fanf  via:hackernews  via:nelson  video  videos  virtualenv  virtualisation  virtualization  vms  web  web-apps  web-services  work  wtf  yelp 

Copy this bookmark:



description:


tags: