jm + infrastructure   23

AWS Lambda Deployment using Terraform – Build ACL – Medium
Fairly persuasive that production usage of Lambda is much easier if you go full Terraform to manage and deploy.
A complete picture of what it takes to deploy your Lambda function to production with the same diligence you apply to any other codebase using Terraform. [...] There are many cases where frameworks such as SAM or Serverless are not enough. You need more than that for a highly integrated Lambda function. In such cases, it’s easier to simply use Terraform.
infrastructure  aws  lambda  serverless  ops  terraform  sam 
5 weeks ago by jm
Physical separation of cyclists from traffic “crucial” to dropping injury rates, shows U.S. study
Citing a further study of differing types of cycling infrastructure in Canada, the editorial writes that an 89% increase in safety was noted on streets with physical separation over streets where no such infrastructure existed. Unprotected cycling space was found to be 53% safer.

In 2014 there were 902 recorded cyclists fatalities in America and 35,206 serious injuries. Per kilometre cycled fatalities per 100 million kilometres cycled sat at 4.7. In the Netherlands and Denmark those rates sit at 1 and 1.1, respectively.
cycling  infrastructure  roads  safety  accidents  cars  statistics  us  canada 
may 2017 by jm
Airflow/AMI/ASG nightly-packaging workflow
Some tantalising discussion on twitter of an Airflow + AMI + ASG workflow for ML packaging:

'We build models using Airflow. We deploy new models as AMIs where each AMI is model + scoring code. The AMI is hence a version of code + model at a point in time : #immutable_infrastructure. It's natural for Airflow to build & deploy the model+code with each Airflow DAG Run corresponding to a versioned AMI. if there's a problem, we can simply roll back to the previous AMI & identify the problematic model building Dag run. Since we use ASGs, Airflow can execute a rolling deploy of new AMIs. We could also have it do a validation & ASG rollback of the AMI if validation fails. Airflow is being used for reliable Model build+validation+deployment.'
ml  packaging  airflow  asg  ami  deployment  ops  infrastructure  rollback 
september 2016 by jm
Amazon S3 Transfer Acceleration
The AWS edge network has points of presence in more than 50 locations. Today, it is used to distribute content via Amazon CloudFront and to provide rapid responses to DNS queries made to Amazon Route 53. With today’s announcement, the edge network also helps to accelerate data transfers in to and out of Amazon S3. It will be of particular benefit to you if you are transferring data across or between continents, have a fast Internet connection, use large objects, or have a lot of content to upload.

You can think of the edge network as a bridge between your upload point (your desktop or your on-premises data center) and the target bucket. After you enable this feature for a bucket (by checking a checkbox in the AWS Management Console), you simply change the bucket’s endpoint to the form No other configuration changes are necessary! After you do this, your TCP connections will be routed to the best AWS edge location based on latency.  Transfer Acceleration will then send your uploads back to S3 over the AWS-managed backbone network using optimized network protocols, persistent connections from edge to origin, fully-open send and receive windows, and so forth.
aws  s3  networking  infrastructure  ops  internet  cdn 
april 2016 by jm
GCHQ intervenes to prevent catastrophically insecure UK smart meter plan - The Inquirer

GCHQ barged in after spooks cast their eyes over the plans and realised that power companies were proposing to use a single decryption key for communications from the 53 million smart meters that will eventually be installed in the UK.

holy crap.
gchq  security  smart-meters  power  uk  electricity  gas  infrastructure 
april 2016 by jm
Dublin is a medium-density city
Comparable to Copenhagen or Amsterdam, albeit without sufficient cycling/public-transport infrastructural investment
infrastructure  density  housing  dublin  ireland  cities  travel  commuting  cycling 
november 2015 by jm
'Hosted Status Pages for Your Company'. We use these guys in $work, and their service is fantastic -- it's a line of javascript in the page template which will easily allow you to add a "service degraded" banner when things go pear-shaped, along with an external status site for when things get really messy. They've done a good clean job.
monitoring  server  status  outages  uptime  saas  infrastructure 
november 2014 by jm
IT Change Management
Stephanie Dean on Amazon's approach to CMs. This is solid gold advice for any company planning to institute a sensible technical change management process
ops  tech  process  changes  change-management  bureaucracy  amazon  stephanie-dean  infrastructure 
october 2014 by jm
Avoiding Chef-Suck with Auto Scaling Groups - forty9ten
Some common problems which arise using Chef with ASGs in EC2, and how these guys avoided it -- they stopped using Chef for service provisioning, and instead baked AMIs when a new version was released. ASGs using pre-baked AMIs definitely works well so this makes good sense IMO.
infrastructure  chef  ops  asg  auto-scaling  ec2  provisioning  deployment 
september 2014 by jm
Code Spaces data and backups deleted by hackers
Rather scary story of an extortionist wiping out a company's AWS-based infrastructure. Turns out S3 supports MFA-required deletion as a feature, though, which would help against that.
ops  security  extortion  aws  ec2  s3  code-spaces  delete  mfa  two-factor-authentication  authentication  infrastructure 
june 2014 by jm
Dan Kaminsky on Heartbleed
When I said that we expected better of OpenSSL, it’s not merely that there’s some sense that security-driven code should be of higher quality.  (OpenSSL is legendary for being considered a mess, internally.)  It’s that the number of systems that depend on it, and then expose that dependency to the outside world, are considerable.  This is security’s largest contributed dependency, but it’s not necessarily the software ecosystem’s largest dependency.  Many, maybe even more systems depend on web servers like Apache, nginx, and IIS.  We fear vulnerabilities significantly more in libz than libbz2 than libxz, because more servers will decompress untrusted gzip over bzip2 over xz.  Vulnerabilities are not always in obvious places – people underestimate just how exposed things like libxml and libcurl and libjpeg are.  And as HD Moore showed me some time ago, the embedded space is its own universe of pain, with 90’s bugs covering entire countries.

If we accept that a software dependency becomes Critical Infrastructure at some level of economic dependency, the game becomes identifying those dependencies, and delivering direct technical and even financial support.  What are the one million most important lines of code that are reachable by attackers, and least covered by defenders?  (The browsers, for example, are very reachable by attackers but actually defended pretty zealously – FFMPEG public is not FFMPEG in Chrome.)

Note that not all code, even in the same project, is equally exposed.    It’s tempting to say it’s a needle in a haystack.  But I promise you this:  Anybody patches Linux/net/ipv4/tcp_input.c (which handles inbound network for Linux), a hundred alerts are fired and many of them are not to individuals anyone would call friendly.  One guy, one night, patched OpenSSL.  Not enough defenders noticed, and it took Neel Mehta to do something.
development  openssl  heartbleed  ssl  security  dan-kaminsky  infrastructure  libraries  open-source  dependencies 
april 2014 by jm
Load Balancer Testing with a Honeypot Daemon
nice post on writing BDD unit tests for infrastructure, in this case specifically a load balancer (via Devops Weekly)
load-balancers  ops  devops  sysadmin  testing  unit-tests  networking  honeypot  infrastructure  bdd 
december 2013 by jm
Scryer: Netflix’s Predictive Auto Scaling Engine
Scryer is a new system that allows us to provision the right number of AWS instances needed to handle the traffic of our customers. But Scryer is different from Amazon Auto Scaling (AAS), which reacts to real-time metrics and adjusts instance counts accordingly. Rather, Scryer predicts what the needs will be prior to the time of need and provisions the instances based on those predictions.
scaling  infrastructure  aws  ec2  netflix  scryer  auto-scaling  aas  metrics  prediction  spikes 
november 2013 by jm
Testing Your Automation [slides]
Test-driven infrastructure, using Chef -- slides from Big Ruby 2013. Tools used: foodcritic (lol), Chefspec, minitest-chef-handler, fauxhai, cucumber chef. This is really good to see -- TDD applied to ops. Video at:
devops  ops  chef  automation  testing  tdd  infrastructure  provisioning  deployment 
april 2013 by jm
WebTechStacks by martharotter - Kippt
A good set of infrastructure/devops tech blogs, collected by Martha Rotter
via:martharotter  blogs  infrastructure  devops  ops  web  links 
november 2012 by jm
Internet Security is a failure
ASF's Paul Querna: 'Security on the Internet sucks, and it is only getting worse. The problem is systemic, with security researchers and developers not producing viable ways for the average user to live on the Internet in a secure fashion without excessive paranoia.'
asf  authentication  infrastructure  tls  internet  security  from delicious
april 2010 by jm
City of Portland develops iPhone app to report city infrastructure problems
ie. take a pic of a pothole and it'll be reported up to the appropriate office quickly and without hassle. wow. are you watching, Dublin?
portland  cool  iphone  apps  potholes  infrastructure  city  from delicious
april 2010 by jm
'an online compiler/interpreter, and a simple collaboration tool. It's a pastebin that executes code for you. You paste your code, and codepad runs it and gives you a short URL you can use to share it.' supports C, C++, D, Haskell, Lua, OCaml, PHP, Perl, Python, Ruby, Scheme, and Tcl code; isolated by a geordi-based supervisor, in turn running inside a firewalled virt, in turn running inside a firewalled dom0. nice work!
codepad  vm  jails  infrastructure  security  via:waxy  c  languages  programming  sandbox  pastebin 
august 2009 by jm
Infrastructures.Org: Best Practices in Automated Systems Administration and Infrastructure Architecture: Gold Server
well-written, and it's good to see version control listed right at the top of the list. But quite dead; interesting for historical reasons only at this stage
via:fanf  deployment  sysadmin  unix  rsync  ssh  cvs  infrastructure  cfengine 
july 2009 by jm

related tags

aas  accidents  airflow  amazon  ami  apps  asf  asg  authentication  auto-scaling  automation  aws  bdd  blogs  bureaucracy  c  canada  canary-requests  cars  cdn  cfengine  change-management  changes  chef  cities  city  clos-networks  code-spaces  codepad  commuting  containers  continuousintegration  cool  cvs  cycling  dan-kaminsky  datacenter  debugging  delete  density  dependencies  deployinator  deployment  development  devops  distcomp  distributed-systems  docker  dublin  ec2  electricity  etsy  extortion  facebook  fat-tree  gas  gchq  heartbleed  honeypot  housing  http  infrastructure  internet  iphone  ireland  jails  lambda  languages  libraries  links  linux  live  load-balancers  lxc  metrics  mfa  ml  monitoring  netflix  networking  networks  open-source  openssl  ops  outages  packaging  pastebin  portland  potholes  power  prediction  process  production  programming  provisioning  roads  rollback  rsync  s3  saas  safety  sam  sandbox  scaling  scryer  security  server  serverless  smart-meters  spikes  ssh  ssl  stack  statistics  status  stephanie-dean  sysadmin  tdd  tech  terraform  testing  tls  tracer-requests  tracing  travel  twitter  two-factor-authentication  uk  unit-tests  unix  uptime  us  via:fanf  via:martharotter  via:waxy  vm  web  zipkin 

Copy this bookmark: