jm + aws   171

Coda Hale's new gig on how they're using Docker, AWS, etc. I like this: "Use containers. Not too much. Mostly for packaging."
docker  aws  packaging  ops  devops  containers  skyliner 
7 weeks ago by jm
AWS Case Study: mytaxi
ECS, Docker, ELB, SQS, SNS, RDS, VPC, and spot instances. Pretty canonical setup these days...
The mytaxi app is also now able to predict daily and weekly spikes. In addition, it has gained the elasticity required to meet demand during special events. Herzberg describes a typical situation on New Year's Eve: “Shortly before midnight everyone needs a taxi to get to parties, and after midnight people want to go home. In past years we couldn't keep up with the demand this generated, which was around three and a half times as high as normal. In November 2015 we moved our Docker container architecture to Amazon ECS, and for the first time ever in December we were able to celebrate a new year in which our system could handle the huge number of requests without any crashes or interruptions—an accomplishment that we were extremely proud of. We had faced the biggest night on the calendar without any downtime.”
mytaxi  aws  ecs  docker  elb  sqs  sns  rds  vpc  spot-instances  ops  architecture 
11 weeks ago by jm
Cross-Region Read Replicas for Amazon Aurora
Creating a read replica in another region also creates an Aurora cluster in the region. This cluster can contain up to 15 more read replicas, with very low replication lag (typically less than 20 ms) within the region (between regions, latency will vary based on the distance between the source and target). You can use this model to duplicate your cluster and read replica setup across regions for disaster recovery. In the event of a regional disruption, you can promote the cross-region replica to be the master. This will allow you to minimize downtime for your cross-region application. This feature applies to unencrypted Aurora clusters.
aws  mysql  databases  storage  replication  cross-region  failover  reliability  aurora 
june 2016 by jm
Green/Blue Deployments with AWS Lambda and CloudFormation - done right
Basically, use a Lambda to put all instances from an ASG into the ELB, then remove the old ASG
asg  elb  aws  lambda  deployment  ops  blue-green-deploys 
may 2016 by jm
3 Reasons AWS Lambda Is Not Ready for Prime Time
This totally matches my own preconceptions ;)
When we at Datawire tried to actually use Lambda for a real-world HTTP-based microservice [...], we found some uncool things that make Lambda not yet ready for the world we live in:

Lambda is a building block, not a tool;
Lambda is not well documented;
Lambda is terrible at error handling

Lung skips these uncool things, which makes sense because they’d make the tutorial collapse under its own weight, but you can’t skip them if you want to work in the real world. (Note that if you’re using Lambda for event handling within the AWS world, your life will be easier. But the really interesting case in the microservice world is Lambda and HTTP.)
aws  lambda  microservices  datawire  http  api-gateway  apis  https  python  ops 
may 2016 by jm
Key Metrics for Amazon Aurora | AWS Partner Network (APN) Blog
Very DataDog-oriented, but some decent tips on monitorable metrics here
datadog  metrics  aurora  aws  rds  monitoring  ops 
may 2016 by jm
Amazon S3 Transfer Acceleration
The AWS edge network has points of presence in more than 50 locations. Today, it is used to distribute content via Amazon CloudFront and to provide rapid responses to DNS queries made to Amazon Route 53. With today’s announcement, the edge network also helps to accelerate data transfers in to and out of Amazon S3. It will be of particular benefit to you if you are transferring data across or between continents, have a fast Internet connection, use large objects, or have a lot of content to upload.

You can think of the edge network as a bridge between your upload point (your desktop or your on-premises data center) and the target bucket. After you enable this feature for a bucket (by checking a checkbox in the AWS Management Console), you simply change the bucket’s endpoint to the form No other configuration changes are necessary! After you do this, your TCP connections will be routed to the best AWS edge location based on latency.  Transfer Acceleration will then send your uploads back to S3 over the AWS-managed backbone network using optimized network protocols, persistent connections from edge to origin, fully-open send and receive windows, and so forth.
aws  s3  networking  infrastructure  ops  internet  cdn 
april 2016 by jm
'AWS Assume Made Awesome' -- 'Here are Trek10, we work with many clients, and thus work with multiple AWS accounts on a regular (daily) basis. We needed a way to make managing all our different accounts easier. We create a standard Trek10 administrator role in our clients’ accounts that we can assume. For security we require that the role assumer have multifactor authentication enabled.'
mfa  aws  awsume  credentials  accounts  ops 
april 2016 by jm
Running Docker on AWS from the ground up
Advantages/disavantages section right at the bottom is good.
ECS, believe it or not, is one of the simplest Schedulers out there. Most of the other alternatives I’ve tried offer all sorts of fancy bells & whistles, but they are either significantly more complicated to understand (lots of new concepts), take too much effort to set up (lots of new technologies to install and run), are too magical (and therefore impossible to debug), or some combination of all three. That said, ECS also leaves a lot to be desired.
aws  docker  ecs  ec2  schedulers 
april 2016 by jm
git for Cloud Storage. Create distributed, decentralized and versioned repositories that scale infinitely to 100s of millions of files and PBs of storage. Huge repos can be cloned on your local SSD for making changes, committing and pushing back. Oh yeah, and it dedupes too due to BLAKE2 Tree hashing.
git  ops  storage  cloud  s3  disk  aws  version-control  blake2 
april 2016 by jm
Charity Majors - AWS networking, VPC, environments and you
'VPC is the future and it is awesome, and unless you have some VERY SPECIFIC AND CONVINCING reasons to do otherwise, you should be spinning up a VPC per environment with orchestration and prob doing it from CI on every code commit, almost like it’s just like, you know, code.'
networking  ops  vpc  aws  environments  stacks  terraform 
march 2016 by jm
Yeobot - CloudNative
'The shared SQL command line for AWS'. it's #chatopsy!
chatops  yeobot  bots  cloudnative  ec2  aws  slack 
march 2016 by jm
AWS Certificate Manager – Deploy SSL/TLS-Based Apps on AWS
Very nifty -- autodeploys free wildcard certs to ELBs and Cloudfront. HN discussion thread is pretty good:
ssl  tls  certificates  ops  aws  cloudfront  elb 
january 2016 by jm
VPC NAT gateways : transactional uniqueness at scale
colmmacc introducing the VPC NAT gateway product from AWS, in a guest post on James Hamilton's blog no less!:
you can think of it as a new “even bigger” [NAT] box, but under the hood NAT gateways are different. The connections are managed by a fault-tolerant co-operation of devices in the VPC network fabric. Each new connection is assigned a port in a robust and transactional way, while also being replicated across an extensible set of multiple devices. In other words: the NAT gateway is internally horizontally scalable and resilient.
amazon  ec2  nat  networking  aws  colmmacc 
january 2016 by jm
2016 Wish List for AWS?
good thread of AWS' shortcomings -- so many services still don't handle VPC for instance
vpc  aws  ec2  ops  wishlist 
december 2015 by jm
Amazon EC2 Container Registry
hooray, Docker registry here at last
ecs  docker  registry  ops  containers  aws 
december 2015 by jm
How to host Hugo static website generator on AWS Lambda
seriously, AWS. editing JSON files in a browser text box is an awful, awful user experience
aws  lambda  hosting  hugo  blogs  website-generators 
december 2015 by jm
Why We Chose Kubernetes Over ECS
3 months ago when we, at, came to evaluate which Docker orchestration framework to use, we gave ECS the first priority. We were already familiar with AWS services, and since we already had our whole infrastructure there, it was the default choice. After testing the service for a while we had the feeling it was not mature enough and missing some key features we needed (more on that later), so we went to test another orchestration framework: Kubernetes. We were glad to discover that Kubernetes is far more comprehensive and had almost all the features we required. For us, Kubernetes won ECS on ECS’s home court, which is AWS.
kubernetes  ecs  docker  containers  aws  ec2  ops 
december 2015 by jm
AWS Api Gateway for Fun and Profit
good worked-through example of an API Gateway rewriting system
api-gateway  aws  api  http  services  ops  alerting  alarming  opsgenie  signalfx 
december 2015 by jm
A Gulp Workflow for Amazon Lambda
'any nontrivial development of Lambda functions will require a simple, automated build/deploy process that also fills a couple of Lambda’s gaps such as the use of node modules and environment variables.'

See also : 'I am psyched about Amazon’s new Lambda service for asynchronous task processing, but the ideal development and testing cycle is really left to the engineer. While Amazon provides a web-based console, I prefer an approach that uses Mocha. Below you will find the gritty details using Kinesis events as a sample input.'
lambda  aws  services  testing  deployment  ops  mocha  gulp  javascript 
december 2015 by jm
Global Continuous Delivery with Spinnaker
Netflix' CD platform, post-Atlas. looks interesting
continuous-delivery  aws  netflix  cd  devops  ops  atlas  spinnaker 
november 2015 by jm
Awesome new mock DynamoDB implementation:
An implementation of Amazon's DynamoDB, focussed on correctness and performance, and built on LevelDB (well, @rvagg's awesome LevelUP to be precise). This project aims to match the live DynamoDB instances as closely as possible (and is tested against them in various regions), including all limits and error messages.

Why not Amazon's DynamoDB Local? Because it's too buggy! And it differs too much from the live instances in a number of key areas.

We use DynamoDBLocal in our tests -- the availability of that tool is one of the key reasons we have adopted Dynamo so heavily, since we can safely test our code properly with it. This looks even better.
dynamodb  testing  unit-tests  integration-testing  tests  ops  dynalite  aws  leveldb 
november 2015 by jm
Valid MFA token does not work during first 1am hour before daylight savings ends and second 1am hour starts · Issue #1611 · aws/aws-cli
Add another one to the "yay for DST" pile. (also yay for AWS using PST/PDT as default internal timezone instead of UTC...)
utc  timezones  fail  bugs  aws  aws-cli  dst  daylight-savings  time 
november 2015 by jm
It's an Emulator, Not a Petting Zoo: Emu and Lambda
a Lambda emulator in Python, suitable for unit testing lambdas
lambda  aws  coding  unit-tests  dev 
october 2015 by jm
Amazon ECS CLI Tutorial - Amazon EC2 Container Service
super-basic ECS tutorial, using a docker-compose.yml to create a new ECS-managed service fleet
ecs  cli  linux  aws  ec2  hosting  docker  tutorials 
october 2015 by jm
Hologram exposes an imitation of the EC2 instance metadata service on developer workstations that supports the [IAM Roles] temporary credentials workflow. It is accessible via the same HTTP endpoint to calling SDKs, so your code can use the same process in both development and production. The keys that Hologram provisions are temporary, so EC2 access can be centrally controlled without direct administrative access to developer workstations.
iam  roles  ec2  authorization  aws  adroll  open-source  cli  osx  coding  dev 
october 2015 by jm
(ARC308) The Serverless Company: Using AWS Lambda
Describing PlayOn! Sports' Lambda setup. Sounds pretty productionizable
ops  lambda  aws  reinvent  slides  architecture 
october 2015 by jm
AWS re:Invent 2015 Video & Slide Presentation Links with Easy Index
Andrew Spyker's roundup:
my quick index of all re:Invent sessions.  Please wait for a few days and I'll keep running the tool to fill in the index.  It usually takes Amazon a few weeks to fully upload all the videos and slideshares.

Pretty definitive, full text descriptions of all sessions (and there are an awful lot of 'em).
aws  reinvent  andrew-spyker  scraping  slides  presentations  ec2  video 
october 2015 by jm
AWS re:Invent 2015 | (CMP406) Amazon ECS at Coursera - YouTube
Coursera are running user-submitted code in ECS! interesting stuff about how they use Docker security/resource-limiting features, forking the ecs-agent code, to run user-submitted code. :O
coursera  user-submitted-code  sandboxing  docker  security  ecs  aws  resource-limits  ops 
october 2015 by jm
The Totally Managed Analytics Pipeline: Segment, Lambda, and Dynamo
notable mainly for the details of Terraform support for Lambda: that's a significant improvement to Lambda's production-readiness
aws  pipelines  data  streaming  lambda  dynamodb  analytics  terraform  ops 
october 2015 by jm
Rebuilding Our Infrastructure with Docker, ECS, and Terraform
Good writeup of current best practices for a production AWS architecture
aws  ops  docker  ecs  ec2  prod  terraform  segment  via:marc 
october 2015 by jm
ECJ ruling on Irish privacy case has huge significance
The only current way to comply with EU law, the judgment indicates, is to keep EU data within the EU. Whether those data can be safely managed within facilities run by US companies will not be determined until the US rules on an ongoing Microsoft case.
Microsoft stands in contempt of court right now for refusing to hand over to US authorities, emails held in its Irish data centre. This case will surely go to the Supreme Court and will be an extremely important determination for the cloud business, and any company or individual using data centre storage. If Microsoft loses, US multinationals will be left scrambling to somehow, legally firewall off their EU-based data centres from US government reach.

(cough, Amazon)
aws  hosting  eu  privacy  surveillance  gchq  nsa  microsoft  ireland 
october 2015 by jm
EC2 Spot Blocks for Defined-Duration Workloads
you can now launch Spot instances that will run continuously for a finite duration (1 to 6 hours). Pricing is based on the requested duration and the available capacity, and is typically 30% to 45% less than On-Demand.
ec2  aws  spot-instances  spot  pricing  time 
october 2015 by jm
Chaos Engineering Upgraded
some details on Netflix's Chaos Monkey, Chaos Kong and other aspects of their availability/failover testing
architecture  aws  netflix  ops  chaos-monkey  chaos-kong  testing  availability  failover  ha 
september 2015 by jm
Summary of the Amazon DynamoDB Service Disruption and Related Impacts in the US-East Region
Painful to read, but: tl;dr: monitoring oversight, followed by a transient network glitch triggering IPC timeouts, which increased load due to lack of circuit breakers, creating a cascading failure
aws  postmortem  outages  dynamodb  ec2  post-mortems  circuit-breakers  monitoring 
september 2015 by jm
How We Use AWS Lambda for Rapidly Intensifying Workloads · CloudSploit
impressive -- pretty much the entire workload is run from Lambda here
lambda  aws  ec2  autoscaling  cloudsploit 
september 2015 by jm
Evolution of Babbel’s data pipeline on AWS: from SQS to Kinesis
Good "here's how we found it" blog post:

Our new data pipeline with Kinesis in place allows us to plug new consumers without causing any damage to the current system, so it’s possible to rewrite all Queue Workers one by one and replace them with Kinesis Workers. In general, the transition to Kinesis was smooth and there were not so tricky parts.
Another outcome was significantly reduced costs – handling almost the same amount of data as SQS, Kinesis appeared to be many times cheaper than SQS.
aws  kinesis  kafka  streaming  data-pipelines  streams  sqs  queues  architecture  kcl 
september 2015 by jm
Spot Bid Advisor
analyzes Spot price history to help you determine a bid price that suits your needs.
ec2  aws  spot  spot-instances  history 
september 2015 by jm
a file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack. S3QL effectively provides a hard disk of dynamic, infinite capacity that can be accessed from any computer with internet access running Linux, FreeBSD or OS-X.
S3QL is a standard conforming, full featured UNIX file system that is conceptually indistinguishable from any local file system. Furthermore, S3QL has additional features like compression, encryption, data de-duplication, immutable trees and snapshotting which make it especially suitable for online backup and archival.
S3QL is designed to favor simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection and error handling have been included from the very first line, and S3QL comes with extensive automated test cases for all its components.
filesystems  aws  s3  storage  unix  google-storage  openstack 
september 2015 by jm
Amazon EC2 2015 Benchmark: Testing Speeds Between AWS EC2 and S3 Regions
Here we are again, a year later, and still no bloody percentiles! Just amateurish averaging. This is not how you measure anything, ffs. Still, better than nothing I suppose
fail  latency  measurement  aws  ec2  percentiles  s3 
august 2015 by jm
Implementing Efficient and Reliable Producers with the Amazon Kinesis Producer Library - AWS Big Data Blog
Good advice on production-quality, decent-scale usage of Kinesis in Java with the official library: batching, retries, partial failures, backoff, and monitoring. (Also, jaysus, the AWS Cloudwatch API is awful, looking at this!)
kpl  aws  kinesis  tips  java  batching  streaming  production  cloudwatch  monitoring  coding 
august 2015 by jm
Amazon S3 Introduces New Usability Enhancements
bucket limit increase, and read-after-write consistency in US Standard. About time too! ;)
aws  s3  storage  consistency 
august 2015 by jm
danilop/runjop · GitHub
RunJOP (Run Just Once Please) is a distributed execution framework to run a command (i.e. a job) only once in a group of servers [built using AWS DynamoDB and S3].

nifty! Distributed cron is pretty easy when you've got Dynamo doing the heavy lifting.
dynamodb  cron  distributed-cron  scheduling  runjop  danilop  hacks  aws  ops 
july 2015 by jm
danilop/yas3fs · GitHub
YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster.
aws  s3  s3fs  yas3fs  filesystems  fuse  sns 
july 2015 by jm
Revised and much faster, run your own high-end cloud gaming service on EC2!
a g2.2xlarge provides decent Windows GPU performance over the internet, at about $0.53 per hour
gaming  games  ec2  amazon  aws  cloud  windows  hacks 
july 2015 by jm
VPC Flow Logs
we are introducing Flow Logs for the Amazon Virtual Private Cloud.  Once enabled for a particular VPC, VPC subnet, or Elastic Network Interface (ENI), relevant network traffic will be logged to CloudWatch Logs for storage and analysis by your own applications or third-party tools.

You can create alarms that will fire if certain types of traffic are detected; you can also create metrics to help you to identify trends and patterns. The information captured includes information about allowed and denied traffic (based on security group and network ACL rules). It also includes source and destination IP addresses, ports, the IANA protocol number, packet and byte counts, a time interval during which the flow was observed, and an action (ACCEPT or REJECT).
ec2  aws  vpc  logging  tracing  ops  flow-logs  network  tcpdump  packets  packet-capture 
june 2015 by jm
Leveraging AWS to Build a Scalable Data Pipeline
Nice detailed description of an auto-scaled SQS worker pool
sqs  aws  ec2  auto-scaling  asg  worker-pools  architecture  scalability 
june 2015 by jm
etcd Clustering in AWS
'a fully-automated solution to build auto-scaling etcd clusters in AWS'
aws  cluster  docker  etcd  asg  autoscaling  ops 
june 2015 by jm
1172401 – Add Amazon root certificates
Well, well -- looks like AWS is about to disrupt PKI, and about time too. If they come up with a Plex-style "provision a cert" API, it'll be revolutionary
pki  ssl  tls  amazon  aws  apis  web-services  ops 
june 2015 by jm
Schedule Recurring AWS Lambda Invocations With The Unreliable Town Clock (UTC)
The Unreliable Town Clock (UTC) is a new, free, public SNS Topic (Amazon Simple Notification Service) that broadcasts a “chime” message every quarter hour to all subscribers. It can send the chimes to AWS Lambda functions, SQS queues, and email addresses.

You can use the chime attributes to run your code every fifteen minutes, or only run your code once an hour (e.g., when minute == "00") or once a day (e.g., when hour == "00" and minute == "00") or any other series of intervals. You can even subscribe a function you only want to run only once at a specific time in the future: Have the function ignore all invocations until it’s after the time it wants. When it is time, it can perform its job, then unsubscribe itself from the SNS Topic.
alestic  aws  lambda  cron  time  clock  periodic-tasks  recurrence  hacks 
may 2015 by jm
Lambda: Bees with Frickin' Laser Beams
a HTTP testing tool in AWS Lambda. nice enough, but still a toy...
lambda  aws  node  javascript  hacks  http  load-testing 
may 2015 by jm
Load data into Redshift from S3 buckets using a pre-canned Lambda function. Looks like it may be a good example of production-quality Lambda
lambda  aws  ec2  redshift  s3  loaders  etl  pipeline 
may 2015 by jm "certificate verification failed" errors due to crappy Verisign certs and overzealous curl policies
Seth Vargo is correct. Its not the bit length of the key which is at issue, its the signature algorithm. The entire keychain for the key is signed with SHA1withRSA:

At issue is that the root verisign key has been marked as weak because of SHA1 and taken out of the curl bundle which is widely popular, and this issue will continue to cause more and more issues going forwards as that bundle makes it way into shipping o/s distributions and aws certification verification breaks.

'This is still happening and curl is now failing on my machine causing all sorts of fun issues (including breaking CocoaPods that are using S3 for storage).' -- @jmhodges

This may be a contributory factor to the issue @nelson saw:

Curl's ca-certs bundle is also used by Node: and doubtless many other apps and packages.

Here's a mailing list thread discussing the issue: -- looks like the curl team aren't too bothered about it.
curl  s3  amazon  aws  ssl  tls  certs  sha1  rsa  key-length  security  cacerts 
april 2015 by jm
'a command line tool that (hopefully) makes it easier to deploy, update, and test functions for AWS Lambda.' much needed IMO -- Lambda is too closed
aws  lambda  mitch-garnaat  coding  testing  cli  kappa 
april 2015 by jm
Cluster-Based Architectures Using Docker and Amazon EC2 Container Service
In this post, we’re going to take a deeper dive into the architectural concepts underlying cluster computing using container management frameworks such as ECS. We will show how these frameworks effectively abstract the low-level resources such as CPU, memory, and storage, allowing for highly efficient usage of the nodes in a compute cluster. Building on some of the concepts detailed in the earlier posts, we will discover why containers are such a good fit for this type of abstraction, and how the Amazon EC2 Container Service fits into the larger ecosystem of cluster management frameworks.
docker  aws  ecs  ec2  ops  hosting  containers  mesos  clusters 
april 2015 by jm
Amazon EC2 Container Service team AmA
a few answers here. Mostly people pointing out shortcomings and the team asking them to start a thread on their forum though :(
ec2  ecs  docker  aws  ops  ama  reddit 
april 2015 by jm
'CredStash is a very simple, easy to use credential management and distribution system that uses AWS Key Management System (KMS) for key wrapping and master-key storage, and DynamoDB for credential storage and sharing.'
aws  credstash  python  security  keys  key-management  secrets  kms 
april 2015 by jm
Run your own high-end cloud gaming service on EC2
Using Steam streaming and EC2 g2.2xlarge spot instances -- 'comes out to around $0.52/hr'. That's pretty compelling IMO
aws  ec2  gaming  games  graphics  spot-instances  hacks  windows  steam 
april 2015 by jm
Microservices and elastic resource pools with Amazon EC2 Container Service
interesting approach to working around ECS' shortcomings -- bit specific to Hailo's microservices arch and IPC mechanism though.

aside: I like their version numbering scheme: ISO-8601, YYYYMMDDHHMMSS. keep it simple!
versioning  microservices  hailo  aws  ec2  ecs  docker  containers  scheduling  allocation  deployment  provisioning  qos 
april 2015 by jm
Subscribing AWS Lambda Function To SNS Topic With aws-cli
how to use the AWS command line tools to do this
aws  aws-cli  cli  lambda  sns  hacks 
april 2015 by jm
AWS Lambda Event-Driven Architecture With Amazon SNS
Any message posted to an SNS topic can trigger the execution of custom code you have written, but you don’t have to maintain any infrastructure to keep that code available to listen for those events and you don’t have to pay for any infrastructure when the code is not being run. This is, in my opinion, the first time that Amazon can truly say that AWS Lambda is event-driven, as we now have a central, independent, event management system (SNS) where any authorized entity can trigger the event (post a message to a topic) and any authorized AWS Lambda function can listen for the event, and neither has to know about the other.
aws  ec2  lambda  sns  events  cep  event-processing  coding  cloud  hacks  eric-hammond 
april 2015 by jm
Amazon Machine Learning
Upsides of this new AWS service:

* great UI and visualisations.

* solid choice of metric to evaluate the results. Maybe things moved on since I was working on it, but the use of AUC, false positives and false negatives was pretty new when I was working on it. (er, 10 years ago!)


* it could do with more support for unsupervised learning algorithms. Supervised learning means you need to provide training data, which in itself can be hard work. My experience with logistic regression in the past is that it requires very accurate training data, too -- its tolerance for misclassified training examples is poor.

* Also, in my experience, 80% of the hard work of using ML algorithms is writing good tokenisation and feature extraction algorithms. I don't see any help for that here unfortunately. (probably not that surprising as it requires really detailed knowledge of the input data to know what classes can be abbreviated into a single class, etc.)
amazon  aws  ml  machine-learning  auc  data-science 
april 2015 by jm
(SEC307) Building a DDoS-Resilient Architecture with AWS
good slides on a "web application firewall" proxy service, deployable as an auto-scaling EC2 unit
ec2  aws  ddos  security  resilience  slides  reinvent  firewalls  http  elb 
april 2015 by jm
S3's "" endpoint
public documentation of how to work around the legacy S3 multi-region replication behaviour in North America
aws  s3  eventual-consistency  consistency  us-east  replication  workarounds  legacy 
april 2015 by jm
When S3's eventual consistency is REALLY eventual
a consistency outage in S3 last year, resulting in about 40 objects failing read-after-write consistency for a duration of about 23 hours
s3  eventual-consistency  aws  consistency  read-after-writes  bugs  outages  stackdriver 
april 2015 by jm
« earlier      
per page:    204080120160

related tags

10/8  aas  accounts  acm  acm-queue  adrian-cockcroft  adrian-cockroft  adroll  advent  advice  alarming  alerting  alestic  alibaba  aliyun  allocation  ama  amazon  ami  analytics  andrew-spyker  anti-fraud  ap  aphyr  api  api-gateway  apis  architecture  asg  asgard  atlas  auc  aurora  authentication  authorization  auto-scaling  autoscaling  availability  aws  aws-cli  awscli  awsume  azul  azure  b2b  b2c  backup  backups  batch  batching  beanstalk  benchmarks  blake2  blogs  blue-green-deployments  blue-green-deploys  boto  bots  bugs  burst  cacerts  campaigns  cap  cassandra  cd  cdn  cep  certificates  certs  chaos-kong  chaos-monkey  chatops  chef  china  chris-newcombe  circuit-breakers  cli  clients  clock  cloud  cloud-connect  cloudformation  cloudfront  cloudnative  cloudsearch  cloudsmith  cloudsploit  cloudwatch  cluster  clustering  clusters  code-spaces  codedeploy  coding  colmmacc  comet  command-line  comparison  conferences  consistency  constant-load  containers  continuous-delivery  corporate-culture  corruption  costs  counters  coursera  cp  cpu  credentials  credstash  cron  cross-region  crypto  culture  curl  danilop  data  data-pipelines  data-protection  data-science  data-structures  databases  datadog  dataman  datawire  daylight-savings  ddos  dedupe  delete  delivery  delta  demo  deploy  deployment  design  dev  devops  dht  disk  disks  distcomp  distributed-cron  distsys  dns  docker  documentation  documents  dos  dropbox  dst  duplicity  duply  dynalite  dynamodb  dynect  ebs  ec2  ecommerce  ecs  elasticache  elb  email  embedded  emr  emrfs  environments  eric-brandwine  eric-hammond  etcd  etl  eu  eu-central-1  eureka  event-processing  event-streaming  events  eventual-consistency  examples  export  extortion  fail  failover  failures  fault-tolerance  ffmpeg  figures  filesystems  firewalls  five-eyes  flow-logs  fluentd  formal-methods  fraud  freebsd  fuse  games  gaming  gce  gchq  germany  gilt  git  github  google  google-storage  gossip  graphics  grey-failures  gulp  h264  ha  hacks  hadoop  hailo  haproxy  hardware  history  hls  hosting  http  https  hugo  hvm  hystrix  iam  incident-response  infrastructure  instances  integration-testing  inter-region  internet  io  iops  iostat  ip-addresses  ipc  ireland  ironfan  james-hamilton  java  javascript  jepsen  jmespath  jobs  json  kafka  kappa  kcl  key-length  key-management  key-rotation  keys  kinesis  kms  knife  kpl  kubernetes  lambda  languages  latency  law  legacy  leveldb  libraries  limits  linux  load  load-balancing  load-testing  loaders  loading  logging  lucene  machine-learning  management  marc-brooker  measurement  memcached  memory  mesos  messaging  metrics  mfa  microservices  microsoft  mit  mitch-garnaat  ml  mocha  mocking  mocks  model-checking  money  mongodb  monitoring  mp4  multi-az  multi-region  mysql  mytaxi  nat  netflix  network  network-partitions  networking  nginx  node  node.js  nosql  notifications  nsa  obama  object-model  onlive  oo  open-source  openstack  ops  opsgenie  optimization  osx  outages  overload  packaging  packet-capture  packets  partitions  pbailis  pdf  percentiles  perf  perfect-forward-secrecy  performance  periodic-tasks  pinterest  piops  pipeline  pipelines  pki  pluscal  post-mortems  postmortem  prediction  presentations  pricing  privacy  prod  production  programming  proving  provisioning  proxying  puppet  pv  pylons  python  qcon  qos  querying  queueing  queues  quora  quorum  r3  rabbitmq  raid  rdbms  rds  read-after-writes  recurrence  reddit  redis  redshift  registry  reinvent  reliability  reliabilty  replication  resilience  resource-limits  ribbon  rightscale  roles  round-trip  route53  rsa  ruby  runjop  s3  s3fs  s3funnel  s3ql  sandboxing  scala  scalability  scale  scaling  scalr  schedulers  scheduling  sched_batch  scraping  scryer  sdk  sdn  search  secrets  security  segment  servers  service-discovery  service-registry  services  ses  sha1  sharding  sift-science  signalfx  simulation  skyliner  slack  slas  slides  smartstack  smtp  smugmug  snapshots  snooping  sns  sockets  solr  spark  speculative-execution  spikes  spinnaker  spot  spot-instances  sql  sqs  ssl  stack-hammer  stackdriver  stacks  startups  steam  storage  streaming  streams  surveillance  survey  sysadmin  system-tests  systemtap  tail  talks  tcp  tcpdump  tellybug  terraform  testing  tests  thrift  time  timezones  tips  tla  tla+  tlc  tls  tools  tornado  tracing  transactions  tunables  tuning  tutorials  two-factor-authentication  ubuntu  ultradns  unit-tests  unix  uploads  us-east  user-submitted-code  utc  version-control  versioning  via:brianscanlan  via:chorn  via:highscalability  via:marc  via:marc-brooker  via:matt-sergeant  via:nelson  via:pdolan  video  virtualization  vlans  vpc  web-services  website-generators  white-papers  whitepapers  windows  wishlist  workarounds  worker-pools  xen  xpath  yas3fs  yeobot  zing  zonify  zookeeper 

Copy this bookmark: