jm + amazon   75

'Lambda and serverless is one of the worst forms of proprietary lock-in we've ever seen in the history of humanity' • The Register
That doesn't mean Polvi is a fan. "Lambda and serverless is one of the worst forms of proprietary lock-in that we've ever seen in the history of humanity," said Polvi, only partly in jest, referring to the most widely used serverless offering, AWS Lambda. "It's seriously as bad as it gets."

He elaborated: "It's code that tied not just to hardware – which we've seen before – but to a data center, you can't even get the hardware yourself. And that hardware is now custom fabbed for the cloud providers with dark fiber that runs all around the world, just for them. So literally the application you write will never get the performance or responsiveness or the ability to be ported somewhere else without having the deployment footprint of Amazon."


Absolutely agreed...
lambda  amazon  aws  containers  coreos  deployment  lockin  proprietary  serverless  alex-polvi  kubernetes 
4 weeks ago by jm
Amazon Shipping Filter - Chrome Web Store
a user script to determine when Amazon.{com,co.uk,fr,de,it,etc} will not deliver to your chosen delivery address, which is a common risk for Irish users
ireland  shipping  amazon  buying  extensions  chrome  userscripts  shopping 
8 weeks ago by jm
How to operate reliable AWS Lambda applications in production
running a reliable Lambda application in production requires you to still follow operational best practices. In this article I am including some recommendations, based on my experience with operations in general as well as working with AWS Lambda.
aws  cloud  lambda  ops  amazon 
8 weeks ago by jm
A Decade of Dynamo: Powering the next wave of high-performance, internet-scale applications - All Things Distributed
A deep dive on how we were using our existing databases revealed that they were frequently not used for their relational capabilities. About 70 percent of operations were of the key-value kind, where only a primary key was used and a single row would be returned. About 20 percent would return a set of rows, but still operate on only a single table.

With these requirements in mind, and a willingness to question the status quo, a small group of distributed systems experts came together and designed a horizontally scalable distributed database that would scale out for both reads and writes to meet the long-term needs of our business. This was the genesis of the Amazon Dynamo database.

The success of our early results with the Dynamo database encouraged us to write Amazon's Dynamo whitepaper and share it at the 2007 ACM Symposium on Operating Systems Principles (SOSP conference), so that others in the industry could benefit. The Dynamo paper was well-received and served as a catalyst to create the category of distributed database technologies commonly known today as "NoSQL."


That's not an exaggeration. Nice one Werner et al!
dynamo  history  nosql  storage  databases  distcomp  amazon  papers  acm  data-stores 
9 weeks ago by jm
Amazon Global Product Price Check
price compare across global Amazon sites, by ASIN. there are some major differences
prices  amazon  via:its  price-check  comparison  shopping  eu  uk  asin 
july 2017 by jm
Open Guide to Amazon Web Services
'A lot of information on AWS is already written. Most people learn AWS by reading a blog or a “getting started guide” and referring to the standard AWS references. Nonetheless, trustworthy and practical information and recommendations aren’t easy to come by. AWS’s own documentation is a great but sprawling resource few have time to read fully, and it doesn’t include anything but official facts, so omits experiences of engineers. The information in blogs or Stack Overflow is also not consistently up to date. This guide is by and for engineers who use AWS. It aims to be a useful, living reference that consolidates links, tips, gotchas, and best practices. It arose from discussion and editing over beers by several engineers who have used AWS extensively.'
amazon  aws  guides  documentation  ops  architecture 
june 2017 by jm
jantman/awslimitchecker

A script and python module to check your AWS service limits and usage, and warn when usage approaches limits.

Users building out scalable services in Amazon AWS often run into AWS' service limits - often at the least convenient time (i.e. mid-deploy or when autoscaling fails). Amazon's Trusted Advisor can help this, but even the version that comes with Business and Enterprise support only monitors a small subset of AWS limits and only alerts weekly. awslimitchecker provides a command line script and reusable package that queries your current usage of AWS resources and compares it to limits (hard-coded AWS defaults that you can override, API-based limits where available, or data from Trusted Advisor where available), notifying you when you are approaching or at your limits.


(via This Week in AWS)
aws  amazon  limits  scripts  ops 
may 2017 by jm
_Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases_
'Amazon Aurora is a relational database service for OLTP workloads offered as part of Amazon Web Services (AWS). In this paper, we describe the architecture of Aurora and the design considerations leading to that architecture. We believe the central constraint in high throughput data processing has moved from compute and storage to the network. Aurora brings a novel architecture to the relational database to address this constraint, most notably by pushing redo processing to a multi-tenant scale-out storage service, purpose-built for Aurora. We describe how doing so not only reduces network traffic, but also allows for fast crash recovery, failovers to replicas without loss of data, and fault-tolerant, self-healing storage. We then describe how Aurora achieves consensus on durable state across numerous storage nodes using an efficient asynchronous scheme, avoiding expensive and chatty recovery protocols. Finally, having operated Aurora as a production service for over 18 months, we share the lessons we have learnt from our customers on what modern cloud applications expect from databases.'
via:rbranson  aurora  aws  amazon  databases  storage  papers  architecture 
may 2017 by jm
Towards true continuous integration – Netflix TechBlog – Medium
Netflix discuss how they handle the eternal dependency-management problem which arises with lots of microservices:
Using the monorepo as our requirements specification, we began exploring alternative approaches to achieving the same benefits. What are the core problems that a monorepo approach strives to solve? Can we develop a solution that works within the confines of a traditional binary integration world, where code is shared? Our approach, while still experimental, can be distilled into three key features:

Publisher feedback — provide the owner of shared code fast feedback as to which of their consumers they just broke, both direct and transitive. Also, allow teams to block releases based on downstream breakages. Currently, our engineering culture puts sole responsibility on consumers to resolve these issues. By giving library owners feedback on the impact they have to the rest of Netflix, we expect them to take on additional responsibility.

Managed source — provide consumers with a means to safely increment library versions automatically as new versions are released. Since we are already testing each new library release against all downstreams, why not bump consumer versions and accelerate version adoption, safely.

Distributed refactoring — provide owners of shared code a means to quickly find and globally refactor consumers of their API. We have started by issuing pull requests en masse to all Git repositories containing a consumer of a particular Java API. We’ve run some early experiments and expect to invest more in this area going forward.


What I find interesting is that Amazon dealt effectively with the first two many years ago, in the form of their "Brazil" build system, and Google do the latter (with Refaster?). It would be amazing to see such a system released into an open source form, but maybe it's just too heavyweight for anyone other than a giant software company on the scale of a Google, Netflix or Amazon.
brazil  amazon  build  microservices  dependencies  coding  monorepo  netflix  google  refaster 
may 2017 by jm
Amazon DynamoDB Accelerator (DAX)
Amazon DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache for DynamoDB that delivers up to a 10x performance improvement – from milliseconds to microseconds – even at millions of requests per second. DAX does all the heavy lifting required to add in-memory acceleration to your DynamoDB tables, without requiring developers to manage cache invalidation, data population, or cluster management.


No latency percentile figures, unfortunately. Also still in preview.
amazon  dynamodb  aws  dax  performance  storage  databases  latency  low-latency 
april 2017 by jm
The Occasional Chaos of AWS Lambda Runtime Performance
If our code has modest resource requirements, and can tolerate large changes in performance, then it makes sense to start with the least amount of memory necessary. On the other hand, if consistency is important, the best way to achieve that is by cranking the memory setting all the way up to 1536MB.
It’s also worth noting here that CPU-bound Lambdas may be cheaper to run over time with a higher memory setting, as Jim Conning describes in his article, “AWS Lambda: Faster is Cheaper”. In our tests, we haven’t seen conclusive evidence of that behavior, but much more data is required to draw any strong conclusions.
The other lesson learned is that Lambda benchmarks should be gathered over the course of days, not hours or minutes, in order to provide actionable information. Otherwise, it’s possible to see very impressive performance from a Lambda that might later dramatically change for the worse, and any decisions made based on that information will be rendered useless.
aws  lambda  amazon  performance  architecture  ops  benchmarks 
march 2017 by jm
Amazon Echo security fail
Ughhhh.
Amazon Echo sends your WiFi password to Amazon. No option to disable. Trust us it's in an "encrypted file"
amazon  echo  wifi  passwords  security  data-privacy  data-protection 
january 2016 by jm
VPC NAT gateways : transactional uniqueness at scale
colmmacc introducing the VPC NAT gateway product from AWS, in a guest post on James Hamilton's blog no less!:
you can think of it as a new “even bigger” [NAT] box, but under the hood NAT gateways are different. The connections are managed by a fault-tolerant co-operation of devices in the VPC network fabric. Each new connection is assigned a port in a robust and transactional way, while also being replicated across an extensible set of multiple devices. In other words: the NAT gateway is internally horizontally scalable and resilient.
amazon  ec2  nat  networking  aws  colmmacc 
january 2016 by jm
RentTheRunway's Engineering Ladder
One of the best things about working at Amazon was having a clear, well-defined career progression, and it's something that's always been absent in startups. Career growth, levelling, and tech management is important, and also helps in hiring by providing clear levels. This is the RentTheRunway engineering ladder, Camille Fournier's team, which they open sourced back in March 2015
engineering  hiring  management  career  renttherunway  camille-fournier  amazon  startups  career-growth  levelling  ladder 
october 2015 by jm
3 Lessons From The Amazon Takedown - Fortune
They are: The leaders we admire aren’t always that admirable; Economic performance and costs trump employee well-being; and people participate in and rationalize their own subjugation.

'In the end, “Amazonians” are not that different from other people in their psychological dynamics. Their company is just a more extreme case of what many other organizations regularly do. And most importantly, let’s locate the problem, if there is one, and its solution where it most appropriately belongs—not with a CEO who is greatly admired (and wealthy beyond measure) running a highly admired company, but with a society where money trumps human well-being and where any price, maybe even lives, is paid for status and success.'

(via Lean)
amazon  work  work-life-balance  life  us  fortune  via:ldoody  ceos  employment  happiness 
august 2015 by jm
Revised and much faster, run your own high-end cloud gaming service on EC2!
a g2.2xlarge provides decent Windows GPU performance over the internet, at about $0.53 per hour
gaming  games  ec2  amazon  aws  cloud  windows  hacks 
july 2015 by jm
1172401 – Add Amazon root certificates
Well, well -- looks like AWS is about to disrupt PKI, and about time too. If they come up with a Plex-style "provision a cert" API, it'll be revolutionary
pki  ssl  tls  amazon  aws  apis  web-services  ops 
june 2015 by jm
Amazon's Drone Delivery Patent Just Feels Like Trolling At This Point
Oh dear, Amazon.
These aren’t actual technologies yet. [...] All of which underscores that Amazon might never ever ever ever actually implement delivery drones. The patent paperwork was filed nearly a year after Amazon’s splashy drone program reveal on 60 Minutes. At the time we called it revolutionary marketing because, you know, delivery drones are technical and logistical madness, not to mention that commercial drone use is illegal right now. Although, in fairness the FAA did just relax some rules so that Amazon could test drones.

At this point it feels like Amazon is just trolling. It’s trolling us with public relations BS about its future drones, and it’s trolling future competitors -- Google is also apparently working on this -- so that if somebody ever somehow does anything relating to drone delivery, Amazon can sue them. If I’m wrong, I’ll deliver my apology via Airmail.
amazon  trolling  patents  uspto  delivery  drones  uavs  competition  faa 
may 2015 by jm
s3.amazonaws.com "certificate verification failed" errors due to crappy Verisign certs and overzealous curl policies
Seth Vargo is correct. Its not the bit length of the key which is at issue, its the signature algorithm. The entire keychain for the s3.awsamazon.com key is signed with SHA1withRSA:

https://www.ssllabs.com/ssltest/analyze.html?d=s3.amazonaws.com&s=54.231.244.0&hideResults=on

At issue is that the root verisign key has been marked as weak because of SHA1 and taken out of the curl bundle which is widely popular, and this issue will continue to cause more and more issues going forwards as that bundle makes it way into shipping o/s distributions and aws certification verification breaks.


'This is still happening and curl is now failing on my machine causing all sorts of fun issues (including breaking CocoaPods that are using S3 for storage).' -- @jmhodges

This may be a contributory factor to the issue @nelson saw: https://nelsonslog.wordpress.com/2015/04/28/cyberduck-is-responsible-for-my-bad-ssl-certificate/

Curl's ca-certs bundle is also used by Node: https://github.com/joyent/node/issues/8894 and doubtless many other apps and packages.

Here's a mailing list thread discussing the issue: http://curl.haxx.se/mail/archive-2014-10/0066.html -- looks like the curl team aren't too bothered about it.
curl  s3  amazon  aws  ssl  tls  certs  sha1  rsa  key-length  security  cacerts 
april 2015 by jm
Amazon Machine Learning
Upsides of this new AWS service:

* great UI and visualisations.

* solid choice of metric to evaluate the results. Maybe things moved on since I was working on it, but the use of AUC, false positives and false negatives was pretty new when I was working on it. (er, 10 years ago!)

Downsides:

* it could do with more support for unsupervised learning algorithms. Supervised learning means you need to provide training data, which in itself can be hard work. My experience with logistic regression in the past is that it requires very accurate training data, too -- its tolerance for misclassified training examples is poor.

* Also, in my experience, 80% of the hard work of using ML algorithms is writing good tokenisation and feature extraction algorithms. I don't see any help for that here unfortunately. (probably not that surprising as it requires really detailed knowledge of the input data to know what classes can be abbreviated into a single class, etc.)
amazon  aws  ml  machine-learning  auc  data-science 
april 2015 by jm
Comparing Message Queue Architectures on AWS
A good overview -- I like the summary table. tl;dr:
If you are light on DevOps and not latency sensitive use SQS for job management and Kinesis for event stream processing. If latency is an issue, use ELB or 2 RabbitMQs (or 2 beanstalkds) for job management and Redis for event stream processing.
amazon  architecture  aws  messaging  queueing  elb  rabbitmq  beanstalk  kinesis  sqs  redis  kafka 
february 2015 by jm
Following Fire Phone Flop, Big Changes At Amazon’s Lab126 | Fast Company | Business + Innovation
as one insider told me, it feels like "Lab126 is in the doghouse" and that "Jeff is taking out his frustration with the failure of the Fire Phone" on upper management.
lab126  amazon  fire-phone  phones  hardware  tech 
january 2015 by jm
Amazon sellers hit by nightmare before Christmas as glitch cuts prices to 1p | Technology | The Guardian
From 7-8pm on Friday, [RepricerExpress] software, used by third-party sellers to ensure their products are the cheapest on the market, went a bit haywire and reduced prices to as little as 1p.
1p  amazon  resellers  repricer-express  fail  price-cutting  automation  risks  undercutting 
december 2014 by jm
AWS Key Management Service Cryptographic Details
"AWS Key Management Service (AWS KMS) provides cryptographic keys and operations scaled for the cloud. AWS KMS keys and functionality are used by other AWS cloud services, and you can use them to protect user data in your applications that use AWS. This white paper provides details on the cryptographic operations that are executed within AWS when you use AWS KMS."
white-papers  aws  amazon  kms  key-management  crypto  pdf 
december 2014 by jm
AWS re:Invent 2014 | (SPOT302) Under the Covers of AWS: Its Core Distributed Systems - YouTube
This is a really solid talk -- not surprising, alv@ is one of the speakers!
"AWS and Amazon.com operate some of the world's largest distributed systems infrastructure and applications. In our past 18 years of operating this infrastructure, we have come to realize that building such large distributed systems to meet the durability, reliability, scalability, and performance needs of AWS requires us to build our services using a few common distributed systems primitives. Examples of these primitives include a reliable method to build consensus in a distributed system, reliable and scalable key-value store, infrastructure for a transactional logging system, scalable database query layers using both NoSQL and SQL APIs, and a system for scalable and elastic compute infrastructure.

In this session, we discuss some of the solutions that we employ in building these primitives and our lessons in operating these systems. We also cover the history of some of these primitives -- DHTs, transactional logging, materialized views and various other deep distributed systems concepts; how their design evolved over time; and how we continue to scale them to AWS. "


Slides: http://www.slideshare.net/AmazonWebServices/spot302-under-the-covers-of-aws-core-distributed-systems-primitives-that-power-our-platform-aws-reinvent-2014
scale  scaling  aws  amazon  dht  logging  data-structures  distcomp  via:marc-brooker  dynamodb  s3 
november 2014 by jm
Amazon announces 300 jobs at Dublin base - RTÉ News
DUB6 is expanding (or is it DUB14 now? can't keep up)
The jobs will be across a variety of positions, including software engineers, technical engineers, technical managers, customer support and IT security.
dublin  amazon  jobs  ireland 
november 2014 by jm
Stephanie Dean on event management and incident response
I asked around my ex-Amazon mates on twitter about good docs on incident response practices outside the "iron curtain", and they pointed me at this blog (which I didn't realise existed).

Stephanie Dean was the front-line ops manager for Amazon for many years, over the time where they basically *fixed* their availability problems. She since moved on to Facebook, Demonware, and Twitter. She really knows her stuff and this blog is FULL of great details of how they ran (and still run) front-line ops teams in Amazon.
ops  incident-response  outages  event-management  amazon  stephanie-dean  techops  tos  sev1 
october 2014 by jm
IT Change Management
Stephanie Dean on Amazon's approach to CMs. This is solid gold advice for any company planning to institute a sensible technical change management process
ops  tech  process  changes  change-management  bureaucracy  amazon  stephanie-dean  infrastructure 
october 2014 by jm
Why Amazon Has No Profits (And Why It Works)
Amazon has perhaps 1% of the US retail market by value. Should it stop entering new categories and markets and instead take profit, and by extension leave those segments and markets for other companies? Or should it keep investing to sweep them into the platform? Jeff Bezos’s view is pretty clear: keep investing, because to take profit out of the business would be to waste the opportunity. He seems very happy to keep seizing new opportunities, creating new businesses, and using every last penny to do it.
amazon  business  strategy  capex  spending  stocks  investing  retail 
october 2014 by jm
Painless, effective peer reviews
This sounds like a nice way to do effective peer-driven team reviews without herculean effort, which were one of the most effective reviewing techniques (along with upwards reviewing of management) I encountered at Amazon. (Yes, the Amazon approach was very time-consuming and universally loathed.)

The potential downside I can see is that it doesn't give the reviewer enough time to revise any review comments they have second thoughts about, whereas written reviews do, but that would be an easy fix at the end of the process. Also, it's worth noting that in most cases, a good review requires a bit of time to marshal thoughts and come up with a coherent review of a peer, so this doesn't completely avoid the impact on effort. Still, a definite improvement I would say.
hr  management  reviews  performance  peer-driven-review  360-reviews  staff  peers  work  teams  amazon 
august 2014 by jm
Latest EBS tuning tips
from yesterday's AWS Summit in NYC:

Cheat sheet of EBS-optimized instances. http://t.co/vmTlhUtpWk
Optimize your queue depth to achieve lower latency & highest IOPS. http://t.co/EO48oa0D6X
When configuring your RAID, use a stripe size of 128KB or 256KB. http://t.co/N0ldtFJ4t6
Use larger block size to speed up the pre-warming process. http://t.co/8UoIeWE2px
ebs  aws  amazon  iops  raid  ops  tuning 
july 2014 by jm
This Internet Millionaire Has a New Deal For You - D Magazine
Good interview with Dave "Woot" Rutledge, who's now well out of Amazon and plans to get back into the crap-clearing business at Meh.com:

'Amazon’s fundamental misunderstanding of what made Woot great can be seen today on the site. It sells many items simultaneously. It’s a marketplace, not an event. The write-ups are cute, not subversively funny. Woot is no longer a bug-eyed beast with eight tentacles. It’s a pancake with two smaller pancakes for Mickey Mouse ears and a smile made of whipped cream. In 2012, two years into his three-year deal with Amazon, Rutledge walked. He won’t say how many millions his early departure cost him, but his contract with Amazon included a three-year non-compete clause from the date of sale, and he was watching the clock.'
amazon  ecommerce  business  b2c  woot.com  meh.com  dave-rutledge  selling  acquisitions 
june 2014 by jm
"Taking the hotdog"
aka. lock acquisition. ex-Amazon-Dublin lingo, observed in the wild ;)
language  hotdog  archie-mcphee  amazon  dublin  intercom  coding  locks  synchronization 
may 2014 by jm
'Monitoring and detecting causes of failures of network paths', US patent 8,661,295 (B1)
The first software patent in my name -- couldn't avoid it forever :(
Systems and methods are provided for monitoring and detecting causes of failures of network paths. The system collects performance information from a plurality of nodes and links in a network, aggregates the collected performance information across paths in the network, processes the aggregated performance information for detecting failures on the paths, analyzes each of the detected failures to determine at least one root cause, and initiates a remedial workflow for the at least one root cause determined. In some aspects, processing the aggregated information may include performing a statistical regression analysis or otherwise solving a set of equations for the performance indications on each of a plurality of paths. In another aspect, the system may also include an interface which makes available for display one or more of the network topology, the collected and aggregated performance information, and indications of the detected failures in the topology.


The patent describes an early version of Pimms, the network failure detection and remediation system we built for Amazon.
amazon  pimms  swpats  patents  networking  ospf  autoremediation  outage-detection 
may 2014 by jm
AWS Elastic Beanstalk for Docker
This is pretty amazing. nice work, Beanstalk team. not sure how well it integrates with the rest of AWS though
aws  amazon  docker  ec2  beanstalk  ops  containers  linux 
april 2014 by jm
Using AWS in the context of Australian Privacy Considerations
interesting new white paper from Amazon regarding recent strengthening of the Aussie privacy laws, particularly w.r.t. geographic location of data and access by overseas law enforcement agencies...
amazon  aws  security  law  privacy  data-protection  ec2  s3  nsa  gchq  five-eyes 
april 2014 by jm
Adrian Cockroft's Cloud Outage Reports Collection
The detailed summaries of outages from cloud vendors are comprehensive and the response to each highlights many lessons in how to build robust distributed systems. For outages that significantly affected Netflix, the Netflix techblog report gives insight into how to effectively build reliable services on top of AWS. [....] I plan to collect reports here over time, and welcome links to other write-ups of outages and how to survive them.
outages  post-mortems  documentation  ops  aws  ec2  amazon  google  dropbox  microsoft  azure  incident-response 
march 2014 by jm
Register article on Amazon's attitude to open source
This article is frequently on target; this secrecy (both around open source and publishing papers) was one of the reasons I left Amazon.
Of the sources with whom we spoke, many indicated that Amazon's lack of participation was a key reason for why people left the company – or never joined at all. This is why Amazon's strategy of maintaining secrecy may derail the e-retailer's future if it struggles to hire the best talent. [...]

"In many cases in the big companies and all the small startups, your Github profile is your resume," explained another former Amazonian. "When I look at developers that's what I'm looking for, [but] they go to Amazon and that resume stops ... It absolutely affects the quality of their hires." "You had no portfolio you could share with the world," said another insider on life after working at Amazon. "The argument this was necessary to attract talent and to retain talent completely fell on deaf ears."
amazon  recruitment  secrecy  open-source  hiring  work  research  conferences 
january 2014 by jm
Chartbeat's Lessons learned tuning TCP and Nginx in EC2
a good writeup of basic sysctl tuning for an internet-facing HTTP proxy fleet running in EC2. Nothing groundbreaking here, but it's well-written
nginx  amazon  ec2  tcp  ip  tuning  sysctl  linux  c10k  ssl  http 
january 2014 by jm
SkyJack - autonomous drone hacking
Samy Kamkar strikes again. 'Using a Parrot AR.Drone 2, a Raspberry Pi, a USB battery, an Alfa AWUS036H wireless transmitter, aircrack-ng, node-ar-drone, node.js, and my SkyJack software, I developed a drone that flies around, seeks the wireless signal of any other drone in the area, forcefully disconnects the wireless connection of the true owner of the target drone, then authenticates with the target drone pretending to be its owner, then feeds commands to it and all other possessed zombie drones at my will.'
drones  amazon  hacking  security  samy-kamkar  aircrack  node  raspberry-pi  airborne-zombies 
december 2013 by jm
AMAZORK - by Zachary Mason
> reorg

Ok, you reorganize all zero of your direct reports. Way to stay out of trouble, Hoss.

Perhaps you'd like to coin an acronym?
amazon  amazork  via:jrauser  sev2s  reorgs  work  zachary-mason  games  interactive-fiction  zork  text-adventures 
december 2013 by jm
10 Things You Should Know About AWS
Some decent tips in here, mainly EC2-focussed
amazon  ec2  aws  ops  rds 
november 2013 by jm
'Experience of software engineers using TLA+, PlusCal and TLC' [slides] [pdf]
by Chris Newcombe, an AWS principal engineer. Several Amazonians sharing their results in simulating tricky distributed-systems problems using formal methods
tla+  pluscal  tlc  formal-methods  simulation  proving  aws  amazon  architecture  design 
october 2013 by jm
The Hole in Our Collective Memory: How Copyright Made Mid-Century Books Vanish - Rebecca J. Rosen - The Atlantic
A book published during the presidency of Chester A. Arthur has a greater chance of being in print today than one published during the time of Reagan.
This is not a gently sloping downward curve. Publishers seem unwilling to sell their books on Amazon for more than a few years after their initial publication. The data suggest that publishing business models make books disappear fairly shortly after their publication and long before they are scheduled to fall into the public domain. Copyright law then deters their reappearance as long as they are owned. On the left side of the graph before 1920, the decline presents a more gentle time-sensitive downward sloping curve.
business  books  legal  copyright  law  public-domain  reading  history  publishers  amazon  papers 
september 2013 by jm
Microsoft CEO Steve Ballmer retires: A firsthand account of the company’s employee-ranking system
LOL MS. Sadly, this talk of "core competencies" and "visibility" is pretty reminiscent of Amazon's review season, too:
This illustrated another problem with [stack ranking]: It destroyed trust between individual contributors and management, because the stack rank required that all lower-level managers systematically lie to their reports. Why? Because for years Microsoft did not admit the existence of the stack rank to nonmanagers. Knowledge of the process gradually leaked out, becoming a recurrent complaint on the much-loathed (by Microsoft) Mini-Microsoft blog, where a high-up Microsoft manager bitterly complained about organizational dysfunction and was joined in by a chorus of hundreds of employees. The stack rank finally made it into a Vanity Fair article in 2012, but for many years it was not common knowledge, inside or outside Microsoft. It was presented to the individual contributors as a system of objective assessment of “core competencies,” with each person being judged in isolation.
When review time came, and programmers would fill out a short self-assessment talking about their achievements, strengths, and weaknesses, only some of them knew that their ratings had been more or less already foreordained at the stack rank. [...] If you did know about the stack rank, you weren’t supposed to admit it. So you went through the pageantry of the performance review anyway, arguing with your manager in the rhetoric of “core competencies.” The managers would respond in kind. Since the managers had little control over the actual score and attendant bonus and raise (if any), their job was to write a review to justify the stack rank in the language of absolute merit. (“Higher visibility” was always a good catch-all: Sure, you may be a great coder and work 80 hours a week, but not enough people have heard of you!)
amazon  stack-ranking  employees  ranking  work  microsoft  core-competencies 
august 2013 by jm
awscli

The future of the AWS command line tools is awscli, a single, unified, consistent command line tool that works with almost all of the AWS services. Here is a quick list of the services that awscli currently supports: Auto Scaling, CloudFormation, CloudSearch, CloudWatch, Data Pipeline, Direct Connect, DynamoDB, EC2, ElastiCache, Elastic Beanstalk, Elastic Transcoder, ELB, EMR, Identity and Access Management, Import/Export, OpsWorks, RDS, Redshift, Route 53, S3, SES, SNS, SQS, Storage Gateway, Security Token Service, Support API, SWF, VPC. Support for the following appears to be planned: CloudFront, Glacier, SimpleDB.

The awscli software is being actively developed as an open source project on Github, with a lot of support from Amazon. You’ll note that the biggest contributors to awscli are Amazon employees with Mitch Garnaat leading. Mitch is also the author of boto, the amazing Python library for AWS.
aws  awscli  cli  tools  command-line  ec2  s3  amazon  api 
august 2013 by jm
stuff Google has learned from their hiring data
A. On the hiring side, we found that [interview] brainteasers are a complete waste of time. How many golf balls can you fit into an airplane? How many gas stations in Manhattan? A complete waste of time. They don’t predict anything. They serve primarily to make the interviewer feel smart.

Instead, what works well are structured behavioral interviews, where you have a consistent rubric for how you assess people, rather than having each interviewer just make stuff up. Behavioral interviewing also works — where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.” The interesting thing about the behavioral interview is that when you ask somebody to speak to their own experience, and you drill into that, you get two kinds of information. One is you get to see how they actually interacted in a real-world situation, and the valuable “meta” information you get about the candidate is a sense of what they consider to be difficult.

This makes sense, and matches what I learned in Amazon. Bad news for Microsoft though! (Correction: Adam Shostack got in touch to note that MS haven't done this for 10+ years either.)

Also, I like this:

A. One of the things we’ve seen from all our data crunching is that G.P.A.’s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there’s a slight correlation. Google famously used to ask everyone for a transcript and G.P.A.’s and test scores, but we don’t anymore, unless you’re just a few years out of school. We found that they don’t predict anything. What’s interesting is the proportion of people without any college education at Google has increased over time as well. So we have teams where you have 14 percent of the team made up of people who’ve never gone to college.
google  hiring  interviewing  interviews  brainteasers  gpa  microsoft  star  amazon 
june 2013 by jm
Schneier on Security: Blowback from the NSA Surveillance
Unintended consequences on US-focused governance of the internet and cloud computing:
Writing about the new Internet nationalism, I talked about the ITU meeting in Dubai last fall, and the attempt of some countries to wrest control of the Internet from the US. That movement just got a huge PR boost. Now, when countries like Russia and Iran say the US is simply too untrustworthy to manage the Internet, no one will be able to argue. We can't fight for Internet freedom around the world, then turn around and destroy it back home. Even if we don't see the contradiction, the rest of the world does.
internet  freedom  cloud-computing  amazon  google  hosting  usa  us-politics  prism  nsa  surveillance 
june 2013 by jm
EC2Instances.info
'Easy Amazon EC2 Instance Comparison'. a nice UI on the various EC2 instance types on offer with their key attributes. Misses out availability of EBS-optimized instances though
amazon  ec2  aws  comparison  pricing 
june 2013 by jm
the infamous 2008 S3 single-bit-corruption outage
Neat, I didn't realise this was publicly visible. A single corrupted bit infected the S3 gossip network, taking down the whole S3 service in (iirc) one region:
We've now determined that message corruption was the cause of the server-to-server communication problems. More specifically, we found that there were a handful of messages on Sunday morning that had a single bit corrupted such that the message was still intelligible, but the system state information was incorrect. We use MD5 checksums throughout the system, for example, to prevent, detect, and recover from corruption that can occur during receipt, storage, and retrieval of customers' objects. However, we didn't have the same protection in place to detect whether [gossip state] had been corrupted. As a result, when the corruption occurred, we didn't detect it and it spread throughout the system causing the symptoms described above. We hadn't encountered server-to-server communication issues of this scale before and, as a result, it took some time during the event to diagnose and recover from it.

During our post-mortem analysis we've spent quite a bit of time evaluating what happened, how quickly we were able to respond and recover, and what we could do to prevent other unusual circumstances like this from having system-wide impacts. Here are the actions that we're taking: (a) we've deployed several changes to Amazon S3 that significantly reduce the amount of time required to completely restore system-wide state and restart customer request processing; (b) we've deployed a change to how Amazon S3 gossips about failed servers that reduces the amount of gossip and helps prevent the behavior we experienced on Sunday; (c) we've added additional monitoring and alarming of gossip rates and failures; and, (d) we're adding checksums to proactively detect corruption of system state messages so we can log any such messages and then reject them.


This is why you checksum all the things ;)
s3  aws  post-mortems  network  outages  failures  corruption  grey-failures  amazon  gossip 
june 2013 by jm
Understanding Elastic Block Store Availability and Performance [slides]
fantastic in-depth presentation on EBS usage; lots of good advice here if you're using EBS volumes with/without PIOPS
piops  ebs  performance  aws  ec2  ops  storage  amazon  presentations 
may 2013 by jm
By the numbers: How Google Compute Engine stacks up to Amazon EC2
Scalr's thoughts on Google's EC2 competitor.
with Google Compute Engine, AWS has a formidable new competitor in the public cloud space, and we’ll likely be moving some of Scalr’s production workloads from our hybrid aws-rackspace-softlayer setup to it when it leaves beta. There’s a strong technical case for migrating heavy workloads to GCE, and I’ll be grabbing popcorn to eagerly watch as the battle unfolds between the giants.
gce  cloud  ec2  amazon  aws  google  scalr 
march 2013 by jm
Algorithmic Rape Jokes in the Library of Babel | Quiet Babylon
Fantastic blog post on the Solid Gold Bomb "Keep Calm And Rape A Lot" algorithmically-generated t-shirt scandal, covering "content spinners", print-on-demand, object spam, Borges' Library Of Babel, "crapjects", and summing up with:
We’re sad for the sick feeling this has brought on! We’re sad for the queasy feeling this has brought on! We’re sad for the ailing feeling this has brought about!


Strange new world for sure.
algorithms  amazon  crapjects  borges  babel  object-spam  spam  content-spinning  rape  scandals  furore  solid-gold-bomb  t-shirts  etailers  print-on-demand 
march 2013 by jm
AWS Advent 2012
'an annual exploration of Amazon Web Services.' Some great hacks here
aws  amazon  advent  sysadmin  s3  ec2  chef  puppet  ops 
december 2012 by jm
James Hamilton - Failures at Scale & How to Ride Through Them - AWS re:Invent 2012 - Cpn208
mostly an update of his classic USENIX paper, but pretty cool to come across a mention of a network monitoring system we've built on page 21 ;)
amazon  james-hamilton  reliabilty  slides  aws 
december 2012 by jm
Weathering the Unexpected - ACM Queue
Failures happen, and resilience drills help organizations prepare for them.


Good write-up on Google's DiRT (Disaster Recovery Test) procedures, clearly based on Amazon's Gameday exercises. ;) See also http://queue.acm.org/detail.cfm?id=2371297 for a moderated discussion including Jesse Robbins and John Allspaw
game-day  tests  disaster-recovery  dirt  exercises  history  amazon  google  etsy  resilience  acm 
september 2012 by jm
South Lake Union Eats
holy moly, that's a lot of food trucks (SLU is the district hosting Amazon's Seattle campus)
food  food-trucks  seattle  restaurants  slu  amazon 
june 2012 by jm
Many Niches » Blog Archive » On Working At Amazon
(catching up on old posts) good article from a recent hire, discussing some unusual aspects of the corporate culture
amazon  culture  work 
june 2012 by jm
Perspectives on the Costa Concordia Incident
hey, co-worker Rory Browne gets namechecked on James Hamilton's blog! woo
costa-concordia  amazon  james-hamilton  disaster  boats  safety  post-mortem 
february 2012 by jm
Amazon hiring embedded OS developers
hey, I know a few of those! 'I need more help on a project I’m driving at Amazon where we continue to make big changes in our datacenter network to improve customer experience and drive down costs while, at the same time, deploying more gear into production each day than all of Amazon.com used back in 2000. It’s an exciting time and we have big changes happening in networking. If you enjoy and have experience in operating systems, networking protocol stacks, or embedded systems and you would like to work on one of the biggest networks in the world, [get in touch].' -- James Hamilton
james-hamilton  aws  jobs  amazon  networking  embedded 
october 2011 by jm
Amazon EC2 outage: summary and lessons learned
Rightscale CTO on last week's outage; pretty detailed, good round-up of useful commentary from around the web, too
ebs  ec2  aws  cloud  availability  slas  rightscale  amazon 
april 2011 by jm
CareerZoo
jobs fair this weekend in Dublin, in The Mansion House -- if you're interested in talking to someone about working for Amazon, come along! (plug plug)
amazon  work  dublin  hiring  careers  jobs  from delicious
january 2011 by jm
That mysterious J
"in e-mail from Microsoft employees, you may find a stray J [...] The J started out its life as a smiley-face. The WingDings font puts a smiley face where the letter J goes. [...] As the message travels from machine to machine, the font formatting may get lost or mangled, resulting in the letter J appearing when a smiley face was intended." aha! mystery solved. Amazon is full of mysterious "J"s in emails, and now I know why
amazon  j  letters  wingdings  microsoft  spoor  fonts  noise  from delicious
november 2010 by jm
CloudSplit – Real Time Cloud Analytics
interesting idea from Joe -- track your cloud-hosting spend in real-time
cloudsplit  hosting  amazon  ec2  azure  joe-drumgoole  analytics  real-time  from delicious
september 2009 by jm

related tags

1p  360-reviews  acm  acquisitions  advent  airborne-zombies  aircrack  alex-polvi  algorithms  amazon  amazork  analytics  api  apis  archie-mcphee  architecture  asin  auc  aurora  automation  autoremediation  autoscaling  availability  aws  awscli  azure  b2c  babel  beanstalk  benchmarks  boats  books  borges  brainteasers  brazil  build  bureaucracy  business  buying  c10k  cacerts  camille-fournier  capex  career  career-growth  careers  ceos  certs  change-management  changes  chef  chrome  cli  cloud  cloud-computing  cloudsearch  cloudsplit  coding  colmmacc  command-line  comparison  competition  conferences  containers  content-spinning  copyright  core-competencies  coreos  corruption  costa-concordia  crapjects  crypto  culture  curl  data-privacy  data-protection  data-science  data-stores  data-structures  databases  dave-rutledge  dax  ddos  delivery  dependencies  deployment  design  dht  dirt  disaster  disaster-recovery  distcomp  docker  documentation  dos  drones  dropbox  dublin  dynamo  dynamodb  ebs  ec2  echo  ecommerce  ecs  elb  embedded  employees  employment  engineering  eric-brandwine  etailers  etl  etsy  eu  event-management  exercises  extensions  faa  fail  failures  fcs  fire-phone  five-eyes  fonts  food  food-trucks  formal-methods  fortune  freedom  funny  furore  gadgets  game-day  games  gaming  garadget  gce  gchq  gifts  google  gossip  gpa  grey-failures  guides  hacking  hacks  happiness  hardware  hiring  history  hosting  hotdog  hr  http  incident-response  infrastructure  interactive-fiction  intercom  internet  interviewing  interviews  investing  iops  iot  ip  ireland  j  james-hamilton  jobs  joe-drumgoole  journalism  kafka  key-length  key-management  kinesis  kms  kubernetes  lab126  ladder  lambda  language  latency  law  legal  letters  levelling  life  limits  linux  lockin  locks  logging  low-latency  lucene  machine-learning  management  meh.com  messaging  microservices  microsoft  ml  monorepo  nat  netflix  network  networking  nginx  node  noise  nosql  nsa  nytimes  object-spam  open-source  ops  ospf  outage-detection  outages  papers  passwords  patents  pdf  peer-driven-review  peers  performance  phones  pimms  piops  pki  pluscal  post-mortem  post-mortems  presentations  price-check  price-cutting  prices  pricing  print-on-demand  prism  privacy  process  proprietary  proving  public-domain  publishers  puppet  queueing  rabbitmq  raid  ranking  rape  raspberry-pi  rdbms  rds  reading  real-time  recruitment  redis  redshift  refaster  reliabilty  renttherunway  reorgs  repricer-express  research  resellers  resilience  restaurants  retail  reviews  rightscale  risks  rsa  s3  safety  samy-kamkar  scale  scaling  scalr  scandals  scripts  sdn  search  seattle  secrecy  security  selling  serverless  sev1  sev2s  sha1  shipping  shopping  simulation  slas  slides  slu  solid-gold-bomb  solr  spam  spending  spoor  spot-fleet  spot-instances  sql  sqs  ssl  stack-ranking  staff  star  startups  stephanie-dean  stocks  storage  strategy  surveillance  swpats  synchronization  sysadmin  sysctl  t-shirts  tcp  teams  tech  techops  tests  text-adventures  tla+  tlc  tls  to-get  tools  tos  trolling  tuning  uavs  uk  undercutting  us  us-politics  usa  userscripts  uspto  via:its  via:johnke  via:jrauser  via:ldoody  via:marc-brooker  via:rbranson  vlans  vpc  warehouses  web-services  white-papers  whitepapers  wifi  windows  wingdings  wirecutter  wired  woot.com  work  work-life-balance  workplace  zachary-mason  zork 

Copy this bookmark:



description:


tags: