The million dollar engineering problem


86 bookmarks. First posted by chris.heisel 12 days ago.


The million dollar engineering problem by and why they use
Terraform  from twitter_favs
2 days ago by kennygatdula
hidden costs scaling amazon services
AWS  lv 
3 days ago by diasyrmus
The Terrible Secret of AWS: sub-linear scaling, aka economies of scale, is what powers growth ever since the Industrial Revolution. But the delightful linear scaling of AWS pricing denies YOU precisely that, and the wedge between the two is why Jeff Bezos is rich.
aws  architecture  cloud  scalability  ovum 
4 days ago by yorksranter
RT : How do you cut a million dollars from your annual bill?
AWS  from twitter
4 days ago by gatestone
Segment is the analytics API you've always wanted. It's the easiest way to install all of your favorite analytics tools at once!
architecture  scalability 
5 days ago by derekharmel
Smart techniques for reducing AWS Dynamo and ECS bill
architecture  cloud  cost  theskinny 
6 days ago by coldclimate
How do you cut a million dollars from your annual bill?
from twitter
6 days ago by peba
Cut AWS costs
aws  business  cloud 
6 days ago by sdellis
The Segment Stack
aws  costs  architecture  business  mezu 
7 days ago by melo
great post.

1. DynamoDB hot shards were a big problem -- and it is terrible that diagnosing this requires a ticket to AWS support! This heat map should be a built-in feature.

2. ECS auto-scaling gets a solid thumbs-up.

3. Switching from ELB to ALB lets them set ports dynamically for individual ECS Docker containers, and then pack as many containers as will fit on a giant EC2 instance.

4. Terraform modules to automate setup and maintainance of ECS, autoscaling groups, and ALBs
terraform  segment  architecture  aws  dynamodb  alb  elb  asg  ecs  docker 
7 days ago by jm
The implication here is that you aren’t paying for total throughput, but rather partition count. And if you happen to have a few keys which saturate the same individual partitions, you have to double capacity to split a single hot partition onto their own partitions rather than scale the capacity linearly. And even there you are limited to the throughput for a single partition.
architecture  business  aws 
7 days ago by oisin
RT : "there's (still) no magic bullet": a deep dive into gory bits of AWS cost and resource optimization by :
from twitter
7 days ago by bob
RT : "there's (still) no magic bullet": a deep dive into gory bits of AWS cost and resource optimization by :
from twitter_favs
7 days ago by roens
“make the easy way, the right way”
from twitter
7 days ago by WibbleFarmer
RT : "there's (still) no magic bullet": a deep dive into gory bits of AWS cost and resource optimization by :
from twitter
7 days ago by pronoiac
# [The million dollar engineering problem](https://segment.com/blog/the-million-dollar-eng-problem/)

> it’s easy to fall into a cycle where the first response to any problem is to spend more money



> we managed to cut our AWS bill by over one million dollars annually



> two thirds of our cost of goods sold (COGS) was the bill from AWS



> hundreds of large EBS drives, over-provisioned cache and RDS instances…Relics left over from incidents of increased load that had not been sized back down



> While we can fix bugs in our own code, we can’t control our customers.



> Because we only have 60% of peak traffic at nights and on weekends, we’ve been able to save substantially by adding auto-scaling, and not have to worry about sudden traffic spikes.



> The general argument is that you can get less steal from noisy neighbors if you are the only one on a physical machine. And there’s more likelihood that you will be the sole VM on a physical machine if you are running at the largest possible instance size.



> From our best estimation, ELBs are essentially built atop an army of small, auto-scaling instances running HAProxy.



> The ALB (application load balancer) allows ECS to set ports dynamically for individual containers, and then pack as many containers as can fit onto a given instance.



> Aurora automatically scales disk capacity in 10GB increments

sysadmin  aws  budget  pricing 
7 days ago by jacklittleton
Favorite tweet: assaf

The million dollar engineering problem https://t.co/k2zjaiF4C9

— Assaf (@assaf) March 19, 2017

http://twitter.com/assaf/status/843496579952754688
IFTTT  twitter  favorite 
7 days ago by tswaterman
"there's (still) no magic bullet": a deep dive into gory bits of AWS cost and resource optimization by :
from twitter_favs
7 days ago by kas
The million dollar engineering problem
from twitter_favs
8 days ago by andydavies
RT : This is the kind of AWS blog post people need

o7 to &co
from twitter
8 days ago by stilkov
Segment is the analytics API you've always wanted. It's the easiest way to install all of your favorite analytics tools at once!
aws 
9 days ago by strzalek
For an early startup, using the cloud isn’t even a question these days. No RFPs, provisioning orders, or physical shipments of servers. Just the promise of…
from instapaper
9 days ago by tmcw
We managed to cut our AWS bill by over one million dollars annually
sysadmin  business 
9 days ago by verbila
The million dollar engineering problem http://bit.ly/2mRwUEb
IFTTT  Buffer 
9 days ago by soofaloofa
The million dollar engineering problem Wise words from on learning to managing infrastructure costs.
from twitter_favs
9 days ago by bf4
The million dollar engineering problem: For an early startup, using the cloud isn’t even a question these days. No RFPs, provisioning orders, or physical shipments of servers. Just the promise of getting up and running on “infinitely scalable” compute power within minutes. via Pocket, added at:March 17, 2017 at 12:14PM
IFTTT  Pocket 
9 days ago by LordEnzo
After a three months of focused work, we managed to cut our AWS bill by over one million dollars annually. Here is the story of how we did it.
aws  scaling  cost  segment 
10 days ago by fjordaan
For an early startup, using the cloud isn’t even a question these days. No RFPs, provisioning orders, or physical shipments of servers. Just the promise of getting up and running on “infinitely scalable” compute power within minutes. via Pocket
business  pinboard  tech 
10 days ago by lfcipriani
"At Segment, we found ourselves in a similar situation near the end of last year. We were hitting the classic startup scaling problems, and our costs were starting to grow a bit too quickly. So we decided to focus on reducing the primary contributor: our AWS bill."
aws  architecture  experience-report  cloud-computing  cost-optimization 
10 days ago by bkittelmann
Segment is the analytics API you've always wanted. It's the easiest way to install all of your favorite analytics tools at once!
10 days ago by akarpo
How a company saved a million dollars a year by looking in depth into their usage of AWS
aws  architecture  cloud 
10 days ago by beebware
RT : Drop everything and carefully read this:
from twitter
10 days ago by rjw1
After being in the weeds for three months, we managed to hit our goal. We eliminated over $1m dollars in annual spend off our AWS bill. And managed to increase our average utilization by 20%.
architecture  aws  cost  performance 
10 days ago by euler
Segment is the analytics API you've always wanted. It's the easiest way to install all of your favorite analytics tools at once!
10 days ago by martinbalfanz
AWS cost cutting exercise
10 days ago by labibrahman
The million dollar engineering problem
from twitter
11 days ago by wschenk