jm + services   20

atlassian/localstack: A fully functional local AWS cloud stack. Develop and test your cloud apps offline!
LocalStack provides an easy-to-use test/mocking framework for developing Cloud applications. Currently, the focus is primarily on supporting the AWS cloud stack.

LocalStack spins up the following core Cloud APIs on your local machine:

API Gateway at http://localhost:4567;
Kinesis at http://localhost:4568;
DynamoDB at http://localhost:4569;
DynamoDB Streams at http://localhost:4570;
Elasticsearch at http://localhost:4571;
S3 at http://localhost:4572;
Firehose at http://localhost:4573;
Lambda at http://localhost:4574;
SNS at http://localhost:4575;
SQS at http://localhost:4576

Additionally, LocalStack provides a powerful set of tools to interact with the cloud services, including a fully featured KCL Kinesis client with Python binding, simple setup/teardown integration for nosetests, as well as an Environment abstraction that allows to easily switch between local and remote Cloud execution.
aws  emulation  mocking  services  testing  dynamodb  s3 
9 days ago by jm
Airfixers - Hosting without the hassle
Full AirBnB property management service in Dublin
dublin  airbnb  management  rental  services 
october 2016 by jm
Charity Majors responds to the CleverTap Mongo outage war story
This is a great blog post, spot on:
You can’t just go “dudes it’s faster” and jump off a cliff.  This shit is basic.  Test real production workloads. Have a rollback plan.  (Not for *10 days* … try a month or two.)

The only thing I'd nitpick on is that it's all very well to say "buy my book" or "come see me talk at Blahcon", but a good blog post or webpage would be thousands of times more useful.
databases  stateful-services  services  ops  mongodb  charity-majors  rollback  state  storage  testing  dba 
october 2016 by jm
AWS Api Gateway for Fun and Profit
good worked-through example of an API Gateway rewriting system
api-gateway  aws  api  http  services  ops  alerting  alarming  opsgenie  signalfx 
december 2015 by jm
A Gulp Workflow for Amazon Lambda
'any nontrivial development of Lambda functions will require a simple, automated build/deploy process that also fills a couple of Lambda’s gaps such as the use of node modules and environment variables.'

See also : 'I am psyched about Amazon’s new Lambda service for asynchronous task processing, but the ideal development and testing cycle is really left to the engineer. While Amazon provides a web-based console, I prefer an approach that uses Mocha. Below you will find the gritty details using Kinesis events as a sample input.'
lambda  aws  services  testing  deployment  ops  mocha  gulp  javascript 
december 2015 by jm
Introducing Netty-HTTP from Cask
netty-http library solves [Netty usability issues] by using JAX-RS annotations to build a HTTP path routing layer on top of netty. In addition, the library implements a guava service to manage the HTTP service. netty-http allows users of the library to just focus on writing the business logic in HTTP handlers without having to worry about the complexities of path routing or learning netty pipeline internals to build the HTTP service.

We've written something very similar, although I didn't even bother supporting JAX-RS annotations -- just a simple code-level DSL.
jax-rs  netty  http  cask  java  services  coding 
december 2015 by jm
wangle/Codel.h at master · facebook/wangle
Facebook's open-source implementation of the CoDel queue management algorithm applied to server request-handling capacity in their C++ service bootstrap library, Wangle.
wangle  facebook  codel  services  capacity  reliability  queueing 
november 2015 by jm
Book a domestic cleaner online in 60 seconds; "like Hailo for cleaners" apparently. Live in Dublin, London, Manchester, Birmingham and Leeds. Use code HASSLEDUBLIN for 15% off
hailo  cleaners  hassle  via:hailo  domestic  home  services  b2c 
july 2014 by jm
Microservices and nanoservices
A great reaction to Martin Fowler's "microservices" coinage, from Arnon Rotem-Gal-Oz:

'I guess it is easier to use a new name (Microservices) rather than say that this is what SOA actually meant'; 'these are the very principles of SOA before vendors pushed the [ESB] in the middle.'

Others have also chosen to define microservices slightly differently, as a service written in 10-100 LOC. Arnon's reaction:

“Nanoservice is an antipattern where a service is too fine-grained. A nanoservice is a service whose overhead (communications, maintenance, and so on) outweighs its utility.”

Having dealt with maintaining an over-fine-grained SOA stack in Amazon, I can only agree with this definition; it's easy to make things too fine-grained and create a raft of distributed-computing bugs and deployment/management complexity where there is no need to do so.
architecture  antipatterns  nanoservices  microservices  soa  services  design  esb 
march 2014 by jm
The Microservice Declaration of Independence
"Microservices" seems to be yet another term for SOA; small, decoupled, independently-deployed services, with well-defined public HTTP APIs. Pretty much all the services I've worked on over the past few years have been built in this style. Still, let's keep an eye on this concept anyway.

Another definition seems to be a more FP-style one: -- where the "microservice" does one narrowly-defined thing, and that alone.
microservices  soa  architecture  handwaving  http  services  web  deployment 
march 2014 by jm
"Dapper, a Large-Scale Distributed Systems Tracing Infrastructure" [PDF]
Google paper describing the infrastructure they've built for cross-service request tracing (ie. "tracer requests"). Features: low code changes required (since they've built it into the internal protobuf libs), low performance impact, sampling, deployment across the ~entire production fleet, output visibility in minutes, and has been live in production for over 2 years. Excellent read
dapper  tracing  http  services  soa  google  papers  request-tracing  tracers  protobuf  devops 
march 2014 by jm
Smart Metering in the UK is FCUKED
Most utilities don’t want smart metering.  In fact they seem to have used the wrong dictionary.  It is difficult to find anything smart about the UK deployment, until you realise that the utilities use smart in the sense of “it hurts”.  They consider they have a perfectly adequate business model which has no need for new technology.  In many Government meetings, their reluctant support seems to be a veneer for the hope that it will all end in disaster, letting them go back to the world they know, of inflated bills and demands for money with menaces. [...]

Even when smart meters are deployed, there is no evidence that any utility will use the resulting data to transform their business, rather than persecute the consumer.  At a recent US conference a senior executive for a US utility which had deployed smart meters, stated that their main benefit was “to give them more evidence to blame the customer”.  That’s a good description of the attitude displayed by our utilities.
smart-metering  energy  utilities  uk  services  metering  consumer 
december 2013 by jm
Airbnb's Smartstack
Service discovery a la Airbnb -- Nerve and Synapse: two external daemons that run on each node, Nerve to manage registration in Zookeeper, and Synapse to generate a haproxy configuration file from that, running on each host, allowing connections to all other hosts.
haproxy  services  ops  load-balancing  service-discovery  nerve  synapse  airbnb 
october 2013 by jm
'a Java port of Twitter's Snowflake thrift service presented as an HTTP-based Dropwizard service'.
an HTTP-based service for generating unique ID numbers at high scale with some simple guarantees. supports returning ID numbers as: JSON and JSONP; Google's Protocol Buffers; Plain text.

At GE, we were more interested in the uncoordinated aspects of Snowflake than its throughput requirements, so HTTP was fine for our needs. We also exposed the core of Snowflake as an embeddable module so it can be directly integrated into our applications. We don't have the guarantees that the Snowflake-Zookeeper integration was providing, but that was also acceptable to us. In places where we really needed high throughput, we leveraged the snowizard-core embeddable module directly.

Odd OSS license, though -- BSDish?
java  open-source  ids  soa  services  snowflake  http 
august 2013 by jm
Latency's Worst Nightmare: Performance Tuning Tips and Tricks [slides]
the basics of running a service stack (web, app servers, data stores) on AWS. some good benchmark figures in the final slides
benchmarks  aws  ec2  ebs  piops  services  scaling  scalability  presentations 
april 2013 by jm
'an expressive toolset for constructing scalable, resilient [service] architectures. It works in the cloud, in the data center, and on your laptop, and it makes your system diagram visible and inevitable. Inevitable systems coordinate automatically to interconnect, removing the hassle of manual configuration of connection points (and the associated danger of human error).' Looks like a pretty neat cluster deployment tool; driven from a single configuration file, using Chef, integrating closely with AWS and providing many useful additional features
chef  deployment  clusters  knife  services  aws  ec2  ops  ironfan  demo 
january 2013 by jm
'a UNIX init scheme with service supervision' - philosophically similar to daemontools, widely packaged, LSB init.d-script-compliant, BSD-licensed
daemon  supervision  services  unix  lsb  server  ops 
january 2013 by jm
What Second Life can teach your datacenter about scaling Web apps
good scaling advice from Linden Labs' Ian Wilkes (who doesn't seem to have a blog, sadly)
linden  ian-wilkes  scaling  datacenters  scalability  deployment  ops  services  from delicious
february 2010 by jm

related tags

advice  airbnb  alarming  alerting  antipatterns  api  api-gateway  apis  architecture  aws  b2c  benchmarks  capacity  cask  charity-majors  chef  cleaners  clusters  codel  coding  consumer  daemon  dapper  databases  datacenters  dba  demo  deployment  design  devops  domestic  dublin  dynamodb  ebs  ec2  emulation  energy  esb  facebook  google  gulp  hacking  hailo  handwaving  haproxy  hassle  home  http  ian-wilkes  ids  internet  ironfan  james-hamilton  java  javascript  jax-rs  knife  lambda  linden  load-balancing  lsb  management  metering  microservices  mocha  mocking  mongodb  nanoservices  nerve  netty  open-source  ops  opsgenie  papers  piops  presentations  protobuf  queueing  reliability  rental  request-tracing  rollback  s3  saas  scalability  scaling  server  service-discovery  services  signalfx  smart-metering  snowflake  soa  software  state  stateful-services  storage  supervision  synapse  testing  tracers  tracing  uk  unix  utilities  via:hailo  wangle  web  webhooks 

Copy this bookmark: