jm + esb   4

WHAT WENT WRONG IN BRITISH AIRWAYS DATACENTER IN MAY 2017?
A SPOF UPS. There was a similar AZ-wide outage in one of the Amazon DUB datacenters with a similar root cause, if I recall correctly -- supposedly redundant dual UPS systems were in fact interdependent, in that case, and power supply switchover wasn't clean enough to avoid affecting the servers.
Minutes later power was restored was resumed in what one source described as “uncontrolled fashion.” Instead of gradual restore, all power was restored at once resulting in a power surge.   BA CEO Cruz told BBC Radio this power surge  caused network hardware to fail. Also server hardware was damaged because of the power surge.

It seems as if the UPS was the single point of failure for power feed of the IT equipment in Boadicea House . The Times is reporting that the same UPS was powering both Heathrow based datacenters. Which could be a double single point of failure if true (I doubt it is)

The broken network  stopped the exchange of messages between different BA systems and application. Without messaging, there is no exchange of information between various applications. BA is using Progress Software’s Sonic [enterprise service bus].


(via Tony Finch)
postmortems  ba  airlines  outages  fail  via:fanf  datacenters  ups  power  progress  esb  j2ee 
may 2017 by jm
insane ESB health and safety policy
Where it is not possible to avoid reversing, it is ESB policy that staff driving on behalf of the company or anybody on company premises should reverse into car spaces/bays, allowing them to drive out subsequently.


BUT WHYYYYYYYYYY
esb  health-n-safety  policies  crazy  funny  driving  reversing  lol  safety 
april 2014 by jm
Microservices and nanoservices
A great reaction to Martin Fowler's "microservices" coinage, from Arnon Rotem-Gal-Oz:

'I guess it is easier to use a new name (Microservices) rather than say that this is what SOA actually meant'; 'these are the very principles of SOA before vendors pushed the [ESB] in the middle.'

Others have also chosen to define microservices slightly differently, as a service written in 10-100 LOC. Arnon's reaction:

“Nanoservice is an antipattern where a service is too fine-grained. A nanoservice is a service whose overhead (communications, maintenance, and so on) outweighs its utility.”

Having dealt with maintaining an over-fine-grained SOA stack in Amazon, I can only agree with this definition; it's easy to make things too fine-grained and create a raft of distributed-computing bugs and deployment/management complexity where there is no need to do so.
architecture  antipatterns  nanoservices  microservices  soa  services  design  esb 
march 2014 by jm
ESB Networks | Power Check | Service Interruptions Map
real-time service outage information on a map, from Ireland's power network
esb  ireland  mapping  data  outages  service  power 
april 2013 by jm

Copy this bookmark:



description:


tags: