scaling   16483

« earlier    

MySQL High Availability at GitHub | Hacker News
Am I mistaken or it seems to me that promoting a slave to a master is kind of fragile operation(detecting that it did really fail). I mean there are more robust methods:
1. There is a bookkeeper cluster providing replicated log. (No Kafka! Hold on a second, move to 4)

2. All nodes of a single MySQL cluster have a service on them that has two responsibilities: 1. accepting writes. 2. following the log. MySQL service only serves reads. Write will be performed indirectly from the writer service which is running besides MySQL service.

3. Bookkeeper (and DistributedLog on top of it) enforces single writer. So essentially one node is appending to log, the others are following it.

4. Whenever one of slaves thinks master is failed (or maybe partially-failed), it issues a FENCE request to bookkeeper, which either stops current master from appending to log, or fails to get majority vote from bookkeeper nodes. In either way log stays consistent. (There exist no two masters at a single point in time)

5. If one slave is able to fence the current master, it just notifies service discovery component to redirect clients. This operation doesn't have to happen atomically. Since old master is fenced and it can't append to log, anyway. So client operation will be safely rejected.

This approach can convert any data store to a consistent, highly available store. It is even capable of running multiple databases grouped as one cluster. Replicated log is a separate module. It is possible to replicate from MongoDB to SQL Server as an example. It has the advantage that, you don't have to adopt any untested databases. Just use good old MySQL/PG/...

I understand that, it requires quite a lot of work to refactor already running infrastructure. But it is more robust and somewhat safer. Bookkeeper's built-in fence mechanism precludes any random writes from any partially failed master.
scaling  architecture  mysql  postgresql  database  high-availability 
12 hours ago by hellsten
Want to work on the "science of " -based programs in w/ rockstars like…
globaldev  evidence-based  scaling  from twitter_favs
2 days ago by Varna
The "Rules" are the greatest -- what does quick-and-dirty look like for today's version of r&d, starting small and scaling?
rules  design  howto  instructions  ideas  csn  scaling  kickstarter  fundraising 
3 days ago by sdean
MIT's Mind-Blowing Solutions to the 9 Hardest Startup Problems |
Bil Aulet from MIT - key points: (1) are you creating a feature a d product or a company (great example for Google). Scale points $1MM, $10MM, $100MM, culture> strategy, other HRM related topics.
ESB6  ch7  ch18  culture  scaling  hiring  Growth 
7 days ago by jeromekatz

« earlier    

related tags

000  2007  2013  2018-04-18-0  2018-04-18  aaron_clauset  accuracy  activismmodel  adaptive  ads  advice  advocacy  agile  algorithms  allometry  amazon  architecture  article  atlassian  automation  autoscale  availability  awesome  aws  bestpractices  bitcoin  brandonpeele  business  cache  capacity  career  ch18  ch7  changemanagement  chriskyle  cities  cloud  cloudformation  cluster  clustering  collection  community  complexity  compose  compute  conference  consistency  container  created-by:ifttt  crisismanagement  criteria  crypto  csn  css  culture  dask  data-pipeline  database  deploy  design  developer  development  devops  devpos  digital  distributed-systems  distributed  diversity  dns  docker-compose  docker  durability  dynamodb  economics  ecs  education  engineering  entrepreneurship  erlang  esb6  evangelizing  evidence-based  evolution  experimentation  facebook  fb  feminism  flask  font-size  fundraising  funny  germany  github-starred  globaldev  golang  google  growth  growthlab  gunicorn  ha  hardware  harvard  hashing  heal  healing  heroku  high-availability  highavailability  hiring  hn  horizontal  hosting  howto  humanfactors  hypergrowth  ideas  idempotency  important  india  instagram  instructions  investors  ipad  iphone  jan  javascript  jupyter  k8  k8s  kellblog  kickstarter  kubernetes  latency  launch  learn  limits  linear  lms  lovefirst  management  marketing  max  message  min  mmm  monitoring  multisig  mysql  nature  netflix  networks  nginx  notebooks  onlineclass  onlinecourses  opensource  optimization  organization  overview  pandas  passiveincome  people  performance  persistence  personalbranding  pinterest  pitching  pooler  pooling  postgres  postgresql  presentations  pricing  product-owner  production  productmanagement  programdesign  programming  prometheus  proxy  python  quality  queue  quorum  rails  randomness  react-native  react  redis  reference  replicas  resource  responsive  route53  ruby  rules  saas  said  sales  sass  scalability  scale  schnorr  scss  security  server  shard  sharding  signature  sla  slicing  sociology  software  sql  stackoverflow  staging  startup  storage  strategy  super  svg  swarm  sysadmin  talk  team  technology  test  todo  tools  training  uber  unix  urban_life  useful  vertical  viewport  wedel  well  whatsapp  work  wow  written   

Copy this bookmark: