failover   1016

« earlier    

github/orchestrator: MySQL replication topology management and HA
MySQL replication topology management and HA. Contribute to github/orchestrator development by creating an account on GitHub.
mysql  replication  golang  failover  database  admin  administration 
november 2018 by pinterb
October 21 post-incident analysis | The GitHub Blog
A network outage caused a split-brain scenario, and their failover system allowed writes to occur in both
regional databases. Once the outage was repaired it was impossible to reconcile writes in an automated fashion as a result.

Embarrassingly, this exact scenario was called out in their previous blog post about their Raft-based failover system at https://githubengineering.com/mysql-high-availability-at-github/ --

"In a data center isolation scenario, and assuming a master is in the isolated DC, apps in that DC are still able to write to the master. This may result in state inconsistency once network is brought back up. We are working to mitigate this split-brain by implementing a reliable STONITH from within the very isolated DC. As before, some time will pass before bringing down the master, and there could be a short period of split-brain. The operational cost of avoiding split-brains altogether is very high."

Failover is hard.
github  fail  outages  failover  replication  consensus  ops 
october 2018 by jm
What's New in Failover Clustering in Windows Server | Microsoft Docs
In Windows Server 2012 R2, if the cluster is configured to use dynamic quorum (the default), the witness vote is also dynamically adjusted based on the number of voting nodes in current cluster membership. If there are an odd number of votes, the quorum witness does not have a vo
cluster  failover  sql  availabilitygroups  reference  Microsoft  sqltact 
september 2018 by wda
4G WAN: Bonding vs Load Balancing
Difference between bonding and load balancing
router  internet  isp  failover 
september 2018 by mjs

« earlier    

related tags

2012r2  2014  6.5  accesspath  activedirectory  admin  administration  always  always_on  alwayson  amazon  aoag  appliance  architecture  aurora  availability  availabilitygroups  aws.regions  aws  aws_rds  azure  backup  based  block  blog  carp  chaos-engineering  chaos  cloud  cloudformation  cluster  clustering  computer  configuration  confluent  consensus  consider  corosync  create  cross-region  cross  csv  d0178  daemon  data  database  databases  datacenter  db  dd-wrt  dedibox  deployment  design  development  devops  dfs  dhcp  dhcpd  digitalocean  direct  discovery  distributed  dns  do  docker  documentation  domain  down  downtime  dual  duty  ebs  elixir  erlang  error  esxi  eureka  fail  failed  failure  fallback  fci  firewall  fix  flocker  floss  force  gif  github  golang  group  groups  gui  guide  ha  hadr  haproxy  heartbeat  high  high_availability  highavailabiity  highavailability  hosting  howto  http  humor  hyper-v  information  instance  internet  ip  isc  isp  java  kafka  keepalived  killswitch  kubernetes  lan  load_balancing  loadbalance  loadbalancing  loss  lvs  make  manual  mdf  memcache  methods  mha  microsoft  mongodb  monitoring  monkey  movement  ms  multi-lan  multi-wan  multiaz  mysql  netflix  network  networking  nginx  nosql  objects  of  omshell  on  online  openbsd  opensource  operations  ops  outage  outages  pager  pagerduty  papers  permission  pfsense  policy  port  possibility  postgis  postgres  postgresql  proxy  raft  ragg  rails  rds  reboot  recovery  redis  redundancy  redundant  reference  reliability  replica  replication  resiliancy  resilience  restart  resume  route53  router  routing  rsynch  s2d  scale  scaling  script  secondary  selective  sentinel  server  service  setup  sfe  shared  sharing  sim  sonicwall  sql  sqlite  sqlserver  sqltact  stack  standalone  storage  stories  strategy  swarm  synology  sysadmin  system  t-sql  technet  test  testing  the  timeouts  tip  tips  tool  topologies  tricks  tutorial  types  ubiquiti  ubnt  ubuntu  unifi  upstream  uptime  use  vcenter  vm  vmware  volume  vpn  wan  wifi  windows  wsfc  zeromq  zookeepr 

Copy this bookmark:



description:


tags: