jm + routing   12

ELS: latency based load balancer, part 1
ELS measures the following things:

Success latency and success rate of each machine;
Number of outstanding requests between the load balancer and each machine. These are the requests that have been sent out but we haven’t yet received a reply;
Fast failures are better than slow failures, so we also measure failure latency for each machine.

Since users care a lot about latency, we prefer machines that are expected to answer quicker. ELS therefore converts all the measured metrics into expected latency from the client’s perspective.[...]

In short, the formula ensures that slower machines get less traffic and failing machines get much less traffic. Slower and failing machines still get some traffic, because we need to be able to detect when they come back up again.
latency  spotify  proxies  load-balancing  els  algorithms  c3  round-robin  load-balancers  routing 
december 2015 by jm
fabio
fast, modern, zero-conf load balancing HTTP(S) router managed by consul; serves 15k reqs/sec, in Go, from eBay
load-balancing  consul  http  https  routing  ebay  go  open-source  fabio 
october 2015 by jm
Baker Street
client-side 'service discovery and routing system for microservices' -- another Smartstack, then
python  router  smartstack  baker-street  microservices  service-discovery  routing  load-balancing  http 
october 2015 by jm
How I doubled my Internet speed with OpenWRT
File under "silly network hacks":
Comcast has an initiative called Xfinity WiFi. When you rent a cable modem/router combo from Comcast (as one of my nearby neighbors apparently does), in addition to broadcasting your own WiFi network, it is kind enough to also broadcast “xfinitywifi,” a second “hotspot” network metered separately from your own.


By using his Buffalo WZR-HP-AG300H router's extra radio, he can load-balance across both his own paid-for connection, and the XFinity WiFi free one. ;)
comcast  diy  networking  openwrt  routing  home-network  hacks  xfinity-wifi  buffalo 
march 2015 by jm
2015-02-19 GCE outage
40 minutes of multi-zone network outage for majority of instances.

'The internal software system which programs GCE’s virtual network for VM
egress traffic stopped issuing updated routing information. The cause of
this interruption is still under active investigation. Cached route
information provided a defense in depth against missing updates, but GCE VM
egress traffic started to be dropped as the cached routes expired.'

I wonder if Google Pimms fired the alarms for this ;)
google  outages  gce  networking  routing  pimms  multi-az  cloud 
february 2015 by jm
Hacker Redirects Traffic From 19 Internet Providers to Steal Bitcoins | Threat Level | WIRED
'The attacker specifically targeted a collection of bitcoin mining “pools”–bitcoin-producing cooperatives in which users contribute their computers’ processing power and are rewarded with a cut of the resulting cryptocurrency the pool produces. The redirection technique tricked the pools’ participants into continuing to devote their processors to bitcoin mining while allowing the hacker to keep the proceeds. At its peak, according to the researchers’ measurements, the hacker’s scam was pocketing a flow of bitcoins and other digital currencies including dogecoin and worldcoin worth close to $9,000 a day. “With this kind of hijacking, you can quite easily grab a large collection of clients,” says Pat Litke, one of the Dell researchers. “It takes less than a minute, and you end up with a lot of mining traffic under your control.”'

'In total, Stewart and Litke were able to measure $83,000 worth of cryptocurrency stolen in the BGP attack [...] but the total haul could be larger'
bitcoin  mining  fraud  internet  bgp  routing  security  attacks  hacking 
august 2014 by jm
Shutterbits replacing hardware load balancers with local BGP daemons and anycast
Interesting approach. Potentially risky, though -- heavy use of anycast on a large-scale datacenter network could increase the scale of the OSPF graph, which scales exponentially. This can have major side effects on OSPF reconvergence time, which creates an interesting class of network outage in the event of OSPF flapping.

Having said that, an active/passive failover LB pair will already announce a single anycast virtual IP anyway, so, assuming there are a similar number of anycast IPs in the end, it may not have any negative side effects.

There's also the inherent limitation noted in the second-to-last paragraph; 'It comes down to what your hardware router can handle for ECMP. I know a Juniper MX240 can handle 16 next-hops, and have heard rumors that a software update will bump this to 64, but again this is something to keep in mind'. Taking a leaf from the LB design, and using BGP to load-balance across a smaller set of haproxy instances, would seem like a good approach to scale up.
scalability  networking  performance  load-balancing  bgp  exabgp  ospf  anycast  routing  datacenters  scaling  vips  juniper  haproxy  shutterstock 
may 2014 by jm
The New Threat: Targeted Internet Traffic Misdirection
MITM attacks via BGP route hijacking now relatively commonplace on the internet, with 60 cases observed so far this year by Renesys
bgp  mitm  internet  security  routing  attacks  hijacking 
november 2013 by jm
Spamhaus victim of BGP route hijacking
Pretty major hi-jinks. Neil Schwartzman says it didn't go on for long, but still, this is crazy antics.

As can seen from the BGP output, we were using a /32 route going over AS 34109. This was highly suspicious for two reasons. First, a /32 route refers only to a single IP address. Except in special cases, routes are normally /24 (256 hosts) or larger. Second, the AS 34109 belongs to CB3ROB which is an Internet provider that has actually been in conflict with Spamhaus (see: spamhaus; allspammedup; theregister). Certainly they weren’t running a legitimate Spamhaus server. It seems clear that the CB3ROB network hijacked one (or more) of the IP addresses of Spamhaus, and installed a DNS server there which incorrectly returns positive results to every query. The result causes harm to Spamhaus users and their customers, making Spamhaus unusable for anyone unable to correct the problem as we did, and perhaps even undermining the credibility of Spamhaus itself.
spamhaus  security  bgp  peering  internet  routing  hacking  dns  dnsbls  cb3rob  as-34109 
march 2013 by jm
Timelike 2: everything fails all the time
Fantastic post on large-scale distributed load balancing strategies from @aphyr. Random and least-conns routing comes out on top in his simulation (although he hasn't yet tried Marc Brooker's two-randoms routing strategy)
via:hn  routing  distributed  least-conns  load-balancing  round-robin  distcomp  networking  scaling 
february 2013 by jm
How did I do the Starwars Traceroute?
It is accomplished using many vrfs on 2 Cisco 1841s. For those less technical, VRFs are essentially private routing tables similar to a VPN. When a packet destined to 216.81.59.173 (AKA obiwan.scrye.net) hits my main gateway, I forward it onto the first VRF on the "ASIDE" router on 206.214.254.1. That router then has a specific route for 216.81.59.173 to 206.214.254.6, which resides on a different VRF on the "BSIDE" router. It then has a similar set up which points it at 206.214.254.9 which lives in another VPN on "ASIDE" router. All packets are returned using a default route pointing at the global routing table. This was by design so the packets TTL expiration did not have to return fully through the VRF Maze. I am a consultant to Epik Networks who let me use the Reverse DNS for an unused /24, and I used PowerDNS to update all of the entries through mysql. This took about 30 minutes to figure out how to do it, and about 90 minutes to implement.
vrfs  routing  networking  hacks  star-wars  traceroute  rdns  ip 
february 2013 by jm

Copy this bookmark:



description:


tags: