jm + netty   22

Neutrino Software Load Balancer
eBay's software LB, supporting URL matching, comparable to haproxy, built using Netty and Scala. Used in their QA infrastructure it seems
netty  scala  ebay  load-balancing  load-balancers  url  http  architecture 
february 2016 by jm
Netty @Apple: Large Scale Deployment/Connectivity [video]
'Norman Maurer presents how Apple uses Netty for its Java based services and the challenges of doing so, including how they enhanced performance by participating in the Netty open source community. Maurer takes a deep dive into advanced topics like JNI, JVM internals, and others.'
apple  netty  norman-maurer  java  jvm  async  talks  presentations 
january 2016 by jm
Introducing Netty-HTTP from Cask
netty-http library solves [Netty usability issues] by using JAX-RS annotations to build a HTTP path routing layer on top of netty. In addition, the library implements a guava service to manage the HTTP service. netty-http allows users of the library to just focus on writing the business logic in HTTP handlers without having to worry about the complexities of path routing or learning netty pipeline internals to build the HTTP service.

We've written something very similar, although I didn't even bother supporting JAX-RS annotations -- just a simple code-level DSL.
jax-rs  netty  http  cask  java  services  coding 
december 2015 by jm
How Netty is used at Layer
pretty conventional HTTP/1.1, WebSockets and HTTP/2 front-end services with modern Netty practices
netty  http  api-services  coding  java  servers 
october 2015 by jm
Netty's async DNS resolver
'Can do ~1M queries to ~3K public DNS servers within ~3 minutes with just a few threads.'

via Trustin Lee. Netty is the business
netty  dns  async  crawlers  resolver  benchmarks  scanning 
june 2015 by jm
an asynchronous Netty based graphite proxy. It protects Graphite from the herds of clients by minimizing context switches and interrupts; by batching and aggregating metrics. Gruffalo also allows you to replicate metrics between Graphite installations for DR scenarios, for example.

Gruffalo can easily handle a massive amount of traffic, and thus increase your metrics delivery system availability. At Outbrain, we currently handle over 1700 concurrent connections, and over 2M metrics per minute per instance.
graphite  backpressure  metrics  outbrain  netty  proxies  gruffalo  ops 
april 2015 by jm
Binary message marshalling, client/server stubs generated by an IDL compiler, bidirectional binary protocol. CORBA is back from the dead!
Intro blog post:

Relevant: Steve Vinoski's commentary on protobuf-rpc back in 2008:
http  rpc  http2  netty  grpc  google  corba  idl  messaging 
february 2015 by jm
Spark 1.2 released
This is the version with the superfast petabyte-sort record:
Spark 1.2 includes several cross-cutting optimizations focused on performance for large scale workloads. Two new features Databricks developed for our world record petabyte sort with Spark are turned on by default in Spark 1.2. The first is a re-architected network transfer subsystem that exploits Netty 4’s zero-copy IO and off heap buffer management. The second is Spark’s sort based shuffle implementation, which we’ve now made the default after significant testing in Spark 1.1. Together, we’ve seen these features give as much as 5X performance improvement for workloads with very large shuffles.
spark  sorting  hadoop  map-reduce  batch  databricks  apache  netty 
december 2014 by jm
Netty: Using as a generic library
Some cool stuff that comes along with Netty: an improved ByteBuffer, a thread-local object pool, a hashed-wheel Timer, and some nice mechanical-sympathy utils.
mechanical-sympathy  netty  java  bytebuffer  object-pools  data-structures  hashed-wheel-timer  algorithms  timers 
november 2014 by jm
This Java library can route paths to targets and create paths from targets and params (reverse routing). This library is tiny, without additional dependencies, and is intended for use together with an HTTP server side library. If you want to use with Netty, see netty-router.
java  jauter  scala  request-routing  http  netty  open-source 
october 2014 by jm
Spark Breaks Previous Large-Scale Sort Record – Databricks
Massive improvement over plain old Hadoop. This blog post goes into really solid techie reasons why, including:
First and foremost, in Spark 1.1 we introduced a new shuffle implementation called sort-based shuffle (SPARK-2045). The previous Spark shuffle implementation was hash-based that required maintaining P (the number of reduce partitions) concurrent buffers in memory. In sort-based shuffle, at any given point only a single buffer is required. This has led to substantial memory overhead reduction during shuffle and can support workloads with hundreds of thousands of tasks in a single stage (our PB sort used 250,000 tasks).

Also, use of Timsort, an external shuffle service to offload from the JVM, Netty, and EC2 SR-IOV.
spark  hadoop  map-reduce  batch  parallel  sr-iov  benchmarks  performance  netty  shuffle  algorithms  sort-based-shuffle  timsort 
october 2014 by jm
Real time analytics with Netty, Storm, Kafka
Arch of a fairly typical Kafka/Storm realtime ad-tracking setup, from eClick/mc2ads, via Trustin Lee
via:trustinlee  kafka  storm  netty  architecture  ad-tracking  ads  realtime 
august 2014 by jm
Moquette MQTT
a Java implementation of an MQTT 3.1 broker. Its code base is small. At its core, Moquette is an events processor; this lets the code base be simple, avoiding thread sharing issues. The Moquette broker is lightweight and easy to understand so it could be embedded in other projects.
mqtt  moquette  netty  messaging  queueing  push-notifications  iot  internet  push  eclipse 
may 2014 by jm
"A New Data Structure For Cumulative Frequency Tables"
paper by Peter M Fenwick, 1993. 'A new method (the ‘binary indexed tree’) is presented for maintaining the cumulative frequencies which are needed to support dynamic arithmetic data compression. It is based on a decomposition of the cumulative frequencies into portions which parallel the binary representation of the index of the table element (or symbol). The operations to traverse the data structure are based on the binary coding of the index. In comparison with previous methods, the binary indexed tree is faster, using more compact data and simpler code. The access time for all operations is either constant or proportional to the logarithm of the table size. In conjunction with the compact data structure, this makes the new method particularly suitable for large symbol alphabets.'

via Jakob Buchgraber, who's implementing it right now in Netty ;)
netty  frequency-tables  data-structures  algorithms  coding  binary-tree  indexing  compression  symbol-alphabets 
may 2014 by jm
Google's Open Bidder stack moving from Jetty to Netty
Open Bidder traditionally used Jetty as an embedded webserver, for the critical tasks of accepting connections, processing HTTP requests, managing service threads, etc. Jetty is a robust, but traditional stack that carries the weight and tradeoffs of Servlet’s 15 years old design. For a maximum performance RTB agent that must combine very large request concurrency with very low latencies, and often benefit also from low-level control over the transport, memory management and other issue, a different webserver stack was required. Open Bidder now supports Netty, an asynchronous, event-driven, high-performance webserver stack.

For existing code, the most important impact is that Netty is not compatible with the Servlet API. Its own internal APIs are often too low-level, not to mention proprietary to Netty; so Open Bidder v0.5 introduces some new, stack-neutral APIs for things like HTTP requests and responses, cookies, request handlers, and even simple HTML templating based on Mustache. These APIs will work with both Netty and Jetty. This means you don’t need to change any code to switch between Jetty and Netty; on the other hand, it also means that existing code written for Open Bidder 0.4 may need some changes even if you plan to keep using Jetty.

[....] Netty's superior efficiency is very significant; it supports 50% more traffic in the same hardware, and it maintains a perfect latency distribution even at the peak of its supported load.

This doc is noteworthy on a couple of grounds:

1. the use of Netty in a public API/library, and the additional layer in place to add a friendlier API on top of that. I hope they might consider releasing that part as OSS at some point.

2. I also find it interesting that their API uses protobufs to marshal the message, and they plan in a future release to serialize those to JSON documents -- that makes a lot of sense.
apis  google  protobufs  json  documents  interoperability  netty  jetty  servlets  performance  java 
april 2014 by jm
Netty Best Practices
'a.k.a. Faster == Better'. Slides from a talk given at Facebook on 19th March 2014 by Norman Maurer
netty  java  performance  optimization  facebook  slides  presentations 
april 2014 by jm
[#1259] Add optimized queue for SCMP pattern and use it in NIO and nativ... · 6efac61 · netty/netty
Interesting -- Netty has imported an optimized ASL2-licensed MPSC queue implementation from Akka (presumably for performance raisins)
performance  optimization  open-source  mpsc  queues  data-structures  netty  akka  java 
march 2014 by jm
Reactor hits GA
'It can't just be Big Data, it has to be Fast Data: Reactor 1.0 goes GA':

Reactor provides the necessary abstractions to build high-throughput, low-latency--what we now call "fast data"--applications that absolutely must work with thousands, tens of thousands, or even millions of concurrent requests per second. Modern JVM applications must be built on a solid foundation of asynchronous and reactive components that efficiently manage the execution of a very large number of tasks on a very small number of system threads. Reactor is specifically designed to help you build these kinds of applications without getting in your way or forcing you to work within an opinionated pattern.

Featuring the LMAX Disruptor ringbuffer, the JavaChronicle fast persistent message-passing queue, Groovy closures, and Netty 4.0. This looks very handy indeed....
disruptor  reactive-programming  reactor  async  libraries  java  jvm  frameworks  spring  netty  fast-data 
november 2013 by jm
NettoSphere demo
'This article will use NettoSphere, a framework build on top of the popular Netty Framework and Atmosphere with support of WebSockets, Server Side Events and Long-Polling. NettoSphere allows [async JVM framework] Atmosphere's applications to run on top of the Netty Framework.'
atmosphere  netty  async  java  scala  websockets  sse  long-polling  http  demos  games 
november 2013 by jm
Making Storm fly with Netty | Yahoo Engineering
Y! engineer doubles the speed of Storm's messaging layer by replacing the zeromq implementation with Netty
netty  async  zeromq  storm  messaging  tcp  benchmarks  yahoo  clusters 
october 2013 by jm
Building a Modern Website for Scale (QCon NY 2013) [slides]
some great scalability ideas from LinkedIn. Particularly interesting are the best practices suggested for scaling web services:

1. store client-call timeouts and SLAs in Zookeeper for each REST endpoint;
2. isolate backend calls using async/threadpools;
3. cancel work on failures;
4. avoid sending requests to GC'ing hosts;
5. rate limits on the server.

#4 is particularly cool. They do this using a "GC scout" request before every "real" request; a cheap TCP request to a dedicated "scout" Netty port, which replies near-instantly. If it comes back with a 1-packet response within 1 millisecond, send the real request, else fail over immediately to the next host in the failover set.

There's still a potential race condition where the "GC scout" can be achieved quickly, then a GC starts just before the "real" request is issued. But the incidence of GC-blocking-request is probably massively reduced.

It also helps against packet loss on the rack or server host, since packet loss will cause the drop of one of the TCP packets, and the TCP retransmit timeout will certainly be higher than 1ms, causing the deadline to be missed. (UDP would probably work just as well, for this reason.) However, in the case of packet loss in the client's network vicinity, it will be vital to still attempt to send the request to the final host in the failover set regardless of a GC-scout failure, otherwise all requests may be skipped.

The GC-scout system also helps balance request load off heavily-loaded hosts, or hosts with poor performance for other reasons; they'll fail to achieve their 1 msec deadline and the request will be shunted off elsewhere.

For service APIs with real low-latency requirements, this is a great idea.
gc-scout  gc  java  scaling  scalability  linkedin  qcon  async  threadpools  rest  slas  timeouts  networking  distcomp  netty  tcp  udp  failover  fault-tolerance  packet-loss 
june 2013 by jm
C500k in Action at Urban Airship
I missed this back in 2010; 500k active TCP connections to a single EC2 large instance using Java and NIO
c10k  java  linux  ec2  scaling  nio  netty  urban-airship 
july 2012 by jm

related tags

ad-tracking  ads  akka  algorithms  apache  api-services  apis  apple  architecture  async  atmosphere  backpressure  batch  benchmarks  binary-tree  bytebuffer  c10k  cask  clusters  coding  compression  corba  crawlers  data-structures  databricks  demos  disruptor  distcomp  dns  documents  ebay  ec2  eclipse  facebook  failover  fast-data  fault-tolerance  frameworks  frequency-tables  games  gc  gc-scout  google  graphite  grpc  gruffalo  hadoop  hashed-wheel-timer  http  http2  idl  indexing  internet  interoperability  iot  jauter  java  jax-rs  jetty  json  jvm  kafka  libraries  linkedin  linux  load-balancers  load-balancing  long-polling  map-reduce  mechanical-sympathy  messaging  metrics  moquette  mpsc  mqtt  netty  networking  nio  norman-maurer  object-pools  open-source  ops  optimization  outbrain  packet-loss  parallel  performance  presentations  protobufs  proxies  push  push-notifications  qcon  queueing  queues  reactive-programming  reactor  realtime  request-routing  resolver  rest  rpc  scala  scalability  scaling  scanning  servers  services  servlets  shuffle  slas  slides  sort-based-shuffle  sorting  spark  spring  sr-iov  sse  storm  symbol-alphabets  talks  tcp  threadpools  timeouts  timers  timsort  udp  urban-airship  url  via:trustinlee  websockets  yahoo  zeromq 

Copy this bookmark: