jm + async   20

'centrally-planned object and thread pools' for java.

'In the default JVM thread pools, once a thread is created it will only be retired when it hasn't performed a task in the last minute. In practice, this means that there are as many threads as the peak historical number of concurrent tasks handled by the pool, forever. These thread pools are also poorly instrumented, making it difficult to tune their latency or throughput. Dirigiste provides a fast, richly instrumented version of a java.util.concurrent.ExecutorService, and provides a means to feed that instrumentation into a control mechanism that can grow or shrink the pool as needed. Default implementations that optimize the pool size for thread utilization are provided. It also provides an object pool mechanism that uses a similar feedback mechanism to resize itself, and is significantly simpler than the Apache Commons object pool implementation.'

Great metric support, too.
async  jvm  dirigiste  java  threadpools  concurrency  utilization  capacity  executors  object-pools  object-pooling  latency 
june 2016 by jm
Netty @Apple: Large Scale Deployment/Connectivity [video]
'Norman Maurer presents how Apple uses Netty for its Java based services and the challenges of doing so, including how they enhanced performance by participating in the Netty open source community. Maurer takes a deep dive into advanced topics like JNI, JVM internals, and others.'
apple  netty  norman-maurer  java  jvm  async  talks  presentations 
january 2016 by jm
Orbit Async
Orbit Async implements async-await methods in the JVM. It allows programmers to write asynchronous code in a sequential fashion. It was developed by BioWare, a division of Electronic Arts.

Open source, BSD-licensed.
async  await  java  jvm  bioware  coding  threading 
june 2015 by jm
Netty's async DNS resolver
'Can do ~1M queries to ~3K public DNS servers within ~3 minutes with just a few threads.'

via Trustin Lee. Netty is the business
netty  dns  async  crawlers  resolver  benchmarks  scanning 
june 2015 by jm
REST Commander: Scalable Web Server Management and Monitoring
We dynamically monitor and manage a large and rapidly growing number of web servers deployed on our infrastructure and systems. However, existing tools present major challenges when making REST/SOAP calls with server-specific requests to a large number of web servers, and then performing aggregated analysis on the responses. We therefore developed REST Commander, a parallel asynchronous HTTP client as a service to monitor and manage web servers. REST Commander on a single server can send requests to thousands of servers with response aggregation in a matter of seconds. And yes, it is open-sourced at

Feature highlights:

Click-to-run with zero installation;
Generic HTTP request template supporting variable-based replacement for sending server-specific requests;
Ability to send the same request to different servers, different requests to different servers, and different requests to the same server;
Maximum concurrency control (throttling) to accommodate server capacity;
Commander itself is also “as a service”: with its powerful REST API, you can define ad-hoc target servers, an HTTP request template, variable replacement, and a regular expression all in a single call. In addition, intuitive step-by-step wizards help you achieve the same functionality through a GUI.
rest  http  clients  load-testing  ebay  soap  async  testing  monitoring 
july 2014 by jm
Little’s Law, Scalability and Fault Tolerance: The OS is your bottleneck. What you can do?
good blog post on Little's Law, plugging quasar, pulsar, and comsat, 3 new open-source libs offering Erlang-like lightweight threads on the JVM
jvm  java  quasar  pulsar  comsat  littles-law  scalability  async  erlang 
february 2014 by jm
Reactor hits GA
'It can't just be Big Data, it has to be Fast Data: Reactor 1.0 goes GA':

Reactor provides the necessary abstractions to build high-throughput, low-latency--what we now call "fast data"--applications that absolutely must work with thousands, tens of thousands, or even millions of concurrent requests per second. Modern JVM applications must be built on a solid foundation of asynchronous and reactive components that efficiently manage the execution of a very large number of tasks on a very small number of system threads. Reactor is specifically designed to help you build these kinds of applications without getting in your way or forcing you to work within an opinionated pattern.

Featuring the LMAX Disruptor ringbuffer, the JavaChronicle fast persistent message-passing queue, Groovy closures, and Netty 4.0. This looks very handy indeed....
disruptor  reactive-programming  reactor  async  libraries  java  jvm  frameworks  spring  netty  fast-data 
november 2013 by jm
NettoSphere demo
'This article will use NettoSphere, a framework build on top of the popular Netty Framework and Atmosphere with support of WebSockets, Server Side Events and Long-Polling. NettoSphere allows [async JVM framework] Atmosphere's applications to run on top of the Netty Framework.'
atmosphere  netty  async  java  scala  websockets  sse  long-polling  http  demos  games 
november 2013 by jm
Making Storm fly with Netty | Yahoo Engineering
Y! engineer doubles the speed of Storm's messaging layer by replacing the zeromq implementation with Netty
netty  async  zeromq  storm  messaging  tcp  benchmarks  yahoo  clusters 
october 2013 by jm
Log4j 2: Performance close to insane
Nice writeup on Log4j 2's new AsyncAppender implementation, based on the LMAX Disruptor. sounds pretty excellent:
“One nice little detail I should mention is that both Async Loggers and Async Appenders fix something that has always bothered me in Log4j-1.x, which is that they will flush the buffer after logging the last event in the queue . With Log4j-1.x, if you used buffered I/O, you often could not see the last few log events, as they were still stuck in the memory buffer. Your only option was setting immediateFlush to true, which forces disk I/O on every single log event and has a performance impact.
With Async Loggers and Appenders in Log4j-2.0 your log statements are all flushed to disk, so they are always visible, but this happens in a very efficient manner.”
logging  java  performance  async  disruptor  low-latency 
july 2013 by jm
Building a Modern Website for Scale (QCon NY 2013) [slides]
some great scalability ideas from LinkedIn. Particularly interesting are the best practices suggested for scaling web services:

1. store client-call timeouts and SLAs in Zookeeper for each REST endpoint;
2. isolate backend calls using async/threadpools;
3. cancel work on failures;
4. avoid sending requests to GC'ing hosts;
5. rate limits on the server.

#4 is particularly cool. They do this using a "GC scout" request before every "real" request; a cheap TCP request to a dedicated "scout" Netty port, which replies near-instantly. If it comes back with a 1-packet response within 1 millisecond, send the real request, else fail over immediately to the next host in the failover set.

There's still a potential race condition where the "GC scout" can be achieved quickly, then a GC starts just before the "real" request is issued. But the incidence of GC-blocking-request is probably massively reduced.

It also helps against packet loss on the rack or server host, since packet loss will cause the drop of one of the TCP packets, and the TCP retransmit timeout will certainly be higher than 1ms, causing the deadline to be missed. (UDP would probably work just as well, for this reason.) However, in the case of packet loss in the client's network vicinity, it will be vital to still attempt to send the request to the final host in the failover set regardless of a GC-scout failure, otherwise all requests may be skipped.

The GC-scout system also helps balance request load off heavily-loaded hosts, or hosts with poor performance for other reasons; they'll fail to achieve their 1 msec deadline and the request will be shunted off elsewhere.

For service APIs with real low-latency requirements, this is a great idea.
gc-scout  gc  java  scaling  scalability  linkedin  qcon  async  threadpools  rest  slas  timeouts  networking  distcomp  netty  tcp  udp  failover  fault-tolerance  packet-loss 
june 2013 by jm
Log4j2 Asynchronous Loggers for Low-Latency Logging - Apache Log4j 2
implemented using the LMAX Disruptor library -- very impressive performance figures. I presume in real-world usage, these latencies are dwarfed by hardware costs, though
disruptor  coding  java  log4j  logging  async  performance 
april 2013 by jm
Functional Reactive Programming in the Netflix API with RxJava
Hmm, this seems nifty as a compositional building block for Java code to enable concurrency without thread-safety and sync problems.
Functional reactive programming offers efficient execution and composition by providing a collection of operators capable of filtering, selecting, transforming, combining and composing Observable's.

The Observable data type can be thought of as a "push" equivalent to Iterable which is "pull". With an Iterable, the consumer pulls values from the producer and the thread blocks until those values arrive. By contrast with the Observable type, the producer pushes values to the consumer whenever values are available. This approach is more flexible, because values can arrive synchronously or asynchronously.
concurrency  java  jvm  threads  thread-safety  coding  rx  frp  fp  functional-programming  reactive  functional  async  observable 
april 2013 by jm
High performance network programming on the JVM, OSCON 2012
by Erik Onnen of Urban Airship. very good presentation on the current state of the art in large-scale low-latency service operation using the JVM on Linux. Lots of good details on async vs sync, HTTPS/TLS/TCP tuning, etc.
http  https  scaling  jvm  async  sync  oscon  presentations  tcp 
july 2012 by jm
Bulletproof Node.js Coding
lots of patterns to write safe node.js code.  Pretty daunting, to be honest
javascript  node.js  coding  programming  async  from delicious
march 2011 by jm
Thousands of Threads and Blocking I/O [PDF]
classic presentation from Paul Tyma of Mailinator regarding the java.nio (event-driven, non-threaded) vs (threaded) model of server concurrency, backing up the scalability of threads on modern JVMs
java  async  io  jvm  linux  performance  scalability  threading  threads  server  nio  paul-tyma  mailinator  from delicious
july 2010 by jm
How do we kick our synchronous addiction?
great post on the hazards of programming in an async framework, and how damn hard it is. good comments thread too (via jzawodny)
via:jzawodny  coding  python  javascript  scalability  ruby  concurrency  erlang  async  node.js  twisted  from delicious
february 2010 by jm
The technology behind Tornado, FriendFeed's web server
more on the new async HTTP server from FriendFeed/Facebook, in Python. looks lovely
async  http  epoll  python  comet  long-poll  facebook  scaling  scalability  web  friendfeed  tornado  opensource  from delicious
september 2009 by jm

related tags

apple  async  atmosphere  atom  await  benchmarks  bioware  c++  c++11  callbacks  capacity  clients  clusters  coding  comet  comsat  concurrency  crawlers  demos  dirigiste  disruptor  distcomp  dns  ebay  epoll  erlang  executors  facebook  failover  fast-data  fault-tolerance  feeds  fp  frameworks  friendfeed  frp  functional  functional-programming  futures  games  gc  gc-scout  http  http-push  https  io  java  javascript  jvm  latency  libraries  linkedin  linux  littles-law  load-testing  log4j  logging  long-poll  long-polling  low-latency  mailinator  messaging  monitoring  netty  networking  nio  node.js  norman-maurer  object-pooling  object-pools  observable  opensource  oscon  packet-loss  paul-tyma  performance  ping  presentations  programming  pubsub  pubsubhubbub  pulsar  python  qcon  quasar  reactive  reactive-programming  reactor  realtime  resolver  rest  ruby  rx  scala  scalability  scaling  scanning  server  slas  soap  spring  sse  standards  storm  sync  talks  tcp  testing  thread-safety  threading  threadpools  threads  timeouts  tornado  twisted  udp  utilization  via:jzawodny  web  websockets  yahoo  zeromq 

Copy this bookmark: