Functional Reactive Programming in the Netflix API with RxJava
7 weeks ago by jm
Hmm, this seems nifty as a compositional building block for Java code to enable concurrency without thread-safety and sync problems.
concurrency
java
jvm
threads
thread-safety
coding
rx
frp
fp
functional-programming
reactive
functional
async
observable
Functional reactive programming offers efficient execution and composition by providing a collection of operators capable of filtering, selecting, transforming, combining and composing Observable's.
The Observable data type can be thought of as a "push" equivalent to Iterable which is "pull". With an Iterable, the consumer pulls values from the producer and the thread blocks until those values arrive. By contrast with the Observable type, the producer pushes values to the consumer whenever values are available. This approach is more flexible, because values can arrive synchronously or asynchronously.
7 weeks ago by jm
The useful JVM options
8 weeks ago by jm
a good reference, with lots of sample output. Not clear if it takes 1.6/1.7 differences into account, though
jvm
reference
java
ops
hotspot
command-line
8 weeks ago by jm
dumping a JVM heap using gdb
12 weeks ago by jm
now this is a neat trick -- having been stuck having to flip to spares and do other antics while a long-running heap dump took place, this is a winner.
heap-dump
gdb
heap
jvm
java
via:peakscale
gcore
core
core-dump
debugging
Dumping a JVM’s heap is an extremely useful tool for debugging problems with a J2EE application. Unfortunately, when a JVM explodes, using the standard jmap tool can take an inordinate amount of time to execute for lots of different reasons. This leads to extended downtime when a heap dump is attempted and even then, jmap regularly fails.
This blog post is intended to outline an alternate method using [gdb] to achieve a heap dump that only requires mere seconds of additional downtime allowing the slow jmap process to happen once the application is back in service.
12 weeks ago by jm
Single Producer/Consumer lock free Queue step by step
march 2013 by jm
great dissection of Martin "Disruptor" Thompson's lock-free single-producer/single-consumer queue data structure, with benchmark results showing crazy speedups. This is particularly useful since it's a data structure that can be used to provide good lock-free speedups without adopting the entire Disruptor design pattern.
disruptor
coding
java
jvm
martin-thompson
lock-free
volatile
atomic
queue
data-structures
march 2013 by jm
From a monolithic Ruby on Rails app to the JVM
march 2013 by jm
How Soundcloud have ditched the monolithic Rails for nimbler, small-scale distributed polyglot services running on the JVM
soundcloud
rails
slides
jvm
scalability
ruby
scala
clojure
coding
march 2013 by jm
Are volatile reads really free?
february 2013 by jm
Marc Brooker with some good test data:
volatile
concurrency
jvm
performance
java
marc-brooker
It appears as though reads to volatile variables are not free in Java on x86, or at least on the tested setup. It's true that the difference isn't so huge (especially for the read-only case) that it'll make a difference in any but the more performance sensitive case, but that's a different statement from free.
february 2013 by jm
Extreme Performance with Java - Charlie Hunt [slides, PDF]
january 2013 by jm
presentation slides for Charlie Hunt's 2012 QCon presentation, where he discusses 'what you need to know about a modern JVM in order
to be effective at writing a low latency Java application'. The talk video is at http://www.infoq.com/presentations/Extreme-Performance-Java
low-latency
charlie-hunt
performance
java
jvm
presentations
qcon
slides
pdf
to be effective at writing a low latency Java application'. The talk video is at http://www.infoq.com/presentations/Extreme-Performance-Java
january 2013 by jm
Cliff Click in "A JVM Does What?"
december 2012 by jm
interesting YouTubed presentation from Azul's Cliff Click on some java/JVM innards
presentation
concurrency
jvm
video
java
youtube
cliff-click
december 2012 by jm
low-gc-membuffers
december 2012 by jm
"This project aims at creating a simple efficient building block for "Big Data" libraries, applications and frameworks; thing that can be used as an in-memory, bounded queue with opaque values (sequence of JDK primitive values): insertions at tail, removal from head, single entry peeks), and that has minimal garbage collection overhead. Insertions and removals are as individual entries, which are sub-sequences of the full buffer.
GC overhead minimization is achieved by use of direct ByteBuffers (memory allocated outside of GC-prone heap); and bounded nature by only supporting storage of simple primitive value (byte, `long') sequences where size is explicitly known.
Conceptually memory buffers are just simple circular buffers (ring buffers) that hold a sequence of primitive values, bit like arrays, but in a way that allows dynamic automatic resizings of the underlying storage. Library supports efficient reusing and sharing of underlying segments for sets of buffers, although for many use cases a single buffer suffices."
gc
java
jvm
bytebuffer
GC overhead minimization is achieved by use of direct ByteBuffers (memory allocated outside of GC-prone heap); and bounded nature by only supporting storage of simple primitive value (byte, `long') sequences where size is explicitly known.
Conceptually memory buffers are just simple circular buffers (ring buffers) that hold a sequence of primitive values, bit like arrays, but in a way that allows dynamic automatic resizings of the underlying storage. Library supports efficient reusing and sharing of underlying segments for sets of buffers, although for many use cases a single buffer suffices."
december 2012 by jm
Everything I Ever Learned About JVM Performance Tuning @Twitter
december 2012 by jm
presentation by Attila Szegedi of Twitter from last year. Some good tips here, well-presented
tuning
jvm
java
gc
cms
presentations
slides
twitter
december 2012 by jm
_The Pauseless GC Algorithm_ [pdf]
december 2012 by jm
Paper from USENIX VEE '05, by Cliff Click, Gil Tene, and Michael Wolf of Azul Systems, describing some details of the Azul secret sauce (via b6n)
via:b3n
azul
gc
jvm
java
usenix
papers
december 2012 by jm
Java tip: How to get CPU, system, and user time for benchmarking
november 2012 by jm
a neat MXBean trick to get per-thread CPU usage in a running JVM (via Tatu Saloranta)
java
jvm
monitoring
cpu
metrics
threads
november 2012 by jm
Cliff Click's 2008 JavaOne talk about the NonBlockingHashTable
october 2012 by jm
I'm a bit late to this data structure -- highly scalable, nearly lock-free, benchmarks very well (except with the G1 GC): http://edwwang.com/blog/2012/02/10/concurrent-hashmap-benchmark/ .
Having said that, it doesn't cope well with frequently-changing unique keys: http://sourceforge.net/tracker/?func=detail&aid=3563980&group_id=194172&atid=948362 .
More background at: http://www.azulsystems.com/blog/cliff/2007-03-26-non-blocking-hashtable and http://www.azulsystems.com/blog/cliff/2007-04-01-non-blocking-hashtable-part-2
This was used in Cassandra for a while, although I think the above bug may have caused its removal?
nonblockinghashtable
data-structures
hashmap
concurrency
scaling
java
jvm
Having said that, it doesn't cope well with frequently-changing unique keys: http://sourceforge.net/tracker/?func=detail&aid=3563980&group_id=194172&atid=948362 .
More background at: http://www.azulsystems.com/blog/cliff/2007-03-26-non-blocking-hashtable and http://www.azulsystems.com/blog/cliff/2007-04-01-non-blocking-hashtable-part-2
This was used in Cassandra for a while, although I think the above bug may have caused its removal?
october 2012 by jm
SnapTree benchmarks
september 2012 by jm
nice concurrent Map data structure for the JVM; beats out ConcurrentHashMap, ConcurrentLinkedHashMap from guava, ConcurrentSkipListMap under both CMS and G1 garbage collectors.
concurrency
benchmarks
hashmap
map
data-structures
java
jvm
snaptree
september 2012 by jm
Locks & Condition Variables - Latency Impact
locking
concurrency
java
jvm
signalling
locks
linux
threading
september 2012 by jm
Firstly, this is 3 orders of magnitude greater latency than what I illustrated in the previous article using just memory barriers to signal between threads. This cost comes about because the kernel needs to get involved to arbitrate between the threads for the lock, and then manage the scheduling for the threads to awaken when the condition is signalled. The one-way latency to signal a change is pretty much the same as what is considered current state of the art for network hops between nodes via a switch. It is possible to get ~1µs latency with InfiniBand and less than 5µs with 10GigE and user-space IP stacks.
Secondly, the impact is clear when letting the OS choose what CPUs the threads get scheduled on rather than pinning them manually. I've observed this same issue across many use cases whereby Linux, in default configuration for its scheduler, will greatly impact the performance of a low-latency system by scheduling threads on different cores resulting in cache pollution. Windows by default seems to make a better job of this.
</blockqote>
september 2012 by jm
Martin "Disruptor" Thompson's Single Writer Principle
september 2012 by jm
Contains these millisecond estimates for highly-contended inter-thread signalling when incrementing a 64-bit counter in java:
Undoubtedly not realistic for a lot of cases, but it's still useful for order-of-magnitude estimates of locking cost. Bottom line: don't lock if you can avoid it, even with 'volatile' or AtomicFoo types.
java
jvm
performance
coding
concurrency
threading
cas
locking
One Thread 300<br>
One Thread with Memory Barrier 4,700<br>
One Thread with CAS 5,700<br>
Two Threads with CAS 18,000<br>
One Thread with Lock 10,000<br>
Two Threads with Lock 118,000<br>
Undoubtedly not realistic for a lot of cases, but it's still useful for order-of-magnitude estimates of locking cost. Bottom line: don't lock if you can avoid it, even with 'volatile' or AtomicFoo types.
september 2012 by jm
HotSpot JVM garbage collection options cheat sheet (v2)
september 2012 by jm
'In this article I have collected a list of options related to GC tuning in JVM. This is not a comprehensive list, I have only collected options which I use in practice (or at least understand why I may want to use them).
Compared to previous version a few useful diagnostic options was added. Additionally section for G1 specific options was introduced.'
hotspot
jvm
coding
gc
java
performance
Compared to previous version a few useful diagnostic options was added. Additionally section for G1 specific options was introduced.'
september 2012 by jm
High performance network programming on the JVM, OSCON 2012
july 2012 by jm
by Erik Onnen of Urban Airship. very good presentation on the current state of the art in large-scale low-latency service operation using the JVM on Linux. Lots of good details on async vs sync, HTTPS/TLS/TCP tuning, etc.
http
https
scaling
jvm
async
sync
oscon
presentations
tcp
july 2012 by jm
Chronon DVR for Java
may 2012 by jm
"record entire execution of your Java app; play it back on any machine". Other features: time-travelling debugger -- step backwards, jump to any point in execution, designed for long running programs; post-execution logging -- add log statements after the program has run, and see what it would have logged. Looks extremely nifty, but I wonder how big those recording files get...
debugging
via:peakscale
eclipse
chronon
dvr
java
coding
logging
jvm
may 2012 by jm
Secret HotSpot option improving GC pauses on large heaps
april 2012 by jm
via Toby DiPasquale. nice tip
java
jvm
gc
performance
hotspot
undocumented
april 2012 by jm
twitter/jvmgcprof - GitHub
february 2012 by jm
'gcprof is a simple utility for profile allocation and garbage collection activity in the JVM [...] Profile allocation and garbage collection activity in the JVM. The gcprof command runs a java command under profiling. Allocation and collection statistics are printed periodically. If -n or -no are provided, statistics are also reported in terms of the given application metric. Total allocation, allocation rate, and a survival histogram is given. The intended use for this tool is twofold: (1) monitor and test garbage allocation and GC behavior, and (2) inform GC tuning.'
gc
java
performance
twitter
jvm
tools
february 2012 by jm
Scott Andreas - Garbage, Garbage Everywhere [slides]
december 2011 by jm
'GC Strategies for Event Processing Systems on the JVM'
gc
java
jvm
event-streams
event-processing
tuning
slides
presentations
scott-andreas
performance
december 2011 by jm
Avoiding Full GCs in HBase with MemStore-Local Allocation Buffers
october 2011 by jm
Fascinating. Evading the Java GC by reimplementing a slab allocator, basically
memory
allocation
java
gc
jvm
hbase
memstore
via:dehora
slab-allocator
october 2011 by jm
peak6/scala-ssh-shell - GitHub
october 2011 by jm
'Backdoor that gives you a scala shell over ssh on your jvm. The shell is not sandboxed, anyone access the shell can touch anything in the jvm and do anything the jvm can do including modifying and deleting files, etc.' nifty!
scala
ssh
repl
interactive
debugging
coding
jvm
java
october 2011 by jm
Scala: The Static Language that Feels Dynamic
june 2011 by jm
a good intro from Bruce Eckel. We need a good excuse to deploy some Scala ;)
scala
actors
java
language
programming
jvm
coding
june 2011 by jm
Thousands of Threads and Blocking I/O [PDF]
july 2010 by jm
classic presentation from Paul Tyma of Mailinator regarding the java.nio (event-driven, non-threaded) vs java.io (threaded) model of server concurrency, backing up the scalability of threads on modern JVMs
java
async
io
jvm
linux
performance
scalability
threading
threads
server
nio
paul-tyma
mailinator
from delicious
july 2010 by jm
Why WeakHashMap Sucks
september 2009 by jm
'SoftReferences are the cheap, crappy caching mechanism [...] perfect for when you'd like your cache to be cleared at random times and in random order.'
softreferences
weakreferences
weak
references
gc
java
jvm
caching
hash
memory
collections
vm
weakhashmap
via:spyced
from delicious
september 2009 by jm
related tags
actors ⊕ allocation ⊕ async ⊕ atomic ⊕ azul ⊕ benchmarks ⊕ bytebuffer ⊕ caching ⊕ cas ⊕ charlie-hunt ⊕ chronon ⊕ cliff-click ⊕ clojure ⊕ cms ⊕ coding ⊕ collections ⊕ command-line ⊕ concurrency ⊕ core ⊕ core-dump ⊕ cpu ⊕ data-structures ⊕ debugging ⊕ disruptor ⊕ dvr ⊕ eclipse ⊕ event-processing ⊕ event-streams ⊕ fp ⊕ frp ⊕ functional ⊕ functional-programming ⊕ gc ⊕ gcore ⊕ gdb ⊕ hash ⊕ hashmap ⊕ hbase ⊕ heap ⊕ heap-dump ⊕ hotspot ⊕ http ⊕ https ⊕ interactive ⊕ io ⊕ java ⊕ jvm ⊖ language ⊕ linux ⊕ lock-free ⊕ locking ⊕ locks ⊕ logging ⊕ low-latency ⊕ mailinator ⊕ map ⊕ marc-brooker ⊕ martin-thompson ⊕ memory ⊕ memstore ⊕ metrics ⊕ monitoring ⊕ nio ⊕ nonblockinghashtable ⊕ observable ⊕ ops ⊕ oscon ⊕ papers ⊕ paul-tyma ⊕ pdf ⊕ performance ⊕ presentation ⊕ presentations ⊕ programming ⊕ qcon ⊕ queue ⊕ rails ⊕ reactive ⊕ reference ⊕ references ⊕ repl ⊕ ruby ⊕ rx ⊕ scala ⊕ scalability ⊕ scaling ⊕ scott-andreas ⊕ server ⊕ signalling ⊕ slab-allocator ⊕ slides ⊕ snaptree ⊕ softreferences ⊕ soundcloud ⊕ ssh ⊕ sync ⊕ tcp ⊕ thread-safety ⊕ threading ⊕ threads ⊕ tools ⊕ tuning ⊕ twitter ⊕ undocumented ⊕ usenix ⊕ via:b3n ⊕ via:dehora ⊕ via:peakscale ⊕ via:spyced ⊕ video ⊕ vm ⊕ volatile ⊕ weak ⊕ weakhashmap ⊕ weakreferences ⊕ youtube ⊕Copy this bookmark: