jm + profiling   15

poor man's profiler
'Sampling tools like oprofile or dtrace's profile provider don't really provide methods to see what [multithreaded] programs are blocking on - only where they spend CPU time. Though there exist advanced techniques (such as systemtap and dtrace call level probes), it is overkill to build upon that. Poor man doesn't have time. Poor man needs food.'

Basically periodically grabbing stack traces from running processes using gdb.
gdb  profiling  linux  unix  mark-callaghan  stack-traces  performance 
9 weeks ago by jm
Java Flame Graphs Introduction: Fire For Everyone!
lots of good detail on flame graph usage in Java, and the Honest Profiler (honest because it's safepoint-free)
profiling  java  safepoints  jvm  flame-graphs  perf  measurement  benchmarking  testing 
september 2017 by jm
Glowroot
"Open source APM for Java" -- profiling in production, with a demo benchmark showing about a 2% performance impact. Wonder about effects on memory/GC, though
apm  java  metrics  measurement  new-relic  profiling  glowroot 
march 2015 by jm
Ever liked a film on Facebook? You’ve given the security services a key to your soul
The researchers started with 86,000 subjects who had filled out the 100-question personality profile – and this, of course, was done as another app on Facebook – and whose personality scores had been matched by algorithms with their Facebook likes. They then found 17,000 who were willing to have a friend or family member take the personality test on their behalf, trying to predict the answers they would give.

The results, from most humans, were stunningly inaccurate. Friends, family and co-workers were all less able to predict how someone would fill out a personality test than the algorithms that had been primed with the subject’s Facebook likes. With only 10 likes to work on, the computer was more accurate than a work colleague would be. With 150 likes, it described the subject’s personality better than a parent or sibling could. And with 300 likes to work on, it was more accurate than a spouse.
likes  facebook  privacy  prism  surveillance  profiling  personality 
january 2015 by jm
coz
A causal profiler for C++.
Causal profiling is a novel technique to measure optimization potential. This measurement matches developers' assumptions about profilers: that optimizing highly-ranked code will have the greatest impact on performance. Causal profiling measures optimization potential for serial, parallel, and asynchronous programs without instrumentation of special handling for library calls and concurrency primitives. Instead, a causal profiler uses performance experiments to predict the effect of optimizations. This allows the profiler to establish causality: "optimizing function X will have effect Y," exactly the measurement developers had assumed they were getting all along.


I can see this being a good technique to stochastically discover race conditions and concurrency bugs, too.
optimization  c++  performance  coding  profiling  speed  causal-profilers 
december 2014 by jm
Speeding up Rails 4.2
Reading between the lines, it looks like Rails 4 is waaay slower than 3....
rails  ruby  performance  profiling  discourse 
december 2014 by jm
Profiling Hadoop jobs with Riemann
I’ve built a very simple distributed profiler for soft-real-time telemetry from hundreds to thousands of JVMs concurrently. It’s nowhere near as comprehensive in its analysis as, say, Yourkit, but it can tell you, across a distributed system, which functions are taking the most time, and what their dominant callers are.


Potentially useful.
riemann  profiling  aphyr  hadoop  emr  performance  monitoring 
august 2014 by jm
A sampling profiler for your daily browsing - Google Groups
via Ilya Grigorik: Chrome Canary now has a built-in, always-on, zero-overhead code profiler. I want this in my server-side JVMs!
chrome  tracing  debugging  performance  profiling  google  sampling-profiler  javascript  blink  v8 
january 2014 by jm
Twitter tech talk video: "Profiling Java In Production"
In this talk Kaushik Srenevasan describes a new, low overhead, full-stack tool (based on the Linux perf profiler and infrastructure built into the Hotspot JVM) we've built at Twitter to solve the problem of dynamically profiling and tracing the behavior of applications (including managed runtimes) in production.


Looks very interesting. Haven't watched it yet though
twitter  tech-talks  video  presentations  java  jvm  profiling  testing  monitoring  service-metrics  performance  production  hotspot  perf 
december 2013 by jm
Creating Flight Recordings
lots more detail on the new "Java Mission Control" feature in Hotspot 7u40 JVMs, and how to use it to start and stop profiling in a live, production JVM from a separate "jcmd" command-line client. If the overhead is small, this could be really neat -- turn on profiling for 1 minute every hour on a single instance, and collect realtime production profile data on an automated basis for post-facto analysis if required
instrumentation  logging  profiling  java  jvm  ops 
september 2013 by jm
Low Overhead Method Profiling with Java Mission Control now enabled in the most recent HotSpot JVM release
Built into the HotSpot JVM [in JDK version 7u40] is something called the Java Flight Recorder. It records a lot of information about/from the JVM runtime, and can be thought of as similar to the Data Flight Recorders you find in modern airplanes. You normally use the Flight Recorder to find out what was happening in your JVM when something went wrong, but it is also a pretty awesome tool for production time profiling. Since Mission Control (using the default templates) normally don’t cause more than a per cent overhead, you can use it on your production server.


I'm intrigued by the idea of always-on profiling in production. This could be cool.
performance  java  measurement  profiling  jvm  jdk  hotspot  mission-control  instrumentation  telemetry  metrics 
september 2013 by jm
Reality, Reactivity, Relevance and Repeatability in Java Application Profiling
this product from JInspired appears to support runtime profiling of java apps with < 5% performance impact
profiling  performance  java  coding  measurement 
april 2013 by jm
Expensive lessons in Python performance tuning
some good advice for large-scale Python performance: prun and guppy for profiling, namedtuples for memory efficiency, and picloud for trivial EC2-based scale-out. (via Nelson)
picloud  prun  guppy  namedtuples  python  optimization  performance  tuning  profiling 
july 2012 by jm
Linux Profiling tools and techniques
great tips for system-level and app-level profiling on Linux from Padraig
profiling  optimization  linux  cache  valgrind 
april 2011 by jm
ioprofile
wraps strace(1) to summarise and aggregate I/O ops performed by a Linux process. looks pretty nifty (via Jeremy Zawodny)
via:jzawodny  io  strace  linux  monitoring  debugging  performance  profiling  sysadmin  ioprofile  unix  tools  from delicious
october 2010 by jm

Copy this bookmark:



description:


tags: