mpm + linux   273

mstat
fine-grained, cgroup-based tool for profiling memory usage over time of a process tree
linux  memory 
8 weeks ago by mpm
Performance Tuning Handbook
This handbook is a guide to achieving consistent and reliable low-latency operation of applications. It explores how modern operating systems schedule user applications and how to observe and modify application operating parameters; and introduces techniques for making sure that system hardware is working cooperatively with your application
linux  performance 
9 weeks ago by mpm
linux-network-performance-parameters
This brief tutorial shows where some of the most used and quoted sysctl/network parameters are located into the Linux network flow
linux  networking  performance 
april 2019 by mpm
The 5-year journey to bring restartable sequences to Linux
With this design, the optimistic case (where the thread isn't interrupted) is extremely fast because expensive locks and atomic instructions can be entirely avoided
linux  concurrency 
february 2019 by mpm
Evolution of the x86 context switch in Linux
Trace the context switch through the Linux kernel from the earliest (0.01) to the most recent LTS release (4.14.67) -- with special emphasis on the first and last versions
linux  concurrency 
february 2019 by mpm
io_submit: The epoll alternative you've never heard about
Disk files are not the same thing as network sockets! Is it even possible to use the Linux AIO API with network sockets in the first place? The answer turns out to be a strong YES
linux  networking 
january 2019 by mpm
MSG_ZEROCOPY
The MSG_ZEROCOPY flag enables copy avoidance for socket send calls. The feature is currently implemented for TCP sockets.
linux  networking  performance  memory 
january 2019 by mpm
Optimizing UDP for content delivery: GSO, pacing and zerocopy
UDP is a popular basis for the experimentation with and rapid deployment of new protocols. Experience shows a large gap in cycle efficiency compared to in-kernel TCP. This talk presents recent optimizations to the UDP stack that narrow this gap. At the core are optimizations long available to TCP: segmentation offload, pacing and zerocopy. We present the recently merged UDP GSO and SO_TXTIME interfaces and discuss select implementation details. We also review partial GSO as it fits in this context and discuss optimizations that are in submission: UDP zerocopy transmit with MSG_ZEROCOPY and UDP GRO.
linux  udp  networking  performance 
january 2019 by mpm
The eXpress Data Path: Fast Programmable Packet Processing in the Operating System Kernel
In XDP, the operating system kernel itself provides a safe execution environment for custom packet processing applications, executed in device driver context.
ebpf  linux  networking  performance 
january 2019 by mpm
A month of /proc
for each day, for 30 days, I’ll be posting a new article about a single file under /proc that should be useful for someone either introspecting a system (or wanting to mutate some kernel param).
linux 
january 2019 by mpm
PSI
Gain insight into resource utilization with new Linux kernel pressure metrics and related tools
monitoring  linux 
november 2018 by mpm
bpftrace
BPFtrace is a high-level tracing language for Linux enhanced Berkeley Packet Filter (eBPF) available in recent Linux kernels
ebpf  linux  observability 
october 2018 by mpm
Toward non-blocking asynchronous I/O
The Linux asynchronous I/O (AIO) layer tends to have many critics and few defenders, but most people at least expect it to actually be asynchronous. In truth, an AIO operation can block in the kernel for a number of reasons, making AIO difficult to use in situations where the calling thread truly cannot afford to block.
linux  io  concurrency 
october 2018 by mpm
The Battle of the Schedulers: FreeBSD ULE vs. Linux CFS
This paper analyzes the impact on application performance of the design and implementation choices made in two widely used open-source schedulers: ULE, the default FreeBSD scheduler, and CFS, the default Linux scheduler.
scheduling  concurrency  linux  bsd 
october 2018 by mpm
Introducing ebpf_exporter
In this blog post we'll talk about how we measure low level metrics and share a tool that can help you to get similar understanding of your systems
linux  ebpf  observability 
october 2018 by mpm
pure-bash-bible
A collection of pure bash alternatives to external processes.
bash  shell  linux 
august 2018 by mpm
Open sourcing oomd, a new approach to handling OOMs
We have developed oomd, a faster, more reliable solution to common out-of-memory (OOM) situations, which works in userspace rather than kernelspace. We designed oomd with two key features: pre-OOM hooks and a custom plugin system. Pre-OOM hooks offer visibility into an OOM before the workload is threatened. The plugin system allows us to specify custom policies that can handle each workload running on a host.
linux  memory 
july 2018 by mpm
Open-sourcing Katran, a scalable network load balancer
Katran offers a software-based solution to load balancing with a completely reengineered forwarding plane that takes advantage of two recent innovations in kernel engineering: eXpress Data Path (XDP) and the eBPF virtual machine.
networking  performance  ebpf  linux 
june 2018 by mpm
ljsyscall
An FFI implementation of the Linux, NetBSD, FreeBSD and OSX kernel ABIs for LuaJIT. This means you will be able to program all the functionality the Unix kernel provides to userspace directly in Lua
lua  linux  bsd 
may 2018 by mpm
gvisor
gVisor is a user-space kernel, written in Go, that implements a substantial portion of the Linux system surface
linux  go  containers 
may 2018 by mpm
Optimizing Linux Memory Management for Low-latency / High-throughput Databases
Details of how the Linux kernel manages virtual memory on NUMA (Non Uniform Memory Access) systems. In a nutshell, certain Linux optimizations for NUMA had severe side effects which directly impacted our latencies
memory  linux 
april 2018 by mpm
linuxkit
LinuxKit, a toolkit for building custom minimal, immutable Linux distributions
linux  deployment  virtualization 
april 2018 by mpm
Linux Raw Sockets
I've learnt a lot about raw socktet programing under Linux and here I want to share a few things with you.
linux  networking 
march 2018 by mpm
bpftrace
DTrace-style dynamic tracing tool for Linux
ebpf  linux  observability 
march 2018 by mpm
How To Measure the Working Set Size on Linux
The Working Set Size (WSS) is how much memory an application needs to keep working. Your app may have populated 100 Gbytes of main memory, but only uses 50 Mbytes each second to do its job. That's the working set size. It is used for capacity planning and scalability analysis.
memory  linux  scalability 
january 2018 by mpm
Linux Load Averages: Solving the Mystery
When Linux load averages increase, you know you have higher demand for resources (CPUs, disks, and some locks), but you aren't sure which. You can use other metrics for clarification
linux  metrics  observability 
september 2017 by mpm
iproute2 cheat sheet
This document aims to provide comprehensive but easy to use documentation for the ip command included in iproute2 package.
linux  networking 
august 2017 by mpm
Dynamic tracing in Linux user and kernel space
Did you forget to insert probe points in your code? No problem. Learn how to insert them dynamically with uprobe and kprobe
observability  linux  debugging  performance 
july 2017 by mpm
The Walking Dead (but with processes)
Let's delve into what exactly a zombie is, and how they should be handled by Linux.
linux 
july 2017 by mpm
An introduction to timekeeping in Linux VMs
Keeping time in Linux is not simple, and virtualization adds additional challenges and opportunities. In this article, I'll review KVM, Xen, and Hyper-V related time-keeping techniques and the corresponding parts of the Linux kernel
linux  time 
june 2017 by mpm
Obscure Synchronization Primitives
Every other synchronization primitive can be built from mutexes and condition variables, so those two are technically all that we require. However, in the search for performance, sometimes it is necessary to investigate how other primitives that might better-fit a given task may be created
concurrency  linux 
may 2017 by mpm
Signals the easy way
Signals are the the necessary evil that almost nobody get right. So this is all about dealing with them in a manageable way that will also make them usable again inside a normal program.
linux  unix 
april 2017 by mpm
Queueing in the Linux Network Stack
This article aims to explain where IP packets are queued in the Linux network stack, how interesting new latency reducing features such as BQL operate and how to control buffering for reduced latency.
linux  networking 
april 2017 by mpm
Shooting yourself in the head with threads
I though I would put together a list of things I have seen over the years of some obvious and some not so obvious issues when using threads on Linux and c or c++. At lot of these issues will also apply to systems based around POSIX as well.
linux  unix  c++  concurrency  time 
march 2017 by mpm
C2C
At a high level, “perf c2c” will show you:
* The cachelines where false sharing was detected.
* The readers and writers to those cachelines, and the offsets where those accesses occurred.
linux  performance  memory 
february 2017 by mpm
Large Pages May Be Harmful on NUMA Systems
On NUMA systems the memory is spread across several physical nodes; using large pages may contribute to the imbalance in the distribution of memory controller requests and reduced locality of accesses, both of which can drive up memory latencies
linux  performance  memory 
february 2017 by mpm
Monitoring and Tuning the Linux Networking Stack: Sending Data
This blog post explains how computers running the Linux kernel send packets, as well as how to monitor and tune each component of the networking stack as packets flow from user programs to network hardware.
linux  networking  performance  monitoring 
february 2017 by mpm
Glances
Glances is a cross-platform system monitoring tool written in Python
linux  monitoring  python 
january 2017 by mpm
Understanding TCP/IP Network Stack & Writing Network Apps
This article will describe the overall operation scheme of the network stack based on data flow and control flow in Linux OS and the hardware layer.
linux  networking  tcp 
january 2017 by mpm
Illustrated Guide to Monitoring and Tuning the Linux Networking Stack: Receiving Data
A series of diagrams aimed to help readers form a more clear picture of how the Linux network stack works.
linux  networking 
january 2017 by mpm
Monitoring and Tuning the Linux Networking Stack: Receiving Data
It is impossible to tune or monitor the Linux networking stack without reading the source code of the kernel and having a deep understanding of what exactly is happening.This blog post will hopefully serve as a reference to anyone looking to do this
linux  networking  monitoring 
january 2017 by mpm
Linux containers in 500 lines of code
I wanted specifically to find a minimal set of restrictions to run untrusted code
linux 
november 2016 by mpm
uftrace
uftrace is a program that simply runs the specified command until it exits. It intercepts and records the user define function calls which are called by the executed process.
linux  performance 
october 2016 by mpm
Packet mmap
This file documents the mmap() facility available with the PACKET socket interface on 2.4/2.6/3.x kernels
linux  networking  performance 
october 2016 by mpm
Capturing Packets in Linux at a Speed of Millions of Packets per Second without Using Third Party Libraries
My article will tell you how to accept 10 million packets per second without using such libraries as Netmap, PF_RING, DPDK and other. We are going to do this with Linux kernel version 3.16 and some code in C and C++.
networking  performance  linux 
october 2016 by mpm
Shared Libraries: Understanding Dynamic Loading
In this post, I will attempt to explain the inner workings of how dynamicloading of shared libraries works in Linux systems.
linux 
september 2016 by mpm
Linux Inside
A book-in-progress about the linux kernel and its insides.
linux 
august 2016 by mpm
Monitoring and Tuning the Linux Networking Stack: Receiving Data
This blog post explains how computers running the Linux kernel receive packets, as well as how to monitor and tune each component of the networking stack as packets flow from the network toward userland programs
linux  networking  monitoring  performance 
june 2016 by mpm
bcc
Observe the execution of any software. These tools use static and dynamic tracing of both user- and kernel-level code (via kprobes, uprobes, tracepoints, and USDT).
linux  performance  monitoring 
april 2016 by mpm
TCPTuner
TCPTuner is TCP congestion control kernel module and GUI packaged together. By loading our kernel module (which is essentially a clone of CUBIC), and running the GUI, users can adjust several parameters of the CUBIC congestion control algorithm
tcp  linux 
april 2016 by mpm
The Definitive Guide to Linux System Calls
This blog post explains how Linux programs call functions in the Linux kernel. It will outline several different methods of making systems calls, how to handcraft your own assembly to make system calls (examples included), kernel entry points into system calls, kernel exit points from system calls, glibc wrappers, bugs, and much, much more.
linux 
april 2016 by mpm
netdata
netdata is a highly optimized Linux daemon providing real-time performance monitoring for Linux systems, Applications, SNMP devices, over the web!
linux  monitoring 
april 2016 by mpm
cpustat
high frequency performance measurements for Linux
linux  performance 
march 2016 by mpm
nicstat
nicstat is a Solaris and Linux command-line that prints out network statistics for all network interface cards (NICs), including packets, kilobytes per second, average packet sizes and more
networking  monitoring  linux 
january 2016 by mpm
libfaketime
libfaketime modifies the system time for a single application
time  testing  linux 
january 2016 by mpm
numfmt
numfmt can operate as a filter, and thus augment other commands, without requiring them to add logic or options to support "human" readable formats. Given the various number formatting options available, it's better to keep all the options involved in a single cohesive command
bash  linux 
november 2015 by mpm
ktap
ktap is a new script-based dynamic tracing tool for Linux, it uses a scripting language and lets users trace the Linux kernel dynamically. ktap is designed to give operational insights with interoperability that allows users to tune, troubleshoot and extend kernel and application. It's similar with Linux Systemtap and Solaris Dtrace
linux  performance  observability  lua 
november 2015 by mpm
linux-insides
A little bit about a linux kernel
linux 
november 2015 by mpm
Snabb Switch: kernel-bypass networking illustrated
Snabb Switch is a networking application that runs on Linux. However, it does not typically use Linux's networking functionality. Instead it negotiates with the kernel to take control of whole PCI network devices and perform I/O directly without using the kernel as a middle-man. This is kernel-bypass networking
linux  networking  performance 
october 2015 by mpm
Per-process netfilter rules
This article documents how the traffic of specific Linux processes can be subjected to a custom firewall or routing configuration, thanks to the magic of cgroups. We will use the Network classifier cgroup, which allows tagging the packets sent by specific processes.
linux  networking 
october 2015 by mpm
Portable umem
This is a port of the Solaris umem memory allocator to other popular operating systems, such as Linux, Windows and BSDish systems (including Darwin/OSX).
memory  linux 
october 2015 by mpm
Dynamic linker tricks: Using LD_PRELOAD to cheat, inject features and investigate programs
I’m going to show you a basic trick that lets you heavily influence the behavior of most applications, which is not only fun, but also, at times, useful.
linux 
september 2015 by mpm
Fun with BPF, or, shutting down a TCP listening socket the hard way
Unlike in BSD, where Berkeley Packet Filter is implemented as a root-only device that attaches to entire network interfaces, on Linux it is implemented in terms of a socket option that usually attaches to AF_PACKET or AF_RAW sockets, however it is a little known fact you can also attach such filters to AF_INET sockets, and better yet, the ability to do so does not require root. Essentially, Linux allows non-root programs to configure their own little private firewall
linux  networking  tcp 
july 2015 by mpm
How to achieve low latency with 10Gbps Ethernet
[How to] optimize our UDP application for latency
linux  networking  performance  udp 
july 2015 by mpm
Linux Inside
A series of posts about the linux kernel and its insides
linux  book 
june 2015 by mpm
perfj
PerfJ is a wrapper of linux perf for java programs
java  performance  linux 
june 2015 by mpm
How to receive a million packets per second
On Linux, how hard is it to write a program that receives 1 million UDP packets per second? Hopefully, answering this question will be a good lesson about the design of a modern networking stack
linux  networking  performance 
june 2015 by mpm
Debug your programs like they're closed source
I’ve started sometimes debugging a new way. With this method, I don’t look at the source code, don’t edit the source code, and don’t use a debugger. I don’t even need to have the program’s source available to me
debugging  linux 
may 2015 by mpm
Systematic Process to Reduce Linux OS Jitter
Finding the cause of hiccups/jitters in a a Linux system is black magic
linux  performance 
april 2015 by mpm
fulltrace
A complete ftrace- and uprobes-based tracer (user, libraries, kernel) for GNU/Linux
linux  performance  testing  managability 
march 2015 by mpm
Command-Line Options
Over time, frequently-used options in well-known Unix programs have established a loose sort of semantic standard for what various flags might be expected to mean. The following is a list of options and meanings that should prove usefully unsurprising to an experienced Unix user
unix  linux  usability 
march 2015 by mpm
DPDK
DPDK is a set of libraries and drivers for fast packet processing
linux  networking  performance 
february 2015 by mpm
PFQ I/O
PFQ is a functional networking framework designed for the Linux operating system that allows efficient packets capture/transmission, in-kernel functional processing and packets steering across sockets/end-points.
linux  networking  functional 
february 2015 by mpm
Linux x86 Program Start Up
This is for people who want to understand how programs get loaded under linux. In particular it talks about dynamically loaded x86 ELF files
linux 
december 2014 by mpm
fastsocket
Fastsocket is a scalable kernel TCP socket implementation and achieves a straight linear performance growth when scaling up to 24 CPU cores
linux  networking  performance  tcp 
october 2014 by mpm
uselessd
uselessd (the useless daemon, or the daemon that uses less... depending on your viewpoint) is a project to reduce systemd to a base initd, process supervisor and transactional dependency system, while minimizing intrusiveness and isolationism. Basically, it’s systemd with the superfluous stuff cut out, a (relatively) coherent idea of what it wants to be, support for non-glibc platforms and an approach that aims to minimize complicated design
linux 
september 2014 by mpm
pcstat
Page Cache stat: get page cache stats for files on Linux
linux  performance  caching 
september 2014 by mpm
Ftrace: The hidden light switch
I've recently found hidden switches that turn on some bright lights, strategically placed by Steven Rostedt and others since the 2.6.27 release. These are the ftrace profilers. I haven't even tried all the switches yet, but I'm stunned at what I've seen so far, and I'm having to rethink what I previously believed about Linux kernel performance analysis.
linux  performance 
august 2014 by mpm
« earlier      
per page:    204080120160

Copy this bookmark:



description:


tags: