perf-tools
A collection of performance analysis tools for Linux using ftrace and perf_events to trace system activity with minimal dependencies.
What is perf-tools?
perf-tools is a collection of command-line performance analysis scripts for Linux systems. It uses the kernel's built-in ftrace and perf_events subsystems to provide detailed observability into disk I/O, system calls, kernel functions, and network activity. The tools help developers and system administrators diagnose performance bottlenecks and anomalous behavior without requiring complex instrumentation or debug symbols.
Linux system administrators, performance engineers, and developers who need low-level observability into kernel and system behavior for debugging and optimization.
It offers a lightweight, dependency-minimal alternative to heavier profiling suites, providing immediate utility on production systems where installing debuginfo or specialized packages isn't feasible. The tools are designed by Brendan Gregg, a renowned performance expert, ensuring practical and battle-tested approaches.
Overview
Performance analysis tools based on Linux perf_events (aka perf) and ftrace
Use Cases
Best For
- Diagnosing slow disk I/O latency and identifying problematic processes
- Monitoring unexpected process executions or file opens in real-time
- Profiling kernel function call frequency and latency for driver development
- Investigating TCP retransmission issues and network stack problems
- Analyzing page cache efficiency and memory subsystem behavior
- Dynamic tracing of kernel or user-space functions without recompilation
Not Ideal For
- Production environments where kernel stability and minimal overhead are non-negotiable
- Teams needing long-term, vendor-supported performance monitoring solutions
- Users on Linux kernels older than 3.2 (e.g., CentOS 6.x) due to risk of panics and bugs
- Developers preferring graphical interfaces or integrated APM suites over command-line scripts
Pros & Cons
Pros
Tools require only basic kernel features like ftrace and perf_events, with no debuginfo or complex packages needed, as highlighted in the Prerequisites section.
Covers disk I/O, system calls, kernel functions, TCP, and cache analysis through scripts like iosnoop and execsnoop, providing comprehensive insights without custom instrumentation.
Created by Brendan Gregg, a renowned performance expert, ensuring tools are battle-tested and follow the Unix philosophy of doing one thing well, with extensive examples and man pages.
Scripts such as opensnoop and killsnoop offer immediate tracing of file opens and signals, enabling live troubleshooting for anomalous system behavior.
Cons
The README explicitly states tools are 'in-development and unsupported', relying on workarounds that can be kernel-version specific and prone to breakage.
Warnings in the README note overhead can slow applications by up to 5x, with risks of kernel panics, especially on older kernels like 2.6.x.
Project describes tools as placeholders until kernel features evolve, pointing to eBPF-based alternatives like bcc for modern use, indicating potential obsolescence.
Frequently Asked Questions
Related Projects
bccBCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more
likwidPerformance monitoring and benchmarking suite
iopingsimple disk I/0 latency measuring tool
jinfoPrints configuration information for a given process
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.