biosnoop - Trace block device I/O and print details incl. issuing PID. biosnoop This tools traces block device I/O (disk I/O), and prints a one-line summary for each I/O showing various details. These include the latency from the time of issue to the device to its completion, and the PID and process name from when the I/O was first created (which usually identifies the responsible process). This uses in-kernel eBPF maps to cache process details (PID and comm) by I/O request, as well as a starting timestamp for calculating I/O latency. This works by tracing various kernel blk_*() functions using dynamic tracing, and will need updating to match any changes to these functions. This makes use of a Linux 4.5 feature (bpf_perf_event_output()); for kernels older than 4.5, see the version under tools/old, which uses an older mechanism Since this uses BPF, only the root user can use this tool. CONFIG_BPF and bcc.

Trace all block device I/O and print a summary line per I/O: # biosnoop

TIME(s) Time of the I/O, in seconds since the first I/O was seen.

COMM Cached process name, if present. This usually (but isn't guaranteed) to identify the responsible process for the I/O.

PID Cached process ID, if present. This usually (but isn't guaranteed) to identify the responsible process for the I/O.

DISK Disk device name.

T Type of I/O: R = read, W = write. This is a simplification.

SECTOR Device sector for the I/O.

BYTES Size of the I/O, in bytes.

LAT(ms) Time for the I/O (latency) from the issue to the device, to its completion, in milliseconds.

Since block device I/O usually has a relatively low frequency (< 10,000/s), the overhead for this tool is expected to be negligible. For high IOPS storage systems, test and quantify before use. This is from bcc. https://github.com/iovisor/bcc

Also look in the bcc distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

Linux Unstable - in development. Brendan Gregg disksnoop(8), iostat(1)