All writing
~/writing/tags
#kernel
2 posts
Systems debugging
4 min
The load average with a seven hour heartbeat
Every seven hours the router's load average hit 1.0 for forty five minutes. CPU was 98% idle. Nothing ran, nothing blocked, nothing was stolen. The culprit was a beat frequency between two five second clocks, and one of them was mine.
#linux#kernel#monitoring#grafana
Systems debugging
9 min
What is putting those bytes on the wire?
The storage interface on a production cluster was pulling 4 to 12x more than the public interface where client requests arrive. Getting to a defensible number meant distrusting everything, including my own profiler, which was off by 30x.
#ceph#ebpf#linux#kernel