現象:AI 推論、編譯或大量 I/O 同時跑時,CPU 可能沒有 100%,但延遲突然變長。只看 top 很難知道工作卡在 CPU、記憶體還是 I/O。
Phenomenon: During inference, builds, or heavy I/O, latency can jump while CPU is below 100%. top alone cannot identify whether work waits on CPU, memory, or I/O.
圖 1|PSI 將想執行卻被延遲的時間彙整成可觀察訊號。
Figure 1 | PSI turns delayed progress into an observable signal.
機制:PSI 量的是有工作卻前進不了 / Mechanism: PSI measures stalled progress
Pressure Stall Information(PSI)統計任務因資源競爭而停滯的時間。some 表示至少部分任務受阻;full 表示一段時間內所有非閒置工作都受阻。Linux 將 CPU、記憶體與 I/O 分別放在 /proc/pressure/。
Pressure Stall Information (PSI) measures time lost to contention. some means some tasks are stalled; full means all non-idle work is stalled for an interval. Linux exposes CPU, memory, and I/O separately under /proc/pressure/.
影響:先分辨壓力類型 / Impact: classify pressure first
邊緣 LLM 若 memory some 持續升高,可能是回收、匿名頁或記憶體頻寬造成延遲;若 io some 升高,增加 CPU 配額未必有效。平均值是時間窗統計,應與請求 p95、cgroup 使用量和核心日誌對齊。
For an edge LLM, sustained memory some may indicate reclaim, anonymous-page, or bandwidth pressure. If io some rises, more CPU quota may not help. Align windowed averages with p95 latency, cgroup usage, and kernel logs.
可操作步驟:建立低風險基線 / Actionable steps: build a safe baseline
cat /proc/pressure/cpu; cat /proc/pressure/memory; cat /proc/pressure/io; cat /sys/fs/cgroup/$CGROUP/memory.current; cat /sys/fs/cgroup/$CGROUP/memory.events先連續記錄 5 分鐘,把 avg10 與服務 p95 畫在同一時間軸。不要立刻加 swap、提高配額或移除限制;先判斷是主機還是 memcg 壓力,再用可回復的小批次驗證。
Record five minutes and align avg10 with p95 latency. Do not immediately add swap, raise quotas, or remove limits; locate host versus memcg pressure, then test a reversible small-batch change.
查核與限制:依本機 Linux 6.18.55 Documentation/accounting/psi.rst 查核(2026-11-15),並對照官方文件。命令只讀取狀態。
Verification and limits: Checked against local Linux 6.18.55 Documentation/accounting/psi.rst on 2026-11-15 and official documentation. Commands are observational.
Linux PSI documentation · cgroup v2 documentation · /proc documentation
留言
張貼留言