跳到主要內容

發表文章

Android 端側 AI 為什麼要在 DMA-BUF 停一下?零拷貝不等於零同步 / Android On-Device AI: DMA-BUF Zero-Copy Still Needs Synchronization

daily-tech:2026-11-19 現象:模型沒有變大,畫面卻開始卡 / Symptom: The model is unchanged, but frames stall 手機把相機畫面交給端側 AI 模型時,最直覺的做法是先複製到 CPU,再複製到 GPU 或 NPU。資料量一大,模型本身可能沒有變慢,真正增加的是搬運、等待與格式轉換。共享 buffer 可以減少複製,但若忽略同步,畫面可能讀到未完成的資料,或因等待 producer 而出現尾端延遲。 When a phone sends camera frames to an on-device AI model, the obvious path copies pixels through the CPU and then to the GPU or NPU. With larger frames, transport, waiting, and format conversion can dominate even when the model is unchanged. Shared buffers can reduce copies, but missing synchronization can expose incomplete data or create tail latency. Android AI buffer path Camera / CPU producer AHardwareBuffer shared handle DMA-BUF fd import + fence GPU / NPU consumer cache sync + fence + lifetime still matter 圖 1|共享 buffer 減少搬運,但 cache、fence 與生命週期仍需管理。 / Figure 1 | Shared buffers reduce copies, but cache, fences, and lifetime still need explicit management. 機制:HardwareBuffer 是入口,DMA-BUF 是共享路徑 / Mechanism: HardwareBuffer is the Android entry, DMA-BUF i...
最近的文章

Android ART 為什麼會突然卡住?看懂 GC、堆積與延遲尖峰 / Android ART GC: Why Short Pauses Become Latency Spikes

daily-tech:2026-11-18 現象:App 沒有崩潰,畫面卻突然停住 / Symptom: The app is alive, but frames suddenly stop Android App 在滑動、開啟相機或載入模型時,可能沒有 Crash,畫面卻短暫停住。很多人先把問題歸咎於 CPU 不夠快,但 ART 的記憶體回收也會讓應用程式執行緒暫停。要找原因,先把配置物件、堆積成長和 GC 暫停分開觀察。 An Android app may not crash, yet briefly freeze while scrolling, opening the camera, or loading a model. CPU speed is an easy suspect, but ART garbage collection can also pause application threads. Separate allocation, heap growth, and GC pauses before changing code or hardware. 機制:ART 如何把垃圾回收放進執行流程 / Mechanism: how ART places GC into execution ART 使用 managed heap 管理 Java 與 Kotlin 物件。程式配置新物件時,堆積會逐步接近成長界線;當回收條件成立,GC 會找出仍被參考的物件,回收不可達空間,並依回收器與階段產生不同程度的工作。背景回收不代表完全沒有停頓:根掃描、執行緒協調與記憶體壓縮仍可能反映在畫面幀時間。 ART manages Java and Kotlin objects in a managed heap. Allocations move the heap toward its growth boundary; when collection is needed, GC identifies reachable objects and reclaims unreachable space. Concurrent work does not mean zero pauses: root scanning, thread coordination, and com...

Linux cgroup v2 記憶體壓力:用 memory.high 先降速,不要等 OOM 才救火 / Linux cgroup v2 Memory Pressure: Throttle Before OOM

現象: 容器裡的 AI 推論、編譯或資料處理突然變慢,主機 RAM 看起來還有剩,最後卻在尖峰時被 OOM Killer 終止。只盯著 free 或 top,常常看不到是哪一個 cgroup 先碰到壓力。 Phenomenon: An AI inference, build, or data-processing container suddenly slows down. The host still appears to have free RAM, yet the workload is eventually killed by the OOM killer. Looking only at free or top hides which cgroup reached pressure first. cgroup v2 memory control memory.current observe usage memory.high reclaim / throttle memory.max hard boundary memory.events latency + logs 圖一|先觀察用量,再以 memory.high 控制回收與降速,最後才把 memory.max 當硬邊界。 Figure 1 | Observe usage, throttle with memory.high, and reserve memory.max for the hard boundary. 機制:三個檔案代表三種控制語意 / Mechanism: three files, three meanings cgroup v2 的 memory.current 是目前用量;memory.high 是「超過後要付出成本」的軟邊界,核心會讓工作進入回收與節流路徑,但不以它直接殺掉整個 cgroup;memory.max 才是硬邊界,回收仍不足時可能觸發 cgroup 內的 OOM。這三者不是三個可互換的 RAM 數字,而是由觀察、降速到最後保護的階梯。 In cgroup v2, memory.current reports current usage. memory.high is a throttle boundary: crossing it pushes tasks throu...

eBPF 為什麼不能隨便讀記憶體?看懂 Linux Verifier 的安全檢查 / Why the Linux eBPF Verifier Refuses Unsafe Memory Access

daily-tech:2026-11-16 現象:一個指標,為什麼不能直接讀? / Symptom: Why can’t one pointer simply be read? 寫 eBPF 時,最容易踩到的坑不是語法,而是「我明明先算出位址,為什麼載入核心就被拒絕?」原因是 eBPF 程式會在核心內執行;Linux 不能等錯誤發生後才補救,因此載入前先由 verifier 檢查指標型別、數值範圍、對齊與控制流程。 When writing eBPF, the frustrating failure is often not syntax: “I calculated the address, so why did the kernel reject the program?” eBPF runs inside the kernel. Linux therefore checks pointer types, value ranges, alignment, and control flow before loading instead of repairing a fault afterward. 機制:verifier 追蹤的是狀態,不是猜測 / Mechanism: the verifier tracks states, not guesses 官方文件把暫存器分成 scalar 與多種 pointer。scalar 會保存最小、最大值與未知位元;指標則帶著來源與可存取範圍。以封包為例,先取出 data 與 data_end,再確認 data 加上標頭長度沒有越過 data_end,之後那個分支內的讀取才被視為安全。若中間把指標做成無法證明範圍的運算,狀態會退化成 scalar,後面的記憶體讀取就會失效。 The kernel documentation describes registers as scalars or several pointer types. A scalar carries bounds and unknown bits; a pointer carries its origin and an accessible range. For packet data, the program loads data and data_end,...

Linux PSI:CPU 沒滿卻變慢?找出記憶體與 I/O 壓力 / Linux PSI: Finding Hidden Contention

現象: AI 推論、編譯或大量 I/O 同時跑時,CPU 可能沒有 100%,但延遲突然變長。只看 top 很難知道工作卡在 CPU、記憶體還是 I/O。 Phenomenon: During inference, builds, or heavy I/O, latency can jump while CPU is below 100%. top alone cannot identify whether work waits on CPU, memory, or I/O. PSI pressure-stall flow runnable work is delayed kernel aggregates some / full stall /proc/pressure/* time windows correlate with p95 tune reversibly 圖 1|PSI 將想執行卻被延遲的時間彙整成可觀察訊號。 Figure 1 | PSI turns delayed progress into an observable signal. 機制:PSI 量的是有工作卻前進不了 / Mechanism: PSI measures stalled progress Pressure Stall Information(PSI)統計任務因資源競爭而停滯的時間。some 表示至少部分任務受阻;full 表示一段時間內所有非閒置工作都受阻。Linux 將 CPU、記憶體與 I/O 分別放在 /proc/pressure/。 Pressure Stall Information (PSI) measures time lost to contention. some means some tasks are stalled; full means all non-idle work is stalled for an interval. Linux exposes CPU, memory, and I/O separately under /proc/pressure/. 影響:先分辨壓力類型 / Impact: classify pressure first 邊緣 LLM 若 memory some 持續升高,可能是回收、匿名頁或記憶體頻寬造成延遲;若 io...

Linux OOM Killer 不只看 RAM 用量:從 oom_score_adj 到 cgroup 邊界的排查路徑 / Linux OOM Killer: Beyond RAM Usage

現象: 服務明明還有一些可用記憶體,Linux 卻突然結束程序,甚至讓容器內的 AI 推論中斷。這不一定是整台機器的 RAM 用光;配置範圍、swap、memcg 限制與保護分數都可能讓核心進入 OOM 路徑。 Phenomenon: Linux may terminate a process even when the machine appears to have free memory. The trigger can be a cgroup, NUMA policy, swap, or allocation scope—not only total RAM. Linux OOM decision path allocation cannot reclaim enough scope: system / memcg / NUMA score memory + oom_score_adj kill selected task; continue 圖 1|Linux OOM killer 的簡化決策路徑:先確認耗盡範圍,再以記憶體用量與 oom_score_adj 評分。 Figure 1 | Simplified Linux OOM-killer path: identify the exhausted scope, then score candidates using memory usage and oom_score_adj. 機制:OOM 先看耗盡範圍,再挑候選者 / Mechanism: scope first, victim second 核心在無法透過 reclaim 或 compaction 滿足配置時,才會進入 OOM killer。 mm/oom_kill.c 的 oom_badness() 依候選程序記憶體用量與 oom_score_adj 計算分數; -1000 代表強力保護。允許使用的記憶體集合取決於當次 OOM 上下文,因此 memcg OOM 不等同於整機 OOM。 The kernel enters the OOM path only after reclaim and compaction cannot satisfy the allocation. In mm/oom_kill.c , oom_badness() ...

Linux io_uring 為什麼仍可能卡住?從固定 Buffer 到 Completion Queue 的 I/O 路徑 / Why Linux io_uring Can Still Stall: Fixed Buffers and Completion Queues

 現象:io_uring 非同步,為什麼服務仍會卡住? / Symptom: Why can an io_uring service still stall? 你把大量檔案讀取改成 Linux io_uring,CPU 使用率下降,卻仍看到請求延遲突然拉高。提交請求變得非同步,不代表儲存裝置立刻完成;queue depth、檔案系統與 block I/O 仍會把等待時間留在 completion 之前。 You moved file reads to Linux io_uring, CPU usage fell, yet request latency still spikes. Submission becomes asynchronous; storage, the filesystem, and block I/O can still delay completion. io_uring 的基本路徑是:應用程式準備 submission queue entry(SQE),核心取出並執行,最後把結果寫成 completion queue entry(CQE)。共享 ring 減少系統呼叫,但 CQE 的 res 才是實際結果;不能只看到 SQE 已提交,就把工作算成完成。 The path is: the application prepares an SQE, the kernel executes it, and a CQE reports the result. Shared rings reduce syscall traffic, but CQE res is the real outcome. Submission is not completion. 圖一|io_uring 的 SQE→核心執行→CQE 路徑;固定 buffer 的生命週期與完成延遲必須分開觀察。 / Figure 1 | The SQE-to-kernel-to-CQE path; buffer lifetime and completion latency are separate signals. Original conceptual diagram. 機制:固定 Buffer 解決什麼,又帶來什麼責任? / Mechanism: what registered bu...