daily-tech:2026-11-20 現象:模型沒有變大,畫面卻開始卡 / Symptom: The model is unchanged, but frames stall 手機把相機畫面交給端側 AI 模型時,最直覺的做法是先複製到 CPU,再複製到 GPU 或 NPU。資料量一大,模型本身可能沒有變慢,真正增加的是搬運、等待與格式轉換。共享 buffer 可以減少複製,但若忽略同步,畫面可能讀到未完成的資料,或因等待 producer 而出現尾端延遲。 When a phone sends camera frames to an on-device AI model, the obvious path copies pixels through the CPU and then to the GPU or NPU. With larger frames, transport, waiting, and format conversion can dominate even when the model is unchanged. Shared buffers can reduce copies, but missing synchronization can expose incomplete data or create tail latency. Android AI buffer path Camera / CPU producer AHardwareBuffer shared handle DMA-BUF fd import + fence GPU / NPU consumer cache sync + fence + lifetime still matter 圖 1|共享 buffer 減少搬運,但 cache、fence 與生命週期仍需管理。 / Figure 1 | Shared buffers reduce copies, but cache, fences, and lifetime still need explicit management. 機制:HardwareBuffer 是入口,DMA-BUF 是共享路徑 / Mechanism: HardwareBuffer is the Android entry, DMA-BUF i...
Android 端側 AI 為什麼要在 DMA-BUF 停一下?零拷貝不等於零同步 / Android On-Device AI: DMA-BUF Zero-Copy Still Needs Synchronization
daily-tech:2026-11-19 現象:模型沒有變大,畫面卻開始卡 / Symptom: The model is unchanged, but frames stall 手機把相機畫面交給端側 AI 模型時,最直覺的做法是先複製到 CPU,再複製到 GPU 或 NPU。資料量一大,模型本身可能沒有變慢,真正增加的是搬運、等待與格式轉換。共享 buffer 可以減少複製,但若忽略同步,畫面可能讀到未完成的資料,或因等待 producer 而出現尾端延遲。 When a phone sends camera frames to an on-device AI model, the obvious path copies pixels through the CPU and then to the GPU or NPU. With larger frames, transport, waiting, and format conversion can dominate even when the model is unchanged. Shared buffers can reduce copies, but missing synchronization can expose incomplete data or create tail latency. Android AI buffer path Camera / CPU producer AHardwareBuffer shared handle DMA-BUF fd import + fence GPU / NPU consumer cache sync + fence + lifetime still matter 圖 1|共享 buffer 減少搬運,但 cache、fence 與生命週期仍需管理。 / Figure 1 | Shared buffers reduce copies, but cache, fences, and lifetime still need explicit management. 機制:HardwareBuffer 是入口,DMA-BUF 是共享路徑 / Mechanism: HardwareBuffer is the Android entry, DMA-BUF i...