]> 2026-09-08T16:38:29+09:00 Lesson:fast on device llm inference with npus deecd917 3000 ko On-device LLM inference on SoCs with CPU, GPU, and NPU. Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior. The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU. S3ResearchAgent medium Venue: ASPLOS. Year: 2025. Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution. Verification: arXiv abstract and DOI metadata; confidence=high. 2026-07-16T14:58:50Z 2461238.1241898 Fast On-device LLM Inference with NPUs. ASPLOS 2025. Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine. Fast On-device LLM Inference with NPUs workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers? lab Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints. Draft 2026-07-18T14:58:31Z 2461240.1239699 2026-07-18T14:58:32Z 2461240.1239815 fast on device llm inference with npus deecd917 3000 ev_e894ca5b738c412a 2026-07-16T14:58:51Z 2461238.1242014 S3ResearchAgent Fast On-device LLM Inference with NPUs. ASPLOS 2025. paper Bibliographic paper record. not_recorded fast on device llm inference with npus deecd917# 30ba25d247e69581158ad3b38eb3464c 3000 verified-content-v1-0113 2026-07-16T18:44:24Z 2461238.2808333 S3ResearchAgent Daliang Xu et al., "Fast On-device LLM Inference with NPUs", ASPLOS 2025. paper Verification: arXiv abstract and DOI metadata; confidence=high. Canonical title: Fast On-device LLM Inference with NPUs Question: How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers? Context: Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution. Method: The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU. Evaluation: workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end Interpretation: Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine. Reusable lesson: Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints. Applicability: On-device LLM inference on SoCs with CPU, GPU, and NPU. Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior. not_recorded fast on device llm inference with npus deecd917# 0f978b0e5bc6b07d9af89d5cb1f41398 3000 canonical-paper-v2-deecd917 2026-07-18T05:21:43Z 2461239.7234144 S3ResearchAgent Daliang Xu et al., "Fast On-device LLM Inference with NPUs", ASPLOS 2025. paper Verification: arXiv abstract and DOI metadata; confidence=medium. Canonical title: Fast On-device LLM Inference with NPUs Question: How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers? Context: Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution. Method: The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU. Evaluation: workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end Interpretation: Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine. Reusable lesson: Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints. Applicability: On-device LLM inference on SoCs with CPU, GPU, and NPU. Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior. official_abstract fast on device llm inference with npus deecd917# 7ef512cad74bfdb99be15b7ecf657895 3000 2026-07-18T14:58:31Z 2461240.1239699 S3ResearchAgent verify_00f345af2ed230b9850f metadata_only context 보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함. inconclusive ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 ev_e894ca5b738c412a 2fd414863da19ffc20707e4e0c17088aad785227b369169032d80aab395c4233 fast on device llm inference with npus deecd917# 0ee6f1e305a13285714b4b38eca231d2 3000 2026-07-18T14:58:31Z 2461240.1239699 S3ResearchAgent verify_5fb4da199226f0834fa5 metadata_only context 보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함. inconclusive ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 verified-content-v1-0113 53d2ff886e3f9b6d82a28698614b19ea2c28bb701de423b0516343ccb6dbbbc0 fast on device llm inference with npus deecd917# 4e570a48a17edb9c8011f3056ef3b571 3000 2026-07-18T14:58:31Z 2461240.1239699 S3ResearchAgent verify_b6731e5f1bfdb7d4d2f4 metadata_only context 보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함. inconclusive ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 canonical-paper-v2-deecd917 5dab486f66f1cb1c2f5399526b372c7ba10bbd7b917785a41af8350148d4b97a fast on device llm inference with npus deecd917# bf302ad414b937dfc0fe5b30c66bdfb6