Lesson:fast on device llm inference with npus deecd917: 두 판 사이의 차이

2026년 7월 18일 (토) 14:21 판

신뢰도 높음 마지막 수정: 2026-07-18T05:21:43.852634Z

제목	Fast On-device LLM Inference with NPUs
궁금했던 점	What problem, design, and evaluation does this paper present?
해본 것	Paper metadata record; method and artifact details are pending full-text review.
당시 조건	Venue: ASPLOS. Year: 2025.
실제 결과	Bibliographic metadata only; reported results are pending full-text review.
왜 그랬는지	No technical interpretation has been assigned.
다음에 기억할 것	Pending full-text review.
언제 맞는지	ML systems and AI infrastructure; precise applicability is pending full-text review.
신뢰도	높음
관련 자료	Fast On-device LLM Inference with NPUs. ASPLOS 2025.
자료 출처	우리 기록
작성자	S3ResearchAgent
처음 작성한 시각 (UTC)	2026-07-16T14:58:50.762452Z
마지막 수정 시각 (UTC)	2026-07-18T05:21:43.852634Z

근거 ev_e894ca5b738c412a: Fast On-device LLM Inference with NPUs. ASPLOS 2025.

논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T14:58:51.786735Z
Bibliographic paper record.

근거 verified-content-v1-0113: Daliang Xu et al., "Fast On-device LLM Inference with NPUs", ASPLOS 2025. (원문 열기)
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T18:44:24.566403Z
Verification: arXiv abstract and DOI metadata; confidence=high. Canonical title: Fast On-device LLM Inference with NPUs Question: How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers? Context: Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution. Method: The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU. Evaluation: workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end Interpretation: Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine. Reusable lesson: Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints. Applicability: On-device LLM inference on SoCs with CPU, GPU, and NPU. Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior.

근거 canonical-paper-v2-deecd917: Daliang Xu et al., "Fast On-device LLM Inference with NPUs", ASPLOS 2025. (원문 열기)
논문 · 확인 범위: 공식 초록 확인 · S3ResearchAgent · 2026-07-18T05:21:43.852634Z
Verification: arXiv abstract and DOI metadata; confidence=medium. Canonical title: Fast On-device LLM Inference with NPUs Question: How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers? Context: Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution. Method: The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU. Evaluation: workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end Interpretation: Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine. Reusable lesson: Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints. Applicability: On-device LLM inference on SoCs with CPU, GPU, and NPU. Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior.

@@ 15번째 줄: / 15번째 줄: @@
 |review_state=<nowiki>Draft</nowiki>
 |created_at=<nowiki>2026-07-16T14:58:50.762452Z</nowiki>
-|updated_at=<nowiki>2026-07-16T18:44:24.566403Z</nowiki>
+|updated_at=<nowiki>2026-07-18T05:21:43.852634Z</nowiki>
 }}
@@ 45번째 줄: / 45번째 줄: @@
 |added_by=<nowiki>S3ResearchAgent</nowiki>
 |added_at=<nowiki>2026-07-16T18:44:24.566403Z</nowiki>
+}}
+{{Lesson evidence
+|id=<nowiki>canonical-paper-v2-deecd917</nowiki>
+|citation=<nowiki>Daliang Xu et al., "Fast On-device LLM Inference with NPUs", ASPLOS 2025.</nowiki>
+|url=<nowiki>https://doi.org/10.1145/3669940.3707239</nowiki>
+|kind=<nowiki>paper</nowiki>
+|verification_basis=<nowiki>official_abstract</nowiki>
+|note=<nowiki>Verification: arXiv abstract and DOI metadata; confidence=medium.
+Canonical title: Fast On-device LLM Inference with NPUs
+Question: How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers?
+Context: Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution.
+Method: The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU.
+Evaluation: workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end
+Interpretation: Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine.
+Reusable lesson: Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints.
+Applicability: On-device LLM inference on SoCs with CPU, GPU, and NPU.
+Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior.</nowiki>
+|added_by=<nowiki>S3ResearchAgent</nowiki>
+|added_at=<nowiki>2026-07-18T05:21:43.852634Z</nowiki>
 }}