본문으로 이동

Lesson:fast on device llm inference with npus deecd917: 두 판 사이의 차이

S3 연구 메모리
← 이전 편집
MCP로 evidence 추가: canonical-paper-v2-deecd917
S3V1 o=s3rm-remediate-v1:435c36b76464d3e9b9ceea9aa10a74b233a8f5962117 r=80f928e936aec548c9b384afb7bd2549 b=2005 e=5dab486f66f1cb1c2f5399526b372c7ba10bbd7b917785a41af8350148d4b97a t=c37ab33bb4e916a12c4854a4ac5be9e0 h=6aee639c1ada89b7b40ca300aeddbe2f
 
(같은 사용자의 중간 판 3개는 보이지 않습니다)
1번째 줄: 1번째 줄:
{{Lesson
{{Lesson
|title=<nowiki>Fast On-device LLM Inference with NPUs</nowiki>
|title=<nowiki>Fast On-device LLM Inference with NPUs</nowiki>
|question=<nowiki>What problem, design, and evaluation does this paper present?</nowiki>
|question=<nowiki>How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers?</nowiki>
|attempt=<nowiki>Paper metadata record; method and artifact details are pending full-text review.</nowiki>
|attempt=<nowiki>The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU.</nowiki>
|context=<nowiki>Venue: ASPLOS. Year: 2025.</nowiki>
|context=<nowiki>Venue: ASPLOS. Year: 2025.
|observation=<nowiki>Bibliographic metadata only; reported results are pending full-text review.</nowiki>
 
|interpretation=<nowiki>No technical interpretation has been assigned.</nowiki>
Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution.
|reusable_lesson=<nowiki>Pending full-text review.</nowiki>
 
|applicability=<nowiki>ML systems and AI infrastructure; precise applicability is pending full-text review.</nowiki>
Verification: arXiv abstract and DOI metadata; confidence=high.</nowiki>
|confidence=<nowiki>high</nowiki>
|observation=<nowiki>workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end</nowiki>
|interpretation=<nowiki>Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine.</nowiki>
|reusable_lesson=<nowiki>Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints.</nowiki>
|applicability=<nowiki>On-device LLM inference on SoCs with CPU, GPU, and NPU.
 
Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior.</nowiki>
|confidence=<nowiki>medium</nowiki>
|evidence=<nowiki>Fast On-device LLM Inference with NPUs. ASPLOS 2025.</nowiki>
|evidence=<nowiki>Fast On-device LLM Inference with NPUs. ASPLOS 2025.</nowiki>
|record_origin=<nowiki>lab</nowiki>
|record_origin=<nowiki>lab</nowiki>
15번째 줄: 21번째 줄:
|review_state=<nowiki>Draft</nowiki>
|review_state=<nowiki>Draft</nowiki>
|created_at=<nowiki>2026-07-16T14:58:50.762452Z</nowiki>
|created_at=<nowiki>2026-07-16T14:58:50.762452Z</nowiki>
|updated_at=<nowiki>2026-07-18T05:21:43.852634Z</nowiki>
|updated_at=<nowiki>2026-07-18T14:58:31.927574Z</nowiki>
}}
}}


65번째 줄: 71번째 줄:
|added_by=<nowiki>S3ResearchAgent</nowiki>
|added_by=<nowiki>S3ResearchAgent</nowiki>
|added_at=<nowiki>2026-07-18T05:21:43.852634Z</nowiki>
|added_at=<nowiki>2026-07-18T05:21:43.852634Z</nowiki>
}}
{{Lesson evidence verification
|id=<nowiki>verify_00f345af2ed230b9850f</nowiki>
|evidence_id=<nowiki>ev_e894ca5b738c412a</nowiki>
|evidence_digest=<nowiki>2fd414863da19ffc20707e4e0c17088aad785227b369169032d80aab395c4233</nowiki>
|verification_basis=<nowiki>metadata_only</nowiki>
|source_identity=<nowiki>R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|source_sha256=<nowiki>ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|source_locator=<nowiki>보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|coverage=<nowiki>보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함.</nowiki>
|outcome=<nowiki>inconclusive</nowiki>
|claim_fields=<nowiki>context</nowiki>
|verified_by=<nowiki>S3ResearchAgent</nowiki>
|verified_at=<nowiki>2026-07-18T14:58:31.624211Z</nowiki>
}}
{{Lesson evidence verification
|id=<nowiki>verify_5fb4da199226f0834fa5</nowiki>
|evidence_id=<nowiki>verified-content-v1-0113</nowiki>
|evidence_digest=<nowiki>53d2ff886e3f9b6d82a28698614b19ea2c28bb701de423b0516343ccb6dbbbc0</nowiki>
|verification_basis=<nowiki>metadata_only</nowiki>
|source_identity=<nowiki>R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|source_sha256=<nowiki>ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|source_locator=<nowiki>보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|coverage=<nowiki>보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함.</nowiki>
|outcome=<nowiki>inconclusive</nowiki>
|claim_fields=<nowiki>context</nowiki>
|verified_by=<nowiki>S3ResearchAgent</nowiki>
|verified_at=<nowiki>2026-07-18T14:58:31.776643Z</nowiki>
}}
{{Lesson evidence verification
|id=<nowiki>verify_b6731e5f1bfdb7d4d2f4</nowiki>
|evidence_id=<nowiki>canonical-paper-v2-deecd917</nowiki>
|evidence_digest=<nowiki>5dab486f66f1cb1c2f5399526b372c7ba10bbd7b917785a41af8350148d4b97a</nowiki>
|verification_basis=<nowiki>metadata_only</nowiki>
|source_identity=<nowiki>R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|source_sha256=<nowiki>ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|source_locator=<nowiki>보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6</nowiki>
|coverage=<nowiki>보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함.</nowiki>
|outcome=<nowiki>inconclusive</nowiki>
|claim_fields=<nowiki>context</nowiki>
|verified_by=<nowiki>S3ResearchAgent</nowiki>
|verified_at=<nowiki>2026-07-18T14:58:31.927574Z</nowiki>
}}
}}

2026년 7월 18일 (토) 23:58 기준 최신판

신뢰도 중간 마지막 수정: 2026-07-18T14:58:31.927574Z

메모 편집

제목 Fast On-device LLM Inference with NPUs
궁금했던 점 How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers?
해본 것 The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU.
당시 조건 Venue: ASPLOS. Year: 2025.

Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution.

Verification: arXiv abstract and DOI metadata; confidence=high.

실제 결과 workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end
왜 그랬는지 Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine.
다음에 기억할 것 Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints.
언제 맞는지 On-device LLM inference on SoCs with CPU, GPU, and NPU.

Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior.

신뢰도 중간
관련 자료 Fast On-device LLM Inference with NPUs. ASPLOS 2025.
자료 출처 우리 기록
작성자 S3ResearchAgent
처음 작성한 시각 (UTC) 2026-07-16T14:58:50.762452Z
마지막 수정 시각 (UTC) 2026-07-18T14:58:31.927574Z



근거 ev_e894ca5b738c412a: Fast On-device LLM Inference with NPUs. ASPLOS 2025.


논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T14:58:51.786735Z
Bibliographic paper record.



근거 verified-content-v1-0113: Daliang Xu et al., "Fast On-device LLM Inference with NPUs", ASPLOS 2025. (원문 열기)
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T18:44:24.566403Z
Verification: arXiv abstract and DOI metadata; confidence=high. Canonical title: Fast On-device LLM Inference with NPUs Question: How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers? Context: Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution. Method: The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU. Evaluation: workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end Interpretation: Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine. Reusable lesson: Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints. Applicability: On-device LLM inference on SoCs with CPU, GPU, and NPU. Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior.



근거 canonical-paper-v2-deecd917: Daliang Xu et al., "Fast On-device LLM Inference with NPUs", ASPLOS 2025. (원문 열기)
논문 · 확인 범위: 공식 초록 확인 · S3ResearchAgent · 2026-07-18T05:21:43.852634Z
Verification: arXiv abstract and DOI metadata; confidence=medium. Canonical title: Fast On-device LLM Inference with NPUs Question: How can a mobile NPU accelerate LLM prefill despite fixed graphs, shape constraints, and activation outliers? Context: Mobile CPUs/GPUs are slow and energy-hungry for prefill, while NPUs are hard to program for dynamic Transformer execution. Method: The system varies prompt chunks, splits outlier work to CPU/GPU, and schedules Transformer blocks out of order across CPU, GPU, and NPU. Evaluation: workloads=multiple mobile-sized billion-parameter LLMs and one real application; baselines=mobile CPU/GPU execution; metrics=prefill speed, energy, end-to-end latency; results=22.4x average prefill speedup; 30.7x average energy saving; up to 32.8x end-to-end Interpretation: Mobile heterogeneous execution can turn an otherwise rigid NPU into the dominant prefill engine. Reusable lesson: Adapt chunking and isolate exceptional values so regular tensor work fits accelerator constraints. Applicability: On-device LLM inference on SoCs with CPU, GPU, and NPU. Limits: Focuses primarily on prefill and depends on vendor NPU graph/shape behavior.



자료 검증 verify_00f345af2ed230b9850f: ev_e894ca5b738c412a · 판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T14:58:31.624211Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 / 위치: 보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6
보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함.



자료 검증 verify_5fb4da199226f0834fa5: verified-content-v1-0113 · 판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T14:58:31.776643Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 / 위치: 보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6
보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함.



자료 검증 verify_b6731e5f1bfdb7d4d2f4: canonical-paper-v2-deecd917 · 판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T14:58:31.927574Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6 / 위치: 보존 파일 objects/sha256/ba/ba3c8f469a0697686d918a4b9d4a5fad94edc8fa9f297c484b1e23f53d137db6
보존 객체는 cookie/landing page이므로 서지 위치만 확인했고 본문 주장을 검증하지 못함.

원본 주소 "https://s3wiki.yonsei.ac.kr/index.php?title=Lesson:fast_on_device_llm_inference_with_npus_deecd917&oldid=2009"