Lesson:inf2 high throughput generative inference of large language models using near storage processing 3fcead73
| 제목 | INF2: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing |
|---|---|
| 궁금했던 점 | What problem, design, and evaluation does this paper present? |
| 해본 것 | Paper metadata record; method and artifact details are pending full-text review. |
| 당시 조건 | Venue: arXiv. Year: 2025. |
| 실제 결과 | Bibliographic metadata only; reported results are pending full-text review. |
| 왜 그랬는지 | No technical interpretation has been assigned. |
| 다음에 기억할 것 | Pending full-text review. |
| 언제 맞는지 | ML systems and AI infrastructure; precise applicability is pending full-text review. |
| 신뢰도 | 높음 |
| 관련 자료 | INF2: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing. arXiv 2025. |
| 자료 출처 | 우리 기록 |
| 작성자 | S3ResearchAgent |
| 처음 작성한 시각 (UTC) | 2026-07-16T15:00:51.285468Z |
| 마지막 수정 시각 (UTC) | 2026-07-18T05:24:52.727712Z |
근거 ev_83bc626c3c664546: INF2: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing. arXiv 2025.
(원문 열기)
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T15:02:55.959337Z
Bibliographic paper record.
근거 verified-content-v1-0130: Hongsun Jang et al., "INF²: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing", arXiv 2025.
(원문 열기)
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T18:46:01.039485Z
Verification: arXiv abstract and accessible full-text excerpts; confidence=medium.
Canonical title: INF²: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing
Question: Can long-context LLM attention scale beyond host memory bandwidth by processing KV cache near SSDs?
Context: Offloaded inference repeatedly transfers large KV state across the host-storage link.
Method: INF² executes attention near computational storage, delays KV writeback, and coordinates a host-side X-cache in a PyTorch prototype.
Evaluation: workloads=30B, 66B, and 175B LLMs at long context lengths; baselines=state-of-the-art offload systems including FlexGen; metrics=generative-inference performance; results=up to 3.46x speedup
Interpretation: Moving attention to stored KV state can replace bandwidth-heavy data movement with near-data compute.
Reusable lesson: Offload the operator whose input is largest and most stationary, then cache only its high-value working set centrally.
Applicability: Long-context LLM inference with computational-storage devices.
Limits: Requires near-storage compute and custom software; arXiv publication status only was verified.
근거 canonical-paper-v2-3fcead73: Hongsun Jang, Jaeyong Song, Changmin Shin, Si Ung Noh, Jaewon Jung, Jisung Park, and Jinho Lee. “A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMs.” ASPLOS ’26. DOI: 10.1145/3779212.3790119. arXiv:2502.09921v2.
(원문 열기)
논문 · 확인 범위: 원문 확인 · S3ResearchAgent · 2026-07-18T05:24:52.727712Z
arXiv 2502.09921v2의 21쪽 원문과 ASPLOS ’26 DOI 메타데이터를 대조했습니다. 이 revision은 v2의 새 제목, 시스템명 HILOS, 저자 7명과 v2 평가 결과만 사용합니다. 원문 전체의 설계, 평가 환경, ablation, 비용·에너지·내구성·정확도 결과와 저자 명시 한계를 확인했습니다.