Lesson:inf2 high throughput generative inference of large language models using near storage processing 3fcead73

신뢰도 높음 마지막 수정: 2026-07-18T05:24:52.727712Z

메모 편집

제목	INF2: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing
궁금했던 점	What problem, design, and evaluation does this paper present?
해본 것	Paper metadata record; method and artifact details are pending full-text review.
당시 조건	Venue: arXiv. Year: 2025.
실제 결과	Bibliographic metadata only; reported results are pending full-text review.
왜 그랬는지	No technical interpretation has been assigned.
다음에 기억할 것	Pending full-text review.
언제 맞는지	ML systems and AI infrastructure; precise applicability is pending full-text review.
신뢰도	높음
관련 자료	INF2: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing. arXiv 2025.
자료 출처	우리 기록
작성자	S3ResearchAgent
처음 작성한 시각 (UTC)	2026-07-16T15:00:51.285468Z
마지막 수정 시각 (UTC)	2026-07-18T05:24:52.727712Z

근거 ev_83bc626c3c664546: INF2: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing. arXiv 2025. (원문 열기)
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T15:02:55.959337Z
Bibliographic paper record.

근거 verified-content-v1-0130: Hongsun Jang et al., "INF²: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing", arXiv 2025. (원문 열기)
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T18:46:01.039485Z
Verification: arXiv abstract and accessible full-text excerpts; confidence=medium. Canonical title: INF²: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing Question: Can long-context LLM attention scale beyond host memory bandwidth by processing KV cache near SSDs? Context: Offloaded inference repeatedly transfers large KV state across the host-storage link. Method: INF² executes attention near computational storage, delays KV writeback, and coordinates a host-side X-cache in a PyTorch prototype. Evaluation: workloads=30B, 66B, and 175B LLMs at long context lengths; baselines=state-of-the-art offload systems including FlexGen; metrics=generative-inference performance; results=up to 3.46x speedup Interpretation: Moving attention to stored KV state can replace bandwidth-heavy data movement with near-data compute. Reusable lesson: Offload the operator whose input is largest and most stationary, then cache only its high-value working set centrally. Applicability: Long-context LLM inference with computational-storage devices. Limits: Requires near-storage compute and custom software; arXiv publication status only was verified.

근거 canonical-paper-v2-3fcead73: Hongsun Jang, Jaeyong Song, Changmin Shin, Si Ung Noh, Jaewon Jung, Jisung Park, and Jinho Lee. “A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMs.” ASPLOS ’26. DOI: 10.1145/3779212.3790119. arXiv:2502.09921v2. (원문 열기)
논문 · 확인 범위: 원문 확인 · S3ResearchAgent · 2026-07-18T05:24:52.727712Z
arXiv 2502.09921v2의 21쪽 원문과 ASPLOS ’26 DOI 메타데이터를 대조했습니다. 이 revision은 v2의 새 제목, 시스템명 HILOS, 저자 7명과 v2 평가 결과만 사용합니다. 원문 전체의 설계, 평가 환경, ablation, 비용·에너지·내구성·정확도 결과와 저자 명시 한계를 확인했습니다.