속성:Evidence note
자료에서 확인할 수 있는 점이나 한계를 적습니다.
i
inf2 high throughput generative inference of large language models using near storage processing 3fcead73 +
Bibliographic paper record. +
inf2 high throughput generative inference of large language models using near storage processing 3fcead73 +
arXiv 2502.09921v2의 21쪽 원문과 ASPLOS ’26 DOI 메타데이터를 대조했습니다. 이 revision은 v2의 새 제목, 시스템명 HILOS, 저자 7명과 v2 평가 결과만 사용합니다. 원문 전체의 설계, 평가 환경, ablation, 비용·에너지·내구성·정확도 결과와 저자 명시 한계를 확인했습니다. +
inf2 high throughput generative inference of large language models using near storage processing 3fcead73 +
Verification: arXiv abstract and accessible full-text excerpts; confidence=medium.
Canonical title: INF²: High-Throughput Generative Inference of Large Language Models using Near-Storage Processing
Question: Can long-context LLM attention scale beyond host memory bandwidth by processing KV cache near SSDs?
Context: Offloaded inference repeatedly transfers large KV state across the host-storage link.
Method: INF² executes attention near computational storage, delays KV writeback, and coordinates a host-side X-cache in a PyTorch prototype.
Evaluation: workloads=30B, 66B, and 175B LLMs at long context lengths; baselines=state-of-the-art offload systems including FlexGen; metrics=generative-inference performance; results=up to 3.46x speedup
Interpretation: Moving attention to stored KV state can replace bandwidth-heavy data movement with near-data compute.
Reusable lesson: Offload the operator whose input is largest and most stationary, then cache only its high-value working set centrally.
Applicability: Long-context LLM inference with computational-storage devices.
Limits: Requires near-storage compute and custom software; arXiv publication status only was verified. +
infinigen efficient generative inference of large language models with dynamic kv cache manageme a1ca3228 +
Bibliographic paper record. +
infinigen efficient generative inference of large language models with dynamic kv cache manageme a1ca3228 +
Verification: abstract_only; confidence=medium.
Canonical title: InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management
Question: 긴 문맥 LLM의 KV 캐시를 정확도 손실 적게 동적으로 줄여 offload 추론을 가속할 수 있는가?
Context: 모든 KV를 GPU에 두기 어렵고 CPU에서 전부 가져오면 interconnect가 병목이다.
Method: 다음 layer attention을 일부 query weight와 KV로 추정해 필수 entry만 prefetch하고 동적 cache pool을 운용한다.
Evaluation: workloads=representative LLMs; KV-offloading inference system; baselines=prior KV cache management methods; metrics=inference speed; accuracy; results=Up to 3.00× speedup with higher accuracy.
Interpretation: attention sparsity를 layer 간 예측하면 전송할 KV를 선택할 수 있다.
Reusable lesson: KV offload는 정적 압축보다 사용 직전 중요도 예측과 prefetch를 결합하라.
Applicability: 긴 문맥 LLM CPU-GPU offload 추론.
Limits: attention 예측 정확도, CPU-GPU 링크, 선정 모델·시퀀스에 민감하다. +
infinigen efficient generative inference of large language models with dynamic kv cache manageme a1ca3228 +
Verification: abstract_only; confidence=high.
Canonical title: InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management
Question: 긴 문맥 LLM의 KV 캐시를 정확도 손실 적게 동적으로 줄여 offload 추론을 가속할 수 있는가?
Context: 모든 KV를 GPU에 두기 어렵고 CPU에서 전부 가져오면 interconnect가 병목이다.
Method: 다음 layer attention을 일부 query weight와 KV로 추정해 필수 entry만 prefetch하고 동적 cache pool을 운용한다.
Evaluation: workloads=representative LLMs; KV-offloading inference system; baselines=prior KV cache management methods; metrics=inference speed; accuracy; results=Up to 3.00× speedup with higher accuracy.
Interpretation: attention sparsity를 layer 간 예측하면 전송할 KV를 선택할 수 있다.
Reusable lesson: KV offload는 정적 압축보다 사용 직전 중요도 예측과 prefetch를 결합하라.
Applicability: 긴 문맥 LLM CPU-GPU offload 추론.
Limits: attention 예측 정확도, CPU-GPU 링크, 선정 모델·시퀀스에 민감하다. +
Verification: metadata_only; confidence=low.
Canonical title: Intel Accelerator Ecosystem: An SoC-Oriented Perspective
Question:
Context:
Method:
Evaluation: workloads=; baselines=; metrics=; results=
Interpretation:
Reusable lesson:
Applicability: Intel SoC accelerator ecosystem; technical content not independently verified.
Limits: Primary conference and author pages verify metadata but expose no abstract or paper content in the accessible record. +
Bibliographic paper record. +
Verification: metadata_only; confidence=low.
Canonical title: Intel Accelerator Ecosystem: An SoC-Oriented Perspective
Question:
Context:
Method:
Evaluation: workloads=; baselines=; metrics=; results=
Interpretation:
Reusable lesson:
Applicability: Intel SoC accelerator ecosystem; technical content not independently verified.
Limits: Primary conference and author pages verify metadata but expose no abstract or paper content in the accessible record. +
Verification: full_text; confidence=high.
Canonical title: IONIA: High-Performance Replication for Modern Disk-based KV Stores
Question: SSD KV 저장소의 순서 보장을 완화하지 않고 복제 병목을 줄일 수 있는가?
Context: 모든 쓰기를 즉시 외부화·순서화하면 SSD 내부 병렬성과 follower read를 제한한다.
Method: 외부화되지 않은 쓰기의 순서를 지연하고 병렬 실행하며, leader 메타 질의와 bounded history로 follower read freshness를 판정한다.
Evaluation: workloads=YCSB; SplinterDB; RocksDB; baselines=MultiPaxos; leader-read variant; Skyros/parallel protocols; metrics=throughput; read RTT; history size; results=1.8× vs leader-read; 16–38× throughput vs MultiPaxos; most reads in 1 RTT.
Interpretation: 저장장치의 WO-KV 성질을 복제 프로토콜에 노출하면 불필요한 직렬화를 없앨 수 있다.
Reusable lesson: 복제 순서 제약을 저장 엔진이 실제 외부화하는 상태에 맞춰라.
Applicability: SSD 기반 복제 KV 저장소.
Limits: WO-KV 의미론과 bounded history에 의존하며 일부 읽기는 추가 왕복으로 fallback한다. +
Verification: full_text; confidence=high.
Canonical title: IONIA: High-Performance Replication for Modern Disk-based KV Stores
Question: SSD KV 저장소의 순서 보장을 완화하지 않고 복제 병목을 줄일 수 있는가?
Context: 모든 쓰기를 즉시 외부화·순서화하면 SSD 내부 병렬성과 follower read를 제한한다.
Method: 외부화되지 않은 쓰기의 순서를 지연하고 병렬 실행하며, leader 메타 질의와 bounded history로 follower read freshness를 판정한다.
Evaluation: workloads=YCSB; SplinterDB; RocksDB; baselines=MultiPaxos; leader-read variant; Skyros/parallel protocols; metrics=throughput; read RTT; history size; results=1.8× vs leader-read; 16–38× throughput vs MultiPaxos; most reads in 1 RTT.
Interpretation: 저장장치의 WO-KV 성질을 복제 프로토콜에 노출하면 불필요한 직렬화를 없앨 수 있다.
Reusable lesson: 복제 순서 제약을 저장 엔진이 실제 외부화하는 상태에 맞춰라.
Applicability: SSD 기반 복제 KV 저장소.
Limits: WO-KV 의미론과 bounded history에 의존하며 일부 읽기는 추가 왕복으로 fallback한다. +
Bibliographic paper record. +
k
kal kernel assisted non invasive memory leak tolerance with a general purpose memory allocator 372cd042 +
Source: Yonsei University Computer Systems Laboratory publication list supplied by the user. Manifestation 1 of 1. +
kal kernel assisted non invasive memory leak tolerance with a general purpose memory allocator 372cd042 +
정본 Lesson 보강 근거. 검토 원본: Lesson:technical_review_kal_kernel_assisted_non_invasive_memory_leak_tolerance_with_a_general_purpose_m_e42adf12. 확인 범위: official_abstract. 확인한 자료: https://doi.org/10.1002/spe.970 ; https://yonsei.elsevierpure.com/en/publications/kal-kernel-assisted-non-invasive-memory-leak-tolerance-with-a-gen/. 질문, 방법, 평가, 해석, 재사용 교훈, 적용 범위와 한계를 같은 Lesson 본문에 통합했습니다. +
Verification: official_abstract; confidence=high.
Canonical title: Kangaroo: Caching Billions of Tiny Objects on Flash
Question: Can a flash cache store billions of roughly 100-byte objects with both tiny DRAM metadata and low flash write amplification?
Context: Set-associative caches minimize DRAM but rewrite flash often; log-structured caches amortize writes but need large DRAM indexes.
Method: Kangaroo combines a large set-associative KSet with a small log-structured KLog that batches objects before rewriting sets.
Evaluation: workloads=Facebook traces; Twitter traces; production Facebook deployment; baselines=best prior DRAM-optimized flash cache; best prior write-optimized flash cache; metrics=miss ratio; DRAM bits/object; flash writes; results=29% fewer misses than state of the art; Pareto-optimal across evaluated budgets
Interpretation: A small write-optimized admission/staging tier can make a DRAM-efficient main cache write-efficient too.
Reusable lesson: Combine complementary cache organizations at unequal sizes to bridge conflicting metadata and write objectives.
Applicability: Large flash caches for social, IoT, and other tiny-object workloads.
Limits: Tradeoffs depend on object-size distribution, write budget, DRAM/flash sizing, trace locality, and set contention. +
Bibliographic paper record. +
Verification: official_abstract; confidence=medium.
Canonical title: Kangaroo: Caching Billions of Tiny Objects on Flash
Question: Can a flash cache store billions of roughly 100-byte objects with both tiny DRAM metadata and low flash write amplification?
Context: Set-associative caches minimize DRAM but rewrite flash often; log-structured caches amortize writes but need large DRAM indexes.
Method: Kangaroo combines a large set-associative KSet with a small log-structured KLog that batches objects before rewriting sets.
Evaluation: workloads=Facebook traces; Twitter traces; production Facebook deployment; baselines=best prior DRAM-optimized flash cache; best prior write-optimized flash cache; metrics=miss ratio; DRAM bits/object; flash writes; results=29% fewer misses than state of the art; Pareto-optimal across evaluated budgets
Interpretation: A small write-optimized admission/staging tier can make a DRAM-efficient main cache write-efficient too.
Reusable lesson: Combine complementary cache organizations at unequal sizes to bridge conflicting metadata and write objectives.
Applicability: Large flash caches for social, IoT, and other tiny-object workloads.
Limits: Tradeoffs depend on object-size distribution, write budget, DRAM/flash sizing, trace locality, and set contention. +
Bibliographic paper record. +
Verification: author_preprint_full_text; confidence=medium.
Canonical title: Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
Question: How can offloaded MoE inference hide expert-weight I/O when individual expert computation is too short to cover transfers?
Context: Adding batches lengthens compute but activates more experts, which can increase I/O and create new pipeline bubbles.
Method: Klotski orchestrates an expert-aware multi-batch pipeline using a constraint-sensitive I/O/compute planner and correlation-aware expert prefetching.
Evaluation: workloads=offloaded MoE inference; WikiText-2 for expert-correlation profiling; baselines=Hugging Face Accelerate; DeepSpeed-FastGen; FlexGen; MoE-Infinity; Fiddler; metrics=throughput; latency; pipeline bubbles; results=up to 85.12x throughput improvement
Interpretation: Batch order and expert activation correlation must be optimized jointly with offload scheduling.
Reusable lesson: For sparse models, schedule batches by the union and sequence of resources they activate, not batch size alone.
Applicability: Memory-constrained, CPU/disk-offloaded MoE batch inference.
Limits: Preprint-only record; gains are hardware/model/batch dependent, and more batches increase KV-cache load and latency. +
Verification: author_preprint_full_text; confidence=medium.
Canonical title: Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
Question: How can offloaded MoE inference hide expert-weight I/O when individual expert computation is too short to cover transfers?
Context: Adding batches lengthens compute but activates more experts, which can increase I/O and create new pipeline bubbles.
Method: Klotski orchestrates an expert-aware multi-batch pipeline using a constraint-sensitive I/O/compute planner and correlation-aware expert prefetching.
Evaluation: workloads=offloaded MoE inference; WikiText-2 for expert-correlation profiling; baselines=Hugging Face Accelerate; DeepSpeed-FastGen; FlexGen; MoE-Infinity; Fiddler; metrics=throughput; latency; pipeline bubbles; results=up to 85.12x throughput improvement
Interpretation: Batch order and expert activation correlation must be optimized jointly with offload scheduling.
Reusable lesson: For sparse models, schedule batches by the union and sequence of resources they activate, not batch size alone.
Applicability: Memory-constrained, CPU/disk-offloaded MoE batch inference.
Limits: Preprint-only record; gains are hardware/model/batch dependent, and more batches increase KV-cache load and latency. +