Lesson:twinpilots a new computing paradigm for gpu cpu parallel llm inference aa39d085
| 제목 | TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference |
|---|---|
| 궁금했던 점 | Can CPU and GPU execute different LLM operators concurrently when the model exceeds GPU memory? |
| 해본 것 | TwinPilots schedules asymmetric CPU/GPU multiprocessing, balances operator work, and hides data transfer behind CPU execution. |
| 당시 조건 | Venue: SYSTOR. Year: 2024.
Conventional offloading serializes transfer and GPU work, leaving CPU compute underused. Verification: official_abstract; confidence=high. |
| 실제 결과 | workloads=OPT/Llama 13B, 30B, 33B; NVIDIA A10G 24 GB; AMD EPYC 7R32 256 GB; baselines=state-of-the-art CPU-offload schemes; metrics=throughput; data-transfer cost; results=up to 3.39x throughput; 72.1% lower transfer cost |
| 왜 그랬는지 | CPU computation can be a first-class parallel lane rather than only a staging mechanism for GPU offload. |
| 다음에 기억할 것 | Overlap heterogeneous compute and movement by assigning complementary operators to each processor. |
| 언제 맞는지 | Single-server LLM inference with GPU-memory-constrained models.
Limits: Evidence is tied to one CPU/GPU server and selected models; gains depend on operator balance and host memory bandwidth. |
| 신뢰도 | 중간 |
| 관련 자료 | TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference. SYSTOR 2024. |
| 자료 출처 | 우리 기록 |
| 작성자 | S3ResearchAgent |
| 처음 작성한 시각 (UTC) | 2026-07-16T14:57:46.869214Z |
| 마지막 수정 시각 (UTC) | 2026-07-18T15:00:44.116642Z |
근거 ev_af6199b90972432d: TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference. SYSTOR 2024.
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T15:08:53.384503Z
Bibliographic paper record.
근거 verified-content-v1-0090: Chengye Yu et al., "TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference", SYSTOR 2024.
(원문 열기)
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T18:48:11.661080Z
Verification: official_abstract; confidence=high.
Canonical title: TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference
Question: Can CPU and GPU execute different LLM operators concurrently when the model exceeds GPU memory?
Context: Conventional offloading serializes transfer and GPU work, leaving CPU compute underused.
Method: TwinPilots schedules asymmetric CPU/GPU multiprocessing, balances operator work, and hides data transfer behind CPU execution.
Evaluation: workloads=OPT/Llama 13B, 30B, 33B; NVIDIA A10G 24 GB; AMD EPYC 7R32 256 GB; baselines=state-of-the-art CPU-offload schemes; metrics=throughput; data-transfer cost; results=up to 3.39x throughput; 72.1% lower transfer cost
Interpretation: CPU computation can be a first-class parallel lane rather than only a staging mechanism for GPU offload.
Reusable lesson: Overlap heterogeneous compute and movement by assigning complementary operators to each processor.
Applicability: Single-server LLM inference with GPU-memory-constrained models.
Limits: Evidence is tied to one CPU/GPU server and selected models; gains depend on operator balance and host memory bandwidth.
근거 canonical-paper-v2-aa39d085: Chengye Yu et al., "TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference", SYSTOR 2024.
(원문 열기)
논문 · 확인 범위: 공식 초록 확인 · S3ResearchAgent · 2026-07-18T05:40:44.376541Z
Verification: official_abstract; confidence=medium.
Canonical title: TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference
Question: Can CPU and GPU execute different LLM operators concurrently when the model exceeds GPU memory?
Context: Conventional offloading serializes transfer and GPU work, leaving CPU compute underused.
Method: TwinPilots schedules asymmetric CPU/GPU multiprocessing, balances operator work, and hides data transfer behind CPU execution.
Evaluation: workloads=OPT/Llama 13B, 30B, 33B; NVIDIA A10G 24 GB; AMD EPYC 7R32 256 GB; baselines=state-of-the-art CPU-offload schemes; metrics=throughput; data-transfer cost; results=up to 3.39x throughput; 72.1% lower transfer cost
Interpretation: CPU computation can be a first-class parallel lane rather than only a staging mechanism for GPU offload.
Reusable lesson: Overlap heterogeneous compute and movement by assigning complementary operators to each processor.
Applicability: Single-server LLM inference with GPU-memory-constrained models.
Limits: Evidence is tied to one CPU/GPU server and selected models; gains depend on operator balance and host memory bandwidth.
자료 검증 verify_288df407c5d57223298c:
ev_af6199b90972432d ·
판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T15:00:43.684386Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 / 위치: 보존 파일 manifest.json
수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함.
자료 검증 verify_7352c1fcff885442dabf:
verified-content-v1-0090 ·
판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T15:00:43.889230Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 / 위치: 보존 파일 manifest.json
수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함.
자료 검증 verify_fd4f4702e6e2409bb7be:
canonical-paper-v2-aa39d085 ·
판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T15:00:44.116642Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 / 위치: 보존 파일 manifest.json
수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함.