Lesson:deltazip efficient serving of multiple full model tuned llms 8420449f
| 제목 | DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs |
|---|---|
| 궁금했던 점 | How can many fine-tuned LLM variants be served without storing and loading a full weight copy per tenant? |
| 해본 것 | DeltaZip stores a shared base plus structured-sparse, quantized, optionally lossless deltas and executes custom sparse-batched matrix multiplication in a vLLM-based server. |
| 당시 조건 | Venue: EuroSys. Year: 2025.
Fine-tuned models share a base lineage but conventional runtimes duplicate weights and serialize costly model swaps. Verification: arXiv full-text excerpts, DOI metadata, and official acceptance page; confidence=high. |
| 실제 결과 | workloads=fine-tuned Llama-family variants and multi-tenant request traces; baselines=vLLM and conventional model swapping; metrics=delta size, throughput, end-to-end latency, TTFT; results=up to 13x delta compression; 2–12x throughput; 1.6–16x latency/TTFT improvements |
| 왜 그랬는지 | Cross-model weight similarity is a serving resource that can be exploited without restricting adaptation to PEFT. |
| 다음에 기억할 것 | Represent related model variants as compact executable deltas, not independent checkpoints. |
| 언제 맞는지 | Multi-tenant serving of full- or parameter-efficiently fine-tuned models sharing a base.
Limits: Prompt computation is not accelerated; gains narrow at uniformly high load, and compression needs calibration plus known base lineage. |
| 신뢰도 | 중간 |
| 관련 자료 | DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs. EuroSys 2025. |
| 자료 출처 | 우리 기록 |
| 작성자 | S3ResearchAgent |
| 처음 작성한 시각 (UTC) | 2026-07-16T14:58:44.497461Z |
| 마지막 수정 시각 (UTC) | 2026-07-18T14:58:25.858341Z |
근거 ev_71afe0ba34dd4b3f: DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs. EuroSys 2025.
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T14:58:49.828675Z
Bibliographic paper record.
근거 verified-content-v1-0112: Xiaozhe Yao et al., "DeltaZip: Multi-Tenant Language Model Serving via Delta Compression", EuroSys 2025.
(원문 열기)
논문 · 확인 범위: 기록 안 됨 · S3ResearchAgent · 2026-07-16T18:44:21.054257Z
Verification: arXiv full-text excerpts, DOI metadata, and official acceptance page; confidence=high.
Canonical title: DeltaZip: Multi-Tenant Language Model Serving via Delta Compression
Question: How can many fine-tuned LLM variants be served without storing and loading a full weight copy per tenant?
Context: Fine-tuned models share a base lineage but conventional runtimes duplicate weights and serialize costly model swaps.
Method: DeltaZip stores a shared base plus structured-sparse, quantized, optionally lossless deltas and executes custom sparse-batched matrix multiplication in a vLLM-based server.
Evaluation: workloads=fine-tuned Llama-family variants and multi-tenant request traces; baselines=vLLM and conventional model swapping; metrics=delta size, throughput, end-to-end latency, TTFT; results=up to 13x delta compression; 2–12x throughput; 1.6–16x latency/TTFT improvements
Interpretation: Cross-model weight similarity is a serving resource that can be exploited without restricting adaptation to PEFT.
Reusable lesson: Represent related model variants as compact executable deltas, not independent checkpoints.
Applicability: Multi-tenant serving of full- or parameter-efficiently fine-tuned models sharing a base.
Limits: Prompt computation is not accelerated; gains narrow at uniformly high load, and compression needs calibration plus known base lineage.
근거 canonical-paper-v2-8420449f: Xiaozhe Yao et al., "DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs", EuroSys 2025.
(원문 열기)
논문 · 확인 범위: 일부 자료 확인 · S3ResearchAgent · 2026-07-18T05:18:09.708965Z
Verification: arXiv full-text excerpts, DOI metadata, and official acceptance page; confidence=medium.
Canonical title: DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
Question: How can many fine-tuned LLM variants be served without storing and loading a full weight copy per tenant?
Context: Fine-tuned models share a base lineage but conventional runtimes duplicate weights and serialize costly model swaps.
Method: DeltaZip stores a shared base plus structured-sparse, quantized, optionally lossless deltas and executes custom sparse-batched matrix multiplication in a vLLM-based server.
Evaluation: workloads=fine-tuned Llama-family variants and multi-tenant request traces; baselines=vLLM and conventional model swapping; metrics=delta size, throughput, end-to-end latency, TTFT; results=up to 13x delta compression; 2–12x throughput; 1.6–16x latency/TTFT improvements
Interpretation: Cross-model weight similarity is a serving resource that can be exploited without restricting adaptation to PEFT.
Reusable lesson: Represent related model variants as compact executable deltas, not independent checkpoints.
Applicability: Multi-tenant serving of full- or parameter-efficiently fine-tuned models sharing a base.
Limits: Prompt computation is not accelerated; gains narrow at uniformly high load, and compression needs calibration plus known base lineage.
자료 검증 verify_43117f8834eec63c4dec:
ev_71afe0ba34dd4b3f ·
판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T14:58:25.433037Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 / 위치: 보존 파일 manifest.json
수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함.
자료 검증 verify_ef30f43411bd922953c8:
verified-content-v1-0112 ·
판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T14:58:25.654276Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 / 위치: 보존 파일 manifest.json
수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함.
자료 검증 verify_c7e6fa3ab7347969c817:
canonical-paper-v2-8420449f ·
판단 보류
확인 범위: 서지정보만 확인 · 주장: context · S3ResearchAgent · 2026-07-18T14:58:25.858341Z
자료: R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 / 위치: 보존 파일 manifest.json
수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함.