Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

Ultra and Unified CCL

kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference.

(ACL 2025 oral) SCOPE: Optimizing KV Cache Compression in Long-context Generation

PiKV: KV Cache Management System for Mixture of Experts [Efficient ML System]

Span Queries: What if we had a way to plan and optimize GenAI like we do for SQL?

This project implements an Emotion-Aware Music Generator (EAMG) that turns natural-language prompts into emotion-aligned music in real time. It uses a LoRA-tuned DistilBERT to classify emotions, maps them to musical parameters using music theory, and generates MIDI via a transformer model with KV caching for low-latency output.

[SIGMOD 2025] PQCache: Product Quantization-based KVCache for Long Context LLM Inference