vLLM

A high-throughput library for LLM inference and serving that utilizes PagedAttention to optimize GPU memory usage.

Explore on Agidapp Global