ExLlamaV2

An ultra-fast inference library specifically optimized for quantized LLMs on Nvidia GPUs.

Explore on Agidapp Global