简介

1. vLLM

项目地址: https://github.com/vllm-project/vllm

2. SGLang

项目地址: https://github.com/sgl-project/sglang

3. LMDeploy

4. Ollama

5. Llama.cpp

6. TensorRT-LLM

7. Hugging Face TGI

8. MLC-LLM