vllm alternatives

vllm-project/vllm (94k stars, +70 this week): A high-throughput and memory-efficient inference and serving engine for LLMs. Why it matters: useful plumbing for shipping LLM-powered features.

Below are 12 other LLM tooling repos tracked by OSS Radar that share at least two GitHub topics with vllm, ordered by how many topics overlap, then by stars gained in the last 7 days. Topic overlap means a similar problem space, not a drop-in replacement, so check each project's README.

1. 1. sgl-project/sglang

   +98/wkest.

   SGLang is a high-performance serving framework for large language models and multimodal models. Why it matters: useful plumbing for shipping LLM-powered features.

   37k stars · Python · shared topics: blackwell, cuda, deepseek, gpt-oss, inference, llama, llm, moe, qwen, transformer

2. 2. hiyouga/LlamaFactory

   +70/wkest.

   Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024). Why it matters: useful plumbing for shipping LLM-powered features.

   75k stars · Python · shared topics: deepseek, gpt, llama, llm, moe, qwen

3. 3. ollama/ollama

   +238/wkest.

   Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. Why it matters: useful plumbing for shipping LLM-powered features.

   183k stars · Go · shared topics: deepseek, gpt-oss, llama, llm, qwen

4. 4. unslothai/unsloth

   +119/wkest.

   Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more. Why it matters: useful plumbing for shipping LLM-powered features.

   78k stars · Python · shared topics: deepseek, llama, llm, openai, qwen

5. 5. huggingface/transformers

   +112/wkest.

   🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models… Why it matters: useful plumbing for shipping LLM-powered features.

   168k stars · Python · shared topics: deepseek, llm, pytorch, qwen, transformer

6. 6. diegosouzapw/OmniRoute

   +658/wkest.

   Never stop coding. Free MIT AI gateway: one endpoint, 359 providers (150+ free), 1200+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek… Why it matters: useful plumbing for shipping LLM-powered features.

   76k stars · TypeScript · shared topics: deepseek, kimi, openai, qwen

7. 7. langgenius/dify

   +147/wkest.

   Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Why it matters: useful plumbing for shipping LLM-powered features.

   158k stars · TypeScript · shared topics: deepseek, gpt, llm, openai

8. 8. AstrBotDevs/AstrBot

   +98/wkest.

   AI Agent Assistant & development framework that integrates lots of IM platforms, LLMs, plugins and AI feature, and can be your openclaw… Why it matters: useful plumbing for shipping LLM-powered features.

   42k stars · Python · shared topics: gpt, llama, llm, openai

9. 9. decolua/9router

   +98/wkest.

   Unlimited FREE AI coding. Connect Claude Code, Codex, Cursor, Cline, Copilot, Antigravity to FREE Claude/GPT/Gemini via 40+ providers. Why it matters: useful plumbing for shipping LLM-powered features.

   31k stars · JavaScript · shared topics: deepseek, llm, openai, qwen

10. 10. dyad-sh/dyad

    +49/wkest.

    Local, open-source AI app builder for power users ✨ v0 / Lovable / Replit / Bolt alternative 🌟 Star if you like it! Why it matters: useful plumbing for shipping LLM-powered features.

    22k stars · TypeScript · shared topics: deepseek, llm, openai, qwen

11. 11. labring/FastGPT

    +35/wkest.

    FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data… Why it matters: useful plumbing for shipping LLM-powered features.

    30k stars · TypeScript · shared topics: deepseek, llm, openai, qwen

12. 12. FlashML-org/FreeToken

    +35/wkest.

    FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently. Why it matters: useful plumbing for shipping LLM-powered features.

    14k stars · Python · shared topics: inference, moe, qwen, transformer

More: all trending LLM tooling repos · compare vllm with the top 2