ai-infra-book alternatives

bojieli/ai-infra-book (6.5k stars, +196 this week): Open-source book that derives LLM inference and training system design from hardware limits, with calculation tools. Why it matters: useful plumbing for shipping LLM-powered features.

Below are 12 other LLM tooling repos tracked by OSS Radar that share at least two GitHub topics with ai-infra-book, ordered by how many topics overlap, then by stars gained in the last 7 days. Topic overlap means a similar problem space, not a drop-in replacement, so check each project's README.

1. 1. mw00/project-maya

   +736/wkest.

   Project Maya - run GLM-5.3-Flash (321B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Why it matters: useful plumbing for shipping LLM-powered features.

   766 stars · C++ · shared topics: llm, llm-inference, mixture-of-experts

2. 2. vllm-project/vllm

   +56/wkest.

   A high-throughput and memory-efficient inference and serving engine for LLMs. Why it matters: useful plumbing for shipping LLM-powered features.

   94k stars · Python · shared topics: deepseek, llm, llm-serving

3. 3. Ebony-Vinyl/dsh-our-free-model

   +4.2k/wkest.

   All you do is install this plugin in dsh: no login, no sign-up, no API key — the frontier models are just there, DeepSeek V4.1 Flash and… Why it matters: useful plumbing for shipping LLM-powered features.

   15k stars · JavaScript · shared topics: deepseek, llm

4. 4. DSH-APP/DSHA

   +210/wkest.

   Android app that runs the DeepSeek Harness agent on phones in a bundled Ubuntu environment, without root or Termux. Why it matters: useful plumbing for shipping LLM-powered features.

   1.2k stars · Java · shared topics: deepseek, llm

5. 5. ollama/ollama

   +168/wkest.

   Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. Why it matters: useful plumbing for shipping LLM-powered features.

   183k stars · Go · shared topics: deepseek, llm

6. 6. langgenius/dify

   +126/wkest.

   Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Why it matters: useful plumbing for shipping LLM-powered features.

   158k stars · TypeScript · shared topics: deepseek, llm

7. 7. unslothai/unsloth

   +84/wkest.

   Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more. Why it matters: useful plumbing for shipping LLM-powered features.

   78k stars · Python · shared topics: deepseek, llm

8. 8. lidge-jun/opencodex

   +84/wkest.

   Universal provider proxy for OpenAI Codex & Claude Code — use any LLM (Claude, Gemini, Grok, DeepSeek, Ollama…) with Codex CLI, App, SDK… Why it matters: useful plumbing for shipping LLM-powered features.

   17k stars · TypeScript · shared topics: deepseek, llm

9. 9. sgl-project/sglang

   +77/wkest.

   SGLang is a high-performance serving framework for large language models and multimodal models. Why it matters: useful plumbing for shipping LLM-powered features.

   37k stars · Python · shared topics: deepseek, llm

10. 10. decolua/9router

    +77/wkest.

    Unlimited FREE AI coding. Connect Claude Code, Codex, Cursor, Cline, Copilot, Antigravity to FREE Claude/GPT/Gemini via 40+ providers. Why it matters: useful plumbing for shipping LLM-powered features.

    31k stars · JavaScript · shared topics: deepseek, llm

11. 11. semantica-agi/semantica

    +77/wkest.

    Graph-Native Infrastructure for Context and Accountable AI Systems. Why it matters: useful plumbing for shipping LLM-powered features.

    14k stars · Python · shared topics: ai-infrastructure, llm

12. 12. huggingface/transformers

    +70/wkest.

    🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models… Why it matters: useful plumbing for shipping LLM-powered features.

    168k stars · Python · shared topics: deepseek, llm

More: all trending LLM tooling repos · compare ai-infra-book with the top 2