OSS Radar
wici-ai avatar

wici-ai/inferssd

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp. Why it matters: helps you run models on your own hardware without an API bill.

GitHub description: Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

Stars

1.2k

Last 24h

+48

Last 7 days

+813

Forks

99

  • llama-modchipsynchopate/llama-modchip
    +494 stars this week

    Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB…

  • project-mayamw00/project-maya
    +199 stars this week

    Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one…

  • PolyStratasimplexai-labs/PolyStrata
    +132 stars this week

    Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith…

  • Atomic AgentAtomicBot-ai/atomic-agent
    +343 stars this week

    Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.

Star history

GitHub stars over time. A steeper line means the repo is gaining attention faster.

Last 21 hours · +727 stars

Hourly star count for wici-ai/inferssd1,184457Nov 12, 3 AM UTCNov 16, 2 AM UTC

Daily, since Nov 2, 2026 · 258 → 1.2k

Daily star count for wici-ai/inferssd1,184258Nov 2, 2026Nov 16, 2026

Discussed on Hacker News

Hacker News is a popular tech discussion site. 1 story links to this repo, with 1 upvotes (points) in total.

Inferssd – Run models larger than RAM or VRAM by streaming MoE experts from SSD1 points · 0 comments · Oct 30, 2026

Maintainer? Add the badge to your README

Shows your live OSS Radar rank and links back here. Updates hourly. How the badge works.

OSS Radar badge preview
[![Trending on OSS Radar](https://ossradar.net/badge/wici-ai/inferssd)](https://ossradar.net/repo/wici-ai/inferssd)
<a href="https://ossradar.net/repo/wici-ai/inferssd"><img src="https://ossradar.net/badge/wici-ai/inferssd" alt="Trending on OSS Radar"></a>

About inferssd

wici-ai/inferssd is a C++ project in the Local LLMs category, created 25 days ago. It has 1.2k GitHub stars and 99 forks, and gained +813 stars over the last 7 days, ranking #2 of 10 Local LLMs repos tracked by OSS Radar this week.

See more trending Local LLMs repos or the top AI repos this week. Looking for something similar? See inferssd alternatives.

Language
C++
Category
Local LLMs
License
Apache-2.0
Created
Oct 22, 2026
Last code push
today (Nov 15, 2026)
Latest release
v0.1.4 6 days ago
  • apple silicon
  • CUDA
  • GGUF
  • inference
  • llama.cpp
  • LLM
  • metal
  • mixture of experts
  • ssd

Rising in Local LLMs this week

  • yogurtFrigadeHQ/yogurt
    +1.7k stars this week

    Private AI meeting notes that stay on your computer. A free, open-source alternative to Granola for Mac, Windows and Linux.

  • smolmaxxinghyperionaidev/smolmaxxing
    +748est. stars this week

    maxxing the ability to run different LLMs on the smallest devices possible.

  • Ollamaollama/ollama
    +675 stars this week

    Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

  • MACaQhuawei-csl/MACaQ
    +314 stars this week

    SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp…

  • MeetilyZackriya-Solutions/meetily
    +153 stars this week

    Privacy-first AI meeting note taker for Windows, macOS and Linux. Free, open source (MIT).

  • CapriCoreCapriChooms/CapriCore
    +134est. stars this week

    Capri — an AI desktop companion for Windows with local/API models, Hugging Face, optional voice, memory and permission-based desktop tools.

inferssd is #190 of 450 on today's leaderboard.