Tools for running AI models locally in 2026

Commercial tools for running AI models locally

1. 1
   LM Studio Element Labs

   Bionic is LM Studio's agent for work and code. Create documents, slides, PDFs, and software with local or frontier open models.

   Complete system Usable You want to run open models privately on your own machine without command-line setup.

   Commercial

New open-source projects

1. 1
   Kaggle-tpu-lab ARahim3

   Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.

   Template or starter Experimental You want to experiment with big open models in a coding agent without paying for GPUs.

   ★ 758 MIT Python Active, pushed 27 days ago
2. 2
   Resin ekzhang

   🧱 Experimental library for building inference engines

   Building block Experimental You are building or researching a custom inference engine for specific hardware.

   ★ 272 Apache-2.0 Python Active, pushed 5 days ago

Open-source tools for running AI models locally

Complete systems you can adopt 4

1. 1
   AnythingLLM Mintplex-Labs

   Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

   Complete system Usable You want private document chat for a small team without building a RAG stack yourself.

   ★ 68k MIT JavaScript Active, pushed today
2. 2
   LocalAI mudler

   LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

   Complete system Usable You want a drop-in local replacement for OpenAI's API covering more than just text.

   ★ 50k MIT Go Active, pushed today
3. 3
   RikkaHub rikkahub

   RikkaHub is an Android APP that supports for multiple LLM providers.

   Complete system Usable You want one Android app for chatting with several AI providers using your own API keys.

   ★ 8.6k AGPL-3.0 Kotlin Active, pushed today
4. 4
   GPUStack gpustack

   A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

   Complete system Usable An organisation pools its own GPUs and wants a control panel for serving models to internal teams.

   ★ 6k Apache-2.0 Python Active, pushed today

Frameworks and platforms to build on 1

1. 1
   BentoML bentoml

   The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

   Framework or platform Production-ready An ML team needs to serve custom models as production APIs without writing serving infrastructure.

   ★ 8.9k Apache-2.0 Python Active, pushed today

Building blocks for your own system 3

1. 1
   Llama.cpp ggml-org

   LLM inference in C/C++

   Building block Production-ready You need efficient local or edge LLM inference and want the engine many other tools are built on.

   ★ 133k MIT C++ Active, pushed today
2. 2
   vLLM vllm-project

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Building block Production-ready You serve open-weight models to many users on your own GPUs and need throughput.

   ★ 96k Apache-2.0 Python Active, pushed today
3. 3
   FastDeploy PaddlePaddle

   High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

   Building block Usable You deploy ERNIE or other PaddlePaddle models, especially on Chinese hardware.

   ★ 3.7k Apache-2.0 Python Active, pushed 39 days ago

Developer tools 2

1. 1
   Ollama ollama

   Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

   Developer tool Production-ready You want to run local LLMs for development, privacy or offline use with minimal setup.

   ★ 186k MIT Go Active, pushed today
2. 2
   OpenLLM bentoml

   Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

   Developer tool Usable You want a quick way to stand up an open model behind an OpenAI-style API.

   ★ 13k Apache-2.0 Python Active, pushed today

All 30 open-source local AI projects

Compared head to head

Looking for an alternative to

Open-source licenses

• Permissive: 29. Use it anywhere, including in closed products. You keep the copyright notice; Apache-2.0 also grants patent rights.
• Network copyleft: 1. Like GPL, and running a changed version as a web service counts as distributing it, so its users get the source too.

Languages

Python 17, TypeScript 4, C++ 3, Go 2, JavaScript 2, Kotlin 1 and Rust 1.