Alternatives to Claude.ai in 2026

Ollama ollama

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

Developer tool Production-ready You want to run local LLMs for development, privacy or offline use with minimal setup.

★ 186k MIT Go Active, pushed today

Llama.cpp ggml-org

LLM inference in C/C++

Building block Production-ready You need efficient local or edge LLM inference and want the engine many other tools are built on.

★ 133k MIT C++ Active, pushed today

vLLM vllm-project

A high-throughput and memory-efficient inference and serving engine for LLMs

Building block Production-ready You serve open-weight models to many users on your own GPUs and need throughput.

★ 96k Apache-2.0 Python Active, pushed today

AnythingLLM Mintplex-Labs

Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

Complete system Usable You want private document chat for a small team without building a RAG stack yourself.

★ 68k MIT JavaScript Active, pushed today

LocalAI mudler

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

Complete system Usable You want a drop-in local replacement for OpenAI's API covering more than just text.

★ 50k MIT Go Active, pushed today

OpenLLM bentoml

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Developer tool Usable You want a quick way to stand up an open model behind an OpenAI-style API.

★ 13k Apache-2.0 Python Active, pushed today

BentoML bentoml

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Framework or platform Production-ready An ML team needs to serve custom models as production APIs without writing serving infrastructure.

★ 8.9k Apache-2.0 Python Active, pushed today

RikkaHub rikkahub

RikkaHub is an Android APP that supports for multiple LLM providers.

Complete system Usable You want one Android app for chatting with several AI providers using your own API keys.

★ 8.6k AGPL-3.0 Kotlin Active, pushed today

GPUStack gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

Complete system Usable An organisation pools its own GPUs and wants a control panel for serving models to internal teams.

★ 6k Apache-2.0 Python Active, pushed today

FastDeploy PaddlePaddle

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

Building block Usable You deploy ERNIE or other PaddlePaddle models, especially on Chinese hardware.

★ 3.7k Apache-2.0 Python Active, pushed 39 days ago

vLLM Ascend vllm-project

Community maintained hardware plugin for vLLM on Huawei Ascend

Integration Usable You have Huawei Ascend accelerators and want to use vLLM on them.

★ 3k Apache-2.0 Python Active, pushed today

Chitu thu-pacman

High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

Building block Usable You serve large models on a mix of Chinese and NVIDIA hardware.

★ 3k Apache-2.0 Python Active, pushed yesterday

Kvcached ovg-project

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Building block Experimental You run multiple models on shared GPUs with bursty traffic and want better utilisation.

★ 2k Apache-2.0 Python Active, pushed today

InferenceX SemiAnalysisAI

Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

Developer tool Usable You compare inference hardware or serving stacks and want reproducible benchmarks.

★ 2k Apache-2.0 Python Active, pushed today

RTP-LLM alibaba

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

Building block Usable You serve Qwen-family models at scale and want an engine proven inside Alibaba.

★ 1.4k Apache-2.0 Python Active, pushed today

Mosec mosecorg

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

Framework or platform Usable You need to put custom PyTorch or TensorFlow models behind an efficient HTTP API with batching.

★ 903 Apache-2.0 Python Active, pushed today

GPT4All nomic-ai

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Complete system Usable Only if you already use it; newer local-LLM apps are better maintained.

★ 77k MIT C++ Quiet, pushed 17 months ago

Kaggle-tpu-lab ARahim3

Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.

Template or starter Experimental You want to experiment with big open models in a coding agent without paying for GPUs.

★ 758 MIT Python Active, pushed 27 days ago

PegaInfer pegainfer-project

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Building block Experimental You want a lean, single-binary inference server for supported models on NVIDIA GPUs.

★ 730 Apache-2.0 Rust Active, pushed today

Dive OpenAgentPlatform

Dive is an open-source MCP Host Desktop Application that seamlessly integrates with any LLMs supporting function calling capabilities. ✨

Complete system Usable You want to use MCP tools with models other than Claude in a desktop chat app.

★ 1.8k MIT TypeScript Slowing, pushed 3 months ago

Parallax GradientHQ

Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere

Building block Experimental You want to experiment with running large models across several consumer machines.

★ 1.4k Apache-2.0 Python Slowing, pushed 4 months ago

ZhiLight zhihu

A highly optimized LLM inference acceleration engine for Llama and its variants.

Building block Experimental You run Llama or DeepSeek models on PCIe consumer or data-centre GPUs and want to benchmark against vLLM.

★ 906 Apache-2.0 C++ Slowing, pushed 8 months ago

Flama vortico

The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native MCP.

Framework or platform Usable You want one framework to expose scikit-learn, PyTorch or LLM models as production APIs with little code.

★ 304 Apache-2.0 Python Active, pushed today

LLM X mrdjohnson

LLMX; Easiest 3rd party Local LLM UI for the web!

Complete system Usable You run Ollama locally and want a no-install chat interface.

★ 296 MIT TypeScript Active, pushed 35 days ago

Resin ekzhang

🧱 Experimental library for building inference engines

Building block Experimental You are building or researching a custom inference engine for specific hardware.

★ 272 Apache-2.0 Python Active, pushed 5 days ago

Llama.ui PromptPasture

A minimal interface for AI Companion that runs entirely in your browser.

Complete system Usable You run llama.cpp or vLLM and want a lightweight chat front end.

★ 221 MIT TypeScript Active, pushed 8 days ago

ServeGen alibaba

A framework for generating realistic LLM serving workloads

Developer tool Experimental You load-test an LLM serving stack and want traffic patterns closer to production than uniform benchmarks.

★ 191 Apache-2.0 Python Active, pushed 2 months ago

Nextjs-vllm-ui yoziru

Fully-featured, beautiful web interface for vLLM - built with NextJS.

Complete system Usable You serve a model with vLLM and want a simple chat UI next to it.

★ 185 MIT TypeScript Active, pushed 3 months ago

Faster Chat 1337hero

A blazingly fast, privacy first & OPEN AI Chat Interface

Complete system Experimental You want a fast, self-hosted multi-provider chat UI without Open WebUI's weight.

★ 175 MIT JavaScript Active, pushed 9 days ago

Superduper superduper-io

Superduper: End-to-end framework for building custom AI applications and agents.

Framework or platform Experimental Rarely now; consider it only if you already depend on it.

★ 5.3k Apache-2.0 Python Quiet, pushed 14 months ago

LM Studio Element Labs

Bionic is LM Studio's agent for work and code. Create documents, slides, PDFs, and software with local or frontier open models.

Complete system Usable You want to run open models privately on your own machine without command-line setup.

Commercial