LocalAI

Drop-in OpenAI replacement

Get it on umbrelOS

Versionv4.1.3

CategoryAI

Source codePublic

Developed byEttore Di Giacinto 1 app

Submitted byhighghlow 6 apps

Compatible withumbrelOS 0.5 or later

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that's compatible with OpenAI API specifications for local inferencing.

It allows you to run LLMs, generate images, audio locally with consumer grade hardware, supporting multiple model families and architectures.

⚠️ Note Before running a model, make sure your device has enough free RAM to support it. Attempting to run a model that exceeds your available memory could cause your device to crash or become unresponsive. Always check the model requirements before downloading or starting it.

What's new

Version v4.1.3+3 weeks ago

This is a patch release with bug fixes and minor improvements.

Key highlights include:

Fixed login via legacy API keys
Fixed Anthropic API compatibility — no longer emits empty tokens, SSE tool calls corrected
Improved GPU detection for MacOS and Thor
Updated llama.cpp, stable-diffusion.cpp, and ace-step.cpp dependencies

Full release notes can be found at https://github.com/mudler/LocalAI/releases

Version v4.1.0+last month

This release turns LocalAI into a production-grade AI platform with distributed clustering, built-in user management, fine-tuning, and much more.

Key highlights include:

Distributed Mode - Run LocalAI as a cluster with smart routing, node groups, drain/resume, and min/max autoscaling
Users & Authentication - Built-in user management with OIDC/OAuth support, invite mode, per-user API keys, and admin impersonation
Quota System - Per-user usage quotas with predictive analytics and usage breakdown dashboards
Fine-Tuning (experimental) - Fine-tune models with TRL, auto-export to GGUF, and import back — all from the UI
Quantization (experimental) - New backend for on-the-fly model quantization
Pipeline Editor - Visual model pipeline editor in the React UI
Standalone Agents - Run agents from the CLI with local-ai agent run
Media History - Browse past generated images and media in Studio pages
Various bug fixes including embeddings base64 encoding, Kokoro TTS phonemization, and gallery tag filtering

Full release notes can be found at https://github.com/mudler/LocalAI/releases

Version v4.0.0+last month

This is a major release that transforms LocalAI into a complete AI orchestration platform with a fully rewritten React UI, native agentic capabilities, and many new backends.

Key highlights include:

Native Agentic Orchestration - Manage, create, and run AI agents with memory and skills via the new UI, plus the new Agenthub community hub for sharing agents
Revamped React UI - Complete frontend rewrite for a faster, modern experience with Canvas mode for previewing code artifacts
MCP Client-Side Support - Full Model Context Protocol support with MCP Apps, tool streaming, and a new LOCALAI_DISABLE_MCP option to disable it
WebRTC Realtime - WebRTC support for low-latency realtime audio conversations
New Backends - Added experimental MLX Distributed, fish-speech, ace-step.cpp, and faster-qwen3-tts
Data Separation - New LOCALAI_DATA_PATH env var to separate persistent data (agents, skills) from configuration
Model RAM Warnings - Visual warnings when a model's storage size exceeds available system RAM

⚠️ Breaking Changes

HuggingFace backend support has been removed

Full release notes can be found at https://github.com/mudler/LocalAI/releases

Version v3.12.1+2 months ago

This is a patch release fixing incompatibilities with Qwen 3 Coder, along with stability improvements and minor enhancements from v3.12.0.

Key highlights include:

Fixed Qwen 3 Coder incompatibilities via updated llama.cpp
Multi-modal Realtime - Send text, images, and audio in real-time conversations
Voxtral Backend - New high-quality text-to-speech backend
Multi-GPU Support - Improved Diffusers performance with multiple GPUs
Improved UI with dark/light theme variants and better navigation
Security fix for SSRF vulnerability in content fetching endpoints
Various realtime stability fixes for audio, image, and model handling

Full release notes can be found at https://github.com/mudler/LocalAI/releases

Version v3.11.0+2 months ago

This is a massive update for audio and multimodal capabilities, introducing realtime audio conversations, music generation, and expanded speech recognition and text-to-speech backends.

Key highlights include:

Realtime Audio Conversations - Native support for fluid voice interactions with agents, enabling low-latency audio-in/audio-out workflows with tool calling support
Music Generation UI - New interface for generating music from text prompts using Ace-Step (MusicGen) directly in the browser
Expanded Speech Recognition - Added WhisperX with speaker diarization, VibeVoice ASR, Qwen-ASR, and Nvidia NeMo backends
TTS Streaming - Text-to-speech now supports streaming mode for lower latency responses
New TTS Backends - Added VoxCPM and Qwen-TTS support, plus most remaining Piper voices from Hugging Face
vLLM Omni Support - New backend integration for vLLM Omni models
Hardware Expansion - Extended support for CUDA 12/13, L4T (Jetson), SBSA, and improved Metal (Apple Silicon) integration with MLX backends

⚠️ Breaking Changes ExLlama (deprecated) and Bark (unmaintained) backends have been removed

Full release notes can be found at https://github.com/mudler/LocalAI/releases

LibreChatEnhanced ChatGPT Clone

ChatBot UIChatGPT but better

OpenClawThe AI that actually does things

Chatpad AIPremium quality UI for ChatGPT

Open WebUIChat with Ollama models like DeepSeek-R1 and LLama, or use an OpenAI API key

OllamaSelf-host open source AI models like DeepSeek-R1, Llama, and more

LocalAI

Related