v3.4.0

[mudler]

🚀 LocalAI 3.4.0

What’s New in LocalAI 3.4.0 🎉

• WebUI improvements: now size can be set during image generation
• New backends: KittenTTS, kokoro and dia now are available as backends and models can be installed directly from the gallery
  Note: these backends needs to be warmed up during the first call to download the model files.
• Support for reasoning effort in the OpenAI chat completion
• Diffusers backend now is available for l4t images and devices
• During backend installation from the CLI can be supplied alias and name (--alias and --name`) to override configurations
• Backends now can be sideloaded from the system: you can drag-and-drop the backends in the backends folder and they will just work!

The Complete Local Stack for Privacy-First AI

LocalAI	The free, Open Source OpenAI alternative. Acts as a drop-in replacement REST API compatible with OpenAI specifications for local AI inferencing. No GPU required. Link: https://github.com/mudler/LocalAI
LocalAGI	A powerful Local AI agent management platform. Serves as a drop-in replacement for OpenAI's Responses API, supercharged with advanced agentic capabilities and a no-code UI. Link: https://github.com/mudler/LocalAGI
LocalRecall	A RESTful API and knowledge base management system providing persistent memory and storage capabilities for AI agents. Designed to work alongside LocalAI and LocalAGI. Link: https://github.com/mudler/LocalRecall

Thank you! ❤️

A massive THANK YOU to our incredible community and our sponsors! LocalAI has over 34,100 stars, and LocalAGI has already rocketed past 900+ stars!

As a reminder, LocalAI is real FOSS (Free and Open Source Software) and its sibling projects are community-driven and not backed by VCs or a company. We rely on contributors donating their spare time and our sponsors to provide us the hardware! If you love open-source, privacy-first AI, please consider starring the repos, contributing code, reporting bugs, or spreading the word!

👉 Check out the reborn LocalAGI v2 today: https://github.com/mudler/LocalAGI

Full changelog 👇

👉 Click to expand 👈

What's Changed

Bug fixes 🐛

• fix(llama.cpp): do not default to linear rope by @mudler in fix(llama.cpp): do not default to linear rope #5982

Exciting New Features 🎉

• feat(webui): allow to specify image size by @mudler in feat(webui): allow to specify image size #5976
• feat(backends): add KittenTTS by @mudler in feat(backends): add KittenTTS #5977
• feat(kokoro): complete kokoro integration by @mudler in feat(kokoro): complete kokoro integration #5978
• feat: add reasoning effort and metadata to template by @mudler in feat: add reasoning effort and metadata to template #5981
• feat(transformers): add support to Dia by @mudler in feat(transformers): add support to Dia #5991
• feat(diffusers): add builds for nvidia-l4t by @mudler in feat(diffusers): add builds for nvidia-l4t #6004
• feat(backends install): allow to specify name and alias during manual installation by @mudler in feat(backends install): allow to specify name and alias during manual installation #5971

🧠 Models

• chore(models): add gpt-oss-20b by @mudler in chore(models): add gpt-oss-20b #5973
• chore(models): add gpt-oss-120b by @mudler in chore(models): add gpt-oss-120b #5974
• feat(models): add support to qwen-image by @mudler in feat(models): add support to qwen-image #5975
• chore(model gallery): add openai_gpt-oss-20b-neo by @mudler in chore(model gallery): add openai_gpt-oss-20b-neo #5986
• fix(harmony): improve template by adding reasoning effort and system_prompt by @mudler in fix(harmony): improve template by adding reasoning effort and system_prompt #5985
• chore(model gallery): add qwen_qwen3-4b-instruct-2507 by @mudler in chore(model gallery): add qwen_qwen3-4b-instruct-2507 #5987
• chore(model gallery): add qwen_qwen3-4b-thinking-2507 by @mudler in chore(model gallery): add qwen_qwen3-4b-thinking-2507 #5988
• chore(model gallery): add huihui-ai_huihui-gpt-oss-20b-bf16-abliterated by @mudler in chore(model gallery): add huihui-ai_huihui-gpt-oss-20b-bf16-abliterated #5995
• chore(model gallery): add openai-gpt-oss-20b-abliterated-uncensored-neo-imatrix by @mudler in chore(model gallery): add openai-gpt-oss-20b-abliterated-uncensored-neo-imatrix #5996
• chore(model gallery): add tarek07_nomad-llama-70b by @mudler in chore(model gallery): add tarek07_nomad-llama-70b #5997
• chore: add Dia to the model gallery, fix backend by @mudler in chore: add Dia to the model gallery, fix backend #5998
• chore(model gallery): add chatterbox by @mudler in chore(model gallery): add chatterbox #5999
• chore(model gallery): add outetts by @mudler in chore(model gallery): add outetts #6000
• chore(model gallery): add impish_nemo_12b by @mudler in chore(model gallery): add impish_nemo_12b #6007
• chore(model-gallery): ⬆️ update checksum by @localai-bot in chore(model-gallery): ⬆️ update checksum #6010

👒 Dependencies

• chore(deps): bump edgevpn by @mudler in chore(deps): bump edgevpn #6001

Other Changes

• docs: ⬆️ update docs version mudler/LocalAI by @localai-bot in docs: ⬆️ update docs version mudler/LocalAI #5967
• chore: ⬆️ Update ggml-org/llama.cpp to 41613437ffee0dbccad684fc744788bc504ec213 by @localai-bot in chore: ⬆️ Update ggml-org/llama.cpp to 41613437ffee0dbccad684fc744788bc504ec213 #5968
• chore(deps): bump torch and diffusers by @mudler in chore(deps): bump torch and diffusers #5970
• chore(deps): bump torch and sentence-transformers by @mudler in chore(deps): bump torch and sentence-transformers #5969
• chore: ⬆️ Update ggml-org/llama.cpp to fd1234cb468935ea087d6929b2487926c3afff4b by @localai-bot in chore: ⬆️ Update ggml-org/llama.cpp to fd1234cb468935ea087d6929b2487926c3afff4b #5972
• chore: ⬆️ Update ggml-org/llama.cpp to e725a1a982ca870404a9c4935df52466327bbd02 by @localai-bot in chore: ⬆️ Update ggml-org/llama.cpp to e725a1a982ca870404a9c4935df52466327bbd02 #5984
• feat(swagger): update swagger by @localai-bot in feat(swagger): update swagger #5983
• chore: ⬆️ Update ggml-org/llama.cpp to a0552c8beef74e843bb085c8ef0c63f9ed7a2b27 by @localai-bot in chore: ⬆️ Update ggml-org/llama.cpp to a0552c8beef74e843bb085c8ef0c63f9ed7a2b27 #5992
• chore: ⬆️ Update ggml-org/whisper.cpp to 4245c77b654cd384ad9f53a4a302be716b3e5861 by @localai-bot in chore: ⬆️ Update ggml-org/whisper.cpp to 4245c77b654cd384ad9f53a4a302be716b3e5861 #5993
• docs: update links in documentation by @lnnt in docs: update links in documentation #5994
• chore: ⬆️ Update ggml-org/llama.cpp to cd6983d56d2cce94ecb86bb114ae8379a609073c by @localai-bot in chore: ⬆️ Update ggml-org/llama.cpp to cd6983d56d2cce94ecb86bb114ae8379a609073c #6003
• fix(l4t-diffusers): add sentencepiece by @mudler in fix(l4t-diffusers): add sentencepiece #6005
• chore: ⬆️ Update ggml-org/llama.cpp to 79c1160b073b8148a404f3dd2584be1606dccc66 by @localai-bot in chore: ⬆️ Update ggml-org/llama.cpp to 79c1160b073b8148a404f3dd2584be1606dccc66 #6006
• chore: ⬆️ Update ggml-org/whisper.cpp to b02242d0adb5c6c4896d59ac86d9ec9fe0d0fe33 by @localai-bot in chore: ⬆️ Update ggml-org/whisper.cpp to b02242d0adb5c6c4896d59ac86d9ec9fe0d0fe33 #6009
• chore: ⬆️ Update ggml-org/llama.cpp to be48528b068111304e4a0bb82c028558b5705f05 by @localai-bot in chore: ⬆️ Update ggml-org/llama.cpp to be48528b068111304e4a0bb82c028558b5705f05 #6012

New Contributors

• @lnnt made their first contribution in docs: update links in documentation #5994

Full Changelog: v3.3.2...v3.4.0

---

This discussion was created from the release v3.4.0.