KubeAI: Private Open AI on K8s

Serve LLMs privately with an OpenAI API compatible API

Visit Website

About	Details
Name:	KubeAI: Private Open AI on K8s
Submited By:	Al Wolf
Release Date	10 months ago
Website	Visit Website
Category	Open Source GitHub

✅️ Drop-in replacement for OpenAI with API compatibility 🚀 Serve OSS LLMs on CPUs or GPUs ⚖️ Autoscaling with scale from 0 🛠️ Zero dependencies (no Istio, Knative, etc.) 🤖 Operates OSS model servers (vLLM and Ollama) 🔋 Chat UI included

Blaze Koepp

Great job on this release! The features to auto scale and the inclusion of a chat UI are fantastic. It’s nice to see a tool that simplifies managing AI models while offering flexibility in deployment.

1 year ago

Buck Yundt

This platform seems like a solid choice for managing AI models on Kubernetes. The drop in replacement for OpenAI and the compatibility with various model servers look very promising.

1 year ago

Conner Jacobs

The ability to scale from zero and support both CPUs and GPUs makes it very versatile. Plus, having no extra dependencies is a huge plus.

1 year ago

Karley Frami

🎊 Congrats on the launch! You’ve created something incredible, and I’m so excited to see it thrive. Well done! 🎉 @samos123

1 year ago