MushyBook
voidllm logo

voidllm

Privacy-first LLM proxy and AI gateway — load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.

AIAI AgentsAi GatewayAnthropicApi GatewayDevOpsGolangHelmGoNOASSERTION 121 14Health 72
endpointreleasevoidllm
Website GitHub

What is it?

What it is

**A privacy-first LLM proxy and AI gateway for teams that take control seriously.**

Why it exists

Privacy-first LLM proxy and AI gateway — load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.

Who should use it

Teams building with Go who want an open-source, self-hosted option.

Who should avoid it

Teams that need a fully managed SaaS with enterprise SLAs out of the box.

How it works

A quick walkthrough in plain English

How voidllm works in 4 steps

Step 1 of 4

Something triggers the flow

A schedule, webhook, or manual click tells voidllm to start.

Features

Ai
Ai Gateway
Anthropic
Api Gateway
Docker
Golang

Advantages

  • Open source (NOASSERTION)
  • Active Go ecosystem
  • Self-hosted deployment options

Disadvantages

  • Requires operational ownership for self-hosted setups
  • Community support varies by project maturity

Installation

cloud

Any OpenAI-compatible SDK works out of the box - just change the base URL to your VoidLLM proxy.

## Features

| Feature | Details |
|---|---|
| OpenAI-compatible proxy | `/v1/chat/completions`, embeddings, images, audio, streaming |
| Multi-provider routing | OpenAI, Anthropic, Azure, Ollama, vLLM, any custom endpoint |
| Load balancing | Round-robin, least-latency, weighted, priority across deployments |
| Automatic failover | Retry on 5xx/timeout, circuit breakers, health-aware routing |
| Web UI | Dashboard, playground, API keys, teams, models, usage, settings |
| RBAC | Org > Team > User > Key hierarchy, 4 roles |
| Rate limits | Requests per minute/day, most-restrictive-wins across levels |
| Token budgets | Daily/monthly limits, real-time enforcement |
| Usage tracking | Tokens, cos

FAQ

What is voidllm?

voidllm is an open-source project licensed under NOASSERTION.

What language is voidllm built with?

Primary language: Go.

Is it free to use?

Yes. Licensed under NOASSERTION. Check the license for commercial use.

Loading documentation…
View on GitHub

Featured in Videos

YouTube tutorials and walkthroughs for voidllm

s7

2,152,967 views

Starts at 13:03

Share

Alternatives

Similar projects ranked by category, topics, and text overlap.

Compare
unsloth logo

unsloth

Python

Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

68.4k·Health 90

Compare
ollama logo

ollama

Go

Run open LLMs locally with one command. CLI and REST API included.

177.1k·Health 90

Compare
LocalAI logo

LocalAI

Go

OpenAI-compatible API for running LLMs and generative models locally.

47.7k·Health 95

Compare
conduit logo

conduit

Dart

Conduit is a Flutter-based mobile application that serves as a native client for Open WebUI, offering real-time streaming, robust authentication (including LDAP, JWT, SSO), productivity features (notes, channels, voice input), rich rendering (code, LaTeX, Mermaid), and platform integrations (widgets, quick actions, share sheet). It prioritizes privacy via secure credential storage and avoids third-party analytics, designed for self-hosted deployments.

1.8k·Health 85

Compare
dao-code logo

dao-code

TypeScript

Dao Code delivers a cost‑effective, Chinese‑first AI coding assistant that runs in the terminal. It streams reasoning, executes tools safely, and reuses DeepSeek’s prefix cache via byte‑stable prefixes and cache‑reusing forks for memory and reflection. The system supports persistent cross‑session memory, layered permissions, long‑task autonomy, and a rich TUI experience, making it suitable for real OSS bug fixes and autonomous project development.

1.5k·Health 87

Compare
voidllm | MushyBook