FreeLLMAPI Tutorial: Stack 16 Free LLM Providers Into One Endpoint for ~1.7B Tokens/Month

Created

Have you ever calculated how many tokens you’re wasting across scattered free-tier quotas? Google Gemini gives you millions per day, Groq offers 30 requests per minute, Cerebras hands you Qwen3 235B inference for free — each tier alone is a toy, but stacked together they add up to roughly 1.7 billion tokens per month of working inference capacity.

The problem is that managing a dozen different SDKs, API keys, rate limits, and failover logic is more tedious than writing your actual code.

That’s what FreeLLMAPI solves. It collapses 16 LLM providers’ free tiers into a single OpenAI-compatible /v1/chat/completions endpoint. Change one base_url, and any OpenAI-compatible client transparently routes across whichever providers you’ve added keys for, with automatic failover when one hits its rate limit.

What is FreeLLMAPI? FreeLLMAPI is an open-source OpenAI-compatible proxy server by tashfeenahmed that aggregates free-tier quotas from 16 LLM providers into a single endpoint, featuring smart routing, automatic failover, and AES-256-GCM encrypted key storage. Licensed under MIT.

Key Data:

  • 🌟 GitHub Stars: 11,998+ (Repository)
  • 🍴 Forks: 1,863+
  • 📦 Current Version: v0.4.1
  • ⚖️ License: MIT
  • 🔧 Core Feature: 16 LLM provider free-tier aggregation, ~1.7B tokens/month
  • 🧠 Core Feature: Smart routing + automatic failover (up to 20 retries)
  • 🔒 Core Feature: AES-256-GCM encrypted key storage
  • 🖥️ Core Feature: Admin dashboard (React + Vite) + Desktop app (macOS / Windows)
  • 🌐 Core Feature: OpenAI / Anthropic / Responses API protocol support
  • 💰 Core Feature: Free tier forever; Premium $19/yr for live model catalog

Prerequisites

Before getting started with FreeLLMAPI, you’ll need:

  • Docker (recommended) or Node.js 20+ (local development)
  • At least one LLM provider API key (Google, Groq, etc. — all free to register)
  • Any OpenAI-compatible client (Python openai library, curl, LangChain, Continue, etc.)

💡 Tip: You don’t need keys from all 16 providers. 2-3 providers give decent failover coverage, and you can add more anytime from the dashboard.

Overview

This tutorial walks you through setting up FreeLLMAPI from scratch:

  1. One-line installation and configuration
  2. Adding provider keys
  3. Sending your first API request
  4. Using the admin dashboard
  5. Installing the desktop app
  6. Integrating with AI tools

Step-by-Step Guide

Step 1: One-Line Installation

FreeLLMAPI provides the simplest possible installation — one command that handles directory creation, key generation, image pulling, and container startup:

1
curl -fsSL https://freellmapi.co/install.sh | bash

This command will:

  • Create a working directory at ~/freellmapi
  • Auto-generate an AES-256 encryption key
  • Pull the latest Docker image
  • Start the container on localhost:3001

⚠️ Security note: The install script preserves your .env file and encryption key on re-runs — it only updates the container to the latest version. You can read the script first at install.sh.

Windows users: Download the .exe desktop installer from GitHub Releases.

Step 2: Manual Installation (Optional)

For more control, use Docker Compose:

1
2
3
4
5
6
7
8
git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi

# Generate encryption key
ENCRYPTION_KEY="$(openssl rand -hex 32)"
printf "ENCRYPTION_KEY=%s\nPORT=3001\n" "$ENCRYPTION_KEY" > .env

docker compose up -d

Environment Variables:

Variable Description Default
ENCRYPTION_KEY AES-256-GCM encryption key (required) None
PORT Service port 3001
HOST_BIND Bind address (0.0.0.0 for LAN access) 127.0.0.1
FREELLMAPI_CONTEXT_HANDOFF Inject context on model switch Empty (off)
DEV_MODE Development mode false

Step 3: Adding Provider Keys

Open http://localhost:3001 in your browser. On first visit, register an admin account (email + password).

Navigate to the Keys page and add your API keys one by one:

Recommended providers to add first:

Provider Free Quota Signup Difficulty Recommended Models
Google Gemini Millions of tokens/day Easy Gemini 2.5 Flash
Groq 30 requests/min Easy Llama 3.3, Llama 4
Cerebras High-speed inference Easy Qwen3 235B
Mistral Millions of tokens/month Easy Large 3, Codestral
OpenRouter 21 free-tier models Easy Multiple choices
GitHub Models Free evaluation quota Easy GPT-4.1, GPT-4o
Cloudflare Workers AI Free tier Medium Kimi K2, GLM-4.7
HuggingFace Free inference API Easy DeepSeek V4, Qwen3

After adding keys, go to Fallback Chain to adjust priority order. Providers at the top are used first; when one returns a 429 or 5xx error, the router automatically falls over to the next.

💡 Tip: Start with 2-3 providers and add more later. All keys are encrypted with AES-256-GCM before storage.

Step 4: Get Your Unified API Key

At the top of the Keys page, you’ll see a unified API key in the format freellmapi-xxx. This is the single credential all your applications use — upstream provider keys are never exposed to your clients.

Step 5: Send Your First Request

Python (Recommended):

1
2
3
4
5
6
7
8
9
10
11
12
13
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",  # let the router pick the best model
    messages=[{"role": "user", "content": "Summarize the fall of Rome in one sentence."}],
)
print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

curl:

1
2
3
4
5
6
7
curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "hi"}]
  }'

Model Parameter Guide:

Value Behavior
auto Router automatically picks the best available model
gemini-2.5-flash Specify a model (router failovers across providers serving it)
gpt-4.1 Specify a model

💡 Tip: The x-routed-via response header tells you which provider handled the request — useful for debugging.

Step 6: Advanced Features

Streaming: Set stream: true for SSE streaming responses. All provider adapters support both streaming and non-streaming.

Tool Calling: OpenAI-style tools / tool_choice requests are passed through, with cross-provider tool_calls message round-trips.

Embeddings: /v1/embeddings supports family-based routing — failover only happens between providers serving the same model (vectors from different models are incompatible).

Image Generation & TTS: /v1/images/generations and /v1/audio/speech also route across providers. Manage them in the dashboard’s Models → Image / Audio tabs.

Sticky Sessions: Multi-turn conversations stay on the same model for 30 minutes to avoid hallucination spikes from mid-conversation model switches.

Context Handoff: Optional feature — set FREELLMAPI_CONTEXT_HANDOFF=on_model_switch to inject a compact system message when a session falls over to a different model.

Step 7: Admin Dashboard

FreeLLMAPI includes a built-in React + Vite admin dashboard with:

  • Key Management: Add, delete, and reorder provider keys
  • Fallback Chain: Drag-and-drop priority ordering
  • Analytics: Request logs, latency, token counts, success rate, per-provider breakdowns
  • Playground: Test prompts directly in the dashboard
  • Model Browser: View all available models including image and audio
  • Dark Mode: Full dark theme support

The dashboard ships in 6 languages: English, 中文 (简体), Français, Español, Português (Brasil), Italiano. It auto-detects your browser language on first load.

Step 8: Desktop App

FreeLLMAPI also offers a native desktop menu-bar app with the entire router + dashboard running locally:

Build from source:

1
2
3
npm install
npm run desktop:dist        # macOS .dmg
npm run desktop:dist:win    # Windows .exe

💡 Tip: Locally built apps are unsigned. Windows SmartScreen may warn on first run (click “More info” → “Run anyway”); macOS launches without Gatekeeper prompts.

Step 9: Integrate with Your AI Tools

FreeLLMAPI works with any OpenAI-compatible client. Here’s how to connect it to popular AI tools:

Universal: Change base_url to http://localhost:3001/v1 and api_key to your unified key.

Claude Code / Anthropic Clients:

FreeLLMAPI supports the Anthropic Messages API (/v1/messages), so Claude Code and the official Anthropic SDKs work directly:

1
2
export ANTHROPIC_BASE_URL=http://localhost:3001
export ANTHROPIC_API_KEY=freellmapi-your-unified-key

Claude model families (opus / sonnet / haiku / default) auto-map to auto or a pinned model from the Keys page.

Codex CLI:

FreeLLMAPI supports the Responses API (/v1/responses), the wire format current Codex CLI versions require:

1
2
export OPENAI_BASE_URL=http://localhost:3001/v1
export OPENAI_API_KEY=freellmapi-your-unified-key

LangChain / LlamaIndex:

1
2
from openai import OpenAI
client = OpenAI(base_url="http://localhost:3001/v1", api_key="freellmapi-...")

Continue (VS Code):

Set apiBase to http://localhost:3001/v1 in Continue settings.

Comparison with Similar Tools

Feature FreeLLMAPI LiteLLM OpenRouter
Deployment Self-hosted Self-hosted / Cloud Cloud service
Free to use ✅ Completely free ✅ Open-source free Partial free models
Providers aggregated 16+ 100+ Multiple providers
Free-tier aggregation ✅ Core feature ❌ Manual config ❌ Pay-per-use
Anthropic API ✅ Native ✅ ✅
Responses API ✅ ❌ ❌
Desktop app ✅ ❌ ❌
Encrypted key storage ✅ AES-256-GCM ❌ N/A (cloud)
Admin dashboard ✅ Built-in ✅ ✅
License MIT MIT Proprietary

FAQ

Q: How do I install FreeLLMAPI?
A: The simplest way is the Docker one-liner: curl -fsSL https://freellmapi.co/install.sh | bash. You can also clone the repo and run docker compose up -d. Windows users can download the .exe installer from GitHub Releases.

Q: Which LLM providers does FreeLLMAPI support?
A: Currently 16 providers: Google Gemini, Groq, Cerebras, OpenCode Zen, Mistral, OpenRouter, GitHub Models, Cloudflare, Cohere, Z.ai (Zhipu), NVIDIA, HuggingFace, Ollama Cloud, Kilo Gateway, Pollinations, LLM7, and OVH AI Endpoints. You can also add any custom OpenAI-compatible endpoint.

Q: Is the 1.7 billion tokens/month claim real?
A: That’s the theoretical maximum when stacking all supported providers’ free tiers. Actual availability depends on how many providers you’ve registered, their free-tier policy changes, and your usage patterns.

Q: How is FreeLLMAPI different from LiteLLM?
A: FreeLLMAPI focuses on the “free-tier aggregation” use case with built-in rate tracking, failover, and encrypted storage. LiteLLM is more general-purpose, supporting 100+ providers but requiring manual routing configuration. FreeLLMAPI also offers a desktop app and native Anthropic/Responses API support.

Q: Is my data secure?
A: All provider API keys are encrypted with AES-256-GCM before storage, only decrypted in memory just before a request. The admin dashboard uses email + password auth (scrypt-hashed), and API endpoints use separate unified-key auth.

Q: Can I use it across devices on my LAN?
A: Yes. Set HOST_BIND=0.0.0.0 at startup for LAN access. Only do this on trusted networks since the proxy is single-user.

Q: Where can I find the FreeLLMAPI tutorial?
A: This article is the complete FreeLLMAPI installation guide and usage tutorial, covering everything from setup to AI tool integration. Official documentation is at freellmapi.co.

Advanced Tips

Custom Providers

Beyond the 16 built-in providers, you can add any OpenAI-compatible endpoint from the Keys page — local llama.cpp, LM Studio, vLLM, or remote gateways.

Premium Model Catalog

The free tier uses a monthly model snapshot. Upgrade to Premium ($19/yr or $49 lifetime) for a live catalog refreshed every 2-3 days — new free models appear in your router the moment they exist. Activate in the dashboard’s Premium page.

Docker Operations

1
2
3
4
5
6
7
8
9
# View logs
docker compose logs -f freellmapi

# Update to latest
docker compose pull && docker compose up -d

# Data persistence
# SQLite data stored in freellmapi-data volume
# Keep .env ENCRYPTION_KEY and data volume when upgrading

Request Analytics

Analytics are retained for 90 days or 100,000 request rows by default. Set REQUEST_ANALYTICS_RETENTION_DAYS=0 or REQUEST_ANALYTICS_MAX_ROWS=0 in .env to disable either limit.

Conclusion

This is the complete FreeLLMAPI installation guide and usage tutorial. From a one-line install to AI tool integration, FreeLLMAPI lets you aggregate 16 LLM providers’ free tiers at zero cost — roughly 1.7 billion tokens per month of inference capacity. Its smart routing and automatic failover ensure your applications never break because of a single provider’s rate limits.

If you’re looking for a free LLM API proxy solution, or want to maximize your free-tier utilization across providers, FreeLLMAPI is worth trying. Visit the GitHub repository and official website for more details.

References

How to cite this article: This article is based on the FreeLLMAPI GitHub Repository (verified 2026-06-24). All commands and configurations have been verified against version v0.4.1.