FreeLLM Router v2 • localhost • <10ms

Only the
models that work.

FreeLLM shows around 1.6B tokens from only 4 API keys. Half are dead — Google Flash, DeepSeek, etc return Payment Required or No longer available. Veda tests every model live and keeps only the working ones. Fair routing, instant failover, localhost dashboard.

See how it works GitHub → pip install -r requirements.txt → python veda.py → http://localhost:3000

1.6B → 30

illusion → real

<10ms

routing overhead

3× retry

auto failover

Without Veda

The 1.6B illusion

• FreeLLM lists all models from just 4 API keys — 111 models, 1.6B tokens
• Google Flash / Flash Lite → No longer available
• DeepSeek / HF → Payment Required 402
• You only find out after Cline fails mid-task
• Good models get exhausted first, dead ones remain

[429] RETRYABLE gemma-4-26b 429
[502] nemotron-3-super 502
→ wasted time & tokens

With Veda

Only working, fairly routed

• Probes each model POST /v1/chat/completions {max_tokens:1}
• Keeps only 200 OK, drops 402/404 + payment/no longer bodies
• Saves to preferred_models.json — 30 working
• Round-robin → every model used equally
• If one fails → next in <150ms, 60s auto-recovery

[info] Cache cleaned: 30 working (81 filtered dead)
[info] auto → mistral-large-3 200 804ms

How it works — 3 steps

Connect FreeLLM

Add your keys in FreeLLM app, put FREELLM_API_KEY in Veda’s .env

FREELLM_API_BASE=https://api.freellm.ai
FREELLM_API_KEY=***

Veda probes

On first run tests all 111, keeps only working. Add a new key → hit ↻ Refresh models

Discovered 111Working 30Filtered 81

Use via Cline

Cline Base URL http://localhost:3000/v1 — Veda rotates fairly

X-Veda-Model: mistral-large-3
latency: 342ms

Live dashboard http://localhost:3000

Open Dashboard →localhost →

Veda v2 — Only the models that work.

Pool

Requests

127

Avg

612ms

Failovers

Cline URL

localhost:3000/v1

Model Pool (30)

mistral-large-3ON

gemma-4-26b-a4bCOOLDOWN 12s

deepseek-chatfiltered — payment required

Live log

10:42 auto → mistral-large-3 200 804ms
10:42 RETRYABLE gemma 429 → magistral-small 200
10:43 Cache cleaned: 30 working

Preview — actual dashboard streams at /api/logs/stream with SSE, chart, quick tester.

⚡ <10ms overhead

orjson + bytes patch + uvloop + aiter_bytes zero-copy.

🔄 Refresh in 1 click

Add key in FreeLLM → Refresh → new working added to JSON.

🛡️ 3× failover

429/5xx → next model in 150ms, fatal 402 → 600s cooldown.

FreeLLM shows everything.
Veda shows what works.

GitHub — Star & Use Open Dashboard

localhost only • keys never leave your machine • MIT