kodebeat / papers / ai
Pooling several model providers behind one local endpoint
A local OpenAI- and Anthropic-compatible rotation proxy.
Fronts a pool of providers behind one stable endpoint, falling through to the next on a quota or auth error. Providers can be HTTP APIs or local CLI tools, and both OpenAI- and Anthropic-SDK clients point at it with no code changes. It binds loopback and refuses any other address without an api_key, because it holds a live key for every provider you configured.
What is in it
- The problem — what goes unanswered without it, and who notices first.
- Why the obvious alternative falls short — stated plainly, including where it is the better choice.
- How it works — the method, not a feature list.
- Concrete use cases — with console output quoted from the repository, never reconstructed.
- The methodology behind any number it emits — every term shown, so the figure survives a question.
- What it deliberately does not do — the section most papers leave out.
Part of the AI infrastructure theme.
Get the PDF
One email with the download link, and this paper already selected. No follow-up sequence.