LiteLLM
LiteLLM is a gateway that lets you call models from Anthropic, OpenAI, Google and many other providers through one consistent API. It runs as a Python library inside your code or as a standalone proxy server in front of your whole stack, with cost tracking, per-key budgets, rate limits and automatic fallbacks for when a provider has a bad day.
View the repo · BerriAI/litellm ↗Anyone building on more than one model, or who suspects they will be within a year, and wants to avoid rewriting application code every time the model landscape shifts. Platform and engineering leads use the proxy to give internal teams governed access to models, with spend visible per team and per key. A single small project calling one provider directly does not need it yet; add it when the second model or the second team arrives.
Install itpip install litellm
# run the proxy:
litellm --model claude-sonnet-4-6The proxy needs a Postgres database for key management and spend tracking, so treat it as a real service with backups rather than a script someone runs. Set budgets and rate limits per key from day one, because an agent stuck in a loop can spend a surprising amount overnight. Provider APIs still differ in the corners, particularly around tool calling and streaming, so test each model you route to instead of assuming the abstraction is perfect.
Where Blash AI comes inIn a Blash build the gateway is the control point for every model call: n8n workflows, retrieval systems on Chroma and local models on Ollama all route through it. We stand it up, set the budgets, fallbacks and logging, and map each workflow to the cheapest model that meets its quality bar, so you can see exactly what every automation costs to run.
When you want this running on your real stack, that is the engagement
Book an AI audit →n8n →
n8n is workflow automation you run on your own infrastructure. It connects the applications a business already uses through a visual editor, with hundreds of prebuilt nodes, a code step for anything bespoke, and native support for calling language models mid-flow. Think of it as a self-hosted alternative to Zapier or Make that you own outright, including every piece of data passing through it.
Ollama →
Ollama runs open-weight language models on your own machine or server. One command downloads a model and serves it behind a local API that most existing AI tooling can talk to, so an application built against a hosted provider can point at your own hardware instead. Sensitive data stays inside your network, there are no per-token fees, and it keeps working with no internet connection at all.
Chroma →
Chroma is an open-source vector database, the store behind retrieval-augmented generation. You embed your documents, Chroma indexes the embeddings, and at question time it returns the passages closest in meaning to the query so a model can answer from your own content with citations. It runs embedded inside a Python process for prototypes or as a standalone server for shared use.