This is a technical diagnosis, not value selling: run the 30 minutes as architecture → failure modes → proof

Asked (summary):

How should I run a 30-minute mock discovery call with a VP/Head of AI Platform Engineering and an AI Platform Engineer evaluating LiteLLM, as a candidate for an enterprise AE role? What questions will they ask, how should I answer, and what should I ask them — given LiteLLM reps sell technically, not with value-based selling, and I'm a non-technical seller?

This playbook is built from 23 de-duplicated factual rows gathered on 2026-09-12 from 8 independent hosts (15 rows of official LiteLLM documentation, 8 independent reviews and production write-ups); no single URL backs more than one row. Official docs govern product facts; independent claims are flagged as evaluator concerns to probe, not verified truth.

The 30-minute call, minute by minute

Hover or tap a phase for its goal; the six panels below carry the full move-by-move plan. The shape of the call: the biggest block (minutes 7–17) is them talking about requirements and failure modes — you asking layered questions, not pitching.

Postgres is not optional in a real deployment: the Admin UI, virtual keys, registries, budget tracking and spend logs all depend on it — budgets are simply not enforced without a connected database. docs.litellm.ai

Redis is the production coordination layer: the router uses it to track cooldowns and shared TPM/RPM usage across replicas — without it, rate limits and router state are per-pod only. docs.litellm.ai

Independent production reports describe real failure patterns to ask about — retry budgets exhausted against a throttled backend, Redis cache poisoning, alias mismatches during rolling restarts, async spend tracking lagging bursts — treat these as discovery questions, not accepted facts. perun.au

Self-hosting is free in license, not in operations: the customer owns HA, scaling, security review, the Postgres/Redis stack, observability and the upgrade lane — say this candidly and it builds credibility. swfte.com

Fundamentals in plain English (learn these cold)

LiteLLM is a self-hosted gateway (proxy): a server that sits between every application and every model provider, exposing one OpenAI-compatible endpoint across 100+ providers. Request path: app → LiteLLM gateway → model provider. Identity, policy and routing happen at the gateway; Postgres stores keys, budgets and usage; Redis coordinates distributed rate limits, router state and cache; telemetry flows out to the customer's existing observability stack.

Proxy/gateway vs SDK — a gateway is a shared server apps call over the network; an SDK is a library inside one app. Gateways centralize control; SDKs don't.

Virtual keys — API keys LiteLLM issues to teams/apps instead of raw provider keys, each with its own budget, rate limit and model access.

Routing / load balancing — spreading requests across multiple deployments of a model and picking the healthy/cheap/fast one.

Retry vs fallback — a retry repeats the same attempt; a fallback moves to a different deployment or model. Different risk, different config.

Redis — a fast in-memory store; here it shares counters, cooldowns and cache between gateway replicas.

Postgres — the relational database; here it persists keys, budgets, spend logs and registries.

RPM / TPM — requests and tokens per minute; the units rate limits are set in.

p95 / p99 latency — the time under which 95% / 99% of requests finish; tail latency is what engineers care about.

RTO / RPO — how fast you recover after failure and how much data you may lose; disaster-recovery targets.

HA (high availability) — surviving the loss of any one component: multiple stateless replicas behind a load balancer, external DB and Redis.

OTel (OpenTelemetry) — the open standard for exporting traces, metrics and logs to observability tools.

Guardrails — checks that run before/after a request (PII, topics, security scans) and can block or log.

Fail-open / fail-closed — when a guardrail is unreachable, do you let traffic through (open) or block it (closed)? A policy decision.

Control plane / data plane — admin instances manage keys/users/config; worker instances serve LLM traffic; LiteLLM can split these across regions.

SLO / SLA — an internal reliability target vs a contractual commitment. Never invent an SLA number; terms are plan-specific.

What they will ask, by role — and how to answer truthfully

The VP will probe operating risk

Standardization & governance: can we enforce one interface, budgets and model access across every team?
Ownership & on-call: who runs the gateway, Postgres and Redis? What does our platform team own?
Security & compliance: SSO/SAML, audit logs, data paths, key encryption, supply-chain posture.
Rollout & change management: upgrade cadence, migrations, rollback.
Commercial support & SLA, build-vs-buy vs writing their own router.

The engineer will probe exactness

Endpoints, providers, streaming, tool calling, structured outputs — do our exact calls translate?
Auth: JWT/SSO, virtual keys, tenancy hierarchy, model aliases.
Routing, retry/fallback semantics, cooldowns, timeout budgets.
Redis/Postgres sizing, HA, multi-region, Kubernetes/Helm, upgrades.
Logs, OTel, Prometheus, guardrails, load testing, provider pass-through escape hatches.

“Is it really OpenAI-compatible for everything we do?”

“LiteLLM exposes an OpenAI-compatible gateway and translates across providers — but let's validate the exact endpoints, models, streaming and tool-call behavior, and any provider-specific parameters you rely on, in your validation. Which native features can't be normalized?” unbiased.ai

“What infrastructure do we actually have to run?”

“Postgres is required for the Admin UI, virtual keys, registries, budgets and persistent usage; Redis becomes important once you run multiple replicas — shared counters, router state, cache. Let's size both from your RPS, concurrency, write volume and log retention.” docs.litellm.ai

“Is it highly available?”

“HA is an architecture, not a checkbox: multiple stateless gateway replicas behind a load balancer, external Postgres and Redis, health checks, controlled migrations, and failure behavior you've actually tested. What are your RTO/RPO targets?” swfte.com

“How do retries and fallbacks work?”

“Retries repeat an attempt; fallbacks move to another deployment or model, with cooldowns when a deployment crosses its allowed failures. I need your latency budget, the error classes you want handled, retry counts, and your rule for when a fallback model is semantically safe.” docs.litellm.ai

“Where does our data go?”

“Because you self-host, prompts don't need to touch LiteLLM's servers — but the providers, logging destinations and guardrails you configure still receive data. Let's map each data path and its retention policy; note guardrails like Akto can be set fail-open or fail-closed when unreachable.” docs.litellm.ai

“What's your SLA and gateway latency overhead?”

“Commercial terms are plan-specific — I'll bring the right document rather than quote from memory. On latency, the honest answer is to benchmark on your topology in the validation; independent posts report degradation at high concurrency and with large Postgres log tables, so let's measure added p95/p99 on your traffic, not accept anyone's number.” dev.to

“How do we isolate teams and stop overspend?”

“Tenancy is Organizations → Teams → (Projects) → Keys: spend rolls up the hierarchy and budgets are enforced inward — a request is blocked when any level on its path is over budget. Organizations/Org Admins and Projects are enterprise features; Projects is beta. What does your team hierarchy and cost-attribution model look like today?” docs.litellm.ai

Discovery questions to ask them, by category

Trigger & current state

Why now — what broke or what's coming?
Are apps calling providers via SDKs directly, or is there a gateway today?
Who owns model access and spend right now?

Workload & API compatibility

Which providers, models and endpoints exactly?
Streaming? Tool calling? Structured outputs?
Any native provider features that cannot be normalized?

Reliability & performance

Current RPS, peak concurrency, token volumes?
Today's p95/p99 and the SLO the gateway must fit inside?
What happens today on a 429 or provider outage?
What counts as a semantically safe fallback model?

Security, identity & data

Identity source (SSO/JWT) and the org/team hierarchy?
Prompt/response retention policy? What may never be logged?
Regions, data residency, egress constraints?

Governance & cost

How is spend attributed to business units today?
Budget enforcement: warn or hard-block?
Who can raise a team's budget?

Deployment & operations

Kubernetes/Helm? Who owns Postgres and Redis?
Release and change-management process; upgrade cadence tolerance?
On-call model — who gets paged when the gateway degrades?

Evaluation & decision

What must a trial demonstrate for a yes?
Who signs off — technical and commercial?
Timeline, procurement steps, security review?
What's the build-vs-buy alternative on the table?

The technical validation you propose in minutes 23–27

Scope

2–3 real applications behind the gateway, on their Kubernetes
2 providers plus one injected failure scenario (429/5xx/timeout)
Representative streaming and tool-call traffic
SSO/JWT auth, team and virtual-key controls, budgets and rate limits
Telemetry to their OTel/Prometheus stack; guardrail behavior incl. fail-open/fail-closed
Load test and a failover/restart/upgrade test

Entry criteria & success metrics (theirs, not yours)

Entry: named owner, deadline, environments, provider keys, traffic sample
Compatibility pass rate on their actual calls
Added gateway latency at p95/p99 under their load
Routing/failover succeeds under injected 429/5xx/timeout
Budgets and rate limits enforce; spend attributes to the right team
No prohibited content in logs; observability completeness
Clean restart, rolling upgrade and migration behavior

Every metric is customer-provided and measured on their topology. You never claim a number; you agree how they will measure it.

Mock-call coaching: weak vs strong

Technique: layer your questions (business/architecture → component → failure mode → metric → proof), synthesize every few minutes (“so what I'm hearing is…”), let the engineer go deep but return to the VP's operating risk, and say “I don't know — I'd verify that in the docs or a build and follow up” rather than bluff. End with a mutual test plan, owners and a date.

Weak

“LiteLLM supports 100+ models, caching, guardrails, budgets, routing, failover, traffic mirroring…” — a feature dump with no question attached.

Strong

“You mentioned Bedrock and Azure. Which endpoints, and do you stream tool calls? That decides what we validate first.” — precise clarifying question.

Weak

“Everything is 100% OpenAI-compatible, it just works.” — an unsupported universal claim an engineer will puncture.

Strong

“The gateway translates across providers; provider-specific parameters are the edge to test. Let's put your exact calls in the trial.” — scoped answer plus caveat plus validation step.

Weak

“How much could you save on LLM spend?” — generic value question this audience did not come for.

Strong

“What happens today when a provider throttles you at peak — and who gets paged?” — failure-mode question that surfaces the trigger.

Weak

“Postgres and Redis are trivial, don't worry about them.” — pretending the customer's operational cost away.

Strong

“You'll own Postgres and Redis — who runs them today, and what's your retention plan for spend logs so the table doesn't grow unbounded?” — candid about ownership.

One-page cheat sheet

Say in the first two minutes

“My goal is to understand your architecture and what a proof would need to show — not to pitch. If it's useful, we scope a validation at the end. Fair?”

Five facts to hold cold

One OpenAI-compatible interface, 100+ providers; virtual keys carry budgets and rate limits
Postgres required for UI, keys, registries, budgets; no DB = no budget enforcement
Redis shares cooldowns and TPM/RPM state across replicas in production
Tenancy: Orgs → Teams → Projects (beta, enterprise) → Keys; budgets enforce inward
Multi-region splits control plane (admin) from data plane (workers) via env flags

Five questions that carry the call

Why now? • SDKs or an existing gateway? • Which endpoints/providers/features exactly? • What's your p95/p99 and SLO, and behavior on 429/outage? • What must a trial demonstrate, who signs off, by when?

Never do

Invent an SLA or performance number • claim universal compatibility • wave away Postgres/Redis ownership • repeat uncorroborated benchmark or security claims as fact — convert them into validation tests

Closing script

“Here's what I heard: [architecture, trigger, three requirements]. I propose a two-week validation: your two busiest apps, two providers, one injected failure, your pass/fail metrics. [Engineer] owns the environment, I own config support and open questions — including the ones I said I'd verify. Can we lock a start date?”

Technical evidence and reading list

All 23 sourced rows. official rows govern product facts; independent rows are evaluator concerns to investigate — including a reported supply-chain incident and high-concurrency benchmarks — not verified product truth.

Type	Fact	Caveat	Source

Method: 23 de-duplicated factual rows on LiteLLM (15 official docs pages, 8 independent implementation reviews) across 8 independent hosts, one row per URL, researched 2026-09-12. Facts are quoted or condensed; official documentation governs product claims and independent claims are labeled as concerns to validate. Full enterprise-relevance text trimmed for space; caveats shortened in the table. Timeline phases are the recommended call plan, not measured data.

This report was generated automatically by Keenable SELECT at a user's request, from publicly available web sources linked herein. Keenable does not review, verify, or endorse its contents and makes no representation as to accuracy, completeness, or timeliness; AI-based extraction may contain errors. Nothing in this report is investment, legal, financial, or other professional advice. All trademarks and referenced content remain the property of their respective owners; no affiliation or endorsement is implied. To report an error, rights concern, or request removal: legal@keenable.ai.