NavyaAI logoNavyaAI
Back to Home

Commercial guide · Last reviewed 2026-08-17

The AI Audit Checklist: Where AI Bills Actually Leak

A practical AI audit checklist from real audits: the questions that find spend leaks in agent loops, retries, routing, serving, and infrastructure — before you buy more capacity.

Short answer

An AI audit is a structured review of what an AI system costs per completed task and where that cost leaks: model routing, retries, agent loops, retrieval overhead, serving utilization, and infrastructure placement. A useful audit produces a written leak map ranked by savings against effort — one of ours cut a production bill from $47K to $28K per month. The checklist below is the same set of questions we ask, published so you can run the first pass yourself.

When it wins

  • Cost per completed task: can you state what one unit of user value costs? If not, that is finding #1 — instrument before optimizing.
  • Model routing: is every workload step on the frontier model, including steps a mid-tier model passes evals on?
  • Retries and loops: what is the retry rate, and do agent loops have budgets and stop conditions — or does failure multiply spend silently?
  • Retrieval overhead: how many tokens does RAG context add per request, and is any of it cached?
  • Serving utilization: if you run GPUs, what fraction of paid capacity does real batching actually use?
  • Idle and oversized capacity: what runs at 3am, and which instances were sized for a launch-day peak that never returned?
  • Egress and data paths: is traffic crossing AZs, regions, or NAT gateways that a same-region design would avoid?
  • Placement math: has anyone computed the API vs self-hosted break-even for your real volume — in either direction?

Comparison table

FactorSelf-audit with this checklistWritten audit by NavyaAI
What you getA first-pass leak map from your own billing and logs.A written leak map with findings ranked by savings vs effort, read personally by the founder.
CostYour time.Free for teams spending $20K+/month; no call required.
Best forTeams under $20K/month, or validating that an audit is worth requesting.Production spend where a 42%-class finding pays for years of attention.
Proof standardDepends on your instrumentation.Same methodology as the published $47K→$28K audit, methods and data public.

Frequently asked questions

What does an AI audit include?

A cost baseline (what the system spends and on what), a leak map across routing, retries, loops, retrieval, serving, and infrastructure, and a ranked fix list — savings against implementation effort. Ours is delivered in writing, so it is useful whether or not you ever engage us.

Is a free AI audit actually free?

Ours is, for teams spending $20K+ per month on inference: you share spend range, provider mix, and workload shape through a short intake, and receive the written leak map with no call required. The business logic is simple — some audited teams become optimization clients, and the audit proves the method either way.

How is an AI cost audit different from a cloud cost review?

A cloud review reads the infrastructure bill. An AI audit reads the workflow: token flows, model choices, retries, agent behavior, retrieval overhead, and serving efficiency — then connects them to the bill. Most AI spend leaks are invisible at the infrastructure line-item level.

How often should AI systems be audited?

At every step-change: traffic growth, new models, new agent features, or a bill that outgrew its explanation. Continuously operated systems fold the audit into monthly cost reporting instead of running it as an event.

References & related

Talk to a human