Managed inference APIs price in their margin
Convenient, until volume grows. Past a measurable break-even, running your own serving stack costs a fraction per token — but only if utilization, batching, and quantization are engineered, not defaulted.
AI Inference Services
Last reviewed AI inference, served and operated — at a cost you can see.
We design, deploy, and operate LLM inference: the serving stack, model routing, autoscaling, quantization, and the cost telemetry that tells you what every million tokens actually costs. In your cloud, on your premises, or at the edge. The numbers behind this service are published: an optimized 70B stack measured at $0.47 per million tokens, and 158 tokens/sec aggregate on a $499 edge board — failures included.
$0.47/M
Measured serving cost, 70B class
158 TPS
Edge inference, measured
2.3×
Throughput from the same GPUs
Default serving configs burn paid-for hardware
FP16 weights, idle KV-cache, no batching strategy: the same GPUs serve a fraction of what they could. Serving is an engineering discipline with a measurable output — cost per million tokens.
Nobody owns inference after launch
Models update, traffic shifts, latency degrades quietly. Operated inference means monitoring, routing policy, capacity planning, and a cost report you can read — not a dashboard nobody checks.
Audit Focus
What we inspect before prescribing a platform change.
The first pass is designed to identify the smallest useful intervention: routing, caching, prompt control, serving tuning, or a deeper break-even audit.
Workload shape: models, token volume, latency targets, concurrency, growth
Break-even math first: API vs cloud GPU vs on-premise, from measured numbers
Serving stack: vLLM-class engines, quantization, KV-cache, continuous batching
Routing: match each request class to the cheapest adequate model
Autoscaling and utilization: capacity that follows traffic, not fear
Cost telemetry: per-model, per-route cost per million tokens, live
Where your inference should run — see the full map
The honest decision table — we deploy and operate all three, so the recommendation follows the math, not the offering.
| Route | Fits when | What we run |
|---|---|---|
| Managed API | Low or spiky volume, frontier-model needs | Routing, caching, and cost telemetry on top of your provider |
| Your cloud GPUs | Steady volume past API break-even | vLLM-class serving, quantization, autoscaling, utilization tuning |
| On-premise / edge | Data residency, GDPR, or unit-economics demands | Full stack in your perimeter — sized from published benchmarks |
How It Works
How the audit works
Step 1
Share spend and workload shape
Submit monthly spend range, provider mix, token volume, and what your inference workloads actually do: chat, RAG, agents, extraction, or batch jobs. Takes minutes, no production access needed.
Step 2
Map cost to workflows
We break the invoice into cost per completed user action: prompt and output tokens, retries, retrieval, tool calls, and orchestration overhead — the 72% of AI cost that hides outside the model call.
Step 3
Rank levers by friction and savings
Each leak gets a lever — routing, caching, prompt compression, retry control, quantization, or a private break-even case — ranked by expected savings against implementation effort.
Step 4
Get the written read, then decide
You receive the first cost-leak read in writing. If the economics justify deeper work, the next step is a scoped engagement; if not, you keep the findings.
Tell us the model, the volume, and the latency target.
A 30-minute consult maps your workload to a serving architecture and a measured cost per million tokens — before any commitment. If the managed API is still your cheapest option, we say so.
$47K → $28K
Case study: a Llama 3 70B production workload moved from 4 GPUs to 2 with INT8 quantization, KV-cache pruning, and serving changes — a 42% monthly cost cut with 2.3x throughput.
Read the full auditFounder experience — US roles & engagements at
Code and Theory (Stagwell Group)ProsciaWolframHeyNeoTeCSAR Lab · UNC Charlotte
Shipped Work
Output that shipped and scaled.
Products we've engineered and clients we've served — real platforms in production, not slideware.
Construction SaaS
BuildUNIX
Construction execution platform for PMC firms — phase-gated workflows, tamper-proof site records, and snag-to-handover tracking that replaces spreadsheets and WhatsApp threads.
Full-stack platform: Next.js, Postgres, async job pipeline, field-team mobile flows.
Enterprise AI SaaS
VectraGPT
Secure AI chatbots for HR, support, and sales — RAG-powered answers from company documents with audit trails, aligned to GDPR and SOC 2 expectations.
RAG platform end to end: ingestion, retrieval, guardrails, multi-tenant serving.
Developer SaaS
XeoRank
SEO, AEO & GEO intelligence engine for developers — triple scoring, Search Console integration, and CLI-first plus MCP agent workflows.
Agent-first product: MCP server, site crawler, scoring engines, WordPress control plane.
AI Creative SaaS
AdFargo
AI creative strategist — learns a brand from its URL and generates on-brand ads, video, and copy built to convert, in minutes.
Generative pipeline: brand ingestion to multi-format creative output.
SEO SaaS
Rankgent
Programmatic SEO platform — generate, preview, and publish thousands of service + location pages in minutes. Trusted by 500+ SEO professionals.
Page-generation engine: templating, bulk preview, and publishing at 50K+ pages scale.
Also trusted by
FAQ
Common questions
What are AI inference services?
The engineering and operation of the systems that run trained models in production: serving frameworks, GPU capacity, quantization, batching, routing, autoscaling, and cost monitoring. Training builds the model; inference is what you pay for every day after — and it is where most AI budgets actually leak.
What does an AI inference platform cost to run?
Measured, not estimated: our published 70B-class stack serves at $0.47 per million tokens after optimization, and an edge-class deployment runs about $14/month per board all-in. Your number depends on models, volume, and latency targets — the consult produces a projected cost per million tokens for your workload before you commit to anything.
Should I use an inference API or run my own serving?
Below break-even volume, the managed API usually wins — and we say so. Past it, owned serving costs a fraction per token. The break-even point is calculable from your bill and workload shape; our free audit and the on-prem cost estimator both compute it from measured numbers.
Which serving stack do you deploy?
vLLM-class engines with continuous batching, quantization tuned to the quality bar your evals set, KV-cache management, and routing across model tiers. The exact stack follows the workload — the same playbook that produced the published 42% cost reduction.
Can you run inference at the edge?
Yes — it is one of our published specialties. We benchmarked LLM serving on a $499 Jetson Orin board at 158 tokens/sec aggregate under 8 watts, raw data included, and the Edge LLM Sizing tool turns those measurements into fleet sizing and cost for your workload.
Who operates it after deployment?
Either side. We hand over with runbooks, monitoring, and training — or operate it under a managed retainer with daily written progress and monthly cost reporting. Your infrastructure and models remain yours in both cases.