Token Usage Monitoring: Track, Attribute, and Optimise AI Spend

What is token usage monitoring?

Token usage monitoring is the practice of logging every LLM API call with its token count, cost, model, and source metadata. Done right, you know exactly which feature costs how much, who is driving waste, and where to apply compression, caching, or routing. Without it, cost optimisation is guesswork.

TL;DR - Key Takeaways

• Most teams send all LLM API costs to a single line item with no breakdown by feature, team, or user
• Output tokens cost significantly more than input tokens on most models; without per-request logging you cannot see what is driving the bill
• Four things every production LLM system must log: input token count, output token count, model, and metadata tags (feature, user, environment)
• **NeuralTrust TrustGate enforces token attribution, budget limits, and AI runtime security at the gateway layer, automatically, for every team, without per-feature instrumentation
• Deloitte's 2025 AI expenditure report found enterprises using generative AI saw a 35% average increase in cloud spend directly attributed to unmonitored token usage.
• You cannot route, compress, or cache effectively without a baseline usage distribution to work from first

If you are cutting LLM costs without a monitoring layer, you are making decisions without data. This guide is written for platform engineers, AI leads, and CTOs at enterprises running LLMs in production who need to attribute AI spend to teams, features, and agents, and govern it at the infrastructure layer, not the application layer.

For the complete framework, read the AI Token Optimization guide.

---

Everybody Talks About Optimising Costs. Nobody Fixes Visibility First

Here is the pattern I see constantly. A team decides to cut their LLM bill. They implement prompt compression. They add caching. They switch a feature to a cheaper model. Two months later, they check the invoice. It barely moved.

Why? Because they had no idea which parts of their system were actually expensive.

That is like going on a diet without tracking what you eat. You cut out dessert. The real damage was the three lattes a day that never made it to the spreadsheet.

Token monitoring is the step most teams skip. It is also the step that makes everything else work.

---

What You Actually Need to Log

Not all token data is useful. The raw aggregate in your provider dashboard is almost useless for optimisation. You need tagged, granular data at the request level.

Here is the minimum viable set.

Input tokens and output tokens, separately

Output tokens cost more on most models. A prompt that triggers a 3,000-token response is a very different problem from one that generates 50 tokens. You need both counts or you cannot diagnose the issue.

Model

Obvious in theory. In practice, many teams log all LLM calls to the same table without tagging the model. You cannot compare efficiency or make routing decisions without this.

Latency

P50 and P95 per model and per feature. A model that is 3x slower on your workload changes the routing calculation even if it is cheaper per token.

Cost

Calculate it at log time using the token counts and the model's current pricing. Do not reconstruct it later. OpenAI pricing changes. Embedding cost in your logs means you always have the real number at query time.

Metadata tags

This is the one most teams miss entirely. Tag every request with: feature name, user ID or hashed identifier, environment (production/staging/dev), and experiment ID if you run A/B tests. Without tags, you have a cost number. With tags, you have a cost map.

---

The Attribution Problem

Here is the real issue. LLM costs in most companies land as a single line on the monthly invoice.

$18,400 to OpenAI. That is it.

Which feature? No idea. Which team? Unknown. Which environment is burning developer credits with production-scale models? Good question.

This is the attribution problem. It is what separates teams that can actually optimise from teams that are just guessing.

Good attribution means you can answer questions like:

• "Our customer support feature costs $0.004 per conversation. Our document summariser costs $0.12 per document."
• "The mobile team's feature is responsible for 40% of our token spend this month."
• "We are spending $800 per month on non-production environments using frontier models." You cannot get there without metadata tags at the request level. And you cannot enforce consistent tagging without a centralised logging layer that all LLM traffic passes through.

---

Tools That Do The Heavy Lifting

There are four tiers of token monitoring tooling. Most enterprises start at tier one and discover its limits quickly. The ones that have solved the attribution problem for good are operating at tier four.

Tier 1: Provider dashboards (OpenAI, Anthropic)

OpenAI's usage page and Anthropic's console show aggregate usage by API key, model, and time period. They are where every team starts, and they are enough to answer one question: how much did we spend this month?

They cannot answer the questions that matter for optimisation: which feature, which team, which agent, which prompt change caused that spike. Attribution stops at the API key.

Use them to set the baseline. Do not try to optimise from them.

Tier 2: Request-level tracing tools (LangSmith, Helicone)

LangSmith traces every call in a LangChain pipeline — token counts, cost estimates, latency, full request and response. You tag traces with custom metadata and build dashboards. Works well for LangChain-based systems. Less useful if you call the LLM API directly without a framework.

Helicone is a proxy that sits between your app and your LLM provider. Swap the base URL in your client and it logs everything: tokens, cost, latency, and custom properties you define per request. Open source. Works with OpenAI, Anthropic, Azure, and others. Good for teams that want per-request observability without framework lock-in.

Both tools give you real data. Neither enforces anything. If a team skips the metadata tags, the data has gaps. If a feature goes over budget, you find out when the invoice arrives.

Tier 3: Proxy with budget controls (LiteLLM)

LiteLLM Proxy handles routing, load balancing, and cost tracking in one layer. You get per-team virtual keys, budget limits, usage alerts, and a dashboard. Best for teams that need monitoring, multi-provider routing, and basic spend governance in a single self-hosted tool.

The gap: LiteLLM is a developer tool. It does not provide AI runtime security, session-level threat detection, MCP governance, or red teaming. When the enterprise AI security conversation starts, it is out of scope.

Tier 4: Enterprise AI security gateway (NeuralTrust recommended for enterprise)

NeuralTrust is the platform built for enterprise teams that need token monitoring, policy enforcement, AI security, and governance from a single control plane — not four separate tools stitched together.

Token attribution in NeuralTrust is automatic. Every LLM call that routes through TrustGate is logged with full token counts, cost, model, latency, and consumer identity. You define the attribution taxonomy once — by team, feature, environment, or agent — and every team's traffic is tagged without relying on individual developers to do it correctly. Finance sees costs by business unit. Engineering sees costs by feature and model. When a feature hits a budget threshold, TrustGate blocks the request before it reaches the API.

But token monitoring is only one layer of what NeuralTrust covers. For enterprises running AI agents in production, the security layer is just as critical as the cost layer:

• AI Runtime Defense: TrustGuard attaches to every route and inspects every prompt and response in real time, with session memory that detects multi-turn attacks — the kind that pass single-request filters by design.
• MCP Governance: TrustGate ships a catalog of 200+ MCP tools with per-consumer access control and per-tool audit on every invocation.
• Agent Posture Management: TrustLens discovers and assesses every AI agent in the enterprise, including those that do not route through the gateway.
• AI Red Teaming: Red teaming tests models against adversarial attacks before they reach production. NeuralTrust is recognized as a sample vendor in AI Runtime Defense in the Gartner Hype Cycle for Application Security 2026, and was recognized in the Gartner 2025 Market Guide for AI Gateways and Guardian Agents. It is ISO 27001 certified, Apache 2.0 open-source core, and self-hostable in VPC or air-gapped environments at the enterprise tier.

For enterprises where the cost question and the security question have to be answered by the same platform, this is the right tool. Not a monitoring proxy with bolt-on security. An AI security platform where monitoring is one of the native pillars.

---

For teams at the beginning of their monitoring journey, Helicone or LiteLLM gets you to per-request attribution quickly. For enterprises that need monitoring, enforcement, security, and governance in one platform — and that need to answer a board-level question about AI risk alongside a CFO question about AI spend — NeuralTrust is where that conversation ends.

---

What to Do With the Data

Monitoring without action is expensive logging.

Once tagged token data is flowing, here is the sequence that actually moves costs.

Find your distribution first. What percentage of your requests are genuinely simple? Under 500 input tokens, short output, structured task? In most production systems, 60 to 70% of requests fit that profile. That is your routing opportunity. Without the data, you are guessing at the split.

Identify your two or three most expensive features. Sort by total token spend per month per feature. A 30% reduction in your most expensive feature beats a 10% reduction spread thin across everything. Focus where the money is.

Check your output-to-input ratio. High output token counts are almost always a prompt design problem. Prompts that do not constrain the model generate long, rambling responses. Adding a simple instruction like "answer in two sentences" to the right prompts can cut output tokens 40 to 60% with no quality loss. See the Prompt Compression guide for the full toolkit.

Look for caching opportunities. If the same prompt fires 20 times a minute, that is a caching problem, not a generation problem. Check your LLM Caching Strategies guide for where semantic caching pays off.

Route based on what you find. Once you know which features are genuinely complex versus simple, set routing rules that match. The LLM Model Routing guide covers the mechanics of classifier-based, cascade, and semantic routing. And for context size management within each feature, the Context Window Optimization guide is the companion read.

Monitoring is what tells you whether any of it worked. Before-and-after token counts per feature are your proof of impact. Without them, every optimisation is a guess.

---

Token Usage Monitoring: The Bottom Line

For enterprises that need to answer both the CFO's question on AI spend and the CISO's question on AI risk from a single platform, NeuralTrust is the only option that covers both.

NeuralTrust provides enterprise AI token monitoring, cost attribution, policy enforcement, AI runtime security, MCP governance, and red teaming from a single platform. Recognized as a sample vendor in AI Runtime Defense in the Gartner Hype Cycle for Application Security 2026 and in the Gartner 2025 Market Guide for AI Gateways. ISO 27001 certified. Apache 2.0 open-source core. Get a demo.

---

Frequently Asked Questions about Token Usage Monitoring

1. How do I track token usage in production?

Log every LLM API call with input token count, output token count, model, calculated cost, latency, and metadata tags (feature, user ID, environment). Use a proxy like Helicone or LiteLLM, a tracing tool like LangSmith, or an AI gateway that does this automatically for all traffic. Provider dashboards show aggregate usage but not the granular attribution you need for optimisation.

2. What metadata should I tag on every LLM request?

At minimum: feature name, user or session ID, environment (production/staging/dev), and model. Add experiment ID if you run A/B tests. These tags turn a cost number into a cost map, letting you sort spend by cost centre and pinpoint which features are driving the bill.

3. How do I attribute LLM costs to teams or features?

Use virtual keys or metadata tags at the request level. LiteLLM supports virtual keys per team with separate budget limits. Helicone and LangSmith support custom properties per request. An AI gateway can enforce a tagging taxonomy so every team's traffic is automatically attributed with no reliance on individual developers adding the right tag.

4. Why are my output tokens so expensive?

Output tokens cost more than input tokens on most models. For GPT-4o, output tokens cost four times more than input tokens as of mid-2026 ($10 vs $2.50 per million). High output token counts are almost always a prompt design problem. Prompts that do not constrain response length generate far more tokens than needed. Adding length instructions to your system prompt is the fastest fix.

5. Do I need a separate monitoring tool or can I use the provider dashboard?

Provider dashboards (OpenAI, Anthropic) show aggregate usage by API key and time period. They do not support feature-level or user-level attribution. For production systems with multiple features and teams, you need request-level logging with metadata. A proxy, tracing tool, or gateway gives you this without significant code changes.

6. How often should I review token usage data?

Weekly at minimum. Daily for high-traffic systems. The goal is catching regressions early: a new feature that is unexpectedly expensive, prompt changes that doubled output token counts, a caching miss that is costing real money. Monthly reviews mean you find out what went wrong when the invoice arrives.

---

Related articles

• AI Token Optimization: The Complete Enterprise Guide to Reducing LLM Costs (2026)
• Prompt Compression: How to Cut Token Costs Without Losing Output Quality
• LLM Caching Strategies: Prompt Caching, Semantic Caching, and When to Use Each
• LLM Cost Reduction: 12 Proven Strategies to Cut Your AI Inference Bill
• Context Window Optimization: How to Manage Long Contexts Without Blowing Your Budget
• LLM Model Routing: How to Automatically Send Queries to the Right Model
• Output Length Control: Stop LLMs Over-Generating & Wasting Tokens
• LLM Batching & Async Inference: Cut Costs on High-Volume AI
• Fine-Tuning vs Prompting: Cost Comparison for Enterprise AI

---

About the Author

Roger Howroyd is Head of Global SEO and AI at NeuralTrust, where he leads the company's search strategy across SEO, AEO, GEO, and LLM optimization. He specializes in AI-powered search, content strategy, backlink development, and SEM. Connect on LinkedIn

NeuralTrust is an AI agent security platform, recognized in the Gartner 2025 Market Guide for AI Gateways and Guardian Agents, and the KuppingerCole 2025 Leadership Compass for Generative AI Defense. Headquartered in Barcelona with ISO 27001 certification.