A normal API call either worked or returned an error, and for debugging that is nearly enough. An LLM call is harder. The response can be valid JSON and still be wrong in meaning, and the bill keeps growing even on calls that succeeded. None of this shows up without a log built for it.
Why LLM calls need their own log layer
Regular application logs record that a call happened and which status code came back. For an LLM that falls short: a conventional API either failed or worked, while a model can quietly get things wrong inside a perfectly successful response.
A dedicated log around every model call does two jobs at once. It lets you reproduce a specific case while debugging, and it shows where the money went, with no spreadsheet reconciliation at month end.
A request id on every call
Write a unique request id first. With thousands of requests a day, nothing else ties together a user's complaint, a line in the application log and the model call behind it.
The best approach is to generate the id where a request enters your system and carry it along the whole path, from the user's action to the API call and back. One support ticket then unfolds into a chain of events instead of "something went wrong somewhere."
Tokens and latency per call
Record input and output tokens every time, and keep them separate. The counts serve both the bill and diagnostics. A sudden jump in input tokens often means extra context slipped into the prompt by accident. A jump in output tokens usually means the model began answering at greater length, sometimes because the prompt or the system instruction changed.
Measure latency on its own, apart from the application's total response time. A slow reply has several possible causes: the model, the network, a queue in front of the call. A separate measurement shows straight away where the delay sits.
Attribution to a key and a user
The most useful step, and the one skipped most often, is linking every call to whoever made it: which API key, which team, which user or feature inside the product. The monthly total answers "how much did we spend." It does not answer "who spent it, and on what."
Without attribution, tracing a cost spike means testing hypotheses one by one. With it, a breakdown is enough: the key or feature that suddenly started making far more calls, or producing much longer answers, stands out.
codexkey already tracks usage per key. If you issue a separate key for each feature or team, the attribution comes free, with no extra code on the product side. Sub-organizations with their own budgets and thresholds exist for this reason: you see which team or product is nearing its limit before the spend turns into a problem.
What to log from the content, and what to leave out
The full prompt and response text helps with debugging, but storing it as is carries risk, especially when a conversation may contain a user's personal information. A reasonable compromise: write the full content to a separate store with restricted access and a short retention period, and keep only a hash or a short excerpt in the main logs and metrics.
Call metadata (the model, the prompt version, the tool name in an agent flow) is worth logging always and without limits. It holds no personal data, and it gives you most of what you need to debug without the conversation itself.
From logs to budgets and alerts
Raw logs solve nothing on their own. What you do with them matters. Compute three numbers from them on a regular schedule: total spend per period for each key, the share of calls with an abnormally high token count, and the share of failed calls that had to be retried.
Those numbers then become thresholds: a notification when you approach the monthly limit, an alert on a sudden latency spike, a flag on a key whose tokens per call suddenly multiplied. codexkey already shows part of this picture at the key and sub-organization level, and the public status page answers "is it us or the provider" without an internal investigation.
A practical case: a key issued to an email-sending feature starts producing messages three times longer than usual after someone edits the system prompt. Without a per-key breakdown, that surfaces only at month end, in the total bill. With one, an alert on the abnormal token jump fires within a day.
Short version
The minimum log around an LLM call: request id, input and output tokens counted separately, latency, attribution to a key and a user. Keep conversation content in a separate, restricted store and log metadata freely. On that base, money is easy to track and bugs are easy to find, so you stop guessing from a total bill at the end of the month.