FJurmanovic/sugarr
1
0
Fork 0
mirror of https://github.com/FJurmanovic/sugarr.git synced 2026-11-11 15:51:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Track cached prompt tokens through to Langfuse

Browse Source 
OpenAI's chat completions response includes prompt_tokens_details.
cached_tokens when prompt caching kicks in - previously dropped
entirely. Now extracted in LlmClient and forwarded to Langfuse as the
flat cache_read_input_tokens usage key.

Used the flat key rather than OpenAI's nested prompt_tokens_details
shape: the installed @langfuse/tracing SDK's usageDetails type is
Record<string, number> | OpenAiUsage (a 3-field camelCase shape, no
_details nesting) even though the wire schema in @langfuse/core
supports nesting - a gap in the SDK's public types, not something
worth fighting with a cast.

Verified: full suite (82/82) passing, tsc/eslint clean.
This commit is contained in:
Fran Jurmanović
2026-08-12 23:15:42 +02:00
parent 6f0300f038
commit df4ca0bf76
5 changed files with 65 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
src/core/langfuse/LangfuseTracer.ts
View File

@@ -16,6 +16,14 @@ import type {
* OpenAI's native snake_case schema (prompt_tokens/completion_tokens/...);
* our own ChatResult.usage uses camelCase for our internal contract, so it
* needs translating at this boundary or Langfuse silently shows zero usage.
*
* Cached tokens use the flat schema's cache_read_input_tokens key rather
* than OpenAI's nested prompt_tokens_details.cached_tokens: the installed
* @langfuse/tracing SDK's usageDetails type is `Record<string, number> |
* OpenAiUsage` (a 3-field camelCase shape with no _details nesting) even
* though the wire schema (@langfuse/core's OpenAiCompletionUsageSchema)
* supports the nested form — keeping everything as flat numbers avoids
* fighting that gap in the SDK's public types.
*/
function toUsageDetails(usage: GenerationOutput['usage']): Record<string, number> | undefined {
if (!usage) return undefined;
@@ -23,6 +31,7 @@ function toUsageDetails(usage: GenerationOutput['usage']): Record<string, number
if (usage.promptTokens !== undefined) details.prompt_tokens = usage.promptTokens;
if (usage.completionTokens !== undefined) details.completion_tokens = usage.completionTokens;
if (usage.totalTokens !== undefined) details.total_tokens = usage.totalTokens;
if (usage.cachedTokens !== undefined) details.cache_read_input_tokens = usage.cachedTokens;
return Object.keys(details).length > 0 ? details : undefined;
}