FJurmanovic/sugarr
1
0
Fork 0
mirror of https://github.com/FJurmanovic/sugarr.git synced 2026-11-11 15:51:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix Langfuse usage tracking: map to snake_case keys it recognizes

Browse Source 
LangfuseTracer was passing our internal camelCase usage fields
(promptTokens/completionTokens/totalTokens) straight through as
usageDetails. Langfuse only recognizes a flat schema (input/output/...)
or OpenAI's native snake_case schema (prompt_tokens/completion_tokens/
total_tokens) for usage/cost calculation - anything else is silently
ignored, so every generation was showing zero usage regardless of
what LlmClient actually reported.

Note: cost still requires a price-list entry for the model name.
Langfuse's built-in list only covers hosted providers (OpenAI,
Anthropic, Google); self-hosted models (e.g. LLM_MODEL=llama3 via
Ollama) need a custom model price added in Project Settings > Models
for a $ figure to show up, even with token counts now correct.

Verified: full suite (80/80) passing, tsc/eslint clean.
This commit is contained in:
Fran Jurmanović
2026-08-12 23:00:04 +02:00
parent e47c57b50d
commit 6f0300f038
2 changed files with 33 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
test/core/langfuse/LangfuseTracer.test.ts
View File

@@ -59,7 +59,10 @@ describe('LangfuseTracer', () => {
const tracer = new LangfuseTracer({ publicKey: 'pk', secretKey: 'sk', baseUrl: 'http://lf' });
const handle = tracer.startTrace('my-pipeline');
const gen = handle.startGeneration({ name: 'step-1', model: 'llama3', input: ['hi'] });
gen.end({ output: 'done', usage: { totalTokens: 5 } });
gen.end({
output: 'done',
usage: { promptTokens: 10, completionTokens: 5, totalTokens: 15 },
});
expect(childStartObservation).toHaveBeenCalledWith(
'step-1',
@@ -67,11 +70,23 @@ describe('LangfuseTracer', () => {
{ asType: 'generation' }
);
expect(generationUpdate).toHaveBeenCalledWith(
expect.objectContaining({ output: 'done', usageDetails: { totalTokens: 5 } })
expect.objectContaining({
output: 'done',
usageDetails: { prompt_tokens: 10, completion_tokens: 5, total_tokens: 15 },
})
);
expect(generationEnd).toHaveBeenCalledTimes(1);
});
it('omits usageDetails entirely when no usage is reported', () => {
const tracer = new LangfuseTracer({ publicKey: 'pk', secretKey: 'sk', baseUrl: 'http://lf' });
const handle = tracer.startTrace('my-pipeline');
const gen = handle.startGeneration({ name: 'step-1', model: 'llama3', input: ['hi'] });
gen.end({ output: 'done' });
expect(generationUpdate).toHaveBeenCalledWith(expect.objectContaining({ usageDetails: undefined }));
});
it('flush() force-flushes the span processor without shutting down the SDK', async () => {
const tracer = new LangfuseTracer({ publicKey: 'pk', secretKey: 'sk', baseUrl: 'http://lf' });
await tracer.flush();