createProviderCounter
Auto-generated API reference for createProviderCounter.
Function: createProviderCounter()
createProviderCounter(
options):TokenCounter
Defined in: token-counter.ts:168
Create a token counter backed by a real tokenizer.
This factory lets you plug in any tokenizer library (tiktoken, Anthropic's
tokenizer, etc.) while conforming to the TokenCounter contract.
#Parameters
#options
#Returns
TokenCounter
#Example
import { createProviderCounter } from '@agentskit/observability'
import { encoding_for_model } from 'tiktoken'
const enc = encoding_for_model('gpt-4o')
const tiktokenCounter = createProviderCounter({
name: 'tiktoken',
tokenize: (text) => [...enc.encode(text)],
})
const tokens = await countTokens(messages, { counter: tiktokenCounter })