Virtualized memory

A chat session with 10k messages shouldn't blow up load(). createVirtualizedMemory wraps any ChatMemory with a fixed "hot" window — recent messages always loaded — and lets you plug in a retriever to surface relevant older messages on demand.

Install

Built into @agentskit/core.

Quick start

import { createInMemoryMemory, createVirtualizedMemory } from '@agentskit/core'

const backing = createInMemoryMemory(history) // 10k messages
const memory = createVirtualizedMemory(backing, { maxActive: 50 })

const visible = await memory.load()      // latest 50
await memory.save([...visible, newMsg])  // cold 9,950 preserved

• maxActive caps the number of recent messages returned.
• Backing store always holds everything — size() / loadAll() expose the full history.
• save merges visible messages with the cold tail, so load → mutate → save doesn't silently truncate history.

Surface older messages on demand

Plug in a retriever. Typical impl: embed the latest user message and hit a vector store for the top-K cold matches.

const memory = createVirtualizedMemory(backing, {
  maxActive: 30,
  maxRetrieved: 5,
  retriever: async ({ hot, cold, maxRetrieved }) => {
    const latest = hot[hot.length - 1]
    const embedding = await embed(latest.content)
    const hits = await vectorStore.search(embedding, { topK: maxRetrieved })
    return cold.filter(m => hits.some(h => h.id === m.id))
  },
})

const merged = await memory.load()
// returns retrieved cold msgs spliced chronologically before hot window

Retrieved messages are spliced in chronological order with the hot window, and duplicates are filtered out.

When to use this

• Long-running assistants where users scroll back weeks later.
• Agents whose task history grows unbounded (background crons).
• Any session that would otherwise OOM on load().

Pair with token budget — virtualized memory caps count, compileBudget caps tokens.

See also

• Persistent memory
• Token budget compiler