Home / Docs / Persistence

Persistence

Persistence

FabrCore provides built-in persistence through Orleans grain state, enabling agents to maintain conversation history and custom data across activations, cluster restarts, and silo migrations.

FabrCoreChatHistoryProvider

FabrCoreChatHistoryProvider extends Microsoft's ChatMessageStore with Orleans-backed persistence. It's automatically wired when using CreateChatClientAgent.

Buffered Writes

Messages are held in memory until flush — fast, no I/O per message.

Lazy Loading

History is loaded from Orleans state on first access, not on initialization.

Automatic Persistence (Recommended)

PersistentAgent.cs 
[AgentAlias("PersistentAgent")]
public class PersistentAgent : FabrCoreAgentProxy
{
    private AIAgent? agent;
    private AgentThread? thread;

    public override async Task OnInitialize()
    {
        (agent, thread) = await CreateChatClientAgent(
            "default",
            threadId: config.Handle);
    }

    public override async Task<AgentMessage> OnMessage(AgentMessage message)
    {
        var response = message.Response();
        var result = await agent!.RunAsync(message.Message, thread);
        response.Message = result.Text;
        return response;
        // No manual FlushAsync() needed!
    }
}
Auto-Flush

After each OnMessage completes, the Orleans grain automatically calls FlushAsync() on all tracked stores. On grain deactivation, any remaining pending messages are flushed to ensure no data loss.

Thread ID Strategies

StrategyThread IDUse Case
One per agentconfig.HandlePersonal assistant, single-user agents
Per-usermessage.FromHandleMulti-user agents
Per-channelmessage.Channel ?? "default"Channel-based chat
Combined$"{fromHandle}:{channel}"User-specific within channels

API Reference

MethodDescription
AddMessagesAsync(messages)Add messages to in-memory buffer (fast, no I/O)
GetMessagesAsync()Get all messages (persisted + pending), lazy-loads from Orleans
FlushAsync()Persist pending messages to Orleans grain state
HasPendingMessagesReturns true if there are unsaved messages
ThreadIdThe unique thread identifier

Custom State

Persist arbitrary typed data (user info, counters, preferences) that survives grain deactivation:

MethodDescription
GetStateAsync<T>(key)Get a strongly-typed value by key
GetStateOrCreateAsync<T>(key, factory)Get existing value or create with factory
HasStateAsync(key)Check if a key exists
SetState<T>(key, value)Set a value (buffered until flush)
RemoveState(key)Remove a key (buffered until flush)
FlushStateAsync()Persist all pending changes to Orleans
Custom State Example 
public override async Task<AgentMessage> OnMessage(AgentMessage message)
{
    var response = message.Response();

    // Get or create pattern
    var stats = await GetStateOrCreateAsync(
        "stats", () => new ConversationStats());
    stats.MessageCount++;
    stats.LastMessage = DateTime.UtcNow;
    SetState("stats", stats);

    // Process message...
    var result = await agent!.RunAsync(message.Message, thread);
    response.Message = result.Text;

    await FlushStateAsync();
    return response;
}
Best Practice

Group related state updates and call FlushStateAsync() once at the end of OnMessage. On grain deactivation, pending changes are auto-flushed to prevent data loss.

Message Compaction

Long conversations fill the model's context window. Compaction automatically summarizes older messages when the conversation grows too large.

How It Works

Message History Token Estimate Threshold Check LLM Summarization Compacted History
  1. Estimate — Token count of all stored messages is estimated before each message
  2. Threshold — If the estimate exceeds the configured threshold (default 75%), compaction triggers
  3. Summarize — Older messages are sent to the LLM for summarization, preserving key decisions, facts, and context
  4. Replace — Older messages are replaced with a single [Compacted History] summary. The most recent messages (default 20) are always kept intact

Configuration

SettingDefaultDescription
CompactionEnabledtrueEnable or disable compaction
CompactionKeepLastN20Recent messages to always preserve
CompactionThreshold0.75Trigger at this ratio of context window

Using TryCompactAsync

Call TryCompactAsync() before model invocations to ensure the history fits within the context window:

Compaction Before Model Invocation 
public override async Task<AgentMessage> OnMessage(AgentMessage message)
{
    // Compact before invoking the model
    var compaction = await TryCompactAsync();
    if (compaction?.WasCompacted == true)
    {
        logger.LogInformation(
            "Compacted: {Original} -> {Compacted} messages",
            compaction.OriginalMessageCount,
            compaction.CompactedMessageCount);
    }

    // Now invoke the model with compacted history
    var response = await agent!.RunAsync(message.Message, thread);
    // ...
}
Why Explicit Compaction?

Compaction invokes an LLM call for summarization, which adds latency. Explicit TryCompactAsync() calls give you control over when this happens — typically before model invocations, not during message addition.

Requires ContextWindowTokens

Compaction requires ContextWindowTokens set on your model in fabrcore.json. Without it, the agent can't determine when the context window is filling up.

Orleans Storage Providers

Message threads and custom state are stored in Orleans grain state. Supported providers:

  • In-memory — Development only, data lost on restart
  • SQL Server — Production, durable
  • Azure Table Storage — Cloud-native
  • ADO.NET providers — PostgreSQL, MySQL, etc.
Documentation