The FabrCore Agent Proxy — Client-Side Pattern with Busy-State Interleaving

The Problem: Single-Threaded Agents and Long-Running LLM Calls

Orleans grains are single-threaded by default. When a grain method is executing, all other calls to that grain queue up and wait. For a typical CRUD grain that finishes in milliseconds, this is fine. For an AI agent that may invoke multiple LLM calls and tool executions in a single turn, it creates a terrible user experience — the second caller blocks until the first call finishes, with no feedback at all.

The naive solution is to make the entire grain reentrant, but that opens the door to concurrent state mutations. If two OnMessage calls run simultaneously and both write to the chat history or modify custom state, you get corrupted data.

FabrCore takes a different approach: the OnMessage method on the underlying IAgentGrain is marked with Orleans' [AlwaysInterleave] attribute. This allows a second call to enter the grain while the first is still running. But instead of letting both calls execute OnMessage, the grain checks whether a primary message is already being processed and routes the second call to a different method: OnMessageBusy.

How Busy-State Interleaving Works

The flow is straightforward:

1. Message A arrives. No primary message is running, so it enters OnMessage normally.
2. While Message A is awaiting an LLM call, Message B arrives.
3. The grain detects that OnMessage is already active and routes Message B to OnMessageBusy instead.
4. Message B gets an immediate response without waiting for Message A to complete.

The default OnMessageBusy implementation returns a simple acknowledgment telling the caller the agent is currently processing another request. You can override it to provide richer context:

public override Task<AgentMessage> OnMessageBusy(AgentMessage message)
{
    var primaryMsg = ActiveMessage;
    return Task.FromResult(new AgentMessage
    {
        ToHandle = message.FromHandle,
        Message = $"I'm currently working on a request"
            + (primaryMsg != null
                ? $" from {primaryMsg.FromHandle}"
                : "")
            + ". I'll be available shortly.",
        Kind = MessageKind.Response,
        TraceId = message.TraceId
    });
}

The ActiveMessage property gives you read-only access to the message currently being processed by the primary OnMessage handler. This lets you tell the caller what the agent is working on without interrupting the work in progress.

There are important safety rules for OnMessageBusy:

• Do not mutate shared state. The primary OnMessage may be at any await point when OnMessageBusy executes. Writing to chat history, custom state, or any shared data structure risks corruption.
• Read-only operations are safe. Reading the agent configuration, checking the active message, or querying external services is fine.
• Chat history is not flushed after OnMessageBusy — only after the primary OnMessage completes.
• No heartbeat or compaction runs for busy-routed messages. They are lightweight by design.

Routing Timer Messages When Busy

Agents that use timers or reminders may receive timer-triggered messages while already processing a user request. You can route these differently in your OnMessageBusy override:

public override Task<AgentMessage> OnMessageBusy(AgentMessage message)
{
    // Timer messages can be identified by their MessageType
    if (message.MessageType?.StartsWith("timer:") == true)
    {
        // Skip timer work when busy — the next tick will catch up
        return Task.FromResult(message.Response());
    }

    // Default busy response for user messages
    return base.OnMessageBusy(message);
}

This keeps timer processing from queuing up behind a long-running user conversation, and prevents timer messages from generating confusing "I'm busy" responses to internal processes.

Stale Message Protection and Monitoring

What happens when an OnMessage call gets stuck — a deadlocked tool, a hung LLM call, or a network timeout that exceeds the configured limit? FabrCore includes stale message protection: if the primary OnMessage has been running for more than 5 minutes, the grain treats the agent as stuck and allows the new message through as a fresh primary instead of busy-routing it. This prevents a single bad call from permanently blocking the agent.

All busy-routed messages are recorded in the agent message monitor with BusyRouted = true, so you can track how often your agents are hitting concurrency and whether your busy responses are meeting user expectations.

On the client side, setting up a context and sending messages is straightforward with IClientContextFactory:

@inject IClientContextFactory ContextFactory

@code {
    private IClientContext? _context;

    protected override async Task OnInitializedAsync()
    {
        _context = await ContextFactory.GetOrCreateAsync("user1");
        _context.AgentMessageReceived += OnAgentMessage;
    }

    private void OnAgentMessage(object? sender, AgentMessage message)
    {
        // Both normal and busy responses arrive here
        InvokeAsync(StateHasChanged);
    }
}

Because SendMessage is fire-and-forget with responses arriving through the AgentMessageReceived event, the client never blocks. Whether the agent responds immediately via OnMessageBusy or after a full LLM turn via OnMessage, the response flows through the same event handler. The client UI can display busy responses differently (a subtle status message vs. a full chat bubble) based on message metadata.