Monitoring

Agent Monitoring

FabrCore provides a pluggable monitoring system for observing message traffic, event streams, and internal LLM calls across your agent system. Messages, events, and LLM calls live in separate buffers with independent query methods and notification events.

IAgentMessageMonitor

The IAgentMessageMonitor interface is the core abstraction for agent monitoring. FabrCore ships with two implementations:

Implementation	Description
`InMemoryAgentMessageMonitor`	Default bounded FIFO buffer for messages, events, and LLM calls
`NullAgentMessageMonitor`	No-op implementation registered when monitoring is not enabled

Enabling Monitoring

Monitoring is opt-in. Enable it in AddFabrCoreServer:

C# — Enable In-Memory Monitor

builder.AddFabrCoreServer(options =>
{
    // Metadata-only LLM capture by default
    options.UseInMemoryAgentMessageMonitor();
});

// Or with full LLM payload capture
builder.AddFabrCoreServer(options =>
{
    options.UseInMemoryAgentMessageMonitor(capture =>
    {
        capture.CapturePayloads = true;
        capture.MaxPayloadChars = 4000;
        capture.MaxToolArgsChars = 2000;
        capture.MaxBufferedCalls = 1000;
        capture.Redact = s =>
            Regex.Replace(s, "sk-[A-Za-z0-9]+", "***");
    });
});

Security: When CapturePayloads = true, the monitor stores actual prompts and responses sent to the LLM. These can contain PII or secrets. Always configure Redact and ensure your storage is trusted. The default is metadata-only.

To use a custom implementation:

C# — Custom Monitor

builder.AddFabrCoreServer(options =>
{
    options.UseAgentMessageMonitor<SqlAgentMessageMonitor>();
});

Monitored Types

MonitoredMessage

Captures every message an agent receives (inbound) and every response it sends (outbound).

Property	Type	Description
`Id`	string	Unique message identifier
`AgentHandle`	string	Fully-qualified agent handle
`Direction`	MessageDirection	Inbound or Outbound
`Timestamp`	DateTimeOffset	When the message was recorded
`Body`	string?	Message body text
`LlmUsage`	LlmUsageInfo?	Token usage for outbound responses

MonitoredEvent

Captures events that reach an agent's OnEvent handler (fire-and-forget).

Property	Type	Description
`Id`	string	Event identifier
`AgentHandle`	string	Agent that received the event
`Type`	string	CloudEvents-compatible event type
`Subject`	string?	Event subject
`Timestamp`	DateTimeOffset	When the event was recorded

MonitoredLlmCall

Captures every internal LLM request/response call an agent makes, regardless of where it originates.

Property	Type	Description
`AgentHandle`	string	Agent that made the call
`Model`	string?	Model used for the call
`InputTokens`	int	Prompt tokens consumed
`OutputTokens`	int	Completion tokens generated
`DurationMs`	long	Call duration in milliseconds
`Streaming`	bool	Whether the call used streaming
`OriginContext`	string	Where the call originated (see below)
`ParentMessageId`	string?	Linked message ID when called from OnMessage
`Timestamp`	DateTimeOffset	When the call was recorded
`RequestMessages`	List?	Prompt payloads (when CapturePayloads enabled)

OriginContext values:

Value	Description
`OnMessage:<id>`	Inside a normal OnMessage flow
`OnEvent:<type>`	Inside an OnEvent handler
`Timer:<name>`	Inside a timer tick
`Reminder:<name>`	Inside a reminder tick
`Compaction`	Inside chat history compaction
`Background`	Outside any scope

Querying Monitored Data

Inject IAgentMessageMonitor and use the three separate query methods:

C# — Querying Messages, Events, and LLM Calls

public class MonitorDashboard
{
    private readonly IAgentMessageMonitor _monitor;

    public MonitorDashboard(IAgentMessageMonitor monitor)
    {
        _monitor = monitor;
    }

    public async Task ShowActivity(string agentHandle)
    {
        // Messages (most recent first)
        var messages = await _monitor.GetMessagesAsync(
            agentHandle: agentHandle, limit: 50);

        // Events delivered to the agent
        var events = await _monitor.GetEventsAsync(
            agentHandle: agentHandle, limit: 50);

        // LLM calls made by the agent
        var llmCalls = await _monitor.GetLlmCallsAsync(
            agentHandle: agentHandle, limit: 20);

        foreach (var call in llmCalls)
        {
            Console.WriteLine(
                $"[{call.Timestamp:HH:mm:ss}] {call.OriginContext} " +
                $"model={call.Model} tokens={call.InputTokens}/{call.OutputTokens} " +
                $"dur={call.DurationMs}ms");
        }
    }
}

Token Tracking

FabrCore tracks LLM token usage at two levels:

Per-Call Tracking

Every LLM call captured by the monitor includes InputTokens and OutputTokens. The TokenTrackingChatClient wraps any IChatClient to automatically record each call to the monitor.

Per-Agent Totals

The AgentTokenSummary provides accumulated totals per agent:

C# — Token Summary

var summary = await _monitor.GetTokenSummaryAsync("user1:my-agent");

Console.WriteLine($"Total input tokens: {summary.TotalInputTokens}");
Console.WriteLine($"Total output tokens: {summary.TotalOutputTokens}");
Console.WriteLine($"Total calls: {summary.TotalCalls}");

LlmUsageScope

LlmUsageScope is an AsyncLocal scope automatically set by OnMessage, carrying the agent handle, parent message ID, trace ID, and origin. This ensures LLM calls made during message processing are correctly attributed.

LlmCallContext

For LLM calls outside of OnMessage (timers, events, background work), use LlmCallContext to tag the origin:

C# — LlmCallContext for Background Work

using (LlmCallContext.Set("Timer:cleanup"))
{
    // LLM calls here will be attributed to "Timer:cleanup"
    var response = await chatClient.GetResponseAsync(messages);
}

Real-Time Notifications

Subscribe to events for real-time updates as messages, events, and LLM calls are recorded:

C# — Event Subscriptions

_monitor.OnMessageRecorded += message =>
{
    // Push to UI when a message is captured
};

_monitor.OnEventRecorded += evt =>
{
    // Push to UI when an event is captured
};

_monitor.OnLlmCallRecorded += call =>
{
    // Push to UI when an LLM call completes
};

Custom Monitor Implementations

Implement IAgentMessageMonitor to build custom monitoring backends (SQL database, external logging service, etc.):

C# — Custom Monitor Interface

public interface IAgentMessageMonitor
{
    // Record operations
    Task RecordMessageAsync(MonitoredMessage message);
    Task RecordEventAsync(MonitoredEvent evt);
    Task RecordLlmCallAsync(MonitoredLlmCall call);

    // Query operations
    Task<IReadOnlyList<MonitoredMessage>> GetMessagesAsync(
        string? agentHandle = null, int? limit = null);
    Task<IReadOnlyList<MonitoredEvent>> GetEventsAsync(
        string? agentHandle = null, int? limit = null);
    Task<IReadOnlyList<MonitoredLlmCall>> GetLlmCallsAsync(
        string? agentHandle = null, int? limit = null);

    // Notification events
    event Action<MonitoredMessage> OnMessageRecorded;
    event Action<MonitoredEvent> OnEventRecorded;
    event Action<MonitoredLlmCall> OnLlmCallRecorded;

    // Configuration
    LlmCaptureOptions LlmCaptureOptions { get; }
}

LlmCaptureOptions

Property	Default	Description
`Enabled`	`true`	Master switch for LLM call capture
`CapturePayloads`	`false`	Store actual prompts and responses
`MaxPayloadChars`	`8000`	Per-field character cap for payloads
`MaxToolArgsChars`	`4000`	Separate cap for tool arguments
`MaxBufferedCalls`	`2000`	FIFO buffer size for LLM calls
`Redact`	`null`	Function to redact sensitive content before storage

Telemetry Docs FabrCore on GitHub

Monitoring

Agent Monitoring

IAgentMessageMonitor

Enabling Monitoring

Monitored Types

MonitoredMessage

MonitoredEvent

MonitoredLlmCall

Querying Monitored Data

Token Tracking

Per-Call Tracking

Per-Agent Totals

LlmUsageScope

LlmCallContext

Real-Time Notifications

Custom Monitor Implementations

LlmCaptureOptions

Documentation