Health Monitoring in FabrCore

Per-Agent Health

Every FabrCore agent exposes health information through the GetHealth method. You can query any agent's health at three detail levels — Basic, Standard, and Detailed — depending on how much information you need:

var health = await agentGrain.GetHealth(HealthDetailLevel.Detailed);
// Returns: State, IsConfigured, AgentType, Uptime, MessageCount,
//          TimerCount, ReminderCount, StreamCount, Configuration

At the Basic level, you get the agent's state (Healthy, Degraded, or Unhealthy) and whether it's configured. Standard adds the agent type, uptime, and message count. Detailed includes everything — timer counts, reminder counts, stream counts, and the full agent configuration.

This is how the DelegateAgent's discovery mechanism works: it calls GetHealth(HealthDetailLevel.Detailed) on each managed agent to determine which ones are healthy and configured, pulling the description from the configuration for routing decisions. You can use the same pattern in your own monitoring code.

Diagnostics Endpoints

FabrCore exposes several HTTP endpoints for monitoring and discovery. These are available automatically when you host FabrCore in an ASP.NET Core application:

/fabrcoreapi/diagnostics/agents — Returns agent counts and per-agent information. Use this for dashboards and operational monitoring. It gives you a snapshot of every active agent in the system, including their handles, types, and health states.

/fabrcoreapi/diagnostics/agents/statistics — Returns aggregate statistics: total agent count, active count, deactivated count, with breakdowns by agent type. This is useful for capacity planning and detecting anomalies — if your active agent count suddenly drops, something may be wrong with the cluster.

/fabrcoreapi/discovery — Returns the full catalog of available agents, plugins, and tools. This is what you'd use to build a management UI or to programmatically discover what capabilities your FabrCore deployment offers.

// These endpoints return JSON — query them from any HTTP client
var client = new HttpClient();

// Get all active agents
var agents = await client.GetFromJsonAsync<AgentDiagnostics>(
    "https://localhost:5001/fabrcoreapi/diagnostics/agents");

// Get aggregate statistics
var stats = await client.GetFromJsonAsync<AgentStatistics>(
    "https://localhost:5001/fabrcoreapi/diagnostics/agents/statistics");

// Discover available capabilities
var discovery = await client.GetFromJsonAsync<DiscoveryResponse>(
    "https://localhost:5001/fabrcoreapi/discovery");

ASP.NET Core Health Checks

FabrCore integrates with the standard ASP.NET Core health check infrastructure. Wire up a health check that queries your FabrCore agents and report their status through the standard /health endpoint:

builder.Services.AddHealthChecks()
    .AddCheck<FabrCoreAgentHealthCheck>("fabrcore-agents");

app.MapHealthChecks("/health");

The FabrCoreAgentHealthCheck implements IHealthCheck and queries the diagnostics infrastructure to determine overall system health. You can customize the health check to match your requirements — check all agents, check only critical agents, or apply custom thresholds for degraded vs. unhealthy states.

This pattern means your FabrCore deployment speaks the same health check language as every other ASP.NET Core service in your infrastructure. Load balancers, orchestrators, and monitoring tools that understand the standard health check protocol work with FabrCore without any special configuration.

Kubernetes Integration

If you're deploying FabrCore to Kubernetes, use the health endpoint for liveness and readiness probes. Kubernetes will automatically restart unhealthy pods and stop routing traffic to pods that aren't ready:

livenessProbe:
  httpGet:
    path: /health
    port: 8080
readinessProbe:
  httpGet:
    path: /health
    port: 8080

The liveness probe tells Kubernetes whether the pod is alive and functioning. If the health check fails repeatedly, Kubernetes kills the pod and starts a new one. The readiness probe tells Kubernetes whether the pod is ready to accept traffic — during startup, while agents are initializing and discovering each other, the pod reports as not ready and doesn't receive requests.

For more granular control, you can expose separate endpoints for liveness and readiness. For example, /health/live might check only that the process is running and Orleans is responsive, while /health/ready might additionally verify that critical agents are configured and healthy.

Learn More

Health monitoring is built into FabrCore at every level — individual agents, system-wide diagnostics, and standard ASP.NET Core integration. Use what's there before building custom solutions.

Check out the full documentation for more details on telemetry, observability, and monitoring configuration.