Testing

Test at the right boundary

Use application-owned substitutes for IFabrCoreAgentHost and IChatClient to test agent logic without a model call. The repository's SDK tests show how to exercise initialization, messages, state and tools. These fixtures are test source, not public SDK types: the NuGet packages do not expose FabrCoreTestHarness or TestFabrCoreAgentHost. The downloadable testing skill includes application-owned source templates for those helpers.

The runtime agent harness provides loops, todos and delegation. It is a separate feature from a test fixture.

Supported in-memory A2A test host

FabrCore.Host.Testing ships FabrCoreA2ATestHost, FakeFabrCoreAgentService and FakeFabrCoreRegistry. It runs real A2A routes, authentication handlers and wire serialization over an in-memory HTTP server. Agent execution and the registry are substituted; this does not start Orleans, exercise SQL persistence or call a model.

This complete console example uses the test host's documented A2A:* configuration. Authentication is disabled only for this isolated test.

using FabrCore.Host.Testing;

await using var host = await FabrCoreA2ATestHost.StartAsync(
    new Dictionary<string, string?>
    {
        ["A2A:Enabled"] = "true",
        ["A2A:Authentication:Mode"] = "None",
        ["A2A:AgentTypes:0"] = "support-agent"
    },
    registry: new FakeFabrCoreRegistry().WithAgentType("support-agent", "Support"));
using var card = await host.GetJsonAsync(
    "/a2a/support-agent/.well-known/agent-card.json");
if (!card.RootElement.TryGetProperty("name", out _))
    throw new InvalidOperationException("Agent card did not include a name.");

Use host.Client for protocol requests and host.AgentService.Sends to assert principal and handle routing. Add negative cases for missing or invalid credentials, cross-principal access, malformed messages and disabled endpoints.

Repository release validation

Run the checked-in scripts from the FabrCore repository root with its pinned SDK. Build.ps1 restores, builds, packs and runs the offline test selection using both existing test runners. Required SQL checks use explicit isolated test databases; missing prerequisites and skipped required SQL tests must fail the release gate. Follow the scripts' prerequisites before running database integration tests.

pwsh ./scripts/Build.ps1
pwsh ./scripts/Test-BuildScripts.ps1
pwsh ./scripts/Test-Documentation.ps1

Use the build guide for SQL validation and package-consumer commands and settings. Consumer checks use produced packages and an isolated cache, not repository project references.

Model evaluations are separate

Offline tests exclude integration and evaluation categories. Paid model evaluations require explicit model credentials, endpoints and budgets; they are not proof of deterministic release readiness. Use the current evaluation projects and their evaluator-specific interpretation rules instead of assuming one numeric threshold applies to every metric.

See the test projects for maintained examples. Published-package consumers should reference matching versions of the supported packages; do not copy internal fixtures as though they were shipped APIs.

Cloud management release checks

Validate admin/user concurrency, reserved-channel spoofing, session ownership, repeated submissions, incomplete receipts, lifecycle conflicts and immutable source histories. Test conditional CRUD, effect-free blueprint previews, ensure/update and multi-silo deployment deduplication. Monitoring/evidence checks need offline hosts, retained cursor changes, SQL outage/saturation/retention and signature-preserving exports.

The 2.0 validation report records executed checks and remaining production-model, sustained SQL and multi-silo stress/failover work. The less-than-5% cloud-read throughput/p95 target requires representative benchmarking; deterministic tests and short local timings do not establish that guarantee.