AI Building AI: How We Cut GraphRAG Ingestion Time by 34%

The coding environment is becoming part of the conversation

There is a particular friction in AI-assisted development: explaining the framework, correcting an outdated pattern, and explaining why a locally plausible change does not fit the rest of the application. Enough of that can turn an assistant into another system to supervise.

In this project, the experience felt different. Useful work happened across C# service lifetimes, LLM response schemas, SQL persistence, embedding batches, and measurements captured by a console application. The assistant could use the environment to connect those pieces. When we asked whether parallel ingestion would help, it also found that the existing telemetry would attribute overlapping requests to the wrong documents.

That observation mattered as much as the parallel loop. Without it, we could have made the implementation faster while making the benchmark less trustworthy.

From the developer's chair, progress can feel exponential because improvements compound: better code understanding makes better experiments possible; better experiments reveal better changes; the findings become repository documentation and reusable skills. This case study does not measure an exponential capability curve or compare generations of coding models. It records a practical shift in how much of the work an AI assistant could help carry through.

The human responsibility stayed concrete: define which facts matter, insist on a fair comparison, and decide what the evidence permits us to ship. Familiarity with an AI coding environment made the loop more productive. It did not make every hypothesis correct.

GraphRAG ingestion is several workloads sharing a finish line

A document becomes useful in several ways. Its chunks support vector retrieval. Its entities become nodes. Its factual relationships become edges. Domains and categories organize the content. Source contributions tell us which document supplied an assertion.

Source text
  +-- Retrieval chunks ---------> Embeddings
  +-- Extraction sections ------> Entities + typed relationships
  +-- Classification evidence --> Domain + category
                                      |
                         Scoped graph + SQL VECTOR storage

We wanted all of those outputs. Replacing typed facts with word co-occurrence would change what the graph means. Making chunks searchable before graph enrichment finishes could improve perceived readiness, but it would answer a different timing question.

We already used SQL Server's vector data type. Nothing in these experiments established a need to replace that storage choice. Before debating a new database, we needed to understand how much time went into generating the data we would store.

The evaluated pipeline separated 500-character retrieval chunks with 100-character overlap from non-overlapping extraction sections of about 2,000 characters, grouped up to eight per LLM batch. Short documents could combine graph extraction and classification; longer documents classified separately while extracting graph batches.

That separation lets us tune LLM work without casually changing the units that make retrieval useful. The eval runner is a .NET console app. FabrCore's services do not require a Blazor UI to run this workload.

An advanced knowledge architecture, with several ways to retrieve evidence

FabrCore's GraphRAG service brings semantic retrieval, typed graph relationships, taxonomy, and document lifecycle management into one .NET service layer. The sophistication is in how those capabilities work together. A passage can be relevant because of what it says, because of the entity it describes, or because that entity connects to something the user already knows.

• Vector search and graph traversal in SQL Server. Native VECTOR(1536) columns store entity and chunk embeddings alongside SQL graph nodes and edges. The search surface supports chunks, entities, relationships, hybrid retrieval, and iterative deep search. An application can start with similar language and explore related entities through the graph.
• Explicit knowledge boundaries. Search requires allowed scopes, and relationship traversal checks both endpoints. Canonical identities help organize entities across scoped records without granting cross-scope access. The host application supplies authorized scopes; domains and categories organize knowledge rather than determine permissions.
• Source-aware maintenance. Documents have identities, content and instruction hashes, versions, and tracked graph contributions. That supports unchanged-document reuse, investigation of extracted facts, and cleanup when a source is deleted. Preserving independent conflicting assertions from multiple sources remains further work.
• Operational visibility and integration. Administration APIs expose documents, graph data, taxonomy, audits, and ingestion metrics. The same service contracts can support a background worker, console process, web application, or FabrCore agent and plugin tools.

These are implemented architectural capabilities, not a claim that this study proved every search mode or enterprise scenario. Our ingestion work focused on feeding that architecture faster while preserving the information that makes it valuable. Rich retrieval has little to follow when ingestion drops the relationship it needs.

Business questions that benefit from connected knowledge

Consider an internal policy assistant asked, “Which team owns this obligation, and which policy establishes it?” Vector retrieval can locate the relevant passages; typed relationships can connect the policy, organization, and responsible team. Source contributions help investigate where those connections came from. Our public-policy experiments tested some of the underlying distinctions, including the difference between a requirement and a recommendation.

The same architecture is a candidate for supplier knowledge, connecting a vendor to services, contract terms, and internal owners; engineering support, connecting a product to systems, runbooks, and known dependencies; and project onboarding, connecting decisions to the teams and documents behind them. Each application needs its own extraction instructions, factual labels, and evaluation corpus. These are potential applications, not customer deployments established by our benchmarks.

For developers, the design question becomes concrete: which relationship would let a user answer a question that a relevant paragraph alone leaves unresolved? That relationship belongs in the evaluation set. The business companion walks through four practical scenarios and the evidence each would need.

A fast graph can still be the wrong graph

We started with technical documents about JSON, CSV, and UUIDs. Then we moved to public policy Markdown: a Project Open Data memo, its governance document, and CISA's open-source policy. The policy corpus contained 59,152 characters and produced 163 retrieval chunks. Pinned source versions kept the text reproducible and avoided document-conversion work.

A search can return the right document even if a relationship inside its graph is missing or reversed. We therefore kept labeled factual checks outside the model's prompt and compared them with persisted edges.

Publication should point from the document to its publisher. Organizational membership should point from a component to its containing organization. An agency required to use metadata should remain the actor; the document describing its responsibility is not a substitute for that agency.

Later labels distinguished mandatory open formats from a recommendation to prioritize non-proprietary formats. A recommendation becoming a requirement is a meaningful error in a policy system, even when the JSON parses and search looks excellent.

The runner captured provider calls, tokens, raw responses, phase timings, graph snapshots, and retrieval results. Tracing raw output into persistence helped distinguish facts the model never emitted from facts lost because their endpoint names could not be resolved.

A green smoke test meant ingestion and retrieval worked. It did not mean every required fact survived. That distinction kept attractive performance numbers from overruling quality.

The biggest cold-ingestion gain came from scheduling

It was tempting to start with model size. Our tested mini configuration remained the useful choice; the smaller and alternative models we tried did not justify replacing it. Larger extraction batches also looked promising. Sixteen sections reduced calls and output, but the apparent latency advantage was too small and factual coverage too weak to justify a new default. We kept eight.

The more convincing change allowed two documents to progress together while preserving one shared limit of four simultaneous chat calls. They reused the same ingestion service. A service per document would have multiplied the request limit and muddied the comparison.

Two concurrent documents reduced batch time by about 34%. Three added little. All six passes retained 163 embedded chunks and returned the correct document at rank one for three retrieval questions. Peak chat concurrency stayed at four.

The gain came from overlapping work. The long memo could take longer individually while competing for shared capacity, yet the corpus finished sooner because shorter documents no longer waited behind it.

We fixed the runner before trusting this result: request samples needed per-document ownership, checkpoint writes needed serialization, and graph checks needed to wait until writers settled. Parallel batch time had to be measured directly. Adding overlapping document durations does not tell you how long the batch took.

Caching answered a different performance question

Three forms of reuse matter: skipping unchanged documents, reusing validated extraction responses during a rebuild, and reusing embeddings for unchanged text. Provider prompt caching is another mechanism entirely. A cached prompt still requires the model to generate its answer.

In the RFC repeat-rebuild experiment, enabling taxonomy and combined-response caching on top of result and embedding caches reduced average forced-repeat time from 8.457 seconds to 0.542 seconds. Fully cached repeats made zero LLM and zero embedding calls, and the checked graph snapshots and vector fingerprints matched.

That is useful for repeated processing. It is not a half-second promise for a new document. Cache namespaces, model identity, instructions, taxonomy context, and the shape of an edit determine what can safely be reused. Representative edits and concurrent cache use remain unfinished evaluations.

Valid JSON was only the first layer of correctness

Schema-enforced JSON addressed a real formatting problem. Asking for JSON in prose had allowed malformed responses, including a trailing comma. Moving the contract into the API made the output shape more dependable. It could not tell us whether an edge was true.

Taxonomy exposed another problem. A prompt displayed Category Name (in Engineering), and the model sometimes treated the domain annotation as part of the name. An experimental structured-name representation separated those fields and rejected unknown names marked as reused. It helped one recurring selection pattern, but existing duplicates remained selectable. The results did not justify adopting it by default.

A constrained policy vocabulary then preserved eight of the original nine facts in both runs, versus six to seven in controls. Average time increased from 17.9 to 20.0 seconds. It still confused action with obligation: agencies REQUIRES metadata appeared where our contract expected agencies USES metadata, with the requirement in the description.

A slower experiment we rejected: separate obligation fields

The next idea sounded sensible: give obligation its own field. Let USES describe the action and required, recommended, or permitted describe its strength. Explicit prohibition fields also had to agree.

For the experiment, qualifiers were stored in a tagged description suffix and decoded separately during evaluation. We excluded the tag from factual scoring: appending required must not manufacture evidence that a required fact survived.

The qualifier survived persistence on all 313 retained edge contributions across the new runs. But both CISA responses contradicted the cross-field contract. One used a general relationship with a prohibition qualifier; another used PROHIBITS while describing required publication. Valid JSON and individually valid enum values had produced an inconsistent assertion.

The parser rejected those responses. The retry path split extraction into smaller batches and regenerated work. Two extra calls pushed average ingestion roughly 61% above same-day controls. Both runs still missed the added open-format and non-proprietary-format facts.

We left the feature off. Extra schema structure can create more validation failures, and the cost of a failure depends heavily on how much work the retry repeats.

What we ended up with: faster ingestion and better evaluation

The outcome was an improved solution: about 34% less policy-batch ingestion time with bounded document concurrency, and about 94% less repeat-rebuild time in the separate full-cache comparison. Chunk embeddings, graph extraction, and SQL vector storage remained part of the pipeline. The slower obligation-field experiment stayed disabled.

We paused with a measured shortlist: mini, eight extraction sections per batch, schema responses for evaluations, and two concurrent documents sharing the existing chat limit. SQL vector storage stayed. Result and embedding caches offered a separate way to avoid repeated work.

We also ended up with a better instrument: a console runner, fixed corpora and labels, raw-response tracing, accurate parallel timing, and a checkpoint for resuming. The latest unit run passed 97 tests. This remains an evaluation checkpoint, not a finished production-quality benchmark or a claim that every promising option became a default.

The remaining work is specific: repair invalid assertions without regenerating entire batches; test realistic edits and concurrent caches; preserve conflicting per-source assertions; evaluate taxonomy against clean candidates; and run larger holdouts and sustained-load tests.

For another team tuning GraphRAG, the order matters. Establish a correct persisted fact. Measure the work that reaches the provider. Separate cold ingestion from reuse. Test bounded scheduling. Then decide whether a more elaborate extraction contract earns its cost.

AI made it practical to move through that loop and leave behind executable checks and reusable instructions. The most valuable outcome was knowing which changes deserved to remain enabled, which belonged behind a flag, and which questions were still open.

Measurement notes and source material

These are FabrCore engineering evaluations from September 5–7, 2026, using the configured gpt-5.4-mini model and 1536-dimensional embeddings. Most settings had two observations, with order reversed to help expose variation. Timings exclude warmup, download, schema setup, and final retrieval verification. Fresh application scopes do not clear provider caches or shared taxonomy.

The last obligation control finished ingestion but hit a report-write error before retrieval. We retained the incomplete artifact, added bounded report-write retries, and ran a replacement. The table uses completed controls. These samples are not a p95 latency estimate, a universal model ranking, or a production load test.

• FabrCore source repository.
• Pinned Project Open Data memo and governance source.
• Pinned CISA open-source policy. Source snapshots support reproducibility, not current-policy guidance.