RAG Chunking Strategies for ASP.NET Core Developers

RAG Chunking Strategies for ASP.NET Core Developers

How to preserve meaning, improve retrieval and choose boundaries that match real business documents

The complete RAG ingestion pipeline follows a source through extraction, normalisation, chunking, metadata, embeddings and indexing. One stage deserves much closer attention:

Where should one searchable piece of content end and the next begin?

That is the chunking problem.

Chunking may look like text preparation, but it directly shapes retrieval quality. A poor boundary can separate a rule from its subject, a table value from its headings or a legal clause from its condition. The search engine may then retrieve technically similar text that is incomplete or misleading.

This guide explains fixed-size, sentence, paragraph, heading-aware, recursive, semantic, parent-child, table, list, legal, spreadsheet and code-aware chunking. It also shows how an ASP.NET Core ingestion pipeline can select, version, test and observe those strategies.

The examples use BuildEstate Pro, my public property-development case study. The RAG design is a teaching extension, not a claim that the repository already contains this production feature.

The governing principle is:

The best chunk is the smallest independently understandable unit that preserves enough meaning to answer the questions users will actually ask.

---

1. What a chunk represents

A chunk is a searchable unit derived from a larger source. A 100-page planning report might become chunks about drainage, highway access, ecology, working hours and landscaping.

Each chunk normally has:

• readable evidence;
• an embedding used for vector search;
• source and section provenance;
• tenant and permission metadata;
• a stable identity and version.

Ideally, one chunk communicates one reasonably complete idea. It must be small enough to match a focused question and large enough to make sense after it leaves the original page.

That balance is the central challenge.

---

2. Why one vector for the whole document is weak

Imagine one report discusses drainage approval, lorry access, noise, ecology, affordable housing and landscaping. One embedding for the complete report blends all those meanings into a broad representation.

The user asks:

Can delivery lorries use the west entrance?

The report is relevant, but the section about construction traffic is far more relevant. Smaller sections create more focused vectors and improve the chance that search ranks the exact evidence highly.

Whole-document retrieval also wastes context. Sending 100 pages to a model when two paragraphs answer the question increases cost, latency and distraction. A larger context window does not remove the need to select good evidence.

---

3. Why one sentence per chunk can be equally weak

Now move to the other extreme:

It must be approved before work begins.

The sentence is grammatically complete, but semantically dependent. What must be approved? Which work? By whom? Under which condition?

Inside its paragraph, the meaning may be obvious. Once indexed independently, it becomes ambiguous.

Very small chunks offer precise matching but lose context. Very large chunks preserve context but mix topics. Good chunking manages this trade-off deliberately instead of choosing a fashionable token number.

---

4. Token limits are guardrails, not meaning

Chunk size can be measured in characters, words, sentences, paragraphs, sections or tokens. Tokens are useful because embedding and generative models enforce token limits.

However, a 500-token counter does not know whether position 500 lies inside:

• a legal clause;
• a table row;
• a numbered procedure;
• a sentence containing a condition;
• an explanation whose conclusion comes next.

Use tokens as a maximum size and budgeting mechanism. Prefer natural structural boundaries wherever the source offers them.

A practical chunking request might contain both:

public sealed record ChunkingOptions(
    int TargetTokens,
    int MaximumTokens,
    int MaximumOverlapTokens,
    int MinimumUsefulTokens,
    bool IncludeDocumentTitle,
    bool IncludeHeadingPath);

TargetTokens guides grouping. MaximumTokens prevents invalid model input. Neither decides what belongs together.

---

5. Preserve structure during extraction

A chunker cannot use headings, tables or page numbers if extraction has flattened everything into one string.

Keep structure in an intermediate model:

public enum ContentBlockKind
{
    Heading,
    Paragraph,
    ListItem,
    Table,
    Code,
    Caption
}

public sealed record ContentBlock(
    ContentBlockKind Kind,
    string Text,
    int PageNumber,
    int HeadingLevel,
    string? HeadingPath,
    IReadOnlyDictionary<string, string> Attributes);

Azure Document Intelligence can preserve headings, paragraphs, tables and layout in structured output such as Markdown. Other sources already have native structure: HTML headings, Word styles, spreadsheet sheets and cells, SQL columns, or syntax trees for code.

Chunking quality begins upstream. Once structure is discarded, rebuilding it reliably is difficult.

---

6. Fixed-size chunking: the baseline

Fixed-size chunking creates blocks of approximately equal length—perhaps 500 tokens each.

It is simple, predictable and useful as a baseline. It can work for sources without dependable structure, including transcripts, informal notes, some logs and long plain-text files.

Its weakness is mechanical boundaries:

Chunk 1: The developer must submit a drainage strategy before construction...
Chunk 2: ...begins. The strategy must describe maintenance and flood controls.

The second chunk refers to “the strategy” without naming it. Retrieval can return a passage that is technically close but incomplete.

Keep fixed-size results as a benchmark. If a more sophisticated strategy does not beat it on real evaluation questions, the extra complexity may not be justified.

---

7. Sliding windows and overlap

Overlap repeats content around boundaries:

Chunk 1: tokens   1–500
Chunk 2: tokens 401–900

The shared 100 tokens can preserve a sentence or explanation split near the boundary.

Overlap also produces costs:

• more chunks and embeddings;
• more index storage;
• repeated search results;
• duplicated prompt content;
• apparent corroboration from what is really one repeated passage.

Deduplicate overlapping results before prompt assembly. A useful context builder can recognise chunks from the same document whose source ranges substantially intersect.

Overlap should repair unavoidable boundaries. It should not compensate for throwing away clear document structure.

---

8. Sentence and paragraph grouping

Sentence-aware splitting avoids cutting grammar in half. It normally groups complete sentences until the target size is reached.

Paragraph-aware splitting respects units that authors often use for complete ideas. It works well when documents are clearly written and extraction preserves real paragraph boundaries.

Both require judgement:

• a complete sentence may depend on the previous sentence;
• one paragraph may be several pages long;
• a PDF extractor may treat every visual line as a paragraph;
• several short paragraphs may form one argument;
• a paragraph may rely on its heading.

A practical strategy combines short neighbouring paragraphs under the same heading and recursively splits an oversized paragraph by sentence.

---

9. Heading-aware chunks preserve the subject

Suppose a report contains:

4. Planning Conditions
4.2 Highway Access
Construction vehicles must not enter from the western road.

Indexing only the sentence loses valuable context. A heading-aware searchable representation becomes:

Planning Officer Report
Planning Conditions > Highway Access
Construction vehicles must not enter from the western road.

The stored evidence should still preserve the original text separately. The title and heading path enrich search without pretending they were part of the paragraph.

Heading-aware chunking is especially useful for policies, manuals, technical documentation, legal guidance, planning reports and structured Markdown.

public sealed record ChunkText(
    string SearchableText,
    string EvidenceText,
    string DocumentTitle,
    string? HeadingPath);

That distinction supports honest citations: retrieval may use enriched text, while the answer cites the original passage and provenance.

---

10. Recursive chunking: structure first, limits last

Recursive chunking tries the strongest boundary available:

1. major section;
2. subsection;
3. paragraph;
4. sentence;
5. token or word boundary only when necessary.

If a section fits the maximum, keep it intact. If it is too large, split by paragraph. If one paragraph remains too large, split by sentence.

public sealed class RecursiveDocumentChunker(ITypedChunker)
{
    public DocumentKind Handles => DocumentKind.StructuredDocument;

    public IReadOnlyList<ChunkDraft> Chunk(
        StructuredDocument document,
        ChunkingOptions options)
    {
        return document.Sections
            .SelectMany(section => SplitSection(section, options))
            .Select((draft, sequence) => draft with { Sequence = sequence })
            .ToArray();
    }
}

Recursive chunking is often a strong general-purpose starting point because it respects meaning where possible while still guaranteeing model-safe sizes.

Its output should be deterministic for identical input and versioned options. Reproducibility helps investigate retrieval changes.

---

11. Semantic chunking detects topic shifts

Semantic chunking examines neighbouring sentences or paragraphs and estimates where meaning changes significantly.

If the first three paragraphs discuss drainage and the next two discuss vehicle access, it can form two groups even if they have different sizes.

This can outperform mechanical splitting when structure is weak, but it introduces:

• additional embedding or model computation;
• variable chunk sizes;
• more ingestion latency and cost;
• thresholds that require tuning;
• harder reproduction when models change;
• a new component that must be evaluated.

Semantic chunking and semantic search are different. Chunking chooses boundaries during ingestion; semantic search finds chunks at query time.

Do not assume “semantic” means better. Compare it with a simpler recursive baseline using the same documents and expected retrieval passages.

---

12. Parent-child chunking combines precision and context

Parent-child chunking stores two levels:

• child chunks are small and precise for matching;
• parent sections contain richer surrounding context for answering.

Parent P42: Highway Access section
  ├─ Child C1: prohibited western entrance
  ├─ Child C2: required eastern entrance and times
  └─ Child C3: traffic-marshal requirement

The user's question may match C2. The retrieval layer can then load P42, or selected siblings, before prompt assembly.

public sealed record ParentChildChunk(
    string ChunkId,
    string ParentId,
    string DocumentId,
    string ChildText,
    int ChildSequence,
    string SectionHeading,
    int PageFrom,
    int PageTo);

This resolves part of the small-versus-large conflict, but retrieval becomes more sophisticated. Avoid blindly returning every parent for every child. A very large parent can reintroduce irrelevant context.

Measure both child retrieval accuracy and the usefulness of expanded parent context.

---

13. Tables need relationships, not loose cells

Consider:

The value £4,000 is meaningless without its row and column headings.

A table-aware strategy may:

• preserve a complete small table;
• repeat headers with each row group;
• convert a row into a faithful sentence;
• split a large table by logical groups;
• store the table title, page and original representation.

Searchable row:

Planning application fees — For ten new homes, the application fee is £4,000 and the processing period is 13 weeks.

Keep the source table for evidence. Generated prose can improve retrieval, but it must not alter values, units or relationships. Validate conversion programmatically where possible.

For analytical questions across thousands of rows, vector chunks may be the wrong tool. A governed SQL query or application tool can calculate an exact answer from the source.

---

14. Lists must keep their introduction

This policy statement loses meaning if its items are separated:

An application is complete only when it includes: 1. the application form; 2. the location plan; 3. the ownership certificate; 4. the required fee.

A chunk containing only the ownership certificate is weak evidence. Preserve the introductory sentence and the complete list when it fits.

For a long list, repeat a concise, original list heading with each group and store item ranges. Avoid presenting one subgroup as though it were the entire list.

Numbering is evidence too. Keep order and item identifiers when users may ask about a particular requirement.

---

15. Legal clauses require references and definitions

Contracts, regulations and policies contain sections, clauses, subclauses, definitions and cross-references.

Subject to clause 7.2, the borrower may...

Retrieving that sentence without clause 7.2 may invert or weaken the intended meaning.

A legal-aware chunk should preserve contract title, clause number, heading, parent clause and referenced definitions. The retrieval layer may expand an explicit cross-reference as additional evidence.

Do not automatically combine distant clauses into rewritten legal advice. Keep passages distinct, cite each source location and make limitations clear. High-stakes legal interpretation requires appropriate professional review.

Arbitrary token splitting is particularly dangerous here because exceptions often follow general rules.

---

16. Spreadsheets are structured data, not flowing prose

A workbook may contain several sheets, column headings, formulas, totals, dates and context hidden in sheet names.

This row is useless alone:

152, Pending, 14 August, Drainage

A faithful searchable representation is:

Outstanding Conditions sheet — Project 152 has status Pending. The status date is 14 August. The outstanding category is Drainage.

Possible spreadsheet units include one record with headings, one small table, a logical row group, one business entity or one reporting period.

Preserve data types and distinguish formula values from labels. If the question requires current totals, comparisons or aggregation, execute an authorised query against structured data rather than embedding a stale spreadsheet snapshot.

RAG retrieves evidence. It should not replace reliable calculation.

---

17. SQL records need an intentional read model

Do not embed arbitrary raw database rows or concatenate every column.

Start with the user questions and build an authorised projection containing meaningful business facts:

public sealed record PlanningConditionKnowledgeRow(
    Guid TenantId,
    Guid ProjectId,
    Guid ConditionId,
    string ProjectName,
    string ConditionNumber,
    string Category,
    string Requirement,
    string Status,
    DateOnly? DueDate,
    DateTimeOffset UpdatedAtUtc);

One chunk can represent one planning condition with stable identity and source metadata. Keep operational values—such as live balances or current stock—behind tools or APIs when freshness and exactness matter more than semantic retrieval.

The question is not “Can this row become a vector?” It is “Is a vector projection the correct way to answer the intended question?”

---

18. Code-aware chunking respects program structure

For source-code search, arbitrary token blocks can cross classes and methods. A syntax-aware chunker can preserve:

• namespace and file path;
• class, interface or record name;
• complete method or property;
• XML documentation and useful comments;
• implemented interfaces and relevant signatures.

A complete C# method is usually a stronger evidence unit than 500 tokens covering the end of one method and start of another.

Very large methods are themselves a signal. The chunker may split by syntax blocks but should retain the containing type and method signature with each child.

Roslyn syntax trees provide reliable C# boundaries. Similar parsers exist for other languages. Regex alone is rarely a robust code parser.

---

19. Contextual enrichment can improve independent meaning

Sometimes the source passage needs compact context:

Original evidence:

The application was refused because the required assessment was missing.

Useful provenance:

Document: Planning Committee Decision
Project: Riverside Development
Section: Environmental Impact
Decision date: 15 July 2026

Searchable text may include trusted title, project and heading information. Preserve the evidence text separately.

An LLM can generate contextual summaries, but that creates derived claims. Store the enrichment method and model version, evaluate factual consistency and never allow generated enrichment to replace the original evidence.

Include metadata that disambiguates meaning, not every database field. Repeating excessive context increases token cost and may overpower the passage itself.

---

20. Select strategies by document type

Represent strategy selection explicitly:

public interface IChunkingStrategy
{
    string Name { get; }
    string Version { get; }
    bool CanHandle(DocumentProfile profile);

    Task<IReadOnlyList<ChunkDraft>> CreateAsync(
        NormalisedDocument document,
        ChunkingOptions options,
        CancellationToken cancellationToken);
}

public sealed class ChunkingStrategyResolver(
    IEnumerable<IChunkingStrategy> strategies)
{
    public IChunkingStrategy Resolve(DocumentProfile profile) =>
        strategies.Single(strategy => strategy.CanHandle(profile));
}

A configuration might route:

Avoid overlapping CanHandle rules that make selection dependent on registration order. Make classification and routing observable.

---

21. Version the strategy and every meaningful input

Suppose version one uses 500-token chunks with 100-token overlap. Version two uses heading-aware parent-child chunks with minimal overlap. They produce different evidence units, IDs and embeddings.

Store:

• strategy name and version;
• options such as target, maximum and overlap;
• normalisation version;
• extraction version;
• embedding model and dimensions;
• source content hash.

public sealed record ChunkingProvenance(
    string Strategy,
    string StrategyVersion,
    string OptionsHash,
    string ExtractionVersion,
    string NormalisationVersion,
    string SourceContentHash);

When strategy changes, build a new index generation, evaluate it against the current version, switch only after it passes, and retain rollback capability. Silent in-place changes make retrieval regressions difficult to diagnose.

---

22. Stable identity should reflect logical content

Sequence-only IDs are easy, but inserting one paragraph near the start can shift every later sequence and make the whole document look changed.

For sources with stable section or clause identifiers, incorporate them:

tenant / document / source version / strategy version / section key / child key

For unstructured content, sequence plus generation may be appropriate. Content hashes help identify unchanged chunks but should not be the sole identity when identical text legitimately appears in different sections.

Identity design affects update cost, deletion, citations and traceability. Choose it with the source lifecycle in mind.

---

23. Evaluate chunks before evaluating generated answers

If retrieval never finds the right passage, prompt engineering cannot repair the missing evidence.

Build a dataset of realistic questions and expected source locations:

public sealed record RetrievalExpectation(
    string Question,
    Guid ExpectedDocumentId,
    string ExpectedSection,
    int? ExpectedPage,
    IReadOnlyCollection<string> RequiredFacts);

Compare strategies using:

• recall at K: did a relevant passage appear in the top K?
• precision at K: how much returned content was useful?
• ranking position: how early did the expected evidence appear?
• context completeness: did the chunk contain the full condition?
• duplication: how many results repeated the same source text?
• citation accuracy: did provenance point to the right page and section?
• cost and latency: how many chunks, tokens and searches were required?

Include hard questions, exact identifiers, paraphrases, negations, tables and cross-references. Use the same dataset to compare fixed, recursive, semantic and parent-child variants.

---

24. Inspect chunks as a human

Metrics do not replace inspection. Build an internal diagnostic view showing:

• original page or source block;
• evidence text;
• enriched searchable text;
• title and heading path;
• parent and neighbouring chunks;
• token count and overlap;
• strategy and pipeline version;
• metadata and permissions;
• retrieval score for a test question.

This makes boundary failures visible. An engineer can see whether a heading was lost, a table row detached, a list duplicated or a parent expanded too broadly.

Never expose sensitive chunks in general logs. The diagnostic tool needs authentication, authorisation and appropriate audit controls.

---

25. BuildEstate Pro worked example

The source section is:

Highway Access Construction vehicles must not enter from the western road. All deliveries must use the eastern service entrance between 9 a.m. and 4 p.m. A traffic marshal must be present during large deliveries.

A poor fixed boundary creates:

Chunk 1: Construction vehicles must not enter from the western road. All deliveries...
Chunk 2: ...must use the eastern service entrance between 9 a.m. and 4 p.m. A traffic marshal...

Neither contains the complete rule.

A heading-aware chunk preserves:

Planning Officer Report > Highway Access
Construction vehicles must not enter from the western road.
All deliveries must use the eastern service entrance between 9 a.m. and 4 p.m.
A traffic marshal must be present during large deliveries.

The user asks:

Where and when can construction deliveries arrive?

The chunk supplies prohibited entrance, required entrance, permitted time and marshal requirement. Its metadata supplies report version, project, page and permissions.

If the whole highway section were much larger, children could represent individual conditions while the parent retained the complete section. Search matches the precise child, then controlled expansion provides the sibling rules needed for a complete answer.

That is a strategy designed around the user's question—not around an arbitrary number.

---

26. Common chunking mistakes

Copying an internet default without evaluation

Five hundred tokens may be reasonable, but it is not evidence that the configuration works for your documents.

Losing title and heading context

Ambiguous pronouns and generic sentences become weak retrieval units.

Using too much overlap

Repeated passages inflate cost and create duplicate evidence.

Flattening tables and lists

Values and items lose the headings that define them.

Treating all sources identically

Contracts, spreadsheets, code and reports have different natural structures.

Mixing generated enrichment with original evidence

Users can no longer tell what the source actually said.

Changing strategy silently

Retrieval behaviour changes without a traceable pipeline version or comparison.

Measuring only final answer fluency

A confident answer can hide missing or incomplete evidence. Inspect retrieval first.

---

27. A practical delivery sequence

1. Select one valuable document type and collect representative samples.
2. Preserve headings, paragraphs, tables, pages and source identifiers during extraction.
3. Write realistic questions with expected passages.
4. Establish a fixed-size baseline.
5. Implement a structure-aware recursive strategy.
6. Add title and heading context while keeping evidence text separate.
7. Measure recall, precision, completeness, duplicates, cost and latency.
8. Add parent-child or semantic chunking only where evidence supports it.
9. Version the selected strategy and publish through a new index generation.
10. Monitor real failed searches and extend the evaluation set.

This turns chunking into an engineering feedback loop rather than a one-time configuration guess.

---

Final mental model

Chunking is the design of evidence boundaries.

Source structure
      ↓
Preserve headings, clauses, rows, lists and code units
      ↓
Choose the smallest independently understandable unit
      ↓
Apply token limits and deliberate overlap
      ↓
Attach parent, provenance, permissions and version
      ↓
Evaluate against real questions
      ↓
Publish a controlled index generation

Large chunks mix unrelated ideas and waste context. Tiny chunks lose the subject and qualifications. Fixed-size chunking is a valuable baseline; recursive heading-aware splitting is often a practical starting point; semantic and parent-child techniques earn their place through measured improvement.

Most importantly, document type matters. A contract clause, spreadsheet row, table, C# method and planning-report section should not automatically pass through the same mechanical splitter.

Continue through the AI Engineering learning journey, review the complete ingestion pipeline, or study embeddings and hybrid retrieval.

If this guide helped you, or you would like to discuss ASP.NET Core, Azure, SQL Server, Microsoft Foundry or RAG engineering, contact me at dotnetdeveloper20xx@hotmail.com.

---

References and further reading

• Chunking content in a RAG solution
• Azure AI Search integrated vectorisation
• Azure Document Intelligence guidance for RAG
• Azure AI Search relevance and ranking overview