All work
132024SCAD

Knowledge-Base Auto-Tagger & Linker

Tagging 40,000+ documents in 36 hours — and making search 3.4× more likely to surface the right one.

Before

An internal SharePoint held 40,000+ documents with inconsistent or missing metadata, which made search effectively useless.

After

A batch embedding + classification pipeline assigns SDG topics, year, language, and confidentiality tier, and suggests related documents. It tagged 40,000+ documents in under 36 hours, lifted search click-through-to-relevant by 3.4×, and now powers the retrieval filters in the main RAG system.

Challenge

Internal SharePoint had 40,000+ documents with inconsistent (or missing) metadata, making search useless.

Approach

Batch embedding + classification pipeline that assigns SDG topics, year, language, and confidentiality tier, plus suggests related documents.

How it was built

  1. 1

    Batch embedding

    Phase 1

    Embedded the whole corpus in batch, the foundation for both classification and related-document suggestions.

  2. 2

    Multi-facet classification

    Phase 2

    Classified each document across SDG topic, year, language, and confidentiality tier — the facets that make filtered search actually work.

  3. 3

    Feed the RAG filters

    Phase 3

    Exposed the tags as retrieval filters in the main document-intelligence system, so this pipeline quietly improved the flagship RAG product too.

Key architecture decisions

Metadata as an enabler for retrieval

Why · Good tags aren't cosmetic — they became the filters that sharpen the main RAG system's retrieval, so the work compounded.

Confidentiality tier as a first-class facet

Why · In a government context, classifying sensitivity up front is what lets downstream systems enforce access safely.

Impact

  • Tagged 40K+ documents in under 36 hours
  • Lifted search click-through-to-relevant by 3.4×
  • Powers the RAG retrieval filters in the main document system
40K+
docs
3.4×
ctr
36 hrs
time

What I'd tell someone building this

  • 01 · Metadata quality is a retrieval feature, not an afterthought.
  • 02 · Batch-tagging an existing corpus can unlock other AI systems that depend on it.
  • 03 · Classify sensitivity early so access control has something to enforce.

Tech stack

text-embedding-3GPT-4Azure AI Search.NET

Ask anything about Knowledge-Base Auto-Tagger & Linker

AI scoped to this project · GPT-OSS 120B