Everything you need to know about how TheHaze fits into your stack, why retrieval deserves its own layer, and what deployment looks like in your VPC.
Copilot, Gemini for Workspace, and similar products bundle retrieval, generation, and UI into a single vendor-controlled stack. That works for general productivity — but it means you're locked into their index, their ranking logic, and their model choices.
TheHaze takes the opposite approach: it's a dedicated retrieval engine that runs inside your VPC and connects to whatever orchestrator or agent framework you already use. You keep full control over which documents are indexed, which model receives the context, and how results are ranked and filtered before they enter a prompt.
The practical difference shows up in accuracy on domain-specific corpora. Generic enterprise AI tools are trained and indexed on broad data; TheHaze is optimized for the long tail of your internal documents — scanned PDFs, legacy slide decks, clinical imaging reports, engineering specs — the 60% of knowledge that never makes it into a clean wiki.
When retrieval and generation are fused into one product, you pay for that coupling in three ways: token cost, data control, and model flexibility.
Token cost: Bundled systems often send large, unfocused context windows to the LLM because they can't fine-tune retrieval per query type. A separate knowledge base lets you retrieve precisely — sending 2,000 relevant tokens instead of 20,000 mediocre ones. At scale, that's a 10× difference in inference spend.
Data control: Your documents stay in your VPC, indexed by infrastructure you audit. Nothing transits through a third-party embedding API unless you choose it. Role-based access, data lineage, and retention policies apply at the retrieval layer — before context ever reaches a model.
Model flexibility: GPT-4 today, Claude tomorrow, an open-weight model next quarter — swap the generator without rebuilding your index. The knowledge base is the durable asset; the model is interchangeable.
Most RAG tutorials assume your corpus is clean Markdown or HTML. Real enterprise data isn't. It's PDFs with embedded charts, PowerPoint decks with speaker notes, scanned contracts, Excel models, and audio transcripts.
Multi-modal retrieval means TheHaze indexes all of these format types through a unified pipeline — extracting text, structure, and visual content from each, then embedding them into a shared semantic space. When a user asks about a revenue trend, the system can return the chart, the narrative paragraph, and the relevant table — ranked by relevance, not by file type.
This isn't OCR stapled onto a text index. It's a purpose-built ingestion and embedding pipeline that preserves the relationships between visual and textual content within each document.
TheHaze ships as a Docker container with a native Model Context Protocol (MCP) server. Your agent framework — whether LangChain, CrewAI, a custom orchestrator, or an internal tool — calls the MCP endpoint with a natural-language query and receives ranked, structured context in return.
The integration is fully customizable. You control ranking thresholds, maximum context length, output format, metadata filters, and access policies. TheHaze doesn't dictate your prompt template or agent architecture — it feeds clean context into whatever you've already built.
Typical deployment: TheHaze container in your VPC → MCP endpoint on an internal port → your orchestrator calls it as a tool → retrieved context injected into the LLM prompt. No SDK required. Setup takes hours, not weeks.
Accuracy in retrieval comes from three layers working together: embedding quality, ranking logic, and backend infrastructure.
Embeddings: TheHaze uses state-of-the-art embedding models tuned for enterprise document types — not generic web text. Multi-modal content gets embedded with models that understand layout, not just character sequences.
Ranking: Raw vector similarity isn't enough. TheHaze applies a multi-stage ranking pipeline — semantic search, cross-encoder reranking, metadata filtering, and deduplication — so the top results are genuinely relevant.
Backend: The indexing and query infrastructure is built for sub-second retrieval at million-document scale. TheHaze's backend keeps the full corpus hot and queryable without compromising latency.
TheHaze uses a diverse set of LLMs, VLMs, and SLMs across the pipeline — for data engineering, embedding, query decomposition, and retrieval. No single model does every job well; we match the right model to each stage of the context layer.
We continuously evaluate open-source, cloud-native, and proprietary models to find the best balance of price, token cost, and performance. We maintain benchmarks on retrieval accuracy across these options and are happy to share our perspective in conversation.
That's a core differentiator: we evaluate constantly, so you don't have to.
A naive RAG pipeline sending top-10 chunks at 512 tokens each delivers 5,120 tokens per query at roughly 60–70% accuracy. A well-architected context engine sends 1,500–2,500 tokens at 85–92% accuracy on the same corpus.
The savings compound: lower input costs, faster inference, cheaper model tiers, and fewer failed queries requiring human correction.
The goal is not to fill the context window. It is to send the smallest amount of information that lets the model answer correctly.
Retrieval-Augmented Generation (RAG) often treated retrieval as a preprocessing step: embed documents, find similar chunks, concatenate, generate.
A context engine treats context selection as a first-class engineering problem with its own optimization targets: ingestion, indexing, retrieval, ranking, and governed packaging — all distinct from generation.
Read more in our blog: The Ideal Context Intelligence Engine. For consulting, contact us at hello@thehaze.ai.
Our solution incorporates best-of-breed retrieval intelligence — the ability to understand a question, identify the evidence required, retrieve it from the correct sources, and assemble a coherent, ranked context package for downstream reasoning.
For a practical walkthrough of how we design that stack — ingestion, multi-representation indexing, hybrid retrieval, and governed packaging — read Building the Ideal Context Intelligence Engine.
No. You can start with data you already govern. Integrate thehaze.ai as the context layer on top of structured warehouses, document repositories, and ontology stores — hosted inside your VPC.
DIY RAG and a production context engine optimize for different outcomes:
| Naive RAG | thehaze.ai Context Engine |
|---|---|
| Only top-k vector similarity | Query decomposition and intent understanding |
| One embedding model for all content | Multi-representation indexing by content type |
| Chunks as the unit of retrieval | Concepts, rules, evidence objects, and documents |
| No ranking beyond cosine distance | Hybrid retrieval + tensor ranking + deduplication |
| Context window as the constraint | Governed context package as the output |
| Accuracy ~60–70% on enterprise corpora | Target 85%+ via retrieval intelligence |
Everywhere you want it to — as long as that's inside your Virtual Private Cloud. TheHaze deploys as a container into AWS, GCP, Azure, or on-premise infrastructure. Your source documents, index, embeddings, and query logs never leave your network perimeter.
There is no shared multi-tenant index, no data sent to external APIs for processing, and no telemetry that includes document content. For regulated industries — healthcare, finance, legal — this isn't a nice-to-have; it's the baseline.
TheHaze ingests and indexes the formats that actually exist in enterprise environments:
PDF (native and scanned), Microsoft Office (Word, Excel, PowerPoint), Google Docs/Sheets/Slides, HTML and Markdown, plain text and CSV, images (PNG, JPEG, TIFF — including charts and diagrams), audio and video transcripts, email archives, and internal wiki formats (Confluence, SharePoint, Notion exports).
New formats can be added via the ingestion pipeline's plugin architecture. If your corpus includes a proprietary or legacy format, talk to us — we've handled worse.