MMF.
• 6 min read

Building QueryNest: Document-Grounded RAG with LangChain & Gemini

How I designed an asynchronous, production-ready Retrieval-Augmented Generation pipeline using LangChain, Google Gemini API, Chroma vector store, and FastAPI.

QueryNest Architecture and Interface

Large Language Models are exceptionally good at synthesis and reasoning, but when applied to private, specialized, or rapidly evolving documents, they frequently encounter two major failure modes: hallucination and knowledge staleness. Fine-tuning an entire model for dynamic document collections is computationally prohibitive and inflexible.

To solve this problem cleanly, I engineered QueryNest: a modular, containerized Retrieval-Augmented Generation (RAG) system. QueryNest allows users to upload technical PDFs, research papers, and policy manuals, index them into high-dimensional embedding spaces, and query them with verifiable, source-attributed responses.

System Architecture Overview

The architecture of QueryNest is built around separation of concerns: document ingestion, vector persistence, semantic retrieval, context distillation, and streaming response generation.

  • Document Ingestion & Chunking: Parses multi-format documents (PDFs, Markdown, TXT) and splits text using recursive character partitioning with configurable overlap.
  • Vector Storage (Chroma DB): Persists high-dimensional embeddings generated by Google Gemini embedding models for low-latency similarity queries.
  • Backend Engine (FastAPI): Asynchronous REST endpoints managing upload pipelines, vector search querying, and conversation state.
  • Orchestration (LangChain): Manages retrieval chains, prompt templates, few-shot conditioning, and output schema parsing.

Chunking Strategy & Embedding Optimization

A common pitfall in naive RAG implementations is poor chunk boundary selection. If chunks are too small, crucial contextual relationships across sentences are lost. If chunks are too large, extraneous noise dilutes the similarity score, degrading retrieval precision.

In QueryNest, I implemented a hierarchical sliding-window chunking strategy with dynamic token budgeting:

Chunk Configuration Parameters:

  • • Chunk Size: 1000 characters
  • • Chunk Overlap: 200 characters
  • • Separators: ["\n\n", "\n", " ", ""]
  • • Metadata Payload: {source_file, page_number, chunk_id}

Each chunk is indexed into Chroma with full metadata provenance, ensuring that every claim generated by the downstream LLM can be mapped back to an exact page number and text segment.

Retrieval & Context Injection

QueryNest utilizes Maximal Marginal Relevance (MMR) alongside standard cosine similarity ranking. When a user submits a query, MMR balances relevance with diversity among retrieved passages, avoiding redundant chunks that say the same thing in different phrasing.

The retrieved chunks are formatted into a strict system prompt that instructs the Gemini model to synthesize answers solely from provided evidence, explicitly stating when information is missing rather than speculating.

Key Engineering Challenges

Building QueryNest involved overcoming several practical production hurdles:

  • Rate Limit & Token Budget Management: Handled through asynchronous queuing and backoff algorithms to respect API thresholds during large batch ingestions.
  • Vector Store Persistence: Managed persistent disk-backed Chroma collections within a Dockerized environment, enabling fast warm starts across deployments.
  • Low Latency Streaming: Built async generator streams in FastAPI to yield synthesized tokens incrementally, significantly improving time-to-first-token UX.

Project Resources

QueryNest is fully open-source with reproducible Docker setup and comprehensive API documentation:

← Back to Blogs