FastGPTFastGPT

Knowledge Base Dual-Engine Principles

Understand storage decoupling, vector retrieval, BM25 full-text indexing, hybrid search, and two-stage reranking mechanisms.

FastGPT employs a dual-engine retrieval architecture, combining semantic vector search with keyword-based full-text indexing to achieve strong generalized understanding alongside exact keyword precision.

Storage Layer Architecture

The platform decouples and partitions knowledge data across distinct physical infrastructure services:

Storage ComponentTechnologyPrimary Content Stored
Operational StoreMongoDBKnowledge base metadata, collection trees, raw chunk texts, background tasks, lease states, and ACL permissions
Vector StorePGVector / Milvus / Zilliz / OceanBaseDense vector embeddings generated by models and associated chunk ID mappings
Full-Text IndexMongoDB Text / Milvus BM25Tokenization dictionaries, inverted indexes, and term-frequency statistical weights

Multi-Vector Mapping and Chunk Storage

In MongoDB's dataset_datas collection, each chunk record stores raw content (q and a) alongside an indexes array, enabling one-to-many vector mapping (a single chunk can bind default text embeddings, custom question embeddings, image description indexes, and image vector embeddings). During retrieval, matched vector IDs map back to their source chunk in dataset_datas; when multiple vectors hit the same chunk, they merge automatically using the highest score, preserving full context alongside targeted recall.


Dual Retrieval Engine Mechanics

1. Vector Search Engine

  • Core Mechanism: Embedding models map text into high-dimensional vector spaces (such as 1536 or 3072 dimensions), using the configured backend metric to rank semantic matches. Dimensions and score distributions depend on the model and vector backend.
  • Key Advantage: Excels at synonymous phrasing, contextual paraphrasing, intent generalization, and cross-lingual queries. Chunks match reliably whenever semantic intent aligns.

2. Full-Text Search Engine

  • Core Mechanism: Tokenizes text and ranks keyword matches. When the vector database is Milvus (≥ 2.5.16), full-text search uses Milvus BM25 (modeldata_v2 single table) with lingua language identification by default for higher accuracy (configurable to MILVUS_LANGUAGE_IDENTIFIER=whatlang for faster throughput); on PG, OceanBase, SeekDB, and openGauss, full-text search uses MongoDB $text indexes with jieba tokenization.
  • Key Advantage: Delivers high recall for serial numbers, legal article numbers, specialized terminology, proper names, and exact error codes.

Select knowledge base retrieval mode


Hybrid Search and Two-Stage Reranking

Text hybrid search follows this pipeline, with reranking controlled by application settings:

Text hybrid retrievalText hybrid retrieval

  1. Stage 1: Multi-Route Recall: The engine queries the vector database and full-text index in parallel, then combines candidate chunks using the configured route weights.
  2. Stage 2: Cross-Encoder Reranking: When enabled, the system passes the query and candidate text to a Rerank model and blends the reranked results. Rerank failures preserve the original recall results. Score semantics depend on the model and provider; calibrate thresholds against labeled examples.
  3. Threshold Filtering and Selection: The engine performs final deduplication, applies the supported score filter, and selects whole ranked chunks within a token budget. The final chunk can take the accumulated size beyond the budget boundary.

Multi-route recall and reranking score distribution


Key Search Parameters

In search testing and application configuration, these parameters govern retrieval quality:

ParameterValid RangeRecommendation and Behavior
Search MethodSemantic search / Full-text search / Hybrid searchHybrid search is the recommended default, providing semantic generalization and exact term matching.
Citation cap (limit)Token budgetControls retrieved context size; tune with chunk size and the application context window.
Lowest correlation0 ~ 1Uses rerank scores when reranking succeeds, or embedding scores in pure semantic search mode. Other modes may skip score filtering; inspect usingReRank and usingSimilarityFilter where exposed.

Search parameters and score threshold configuration


Image Retrieval Architecture

Knowledge base retrieval supports image inputs across two indexing and recall paths depending on the models configured on the knowledge base:

  1. Image Description Retrieval: With a Vision Language Model (VLM) configured, the system generates text descriptions of images during ingestion and query time, matching them through standard text retrieval.
  2. Image Vector Retrieval: With a multimodal Embedding model that accepts image inputs, the system encodes images directly into vectors and matches them against stored image embeddings.
Knowledge Base Model CombinationText-Only QueryImage-Only QueryText + Image Hybrid Query
Standard Text Embedding (No Vision Model)Standard text retrievalNo image retrieval pathUses text input only
Standard Text Embedding + Vision ModelStandard text retrieval (matches ingested image descriptions)Converts query image to description for text retrievalCombines query text and image description
Multimodal Image Embedding (No Vision Model)Standard text retrievalDirect image vector similarity searchFuses text retrieval and image vector search
Multimodal Image Embedding + Vision ModelText retrieval (matches ingested image descriptions)Dual-route image description + image vector searchMulti-route RRF fusion across text, image description, and image vector

Common Retrieval Symptoms and Tuning Matrix

SymptomRecommended Checks and Adjustments
No relevant chunks returnedConfirm collection indexing is complete; lower Lowest correlation; raise the Citation cap (limit); check whether the query is too short or missing subject context
Results are overly broad or off-topicRaise Lowest correlation; reduce Citation cap to trim noisy chunks; refine chunk splitting size; apply Tag Filters to narrow the collection scope
Serial numbers, article codes, or error codes missSwitch to Hybrid search or Full-text search; increase full-text weight; disable query rewriting on exact code queries to preserve literal terms
Conversational or paraphrased queries missEnable Hybrid search with a Rerank model; enable query rewriting to resolve multi-turn coreferences; add custom index questions to high-value chunks
Noticeable latency after enabling query rewritingQuery rewriting adds an LLM call before retrieval; switch to a lower-latency model or enable it selectively for multi-turn follow-up scenarios
Ranking remains inaccurate with Rerank enabledVerify that the user query forms a complete statement; confirm first-stage recall returns enough candidates; calibrate similarity thresholds against run details
Weak image-to-image recallConfirm the knowledge base uses a multimodal Embedding model or Vision model and that images generated valid image indexes during import
Unstable text + image hybrid resultsClarify the primary query signal; upload the image alone when searching purely for visual similarity so extra generic text does not dilute RRF ranking

Next Steps

Edit on GitHub

Last updated

On this page

Put your business knowledge to work with AI

Import company documents and build accurate, reliable, traceable AI answers.