FastGPTFastGPT

RAG Architecture and Anti-Hallucination

Master the RAG execution loop, context assembly, truncation strategies, evaluation metrics, and anti-hallucination engineering controls.

Retrieval-Augmented Generation (RAG) is a critical architectural pattern combining static knowledge assets with large language models. FastGPT combines retrieval, reference prompts, and citations to reduce unsupported answers. Validate factual accuracy with a representative test set.

The Complete RAG Execution Loop

A configured RAG pipeline can contain the following phases. Query rewriting, hybrid search, and reranking depend on application settings:

RAG question-answering flowRAG question-answering flow


Core Evaluation Metric Framework

Evaluating knowledge base RAG performance centers around four key dimensions:

MetricDefinition and FocusOptimization Levers
RecallWhether the chunk containing the ground truth was successfully retrievedAdjust chunk boundaries, enable hybrid search, tune embedding models
PrecisionProportion of retrieved chunks that provide direct factual valueIntegrate Rerank models for second-pass scoring; increase Lowest correlation thresholds
Context RedundancyDensity of irrelevant information in the model contextRefine chunk size limits, lower citation token budgets, eliminate duplicate data chunks
Factual FaithfulnessDegree of alignment between model statements and cited reference materialStrengthen prompt constraints, select models with strong instruction-following capabilities

Context Assembly and Token Budgeting

Large language models feature maximum context window boundaries, and model attention degrades across lengthy prompts. FastGPT enforces structured context assembly rules:

  1. Token Allocation: The system partitions context into four allocations: global system prompt, multi-turn history, retrieved knowledge chunks, and completion token headroom.
  2. Relevance Ordering: Candidate order follows the active retrieval and reranking configuration. Inspect the returned references to confirm that relevant evidence survives selection.
  3. Safe Boundary Truncation: Retrieval keeps whole chunks and can cross the token-budget boundary with its final chunk. The AI Chat stage applies its own reference limits; inspect the final assembled context.

Prompt context assembly and parameter configuration


Three Pillars of Anti-Hallucination Engineering

Use the following controls and verify their behavior in the published application:

1. Grounded System Instructions

Define explicit operational boundaries within the prompt:

You are a corporate policy assistant.
Answer user questions strictly using the retrieved reference material below.
When the references do not record the requested information, respond that the documentation contains no record, identify the missing details, and direct the user to the appropriate department.
Do not infer unrecorded figures, deadlines, or promises from external knowledge.

2. Hard Lowest Correlation Threshold Filtering

Calibrate Lowest correlation thresholds using labeled queries and verify the active score-filter mode. Configure and test an empty-result branch or insufficient-evidence prompt in the application.

3. End-to-End Citation Transparency

During response generation, the engine tracks and outputs the source file name, chunk text, and collection ID for each used candidate. Front-end chat widgets render clickable citation pills, enabling business users to verify statements against source text.

Citation verification against source material

Next Steps

Edit on GitHub

Last updated

On this page

Put your business knowledge to work with AI

Import company documents and build accurate, reliable, traceable AI answers.