Dataset search citation merge
Combine, deduplicate, and rerank retrieval results from multiple knowledge base searches in a FastGPT workflow.

What It Does
Combines citation lists from multiple upstream Dataset Search nodes into a single datasetQuote array, deduplicates chunks by ID, re-ranks results using RRF (Reciprocal Rank Fusion), and trims the combined context to the configured Maximum Tokens budget.
Inputs, Configuration, and Outputs
| Item | Internal Key | Value Type | Description |
|---|---|---|---|
| Maximum Tokens | limit | Number | Maximum token budget retained after merging; lower-ranked RRF items beyond this cap are trimmed |
| Citation-1, Citation-2… (Dynamic) | system_datasetQuoteList | DatasetQuote | Connects two or more quoteQA outputs from upstream Dataset Search nodes |
| Dataset citations | quoteQA | DatasetQuote | Deduplicated, RRF-ranked, and token-trimmed citation list ready for AI Chat |
Usage
AI Chat accepts a single citation list. Connect the outputs of multiple search nodes to Dataset search citation merge, then pass the combined citations to AI Chat.
This verified example supplies two demonstration lists containing three entries, including one duplicate. The output contains two citations with RRF ranking scores:

Open Run Details and select Dataset search citation merge to verify the merged total. This run shows 2:

Example Use Cases
- After question classification, search different Datasets per category, then feed the merged results to a single AI Chat node. This avoids adding a separate AI Chat node to each branch.
- Run parallel searches using both the original user question and an expanded sub-query, then merge and deduplicate the recalled chunks with RRF before answer generation.
Last updated