App Evaluation (Beta)
Evaluate FastGPT applications with test datasets, automated scoring metrics, and quality regression reports.
Starting from FastGPT v4.11.0, batch app evaluation is supported. By providing multiple QA pairs, the system automatically scores your app's responses, enabling quantitative assessment of app performance.
The system automatically scores each QA pair in the dataset and summarizes the result as an Overall Score in the evaluation list.
Create an App Evaluation
Go to the Evaluation Page

App evaluation is available only in the commercial edition. In the left Studio sidebar, click App Evaluation(Beta) to open the App evaluation page, then click Create a task in the upper-right corner.
Fill in Evaluation Details

On the task creation page, provide the following:
- Task name: A label to identify this evaluation
- Evaluation model: The model used for scoring
- Evaluation app: The app to be evaluated (supports simple Chat Agents and Workflows without user interaction nodes; plugins and Chat Agent V2 are not supported)
Prepare Evaluation Data
Click Click to download the CSV template for this app in the dialog to download the CSV template. The template includes these fields:
- Global variables
- q (question)
- a (expected answer)
- Chat history
Notes:
- Maximum of 1,000 QA pairs
- Follow the template format when filling in data
Upload the completed file and click Start the evaluation to create the task.
View Evaluation Results
Evaluation List

The evaluation list shows all tasks with key information:
- Progress: Current execution status
- Executor: The user who created the task
- Evaluation app: The app being evaluated
- Start/End Time: Execution time range
- Overall Score: The task's aggregate score
Use this to compare results across iterations as you improve your app.
Evaluation Details

The screenshot shows an evaluation interrupted by a model HTTP 429 rate limit. Failed records display the error; resolve the rate or quota limit and retry before comparing valid scores.
Click Detail to open the Task Details dialog:
Task Overview: The top of the dialog shows overall task information, including evaluation configuration and summary statistics.
Detailed Results: The lower part of the dialog lists each QA pair with its score, showing:
- User question
- Standard output
- App output
Last updated