FastGPTFastGPT

Model Troubleshooting

Diagnose and fix LLM, embedding, rerank, and tool-calling errors in self-hosted FastGPT deployments.

(1) How to check model availability issues

  1. For privately deployed models, first confirm whether the deployed model is normal.
  2. Directly test whether the upstream model is running normally through CURL request (cloud model or private model are both tested).
  3. Test the AI Proxy channel on the Model provider tab of Admin → System Resources → System Models. If that channel uses a relay such as OneAPI, inspect the relay separately.
  4. Use the model for testing in FastGPT.

These examples call upstream services directly. Replace the endpoint, model, and credentials with values supported by your channel. Remove credentials from recorded errors. See model configuration.

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'
curl https://api.openai.com/v1/embeddings \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "The food was delicious and the waiter...",
    "model": "text-embedding-ada-002",
    "encoding_format": "float"
  }'
curl --location --request POST 'https://xxxx.com/api/v1/rerank' \
--header 'Authorization: Bearer {{ACCESS_TOKEN}}' \
--header 'Content-Type: application/json' \
--data-raw '{
  "model": "bge-rerank-m3",
  "query": "Who is the director",
  "documents": [
    "Who are you?\nI am the assistant of the movie 'Suzume'"
  ]
}'
curl https://api.openai.com/v1/audio/speech \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tts-1",
    "input": "The quick brown fox jumped over the lazy dog.",
    "voice": "alloy"
  }' \
  --output speech.mp3
curl https://api.openai.com/v1/audio/transcriptions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F file="@/path/to/file/audio.mp3" \
  -F model="whisper-1"

(2) Error - Model response is empty/Model error

Empty output can result from upstream rejection, incompatible parameters, or an event stream ending early. Inspect model-call logs and compare the actual request with the upstream response. Test both JSON and streaming modes; stream: false may return a more complete error.

Common causes:

  1. The upstream provider blocked the request due to content safety filters.
  2. Unsupported model parameters: test with messages and required fields first, then add optional parameters one by one.
  3. Parameter values out of range for the model: for example, models that reject temperature: 0 or two decimal places, max_tokens exceeding the model limit, or context length overflow.
  4. Incompatible streaming implementation in a self-hosted model server.

You can copy the request body from the error log and test it directly with curl:

curl --location --request POST 'https://api.openai.com/v1/chat/completions' \
--header 'Authorization: Bearer sk-xxxx' \
--header 'Content-Type: application/json' \
--data-raw '{
  "model": "xxx",
  "temperature": 0.01,
  "max_tokens": 1000,
  "stream": true,
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ]
}'

(3) How to test whether a model supports tool calling

The model and channel must both support tool calling. Verify ordinary chat first, then the two-round protocol below. The sample address represents an OpenAI-compatible model endpoint or relay.

1. Request a tool call

curl --location --request POST 'https://model.example.com/v1/chat/completions' \
--header 'Authorization: Bearer sk-xxxx' \
--header 'Content-Type: application/json' \
--data-raw '{
  "model": "gpt-5",
  "temperature": 0.01,
  "max_tokens": 8000,
  "stream": true,
  "messages": [
    {
      "role": "user",
      "content": "What time is it?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_time",
        "description": "Get the current time in the user timezone.",
        "parameters": {
          "type": "object",
          "properties": {},
          "required": []
        }
      }
    }
  ],
  "tool_choice": "auto"
}'

2. Inspect the response

When the model selects a tool, choices[0].delta.tool_calls carries the call. Merge function-name and argument fragments by index and retain the actual id. This example shows a complete fragment shape:

{
  "id": "example-chat-id",
  "object": "chat.completion.chunk",
  "choices": [{
    "index": 0,
    "delta": {
      "tool_calls": [{
        "index": 0,
        "id": "example-call-id",
        "type": "function",
        "function": { "name": "get_time", "arguments": "{}" }
      }]
    },
    "finish_reason": null
  }]
}

3. Send the tool result in a second streamed request

Send the tool result using the actual call ID, function name, and arguments returned in the first round. example-call-id is a placeholder. Expect the model to continue using the tool result; it may request another tool.

curl --location --request POST 'https://model.example.com/v1/chat/completions' \
--header 'Authorization: Bearer sk-xxx' \
--header 'Content-Type: application/json' \
--data-raw '{
  "model": "gpt-5",
  "temperature": 0.01,
  "max_tokens": 8000,
  "stream": true,
  "messages": [
    {
      "role": "user",
      "content": "What time is it?"
    },
    {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "example-call-id",
          "type": "function",
          "function": {
            "name": "get_time",
            "arguments": "{}"
          }
        }
      ]
    },
    {
      "tool_call_id": "example-call-id",
      "role": "tool",
      "name": "get_time",
      "content": "{\n  \"time\": \"2024-09-14 22:59:21 Sunday\"\n}"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_time",
        "description": "Get the current time in the user timezone.",
        "parameters": {
          "type": "object",
          "properties": {},
          "required": []
        }
      }
    }
  ],
  "tool_choice": "auto"
}'

(4) Vector search scores exceed 1

Check embedding normalization, the vector backend's similarity metric, and reranking. Score meaning depends on the backend and model. Verify ranking and thresholds against fixed samples; see retrieval settings.


(5) "Current group upstream load is saturated, please try again later"

This error comes from the upstream configured in your AI Proxy channel (such as a relay like OneAPI or the model provider). Check channel concurrency, rate limits, quota, and service health. Record request identifiers for the service administrator. Verify capabilities and application behavior before switching models.


(6) "Connection Error" in logs when using the API

From the service making the request, check DNS, ports, TLS, proxy address, and network policy, then credentials. Test the upstream and AI Proxy channel separately to identify the failing hop.


(7) Enable image indexing reports 400

Inspect the specific error and the visual/OCR capability, file format, model configuration, and quota required by image processing. Confirm availability according to the deployment's system configuration and import mode.

Edit on GitHub

Last updated