# Integrating Local Models with Ollama

> Deploy local open-source LLMs and embedding models with Ollama and connect them to your self-hosted FastGPT instance.

Source: https://doc.fastgpt.io/self-host/custom-models/ollama
Last Updated: 2026-11-12

[Ollama](https://ollama.com/) is an open-source AI model deployment tool focused on simplifying the deployment and usage of large language models. It supports one-click download and running of various LLMs.

## Installing Ollama

Ollama supports multiple installation methods, but Docker is recommended. If you install Ollama directly on your host machine, you'll need to figure out how to let the FastGPT Docker container access Ollama on the host, which can be tricky.

### Docker Installation (Recommended)

Use Ollama's official Docker image for one-click installation and startup (make sure Docker is installed on your machine):

```bash
docker pull ollama/ollama
docker run --rm -d --name ollama -p 11434:11434 ollama/ollama
```

If your FastGPT is deployed in Docker, put the Ollama container on the same network as the AI Proxy container (`fastgpt-aiproxy`). Starting with v4.17, AI Proxy sends model requests, and in the official Compose file AI Proxy joins only the `fastgpt_aiproxy` network:

```bash
docker run --rm -d --name ollama --network fastgpt_aiproxy -p 11434:11434 ollama/ollama
```

### Host Installation

If you prefer not to use Docker, you can install directly on the host machine.

#### MacOS

If you're on macOS with Homebrew installed:

```bash
brew install ollama
ollama serve # Start the service after installation
```

#### Linux

On Linux, you can use a package manager. For Ubuntu:

```bash
curl https://ollama.com/install.sh | sh # Downloads and runs the official install script
ollama serve # Start the service after installation
```

#### Windows

On Windows, download the installer from the Ollama official website. Run the installer and follow the wizard. After installation, start the service in Command Prompt or PowerShell:

```bash
ollama serve # After installation, visit http://localhost:11434 in your browser to verify Ollama is running
```

#### Additional Notes

When running Ollama as a host application, configure Ollama to listen on 0.0.0.0.

##### 1. Linux

If Ollama runs as a systemd service, edit the service file with `sudo systemctl edit ollama.service`. Add `Environment="OLLAMA_HOST=0.0.0.0"` under the \[Service] section. Save and exit, then run `sudo systemctl daemon-reload` and `sudo systemctl restart ollama` to apply.

##### 2. MacOS

Open a terminal and run `launchctl setenv OLLAMA_HOST "0.0.0.0"`, then restart the Ollama application.

##### 3. Windows

Open "Edit system environment variables" from the Start menu or search bar. In "System Properties", click "Environment Variables". Under "System variables", click "New" and create a variable named OLLAMA_HOST with value 0.0.0.0. Click "OK" to save, then restart Ollama from the Start menu.

### Pull Model Images

After installing Ollama, no models are available locally -- you need to pull them:

```bash
# For Docker deployment, enter the container first: docker exec -it [Ollama container name] /bin/sh
ollama pull [model name]
```

![Pull Model Images — interface example](https://cdn.jsdelivr.net/gh/labring/fastgpt-img@main/optimized/Ollama-pull.png.webp)

### Test Communication

After installation, verify connectivity by starting a temporary curl container on the `fastgpt_aiproxy` network used by AI Proxy and trying to reach Ollama:

```bash
docker run --rm --network fastgpt_aiproxy curlimages/curl http://XXX.XXX.XXX.XXX:11434  # Container: "http://[container name]:[port]", Host: "http://[host IP]:[port]" (host IP cannot be localhost)
```

If you see that the Ollama service is running, communication is working.

## Integrating Ollama with FastGPT

### 1. Check Available Models

First, check which models Ollama has:

```bash
# For Docker-deployed Ollama: docker exec -it [Ollama container name] /bin/sh
ollama ls
```

![1. Check Available Models — interface example](https://cdn.jsdelivr.net/gh/labring/fastgpt-img@main/optimized/Ollama-models1.png.webp)

### 2. AI Proxy Integration

If you're using FastGPT's default configuration from [here](https://doc.fastgpt.io/self-host/deploy/docker), AI Proxy is enabled by default.

Make sure AI Proxy can access the Ollama container. If not, refer to the [installation section](#installing-ollama) above -- check that the host listens on 0.0.0.0 and that the Ollama container joined the `fastgpt_aiproxy` network.

Run FastGPT, then go to **Admin -> System Resources -> System Models -> Model provider -> New**. Select Ollama as the protocol type, add your pulled model, and fill in the proxy address (`http://[container name]:[port]` for container deployments, or `http://[host IP]:[port]` for host installations using a routable host IP).

![2. AI Proxy Integration — interface example](https://cdn.jsdelivr.net/gh/labring/fastgpt-img@main/optimized/Ollama-aiproxy3.png.webp)

After saving and testing the channel, continue to [5. Model Addition and Usage](#5-model-addition-and-usage) to add and enable the model under **Model configuration**.

### 3. OneAPI Integration

This section applies when you already run a OneAPI service. Deploy OneAPI according to its official documentation and make sure AI Proxy can reach it. For new deployments, use [AI Proxy Integration](#2-ai-proxy-integration) above or [Direct Integration](#4-direct-integration) below.

In the OneAPI page, add a new channel with type Ollama. Enter your Ollama model name (must match exactly), then fill in the Ollama proxy address below -- default is [http://address:port](http://address:port), without /v1. Test the channel after adding. This example uses Docker-deployed Ollama; for host-installed Ollama, use http\://\[host IP]:\[port].

![3. OneAPI Integration — interface example](https://cdn.jsdelivr.net/gh/labring/fastgpt-img@main/optimized/Ollama-oneapi1.png.webp)

After adding the channel, click Token -> Add Token, fill in the name, and configure as needed.

![3. OneAPI Integration — interface example](https://cdn.jsdelivr.net/gh/labring/fastgpt-img@main/optimized/Ollama-oneapi2.png.webp)

FastGPT routes model requests through AI Proxy. Keep `AIPROXY_API_ENDPOINT` and `AIPROXY_API_TOKEN` configured. In Admin, add an OpenAI-compatible model provider with your OneAPI endpoint (`http://address:port/v1`) and OneAPI token, then associate the models.

Continue to [5. Model Addition and Usage](#5-model-addition-and-usage) to add and use models.

### 4. Direct Integration

To use Ollama without OneAPI, configure its OpenAI-compatible endpoint (`http://address:port/v1`) as a model provider in Admin. Ollama does not require authentication by default; if you enabled authentication, supply the corresponding key. FastGPT still routes requests through AI Proxy, so keep `AIPROXY_API_ENDPOINT` and `AIPROXY_API_TOKEN` configured. For a host deployment, use the host IP; for a container deployment, use an address reachable from AI Proxy.

Continue to [5. Model Addition and Usage](#5-model-addition-and-usage) to add and use models.

### 5. Model Addition and Usage

In FastGPT, go to **Admin -> System Resources -> System Models -> Model configuration -> Add model**. Make sure the model identifier matches the model name configured in the channel. See [Model configuration](https://doc.fastgpt.io/self-host/config/model/intro) for details.

![5. Model Addition and Usage — interface example](https://cdn.jsdelivr.net/gh/labring/fastgpt-img@main/optimized/Ollama-models2.png.webp)

![5. Model Addition and Usage — interface example](https://cdn.jsdelivr.net/gh/labring/fastgpt-img@main/optimized/Ollama-models3.png.webp)

Create an app in Studio and select the model you added. The model name shown is the alias you set. Note: the same model cannot be added multiple times -- the system uses the alias from the most recent addition.

![5. Model Addition and Usage — interface example](https://cdn.jsdelivr.net/gh/labring/fastgpt-img@main/optimized/Ollama-models4.png.webp)

### 6. Additional Notes

For the Ollama proxy addresses above: host-installed Ollama uses "http\://\[host IP]:\[port]", container-deployed Ollama uses "http\://\[container name]:\[port]".
