# Run tasks on a local model with Ollama

Point a Borago Board project at Ollama on your own machine or another one on your network, run a task on a local model, and see why the context window matters.

Borago Board can run a task on a model served from hardware you own, instead of on a hosted provider. There is no API key, nothing goes to a provider, and a run costs nothing. This tutorial installs Ollama, adds it to the app as a self-hosted service, points one project at it, and runs a small task. It ends with the setting that decides whether a local model works well, the context window it is served with.

## You will build

A self-hosted service named after your machine, pointing at Ollama. A project whose task runner is that service. One finished task, and a usage line with tokens and time but no cost. Optionally, the same service reached from a second computer at a different address.

## You need

- The desktop app, with a project whose container runs, see [Creating a project](/docs/guides/creating-a-project/).
- A computer that can run a model. [Using a local AI model](/docs/local-models/) gives the hardware guidance. In short, a model that calls tools reliably wants a graphics card with about 24 GB of memory, or about 32 GB of unified memory on an Apple silicon Mac. Smaller models can be selected, and the last section says what to expect from them.
- About 20 GB of free disk for the model this tutorial suggests.

## Steps

### 1. Install Ollama and download a model

Ollama is the model server. Install it from [ollama.com/download](https://ollama.com/download) on the computer that will run the model. Then download a model that can call tools. The [local models page](/docs/local-models/) suggests this one as a starting point for a 24 GB card:

```
ollama pull qwen3-coder:30b
```

What happens. Ollama keeps the model on disk and serves it at `http://localhost:11434/v1`, which speaks the OpenAI chat completions protocol. The app sends its requests there itself. The project container never talks to the model server and needs no access to it.

### 2. Raise the context window Ollama serves

Ollama serves every model with a small context window by default, whatever the model supports. Raise it before adding the service. On Linux, run the first command and add the two lines it opens, then restart Ollama:

```
sudo systemctl edit ollama.service
```

```
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"
```

```
sudo systemctl daemon-reload
sudo systemctl restart ollama
```

On macOS, set the variable and restart the Ollama app. This lasts until the next reboot:

```
launchctl setenv OLLAMA_CONTEXT_LENGTH 32768
```

On Windows, add `OLLAMA_CONTEXT_LENGTH` with the value `32768` as a user environment variable, then quit Ollama from the tray and start it again.

What happens. A larger window uses more graphics memory. 32768 is a sensible start. The app cannot set this for you, because the protocol it uses to talk to the server has no way to ask for a window size. When a prompt is longer than the window, Ollama cuts the beginning off without an error. The model then loses its instructions and looks confused rather than misconfigured. The app detects the window Ollama really serves and sizes its prompts to fit, so nothing is silently cut, but a small window still leaves the model little room to work.

### 3. Add the server under Settings

Open Settings from the gear in the top bar and choose **Self-hosted services**. Press **Add a self-hosted service**. The editor opens with **Name** set to **Self-hosted AI service** and **Server address** set to `http://localhost:11434/v1`. Give it a name you will recognise, such as the machine's name. Leave the address when Ollama runs on this computer. Leave **API key (optional)** empty, because Ollama needs none. Press **Add**.

What happens. The service belongs to your account, not to one project. Its name, its models, its settings and its address sync to every device signed in to the account. Projects refer to it by name. The list row shows the address and a dot that turns green when the server answers.

### 4. Detect the models and verify

Press **Edit** on the new row. The section **Models on this server** fills in by itself with what Ollama has, and **Refresh** asks again. Press **Verify**.

What happens. Verify asks the server a few questions with a model it detected, and prints a card with the server kind, whether the model is installed, whether it makes tool calls, whether it accepts images, the context window, and how long a trivial prompt took. The context line reads, for example, 32,768 tokens, and adds "of" and the model's own larger figure when the server serves less than the model supports. A 40-second answer to a trivial prompt means a real task will take hours. When Ollama is installed but not running, the editor offers **Start it**. When the model is missing, it offers a button that downloads it, with progress. Leave **Tool calling** on **Automatic (recommended)**. Press **Save**, or close the editor if nothing changed.

### 5. Point the project's task runner at it

Press **Edit project** among the project's controls, then open the **AI Roles** tab. In **Task runner**, choose your service by name from the group **Self-hosted AI services (your hardware)**. In **Model**, choose the model from the group headed **On** and the service's name. Press **Save changes**.

What happens. Each AI role picks its own service, so the task runner can use the local model while chat or the reviews use another. Roles left on their defaults follow the task runner's service. A planning role makes one completion with no tools, so a smaller general model is enough there. The project's **AI Auth** tab lists the account's servers with the same online dot and a **Manage in Settings** button. It has no key to enter for them.

### 6. Run a small task

Add a task in the **Planning** column and paste this as its description:

```
Create a file named hello.txt in the project's root folder containing the
single line "Hello from a local model". Then list the root folder to confirm
the file is there.
```

Move the card to **To Do**. The queue starts it, the card moves to **In Progress** with a timer, and the output streams in the task dialog.

What happens. The run is the same agent loop the hosted providers use, with a shell and a file editor inside the project container. What differs is the ceiling. There is nothing to spend, so a self-hosted run has no cost budget. It stops at **Time limit per task (min)** on the service, 120 minutes by default, or when it produces nothing for 20 minutes. A server that takes longer than 5 minutes to start answering is stopped too. There is no web search, no page fetch and no investigator on a self-hosted run.

### 7. Read the usage line

Open the finished task and find **Token usage** near the bottom. Press it to expand the runs.

What happens. The heading shows tokens in and out, the number of runs and the total run time. There is no dollar amount, because a self-hosted run costs nothing, and on a local model time is the only price. Token counts appear when the server reports them. No cache figures are shown for self-hosted runs. The spend in the top bar does not move.

### 8. Reach the server from another computer

This step is for a model on one machine and the desktop app on another. Make Ollama listen on the network. On Linux, add a second line to the same override from step 2 and restart Ollama:

```
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_CONTEXT_LENGTH=32768"
```

Ollama has no password, so allow only the computer that runs the app. Replace both addresses with your own:

```
sudo ufw allow from 192.168.1.10 to any port 11434 proto tcp
```

In the app, edit the service and set **Server address** to the model machine's address, such as `http://192.168.1.50:11434/v1`. If one of your devices reaches the server differently, for example the model machine itself, which reaches it at `localhost`, press **Use a different address on this device** on that device and fill **Address on this device**. **Use the shared one** removes it.

What happens. The server address is shared by every device on the account. The per-device address is the one field that never syncs, so each machine can reach the same server by the route that works from there, and projects still refer to it by one name. Never forward the port to the internet. For access from outside your network, use a VPN and put the VPN address in the app.

## Check it worked

The file `hello.txt` exists in the project's root, which the terminal or the file browser shows. The task is in **Done** with a summary. **Token usage** shows one run, a run time, and no dollar amount. In Settings, **Self-hosted services** shows the service with a green dot.

## What the context window does to task sizing

The app fits each run to the window the server serves, so a small window means a weaker run. When the served window is far smaller than the model supports, a notification says so and names `OLLAMA_CONTEXT_LENGTH`. Chat plans for the runner too. When chat proposes tasks for a project on a local model, it aims for small, tightly scoped tasks, because a failed run is re-run from the start. The line above the proposals says which service and window it sized for, and **Fewer, bigger** re-asks. Only Ollama reports its window. For vLLM, llama-server or LM Studio, type the window you started the server with into **Context window (tokens)** on the service. Too high is the harmful direction.

## When a model is too small to call tools

Small models often cannot make proper tool calls. With **Tool calling** on **Automatic (recommended)**, the app notices and switches that run to a plain text format for commands. A model that loops or answers nothing is stopped with a message saying so. A failed run returns the task to To Do to be retried, and it moves to Blocked with the reason after 6 attempts, or earlier when the check before a retry decides another attempt cannot help. An unreachable server or a missing model is not retried. The task stops at once with the address and the fix. None of this breaks the board or wedges the queue, but a small model will not finish real work, and it may claim a task is done when it is not. Read the output before trusting a Done card from a small model.

## Where to go next

- [Plan a feature with chat, then run the cards](/docs/tutorials/plan-with-chat/) sizes tasks for this runner.
- [Set up Ollama](/docs/tutorials/keys/ollama/) is the short setup page, and the other self-hosted servers have their own.
- [Choosing an AI provider](/docs/guides/choosing-an-ai-provider/) compares the hosted providers with self-hosted servers.

Related: [Using a local AI model](/docs/local-models/), [Choosing an AI provider](/docs/guides/choosing-an-ai-provider/), [Set up Ollama](/docs/tutorials/keys/ollama/), [Cost and usage](/docs/guides/cost-and-usage/), [How a run works](/docs/reference/how-a-run-works/)