# Set up llama.cpp

Serve a GGUF model with llama.cpp's llama-server, add it to Borago Board as a self-hosted service, set its context window and run a first task.

llama.cpp's `llama-server` serves a GGUF model file over the OpenAI chat completions protocol. Borago Board runs tasks on it with no API key and no cost per run. The concepts are the same as in [Run tasks on a local model with Ollama](/docs/tutorials/local-model-with-ollama/).

## You need

- The desktop app and a project, see [Creating a project](/docs/guides/creating-a-project/).
- llama.cpp installed on the model machine, from [its repository](https://github.com/ggml-org/llama.cpp).

## Steps

### 1. Start the server

`--jinja` turns on tool calling, which a task needs. `-c` sets the context window. `-hf` downloads a model from Hugging Face.

```
llama-server --jinja -c 32768 -hf unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M
```

The server listens at `http://localhost:8080/v1` by default. Without `-c` it uses the model's full trained window, which may not fit in memory.

### 2. Add the service

Open Settings from the gear and choose **Self-hosted services**. Press **Add a self-hosted service**. Name it, set **Server address** to `http://localhost:8080/v1`, or the model machine's address on your network, and press **Add**. Fill **API key (optional)** only if you started the server with `--api-key`. The key is entered in the desktop app. In the cloud browser and on the phone the field is disabled, and a stored key stays stored.

### 3. Enter the context window

Press **Edit** on the row. Type the number you gave `-c` into **Context window (tokens)**. llama-server does not report it, and a figure that is too high makes the server cut prompts short without an error. Press **Verify** to check the model makes tool calls, then **Save**. If the model can read images, turn on **Image attachments** yourself, since llama-server does not say.

### 4. Choose it for the runs

In the project, press **Edit project**, open **AI Roles**, and in **Task runner** choose your service from **Self-hosted AI services (your hardware)**. Choose the model and press **Save changes**.

What happens. Self-hosted runs have a shell and a file editor in the project container, a code search and task breakdown. They have no web search, no page fetch and no investigator. With **Tool calling** on **Automatic (recommended)**, a model whose tool calls fail is switched to a plain text format for that run.

## First task to try

```
Create a file named hello.txt in the project's root folder containing the
single line "Hello from a local model". Then list the root folder.
```

Related: [Using a local AI model](/docs/local-models/), [Run tasks on a local model with Ollama](/docs/tutorials/local-model-with-ollama/), [Set up LM Studio](/docs/tutorials/keys/lm-studio/), [Choosing an AI provider](/docs/guides/choosing-an-ai-provider/)