Set up LM Studio
Serve a model from LM Studio's local server, add it to Borago Board as a self-hosted service, set its context window and run a first task.
LM Studio is a desktop app for downloading and serving models, with a local server that speaks the OpenAI chat completions protocol. Borago Board runs tasks on it with no API key and no cost per run. The concepts are the same as in Run tasks on a local model with Ollama.
You need
- The desktop app and a project, see Creating a project.
- LM Studio from lmstudio.ai, with a model downloaded that supports tool use.
Steps
1. Load the model with a larger context window
The window is set when the model loads. In LM Studio, set Context Length in the model's load settings, or from the command line with LM Studio's lms tool. The first command lists your models, and the second loads one by that name.
lms ls
lms load <model> --context-length 32768Changing the window means loading the model again.
2. Start the server
Start the server from the Developer tab, or from the command line.
lms server startThe server listens at http://localhost:1234/v1 by default.
3. Add the service
Open Settings from the gear and choose Self-hosted services. Press Add a self-hosted service. Name it, set Server address to http://localhost:1234/v1, or the machine's network address if LM Studio serves on the network, and press Add.
4. Enter the context window
Press Edit on the row. Type the Context Length you loaded with into Context window (tokens). LM Studio does not report it, and a figure that is too high makes the server cut prompts short without an error. Press Verify to check the model makes tool calls, then Save. If the model can read images, turn on Image attachments yourself.
5. Choose it for the runs
In the project, press Edit project, open AI Roles, and in Task runner choose your service from Self-hosted AI services (your hardware). Choose the model and press Save changes.
What happens. Self-hosted runs have a shell and a file editor in the project container, a code search and task breakdown. They have no web search, no page fetch and no investigator.
First task to try
Create a file named hello.txt in the project's root folder containing the
single line "Hello from a local model". Then list the root folder.Related: Using a local AI model, Run tasks on a local model with Ollama, Set up llama.cpp, Choosing an AI provider