Run the same task on three providers
Run one small task on Claude, OpenAI and Gemini in Borago Board and compare the tokens, cost, model and summary of each run side by side.
Borago Board runs each task on your own API key, and any task can override the project's provider. That makes a fair comparison easy. You write one task, make identical copies, point each copy at a different provider, and read the results side by side. This tutorial uses Claude, OpenAI and Gemini. Grok works the same way.
You will build
Three identical tasks on one project, each run by a different provider, with their usage and their answers ready to compare.
You need
- The desktop app with a project whose container starts, see Creating a project.
- API keys for two or three providers. See Claude, OpenAI, Gemini or Grok.
Steps
1. Add the keys
Press Edit project and open the AI Auth tab. Under AI credentials (this project) there is one card per provider, titled Claude account (Anthropic), Codex account (OpenAI), Gemini account (Google) and Grok account (xAI). Paste each key on its card and save.
What happens. The key is stored encrypted and never shown again. A project can hold a key for every provider at once, because any single task may override the project's provider. The app calls its OpenAI runner Codex, so that is the name on the card and in every picker. Each card also carries its own cost budget, see Set a cost budget and read every run's usage.
2. Write one small, read-only task
Press Add task in Planning. Paste this as the description:
Read the source code and explain, in at most ten bullet points, how a request
travels from the entry point to the code that handles it. Name the files
involved. Do not change, create or delete any files.Leave AI service on Default and Model override on Inherit - project default. Save it.
What happens. The task is read-only on purpose. One container serves every task in a project, so a task that edits files would leave the next copy starting from a different repository. A read-only task gives every provider the same starting point.
3. Make two copies for the other providers
Press Add task twice more and paste the same description each time. In the first copy set AI service to Codex. In the second set it to Gemini. Leave Model override on its inherit option, or pick a model from the list.
What happens. Model override clears whenever AI service changes. A model name belongs to one provider, so a Claude model kept on an OpenAI task would be meaningless and is refused when saved. The model list always shows the chosen provider's own models.
4. Run all three
Drag the three cards to To Do.
What happens. The queue runs them one after another, never two at once, because a project runs one task at a time. Each run starts from nothing. There is no session resume, so every provider reads the code for itself.
5. Compare the usage
Open each finished task and expand Token usage.
What happens. Each run line shows its tokens in and out, cache reads, estimated cost, duration and the model that served it. The model is the one the provider reported, not the one requested. The cost is an estimate from the provider's list price. Put the three blocks side by side and compare cost, time and tokens.
6. Compare the answers
In each task, read the run's output under the description, and expand Context Summary where it appears.
What happens. The output is the latest run's text. Run History lists earlier runs of the same task, if you ran one twice. Judge the three answers against the code yourself.
Check it worked
Each of the three tasks names a different provider's model in Token usage, and each has an answer.
Why the runs differ
The providers do not get identical tools. Every run has the shell, the file editor and the repository search. The investigation tool runs on Claude, OpenAI, Grok and Gemini. Web search runs on Claude, OpenAI and Gemini, and is counted and priced into the run. Reading a web page by address runs on Claude and Gemini only. A task that needs neither search nor pages, like the one above, is the fairest comparison. See the agent for the full table.
Where to go next
Make the provider that suits the project its default under Task runner in the AI Roles tab. Keep the per-task picker for the odd task that fits another provider better.
Related: Choosing an AI provider, Cost and usage, Set a cost budget and read every run's usage, How a run works