29 Local AI with Ollama

The metacheck power module can use an AI language model to read sections of a research paper and extract information about power analyses — things like which statistical test was used, what sample size was planned, and what effect size was assumed. How well this works depends on the AI model you use, and the general ability of the model to understand scientific text.

By default, Metacheck expects a cloud API to do this. But there can be cases where you want to make sure that data does not leave your computer. This guide explains how to run AI models locally on your own computer, using a free, open-source tool called Ollama. Running locally means:

No internet connection required after the initial setup
No API keys or accounts needed
No usage limits or costs
Your papers and any information processed by the AI stay on your own machine

The trade-off is that your computer does the work, so it will be slower — especially the first time, while the model loads into memory.

29.1 Setup

You need to repeat these step every time you want to use Ollama with Metacheck on a new machine.

29.1.1 What you need

A computer with at least 8 GB of RAM (16 GB recommended)
R with the metacheck package installed
At least 2–5 GB of free disk space for the AI model (and more if you want to try larger models)

You do not need a graphics card (GPU). Ollama works on ordinary laptop hardware.

29.1.2 Step 1: Install Ollama

Ollama is a free program that lets you download and run AI models on your computer. It runs in the background like a web server, and R talks to it automatically.

Windows:

Go to https://ollama.com/download
Click Download for Windows
Run the installer and follow the prompts
Once installed, Ollama starts automatically in the background. You will see its icon in the system tray (bottom-right corner of your screen)

Mac:

Go to https://ollama.com/download
Click Download for Mac
Open the downloaded .dmg file and move Ollama.app to your Applications folder
You can eject the disk image and delete it now.
Open Ollama from Applications — it will appear in your menu bar
When you first open Ollama, it will add background items that you can manage in Login Items & Extensions
Ignore the “Launch” interface and open a Terminal window (or use the terminal in RStudio – not the console) and use the terminal instructions below.

29.1.3 Step 2: Download an AI model

Ollama needs to download the actual AI model before it can be used. Think of this like downloading an app: you do it once, and then it is stored on your computer.

The simplest way to do this is in the app itself: click the “Add Model” button, search for “qwen2.5”, and click “Download” next to the qwen2.5:3b model (or select the model, and ask a question).

If you feel comfortable using the command line, you can also do this with a terminal command. Open a Terminal (Mac) or Command Prompt / PowerShell (Windows) and run:

ollama pull qwen2.5:3b

This downloads the Qwen 2.5 (3 billion parameter) model — a compact, capable model that works well for the power module and is about 2 GB in size. The download may take a while depending on your internet connection.

Tip: If you have more RAM (16 GB+) and want better accuracy, you can use a larger model (but it will be slower and take more disk space). For example: - ollama pull qwen2.5:7b – 4.7 GB, noticeably more capable - ollama pull llama3.1 – 4.7 GB, good general-purpose alternative - ollama pull smollm:135m – 91 MB, tiny model for quickly testing workflows

To check which models you have downloaded, run:

ollama list

29.1.4 Step 3: Verify Ollama is running

Ollama should start automatically when you install it. Anytime you want to use ollama from Metacheck, make sure it is running in the background. You can check this by looking for the Ollama icon (in the system tray on Windows, or menu bar on Mac).

To check, open your web browser and visit:

http://localhost:11434

If you see a message like Ollama is running, you are ready to go.

If nothing appears, open the Ollama application manually (from Applications on Mac, or search for “Ollama” in the Windows Start menu).

29.1.5 Step 4: Configure Metacheck to use Ollama

In R, run the following three lines before using the power module. These tell Metacheck to use the AI, which model to use, and how many calls it is allowed to make (which in the case of a free local model can be set as high as you want):

llm_use(TRUE)                          # turn on LLM support
llm_model("ollama/qwen2.5:3b")         # use the local Ollama model
llm_max_calls(5000)                     # maximum number of AI queries per run

The model name must start with ollama/ — this tells Metacheck to send queries to your local Ollama installation rather than a cloud service.

Note: If you downloaded a different model in Step 2, replace qwen2.5:3b with the name you used (for example, "ollama/qwen2.5:7b" or "ollama/llama3.1").

29.2 A simple first example

Here is a quick example to confirm that everything is working. We will search a paper for any paragraph that mentions participants, then ask the local AI model to describe who those participants were.

library(metacheck)

llm_use(TRUE)
llm_model("ollama/qwen2.5:3b")
llm_max_calls(5000)

# Load a single paper from the built-in psychsci dataset
paper <- psychsci[[84]]

# Find all paragraphs in the paper that mention participants
participant_paragraphs <- search_text(paper, "particip", return = "paragraph")

# Combine them into one block of text to send to the AI
combined_text <- paste(participant_paragraphs$text, collapse = "\n\n")

# Ask the AI to describe the participants
result <- llm(
  text = combined_text,
  system_prompt = "Based on these excerpts from a scientific paper, briefly describe the study participants: how many there were, who they were (e.g. students, patients, age group), and any other relevant characteristics reported."
)

cat(result$answer)

The first query will feel slow — this is normal. Ollama needs a moment to load the model into memory the first time. Subsequent queries in the same R session will be much faster.

If you see a short paragraph describing the participants, your setup is working correctly and you are ready to move on to the power module.

29.3 Run the power module

29.3.1 On a single paper

library(metacheck)

llm_use(TRUE)
llm_model("ollama/qwen2.5:3b")
llm_max_calls(5000)

# Load a single paper (this uses paper number 84 from the built-in psychsci dataset)
paper <- psychsci[[84]]

# Run the power module
result <- module_run(paper, "power")

# See the result
result$traffic_light   # "green", "red", or "na"
result$summary_text    # a plain-English summary
result$table           # the full extracted data

The first query will feel slow — this is normal. Ollama needs a moment to load the model into memory. Subsequent queries on the same session will be much faster.

29.3.2 On the full psychsci dataset

library(metacheck)

llm_use(TRUE)
llm_model("ollama/qwen2.5:3b")
llm_max_calls(5000)

paper <- psychsci
result <- module_run(paper, "power")

Processing the full dataset takes considerably longer (and could take hours, depending on your hardware, the number of power related sentences detected, and model size).

29.4 Running RegCheck locally

The RegCheck module compares a paper against its preregistration. By default it sends text to a hosted RegCheck app, but you can run the whole comparison on your own machine with Ollama, so no paper or preregistration text leaves your computer. This needs a one-time setup of a small RegCheck server (in addition to Ollama).

1. Pull the required Ollama models. RegCheck uses a fixed embedding model plus a language model of your choice. In a terminal:

ollama pull nomic-embed-text-v2-moe
ollama pull llama3.2          # or llama3.1, mistral, qwen3.5:9b, etc.

Use the largest language model that fits your hardware — bigger models generally give better judgements.

2. Start the local RegCheck server. There are two ways. The Docker path is recommended and needs no Python:

# Docker (recommended) — first run builds the image (~5 minutes), then it is instant
regcheck_start_local(method = "docker")

The Python path needs Python 3.12+ and a one-time environment setup:

regcheck_setup_local()                    # once: build the Python environment
regcheck_start_local(method = "python")   # start the server

Either way the server runs at http://localhost:8000, Ollama must be running with the models pulled, and the API token is set automatically to "metacheck-local" — no token configuration needed. By default regcheck_start_local() auto-selects the largest language model you have pulled; pass model = "..." to choose a specific one.

3. Run reg_check — it uses the local server automatically. With the local server running, the default client = "ollama" routes the comparison through it:

module_run(demopaper(), "reg_check")   # client = "ollama" by default → local server

4. Stop the server when finished:

regcheck_stop_local()

See the RegCheck module chapter for what the comparison returns and how to read the results.

29.5 Troubleshooting

29.5.1 “Ollama is not running” or connection errors

Open the Ollama application and wait a few seconds for it to start
On Windows, look for the Ollama icon in the system tray; right-click and choose “Start”
On Mac, launch Ollama.app and click the Ollama icon in the menu bar

29.5.2 The model seems to freeze or takes very long

This is normal on the first query while the model loads. Wait at least 60 seconds before concluding something has gone wrong
Larger models take longer; try qwen2.5:3b if you just want to check if your setup is working, as it is faster than the larger models.

29.5.3 “Model not found” error

Make sure the model name in llm_model() exactly matches the name shown in ollama list
The name must start with ollama/, for example "ollama/qwen2.5:3b"

29.5.4 Out of memory errors

Try a smaller model: ollama pull qwen2.5:3b and use llm_model("ollama/qwen2.5:3b")
Close other applications to free up RAM

29.5.5 Switching back to a cloud model

llm_model("groq")   # or another model name

29.6 Quick reference

# Minimal setup to run the power module locally
library(metacheck)
llm_use(TRUE)
llm_model("ollama/qwen2.5:3b")
llm_max_calls(5000)
paper <- demopaper()
result <- module_run(paper, "power")