6 Caching and Reuse

The two slowest and most expensive parts of a Metacheck run are querying a language model and downloading files from an online repository. Both are re-encountered constantly: you re-run a report after tweaking a module, you process the same paper in several modules, or you build an archive from hundreds of repositories overnight and it stops halfway. To avoid paying for the same work twice, Metacheck keeps two on-disk caches — one for LLM responses, one for downloaded repository files — plus a small in-memory cache used by a single module.

This chapter explains what each cache stores, where it lives, how entries are keyed (i.e. what counts as “the same” request), when it is reused, and how to inspect or clear it.

The short version

LLM responses are cached automatically (on by default). Re-running a report on the same paper replays the stored answers instead of re-billing the provider. Toggle with llm_cache(); wipe with llm_cache_clear().
Downloaded repository files are cached automatically (always on). The same file is fetched once and reused by every module and every later session. Clear it by deleting the cache directory.
Both caches live in a persistent per-user location (via rappdirs), so they survive across R sessions.

6.1 Where the caches live

Both on-disk caches are stored in the standard per-user cache location for your operating system, resolved by the rappdirs package under scienceverse/metacheck. You never need to create or manage these directories by hand — Metacheck creates them on first use.

Cache	Directory (under the user cache dir)	Contents
LLM responses	`metacheck/llm/`	one `.rds` per distinct LLM call
Repository files	`metacheck/repo_files/`	one sub-folder per repository, mirroring its files

You can see the exact paths on your machine with:

rappdirs::user_cache_dir("metacheck/llm", "scienceverse")
rappdirs::user_cache_dir("metacheck/repo_files", "scienceverse")

On Windows these sit under …/AppData/Local/scienceverse/metacheck/Cache/, on macOS under ~/Library/Caches/…, and on Linux under ~/.cache/….

6.2 The LLM response cache

6.2.1 Why it is safe to cache an LLM answer

Normally you would not cache a language-model call: ask the same question twice and you can get two different answers. Metacheck sidesteps this by forcing temperature 0 on every llm() call. At temperature 0 the model is (as close as providers allow) deterministic: the same inputs produce the same output. That makes the answer reproducible, and a reproducible answer is one you can safely store and replay. The cache is therefore not a shortcut that changes results — replaying a cached answer gives you the same thing a fresh call would have.

This is the same property that makes Metacheck’s LLM use reproducible in the first place: it is documented alongside the LLM workflow in the Using Large Language Models chapter.

6.2.2 What is stored, and how entries are keyed

The cache is keyed by everything that determines the answer. When llm() is about to make a call, it builds a key from five inputs:

the model name,
the system prompt,
the input text (the row of text being classified),
the type specification (the structure the answer must take), and
any params (e.g. a seed).

These are serialised to a canonical byte stream and reduced to an MD5 hash, which becomes the filename (<hash>.rds). Two details make the key robust:

params are sorted by name before hashing, so list(seed = 1, top_p = 1) and list(top_p = 1, seed = 1) hit the same entry — list order does not matter.
The type spec is reduced to its printed form, so if you change the requested output structure (add a field, change a description), the key changes and you get a fresh call rather than a stale answer for the old structure.

Change any of these five inputs and the key changes, so the cache misses and a real call is made. This is exactly what you want: a different question is a different question.

Each entry stores not just the parsed answer (the data frame llm() returns) but also the raw provider result and, where the provider exposes it, the model’s reasoning/thinking trace — so nothing is lost by going through the cache.

6.2.3 Turning it on and off

The cache is on by default. Query or set it with llm_cache():

llm_cache()          # is the cache on? (TRUE by default)
llm_cache(FALSE)     # force every call to hit the provider afresh
llm_cache(TRUE)      # re-enable

Disabling it is mostly useful when you are deliberately testing provider behaviour or measuring latency, and want to be sure no answer is being replayed.

6.2.4 Clearing it

To wipe every stored answer — for example after upgrading a model, or to reclaim disk space — use llm_cache_clear(), which deletes all cache entries and returns how many it removed:

llm_cache_clear()    # returns the number of entries deleted

You do not normally need to clear the cache to pick up a changed prompt or model: because those are part of the key, a change produces a fresh call automatically. Clearing is for reclaiming space or forcing a clean slate.

6.2.5 What this means in practice

Re-running a report on the same paper is close to free on the LLM side: the first run populates the cache, and every subsequent run replays it. This is what makes it comfortable to iterate on a module’s non-LLM code, or to regenerate a report, without watching your API bill climb. It also means the llm_max_calls() cap counts actual provider calls — cached replays do not consume it.

6.3 The repository-file download cache

6.3.1 What it does

Modules that read file contents — data_check and code_check — need local copies of the files a repository shares. repo_check only lists the files on OSF, GitHub, and Zenodo (it does not fetch them). When a content-reading module needs a file, it is downloaded once into a shared cache and read from there. Every later read — by another module in the same run, or by any module in a later session — reuses the cached copy instead of downloading again.

This is why, in a long archive-building run, an interruption is cheap to recover from: the files already fetched are still on disk, and resuming skips straight past them.

6.3.2 How entries are keyed

Each repository gets its own sub-folder, named by a filesystem-safe encoding of its URL (the scheme is stripped and any awkward characters are replaced), so two different repositories never collide and the same repository always resolves to the same folder across sessions. Inside that folder, each file is stored at its repository-relative path, not just its basename — so two files both called data.csv in different sub-folders are kept apart rather than overwriting each other.

Reuse is decided by a simple, robust test: if the target path already exists and is non-empty, it is treated as already downloaded and reused. A zero-byte file (the fingerprint of a download that failed partway) is not trusted — it is re-fetched. This means a broken partial download never silently poisons later runs.

6.3.3 It is always on

Unlike the LLM cache, the download cache has no on/off switch — reuse of already-fetched files is always active, because there is no downside: the file on disk is a byte-for-byte copy of the remote file. What you can control is how much gets downloaded in the first place, through the data_check arguments described below.

6.3.4 Controlling what gets downloaded

The download cache stores whatever the modules fetch, and data_check gives you fine control over that:

download — "data" (the default) fetches only the machine-readable files the checks analyse (tabular data plus codebook/README files); "all" fetches every file in the repository (the right choice when building a complete archive with convert_psychds()); FALSE/"none" downloads nothing and classifies files by name only.
skip_types — a vector of file types never to download even under download = "all" — e.g. "asset" for stimuli and media a release links to rather than mirrors.
peek_zips — look inside each .zip using an HTTP range request (without downloading the whole archive) and only fetch zips that actually contain data or a codebook, leaving a zip of pure stimuli to be linked instead.
max_file_size / max_download_size — size caps (in MB) that gate what may be fetched, so one runaway file or one enormous repository cannot fill your disk. These are an upfront, all-or-nothing gate per repository: if a file or a repository’s total exceeds the cap, that repository is refused (nothing downloaded) and the reason is reported.

These arguments decide what enters the cache; once a file is in, it stays and is reused. See the data_check chapter for full details and examples.

6.3.5 Clearing it

There is no dedicated clearing function; the cache is just files on disk. To reclaim space or force fresh downloads, delete the directory (or a single repository’s sub-folder within it):

# the whole download cache
unlink(rappdirs::user_cache_dir("metacheck/repo_files", "scienceverse"),
       recursive = TRUE)

Because entries are keyed by repository URL and relative path, deleting is safe: anything still needed is simply re-downloaded on the next run.

6.4 How the caches work together in a full run

When you run several modules on a paper — or build an archive across a corpus — the two caches complement each other:

repo_check lists the repository’s files (no download).
data_check / code_check download the files they need into the repository-file cache; a second module reading the same file reuses it.
When an LLM is enabled, data_check, codebook_check, and other LLM-using modules send text to llm(), whose answers are stored in the LLM response cache.
Re-running the whole pipeline, or resuming an interrupted archive build, replays both caches: files are read from disk and LLM answers are replayed, so only genuinely new work is done.

Modules such as codebook_check do not have a separate results cache of their own — their re-run savings come entirely from these two shared caches (the files they read and the LLM answers they request). This keeps the caching story simple: there are two on-disk caches, and every module benefits from them without needing its own.

6.5 A note on the in-memory synonym cache

One module, funding_check, builds a list of organisation-name synonyms once and keeps it in memory for the rest of the session (an ordinary in-memory cache, not on disk). It is an implementation detail with no user-facing controls: it simply avoids rebuilding the same lookup repeatedly within a single R session, and disappears when the session ends. You never need to manage it.

6.6 Summary

Cache	Scope	On by default	Key	Toggle	Clear
LLM responses	on disk, cross-session	yes	model + prompt + text + type + params	`llm_cache()`	`llm_cache_clear()`
Repository files	on disk, cross-session	yes (always on)	repo URL + relative path	— (control via `data_check` download args)	delete the cache directory
Funding synonyms	in memory, per session	yes	—	—	ends with the session

The guiding principle is the same for both on-disk caches: cache only what is reproducible, and key it on everything that could change the result. LLM answers are cached because temperature 0 makes them reproducible; downloaded files are cached because a byte-for-byte copy of a remote file is trivially reproducible. In both cases, changing the thing that matters — a prompt, a model, a repository — produces a fresh result automatically, so the caches speed you up without ever giving you a stale answer.