7 Caching and Reuse

The two slowest and most expensive parts of a Metacheck run are querying a language model and talking to an online repository — both listing what it contains and downloading its files. Both are re-encountered constantly: you re-run a report after tweaking a module, you process the same paper in several modules, or you build an archive from hundreds of repositories overnight and it stops halfway. To avoid paying for the same work twice, Metacheck keeps four on-disk caches — LLM responses, downloaded repository files, repository listings, and peeked-inside zip archives — plus a small in-memory cache used by a single module.

This chapter explains what each cache stores, where it lives, how entries are keyed (i.e. what counts as “the same” request), when it is reused, and how to inspect or clear it.

The short version

LLM responses are stored automatically, but by default only for the R session: within a session, asking the same question again replays the stored answer instead of re-billing the provider. llm_cache(TRUE) keeps the answers between sessions, llm_cache(FALSE) stores none; wipe them with llm_cache_clear().
Downloaded repository files, repository listings, and peeked zip archives are not cached by default. data_check, code_check, and repo_check otherwise redo this work every run. Pass cache = TRUE to keep it instead.
A cache you ask to keep lives in the current working directory, so a project’s cache sits with the project. See where most of them are with metacheck_cache_info().

7.1 Where the caches live

Nothing is kept between sessions unless you ask for it: without cache = TRUE or llm_cache(TRUE), Metacheck writes only to the R session’s temporary folder, which R removes when the session ends. A cache you ask to keep goes in a folder in your current working directory, rather than a hidden per-user location elsewhere on the machine. This is deliberate: these caches are per-analysis rather than shared between projects, so keeping them beside the project makes them easy to find, easy to inspect, and easy to delete, and they disappear when you delete the project folder. A hidden cache is exactly the kind that gets silently lost.

Cache	Folder (in the working directory)	Contents
LLM responses (with `llm_cache(TRUE)`)	`.metacheck_llm_cache/`	one `.rds` per distinct LLM call
Repository files	`.metacheck_repo_cache/`	one sub-folder per repository, mirroring its files
Repository listings	`.metacheck_repo_info_cache/`	one `.rds` per repository, caching what the host’s API said it contains
Peeked zip archives	`.metacheck_zip_peek_cache/`	one `.rds` per zip URL, caching what a range-request peek found inside it

You never need to create these directories by hand; Metacheck creates them the first time it stores something in them. To see where they are and how much space they take, use metacheck_cache_info(), which returns the location and size of the LLM, repository-file, and repository-listing caches at once (while llm_cache() is FALSE, the LLM row shows this session’s temporary folder) (the zip-peek cache is not currently included in this summary — see Clearing it below for how to inspect or clear that one on its own):

metacheck_cache_info()

Because the folder names begin with a dot, they are hidden in most file browsers, and because they sit inside your project you may want to keep them out of version control. Metacheck does not write to your .gitignore for you, so add a line yourself if you use git:

.metacheck_*

You can move all four caches somewhere else by setting one option, which is useful if you want a single shared cache across several projects:

options(metacheck.cache.dir = "~/metacheck_caches")

Each cache also has its own override for finer control: the metacheck.repo_cache.dir option for the repository files, and the METACHECK_LLM_CACHE_DIR environment variable for the LLM responses.

7.2 The LLM response cache

7.2.1 Why it is safe to cache an LLM answer

Normally you would not cache a language-model call: ask the same question twice and you can get two different answers. Metacheck sidesteps this by forcing temperature 0 on every llm() call. At temperature 0 the model is (as close as providers allow) deterministic: the same inputs produce the same output. That makes the answer reproducible, and a reproducible answer is one you can safely store and replay. The cache is therefore not a shortcut that changes results — replaying a cached answer gives you the same thing a fresh call would have.

This is the same property that makes Metacheck’s LLM use reproducible in the first place: it is documented alongside the LLM workflow in the Using Large Language Models chapter.

7.2.2 What is stored, and how entries are keyed

The cache is keyed by everything that determines the answer. When llm() is about to make a call, it builds a key from five inputs:

the model name,
the system prompt,
the input text (the row of text being classified),
the type specification (the structure the answer must take), and
any params (e.g. a seed).

These are serialised to a canonical byte stream and reduced to an MD5 hash, which becomes the filename (<hash>.rds). Two details make the key robust:

params are sorted by name before hashing, so list(seed = 1, top_p = 1) and list(top_p = 1, seed = 1) hit the same entry — list order does not matter.
The type spec is reduced to its printed form, so if you change the requested output structure (add a field, change a description), the key changes and you get a fresh call rather than a stale answer for the old structure.

Change any of these five inputs and the key changes, so the cache misses and a real call is made. This is exactly what you want: a different question is a different question.

The key does not depend on the version of R. Before Metacheck 0.4.1 it did, by accident (R’s serialised bytes record the version of R that wrote them), so upgrading R emptied the cache. Because the key changed once with that fix, answers cached by an earlier version of Metacheck are not found after updating, and the first run after the update makes real calls again.

Each entry stores not just the parsed answer (the data frame llm() returns) but also the raw provider result and, where the provider exposes it, the model’s reasoning/thinking trace — so nothing is lost by going through the cache.

7.2.3 Turning it on and off

By default the answers are stored in the R session’s temporary folder, so they are replayed within the session and removed when R closes. Query or set this with llm_cache():

llm_cache()          # are answers kept between sessions? (FALSE by default)
llm_cache(TRUE)      # keep answers between sessions, in .metacheck_llm_cache/
llm_cache(FALSE)     # store no answers: every call asks the provider again
options(metacheck.llm.cache = NULL)  # back to the default: this session only

Before Metacheck 0.4.3 the answers were kept between sessions by default; this changed because CRAN does not allow a package to keep files outside the session’s temporary folder unless the user asks for it. Turning the cache off is mostly useful when you are deliberately testing provider behaviour or measuring how long calls take, and want to be sure no answer is being replayed.

7.2.4 Clearing it

To wipe every stored answer — for example after upgrading a model, or to reclaim disk space — use llm_cache_clear(), which deletes all entries of the cache in use (the lasting one when llm_cache() is TRUE, otherwise this session’s temporary one) and returns how many it removed:

llm_cache_clear()    # returns the number of entries deleted

You do not normally need to clear the cache to pick up a changed prompt or model: because those are part of the key, a change produces a fresh call automatically. Clearing is for reclaiming space or forcing a clean slate.

7.2.5 What this means in practice

Re-running a report on the same paper is close to free on the LLM side: the first run fills the cache, and every later run in the same session replays it, as does every run in a later session when you set llm_cache(TRUE). This is what makes it comfortable to iterate on a module’s non-LLM code, or to regenerate a report, without watching your API bill climb. Note that the llm_max_calls() cap still counts cached replays: llm() checks the number of texts against the cap before it looks in the cache.

7.3 The repository-file download cache

7.3.1 What it does

Modules that read file contents — data_check and code_check — need local copies of the files a repository shares. repo_check only lists the files on the repositories a paper links to, such as OSF, GitHub, Zenodo, Dataverse, Figshare, Dryad, and PsychArchives (it does not fetch them). When a content-reading module needs a file, it is downloaded once and read from there, and every later read within the same run reuses that copy instead of downloading again.

Whether those copies survive the session is up to you, and by default they do not. With cache = TRUE, the files are kept in the cache folder and reused by later sessions as well, which is what makes a long archive-building run cheap to recover from after an interruption: the files already fetched are still on disk, and resuming skips straight past them.

7.3.2 How entries are keyed

Each repository gets its own sub-folder, named by a filesystem-safe encoding of its URL (the scheme is stripped and any awkward characters are replaced), so two different repositories never collide and the same repository always resolves to the same folder across sessions. Inside that folder, each file is stored at its repository-relative path, not just its basename — so two files both called data.csv in different sub-folders are kept apart rather than overwriting each other.

Reuse is decided by a simple, robust test: if the target path already exists and is non-empty, it is treated as already downloaded and reused. A zero-byte file (the fingerprint of a download that failed partway) is not trusted — it is re-fetched. This means a broken partial download never silently poisons later runs.

7.3.3 Turning it on

Unlike the LLM cache, which stores answers for the session by default, the download cache keeps nothing between sessions by default. Both data_check and code_check take a cache argument that is FALSE unless you say otherwise:

# the default: files go to a temporary folder, discarded when R exits
module_run(paper, "data_check")

# keep the files, and reuse them in later sessions
module_run(paper, "data_check", cache = TRUE)

With cache = FALSE the files are still downloaded and still read; they are simply written to a session folder that R removes on exit, so nothing accumulates on your disk. Files fetched earlier in the same session are reused either way.

Turn it on when you are checking the same repositories repeatedly, or building an archive over several runs, and the download is worth keeping. Leave it off for a one-off check of somebody else’s repository, where you would otherwise be left with copies of their files you never asked to keep.

Reuse of a cached file is safe because a downloaded file is a byte-for-byte copy of the remote one. What you can control separately is how much gets downloaded in the first place, through the arguments described below.

7.3.4 Controlling what gets downloaded

The download cache stores whatever the modules fetch, and data_check gives you fine control over that:

download — "data" (the default) fetches only the machine-readable files the checks analyse (tabular data plus codebook/README files); "all" fetches every file in the repository, which is the right choice when building a complete archive; FALSE/"none" downloads nothing and classifies files by name only.
skip_types — a vector of file types never to download even under download = "all". When left as NULL, the default, "materials" and "unknown" are skipped: stimuli and software, and the classifier’s collection of manuscripts, logs and unrecognised content, which would only inflate an archive. Pass an explicit vector to override this, or character(0) to download every type. The six types are "data", "code", "documentation", "materials", "output" and "unknown". Skipped files still appear in the manifest, with the reason.
peek_zips — look inside each .zip using an HTTP range request, without downloading the whole archive, and only fetch zips that actually contain data or a codebook, leaving a zip of pure stimuli to be linked instead. It defaults to TRUE in both data_check and repo_check. In repo_check the peek is used slightly differently: it lists each archive’s contents in place of the archive itself, without downloading anything. Peeking itself has its own cache — see below.
max_file_size / max_download_size — size limits (in MB) on what may be downloaded, so that one very large file or one enormous repository cannot fill your disk. A file larger than max_file_size is skipped, and the rest of the repository is still downloaded. max_download_size limits the total size of a repository’s files on your disk: files already saved are kept, and then the smallest of the remaining files are added until the next one would pass the limit. A file whose size the repository does not give is also downloaded, but the download is stopped when the file becomes larger than max_file_size or than what is left of max_download_size, and the file is then reported as too large, like a file whose size was known. Every skipped file appears in the manifest, with the reason.

These arguments decide what enters the cache; once a file is in, it stays and is reused. See the data_check chapter for full details and examples.

7.3.5 Inspecting and clearing it

Three functions report on the cache and one clears it:

repo_cache_dir()     # where it is
repo_cache_size()    # how large it is, in bytes
metacheck_cache_info()  # location and size of the LLM, repo-file, and repo-info caches at once

repo_cache_clear()   # delete everything in it

repo_cache_clear() also takes a single repository URL, so you can discard one repository’s files while keeping the rest — useful when a repository has changed on the server and you want its files fetched afresh. A quiet argument (default FALSE) suppresses its usual message about what was removed:

repo_cache_clear("https://osf.io/abc123")
repo_cache_clear(quiet = TRUE)   # delete everything, without the message

Because entries are keyed by repository URL and relative path, clearing is always safe: anything still needed is simply downloaded again on the next run.

7.4 The repository-listing cache

Downloading a repository’s files is not the only repeated cost: listing what a repository contains — asking Dryad’s or Zenodo’s API “what files does this record have, and what do they weigh” — is a separate API call with its own cost and its own rate limit, paid by repo_check every time it runs, even for a repository whose files are already fully downloaded and cached.

This matters most when a long run is interrupted and restarted: restarting re-issues every listing call again from scratch, which can exhaust a host’s quota (Dryad, in particular, applies a strict one) far faster than the real workload — listing each repository once — would ever need. The repository-listing cache exists specifically to make a restart free for anything already listed once.

It is a separate cache from the repository-file cache described above: one caches what a host’s API said a repository contains, the other caches the files themselves once downloaded.

Off by default, like the file-download cache. Turn it on by passing cache = TRUE to repo_check, or to code_check or data_check, which pass it on to repo_check. Clear it with repo_info_cache_clear():

module_run(paper, "repo_check", cache = TRUE)   # reuse listings across restarts
repo_info_cache_clear()                         # delete every cached listing

A cached listing is not refreshed automatically — a published dataset’s metadata rarely changes once public, but if you know a paper’s repository has genuinely been updated, clear it explicitly rather than relying on the cache to notice.

7.5 The zip-peek cache

Looking inside a .zip without downloading it (the peek_zips behaviour described above) still costs an HTTP range request per zip. Like repository listings, a peeked zip’s contents rarely change once a repository is public, so repeated peeks of the same zip across restarts are also cached on disk, mirroring the repository-listing cache.

As with the repository-listing cache, there is no on/off toggle function — peeking is cached only when you explicitly pass cache = TRUE to the function doing the peeking (zip_peek(), or indirectly through data_check/code_check/repo_check’s own cache argument). To clear it:

zip_peek_cache_clear()   # delete every cached zip peek

7.5.1 Clearing the zip-peek cache

metacheck_cache_info() does not currently report on this cache the way it does the other three — use zip_peek_cache_clear() to clear it, or inspect .metacheck_zip_peek_cache/ directly if you want to see its size first.

7.6 How the caches work together in a full run

When you run several modules on a paper — or build an archive across a corpus — the four caches complement each other:

repo_check lists the repository’s files. With cache = TRUE, a restart reuses that listing instead of re-querying the host.
If repo_check needs to look inside a .zip (peek_zips = TRUE, the default), that peek is itself cached when cache = TRUE is passed through.
data_check / code_check download the files they need; a second module reading the same file in the same run reuses that copy. With cache = TRUE the copies are kept for later sessions as well.
When an LLM is enabled, data_check, codebook_check, and other LLM-using modules send text to llm(), whose answers are stored in the LLM response cache.
Re-running the whole pipeline in the same session replays the LLM cache, so no answer is paid for twice; in a later session it does so only with llm_cache(TRUE). It replays the listings, zip peeks, and downloads too if you ran with cache = TRUE; without it, all three are redone, because the previous session’s copies were discarded. This is the main reason to turn caching on for an archive build you expect to resume.

Modules such as codebook_check do not have a separate results cache of their own — their re-run savings come entirely from these shared caches (the listings, peeks, files, and LLM answers they request). This keeps the caching story simple: there are four on-disk caches, and every module benefits from the ones relevant to it without needing its own.

7.7 A note on the in-memory synonym cache

One module, funding_check, builds a list of organisation-name synonyms once and keeps it in memory for the rest of the session (an ordinary in-memory cache, not on disk). It is an implementation detail with no user-facing controls: it simply avoids rebuilding the same lookup repeatedly within a single R session, and disappears when the session ends. You never need to manage it.

7.8 Summary

Cache	Scope	On by default	Key	Toggle	Clear
LLM responses	on disk; for the session, or between sessions with `llm_cache(TRUE)`	for the session only	model + prompt + text + type + params	`llm_cache()`	`llm_cache_clear()`
Repository files	on disk, cross-session	no (`cache = TRUE` to enable)	repo URL + relative path	`cache` argument of `data_check` / `code_check`	`repo_cache_clear()`
Repository listings	on disk, cross-session	no (`cache = TRUE` to enable)	host + repository id	`cache` argument of `repo_check` / `code_check` / `data_check`	`repo_info_cache_clear()`
Peeked zip archives	on disk, cross-session	no (`cache = TRUE` to enable)	zip URL	`cache` argument of the peeking function	`zip_peek_cache_clear()`
Funding synonyms	in memory, per session	yes	—	—	ends with the session

The guiding principle is the same for all four on-disk caches: cache only what is reproducible, and key it on everything that could change the result. LLM answers are cached because temperature 0 makes them reproducible; downloaded files, repository listings, and zip peeks are cached because each is a faithful copy of something that rarely changes once a repository is public. In every case, changing the thing that matters — a prompt, a model, a repository — produces a fresh result automatically, so the caches speed you up without ever giving you a stale answer.