This module has not been validated: its checks have unknown error rates, so evaluate its output carefully.
24.1 The basics
A paper’s data is usually shared as a set of files in an online repository or a folder — spreadsheets, CSV files, maybe some code. data_check looks at all of those files, works out which ones actually contain data, reads the ones it can, and describes every column: what it is called, how it is stored, and whether anything about it looks wrong (a column that is entirely empty, a category typed two different ways, and so on). It does several related jobs at once rather than one single check, so think of it as a first inventory of a paper’s data, not a one-line verdict.
Here the module is run on a small folder with one data file (study.csv) and one code file, built just for this example. test_paper() gives a placeholder paper with no online repository, and local_only = TRUE tells the module to look only at this local folder:
#>
#> - We classified 1 file: 1 data.
#> - We found 1 tabular data file and extracted 4 columns from 1 of them.
#> - We grouped the files into 1 study group (ex1). We ran automated data-quality checks on 4 columns and found no issues. We detected demographic column for Age.
Running a module always works the same way: module_run(paper, "module_name", ...), here with the two extra arguments that point it at a local folder instead of an online repository. Assigning the result to a variable (mo, short for “module output”) lets you look at specific pieces of it instead of only reading the printed summary. The most central piece for data_check is mo$table, which has one row per data column:
A green traffic light here means the files could be read and no data-quality problems were flagged — it does not mean the data itself is correct, complete, or collected the way the paper describes. data_check can tell you that a column called age holds numbers between 18 and 65, but it cannot tell you whether those ages were recorded accurately. Classifying a file as “data” also does not guarantee the file is the right data for the reported analysis, only that it looks like a dataset. This module does much more than this simple example shows — file classification, study grouping across multiple experiments, Qualtrics handling, and more — covered in the rest of this chapter; the automated data-quality checks it runs on each column have their own chapter, Data Quality Checks.
24.2 What it checks
The data_check module is the foundation for all of Metacheck’s data-oriented checks. It takes the files shared with a paper — from a linked repository or a local folder — and does three things:
Classifies every file into a semantic type: tabular data, code, documentation (README, codebook, supplemental material, licence), materials (images/audio/video/stimuli, software), output, or unknown. Within documentation, a separate doc_role column distinguishes a README from a codebook from other supplemental material.
Reads each readable tabular data file and describes every column along several independent facets — how it is stored, its measurement level, what it measures, how it functions, and its unit — together with basic summary statistics.
Runs data-quality checks on those columns and on the raw workbook of every spreadsheet, and checks whether any .csv file is separated by spaces instead of commas, described in Data Quality Checks. These were formerly two separate modules, data_validate and excel_check; both are now part of data_check.
Makes the extracted columns and read data frames available to the other data modules — codebook_check builds on data_check’s output rather than re-reading the files, psychds_check uses its classification and study grouping to compare the repository with the Psych-DS layout, and convert_psychds() (not yet available) is designed to use them to build a Psych-DS archive.
On its own, data_check gives you a machine-readable inventory of what was shared and what each data column contains. It is also the first module in the data pipeline: run it once, and the downstream modules reuse its results.
Note
This chapter runs entirely offline. We build small example repositories in a temporary folder and point data_check at them with local_path. The same calls work against a real downloaded repository — see the Local Files chapter. If you have no manuscript at all, report_repository("path/to/folder") runs data_check together with repo_check, code_check, and codebook_check on the folder in one step — see Getting Started.
Run Metacheck on a Repository: choose a folder or a .zip file, or paste a link to an online repository. The website runs repo_check, code_check, and data_check on it, and can also run the R and Python code (Reproducibility Check).
Run Metacheck on a Paper: tick the module under Checks to run it on the repositories the paper links to, or tick Also run the code in the paper’s repository.
A folder or .zip file you choose is uploaded to the server. To keep it on your own computer, use the Docker installation, which offers the same website. The website runs the modules with their default settings; to change them, run them in R as described in this chapter.
24.3 A worked example
We create a tiny study folder with one data file and a code file:
Now run the module. test_paper() supplies a minimal paper object with no repository links, and local_only = TRUE means only the local folder is inspected.
#>
#> - We classified 2 files: 1 data, 1 code.
#> - We found 1 tabular data file and extracted 6 columns from 1 of them.
#> - We grouped the files into 1 study group (ex1). We ran automated data-quality checks on 6 columns and found no issues. We detected demographic column for Age.
24.4 File classification
The first thing data_check produces is a classification of every file it found, available in the structure element:
a README, a codebook (including a Qualtrics .qsf survey definition), a licence file, and plain supplemental text (preprints, slide decks, Word docs, and preregistrations)
materials
anything a participant or experimenter interacts with directly rather than an analytic artifact: images, audio, video, fonts (stimuli) and executables, installers, and config files (runnable software)
output
rendered results (.spv, figures)
unknown
anything no rule places (an LLM may reclassify these); an unopenable archive also falls back here
documentation is deliberately coarse: it answers “is this file documentation of some kind,” which is what drives Psych-DS folder placement and report grouping. A separate, orthogonal column, doc_role, answers the finer question of which documentation artifact a file is — "readme", "license", "codebook", or "supplemental" — and is what codebook_check uses to decide which files to parse.
Files that software creates by itself, and that hold nothing a reader can use, are left out of the file list, so they are never classified or read: macOS’s __MACOSX/ folder and ._ files, .DS_Store, Windows’s Thumbs.db and desktop.ini, RStudio’s .Rproj.user/ folder, Jupyter’s .ipynb_checkpoints/ folder, compiled Python files (__pycache__/, .pyc), the lock files Office and LibreOffice leave while a document is open, and a few similar files. repo_check, which supplies data_check’s list of files, removes them, and data_check removes them again from the contents of any archive it unpacks. The Repository Check chapter describes these files in full.
24.4.1 How the classifier works: data_classify_files()
Classification is done by the exported helper data_classify_files(), which layers several rule sets in order. Later layers only fire when nothing earlier has already decided a file’s type:
Format rules (file_category()’s hard rules, translated into this vocabulary): a file whose format unambiguously says what it is — a README, a codebook, or a fixed extension such as .qsf (a Qualtrics survey definition, itself a codebook), .npy/.h5/.parquet/.feather (data), or .iqdat/.edat/.edat2 (trial-level task exports, data) — is locked in here and nothing downstream can override it.
Georeferenced rasters: an image (.tif, .jpg, .png, .jp2) with a matching world file or .aux.xml sidecar is map data, not a photograph or stimulus.
JSON/XML name rules: since .json/.xml can hold anything, these are classified by name alone — dataset_description.json, .zenodo.json, datacite.xml, and similar are documentation; package.json/package-lock.json/tsconfig.json are code.
Category words in the file or folder name: an explicit word like “codebook”, “materials”, “pre-reg”, or “scripts” in the path is a more deliberate signal than a generic extension, and is checked name-first, then folder, then the folders above it.
The coarse file_types fallback (.file_type_crosswalk): whatever is still unresolved falls back to metacheck’s general file-type table (e.g. image → materials, text → documentation).
A ro-crate-metadata.json is always documentation (with doc_role == "readme", since it is collection-level like a README), whatever any earlier rule said.
For files classified as data, data_check records a second field, data_format, which is "tabular" or "raw". This is decided by the exported helper data_format() against a list of raw extensions (EEG/physiological, neuroimaging, motion capture, array formats, eye-tracking, audio/video, and generic binaries/documents). Only tabular files are read for column extraction; raw data files are classified and listed but never parsed as a table. Unknown extensions fall back to raw, since Metacheck has no reader for an unrecognised format.
To make the layout of a repository easy to see at a glance, data_check‘s report includes a data tree — the folder structure of each repository drawn as an indented tree, under the heading “Data Tree”, one per repository when several are linked. Each file is followed by how it was classified, the study it belongs to, and any file-naming problem repo_check found. The tree shows everything in a repository: a local folder is listed itself, with hidden files, files that do not belong in a repository (see Repository Check) and empty folders, and a repository link shows those unwanted files too. Unwanted files are marked “junk: remove” or “junk: keep only on purpose”, a folder such as .git/, __pycache__/ or __MACOSX/ is one line with the number of files inside, other hidden files are marked “hidden file”, and empty folders “empty folder”. Folders, empty ones included, are listed before files, and both are sorted alphabetically without regard to capitals, so the same repository always produces the same tree. A local folder is named by its own name (“Repository: my_study”), not by its full path on the computer that ran the check. The tree sits in a box with its own scroll bar, so a large repository does not make the report endless. The repository apps’ brief report shows the same tree.
tree_lines<-grep("^[│├└ ]", unlist(strsplit(mo$report, "\n")), value =TRUE)cat(tree_lines, sep ="\n")
On a real repository this is often the quickest way to spot problems: data scattered across many ad-hoc folders, deeply nested paths, or files sitting loose at the top level. This same tree is the starting point for the psychds_check module, which compares a repository’s actual tree against the tree the Psych-DS standard says it should have.
24.6 Column facets
For every readable tabular file, data_check reads the data and describes each column. Earlier versions gave each column a single col_type value — but that conflated things that are not really alternatives: how a value is stored, what level it is measured on, and what it actually measures. Following the DDI metadata standard, data_check now describes each column with several orthogonal facets, computed by the exported helper data_col_facets():
how it functions in the dataset (DDI VariableRole)
unit
seconds, milliseconds, years, …
the unit of measure (DDI UnitType)
quality
ok, empty, constant
a data-quality state, kept separate from type
parse_note
NA, comma_decimal, mostly_numeric
a representation quirk (was a fake col_type)
Because the facets are independent, an age column is numericandratioand the age concept and measured in years — each fact recorded in its own field rather than crammed into one label. A participant_id is text with the identifier role and the id concept; a Likert item is numeric, ordinal, with the likert concept.
24.6.1 How facets are derived
data_col_facets() builds on two lower-level pieces:
data_col_type() — the rules-only “primitive”. It classifies a column in a fixed order: all-NA → empty; an ID name pattern → id; one unique value → constant; two unique → binary; ≥70% date-parseable → date; median string length > 40 → text; numeric with non-integers or >20 unique → continuous; an ambiguous integer (3–20 unique) is flagged ambiguous = TRUE for the LLM; comma-decimal text ("3,14") → continuous_comma_decimal / continuous_outliers_excluded. It also returns numeric_values (a clean numeric vector for statistics) and is_numeric. This primitive is deliberately kept internal so its battle-tested edge cases survive.
data_col_concept() — the content classifier for the concept facet, using name-and-value agreement: a column is only tagged (say) reaction_time when its name looks like a reaction time (.concept_is_rt()) and its values are plausible non-negative durations. It detects reaction_time, accuracy (.concept_is_accuracy() — a 0/1 or true/false column named acc/correct/hit/error), the three demographic concepts via data_check_demographic(), timestamp (.concept_is_timestamp() — a datetime name whose values carry an HH:MM component), and condition (.concept_is_condition() — a name-driven condition/group/treatment/arm).
data_col_facets() then untangles the primitive’s col_type into representation + measurement_level (via .coltype_to_facets()), assigns the role (an id column is an identifier, a datetime is a timestamp, a condition concept is a condition, everything else is a measure), fills structural concepts (id, date, timestamp) that follow from other facets, and detects a likert concept from an ordinal-looking ambiguous integer. It seeds a unit for the two concepts that imply one: reaction_time becomes milliseconds when the median value is ≥100 else seconds, and age becomes years.
# RT: representation "numeric", concept "reaction_time", unit "milliseconds"
# subject_id: representation "text", role "identifier", concept "id"
Concepts detected by rules keep the whole rule-based path fully offline and deterministic. When you enable an LLM (see below), it fills in the concept and measurement_level for columns the rules left blank — for example a reaction time hidden behind a cryptic name like q3.
Note
The old single col_type field has been replaced by these facets. If you have code that read mo$table$col_type, read the relevant facet instead — most often representation (numeric vs text) or concept.
The three demographic concepts — age, gender/sex, race/ethnicity — are detected by the exported helper data_check_demographic(), which is the single rule reused by data_col_concept() (here), by the facet model, and by the demographic inventory. It requires the column name and values to agree: an age column is only tagged when its name looks like age and its values fall in a plausible human range, so a condition column coded 1/2 is never mistaken for gender.
Alongside the facets, data_col_stats() computes a one-row block of statistics for each column: n, n_missing, n_unique, mean, sd, se, median, min, max, range, p25, p75, iqr, skewness, and kurtosis. Non-numeric or empty columns return n/n_missing/n_unique with the numeric statistics as NA. These feed the report’s Descriptives Overview (one table per source file) and are carried into the Psych-DS variableMeasured block. The data-quality checks do not use them; they read the values themselves.
24.8 Study groups
A single repository often bundles several studies (Experiment 1, Study 2a, a pilot). data_check assigns every analysable file a study group (ex1, ex2a, pilot1, …) so a multi-study repository can be split into self-contained Psych-DS datasets (by convert_psychds(), not yet available). This is done by data_group_llm(), but — despite the name — it is deterministic wherever the evidence allows, and only falls back to the model for files nothing else can place. Four passes run in order, each more specific overriding the last:
By source repository (.data_group_from_repo()): a paper that links several repositories with different files is multi-study, one study per repository. Repositories whose file sets overlap ≥90% (Jaccard) are treated as mirrors of one study, not two.
By path (.data_group_from_path()): a filename or folder that names its study outright — Experiment 1/, study2a_data.csv, even smashed-together names like …experiment1creplication… — is grouped by a regex that reads such names more reliably than a small LLM.
By code reference (.data_code_refs()): a script names the data it reads and writes (read_csv("raw/x.csv")), which is hard evidence that the script and those files belong to the same study. This rescues data files whose own path only names a processing stage (raw/, processed/).
By LLM (only when llm_use(TRUE)): the last resort, seeing only files the deterministic passes could not place. When those placed everything — the common multi-repo case — no call is made at all.
Two guards make the result trustworthy. data_study_roster() reads the manuscript for the studies it names (“Experiment 1, 2a, 3”) — the authoritative list — and relabels slot-named groups to the authors’ labels when the counts match; .data_group_check_roster() reports any disagreement rather than guessing. And every file belongs to exactly one study: there is no shared group. A file that no pass placed falls back to its repository’s study, then to the sole study when there is only one, then to ex1 when no study was found at all, and otherwise to the first study. Only collection-level files, such as a top-level README, ro-crate-metadata.json or LICENSE, stay ungrouped. When no evidence anywhere names a study, every file ends up in ex1, and psychds_check notes that subgrouping could not be detected.
24.9 Reading files: format tolerance
data_check reads tabular files with data_read_head() (n_rows = Inf reads the whole file), which is deliberately tolerant of the messy encodings and formats research data actually ships in:
Delimited text (.csv/.tsv/.txt/.dat): a .tsv is read as tab-separated; for the other files .sniff_delimiter() counts the commas, semicolons, tabs and vertical bars (|) in the first line that is neither blank nor a comment (starting with #) and uses the most frequent one, or a comma when none is present. A space is never used as the separator, so a .csv separated by spaces reads as one column of text (data_check reports such files; see Data Quality Checks). Header presence is auto-detected (.detect_header() — a file whose first two rows are both all-numeric is treated as headerless). Reading uses data.table::fread when available (orders of magnitude faster on awkward quoted fields) and falls back to read.delim.
A “single big field” blob (a JSON/XML document dumped under one header) is detected from the first two lines (.is_single_field_blob()) and skipped — it is not a table, and parsing it is pathologically slow.
Spreadsheets (.xlsx/.xls via readxl, and .ods when the readODS package is installed): every sheet that holds a table is read, not only the first. A sheet named README, notes, legend, info, metadata, about, description, codebook or dictionary, or laid out as a codebook, is not taken for data when the file has another table. SPSS/Stata/SAS (.sav/.dta/.sas7bdat) via haven (preserving value labels and declared missing codes for codebook_check); JASP (.jasp) and jamovi (.omv) return a labelled data frame like a .sav.
R objects: .rds yields its data frame; an .RData/.rda workspace is restored in an isolated subprocess (.read_rdata_isolated()) because loading a saved model that references an uninstalled package can print C-level diagnostics or crash. A workspace that holds no reusable data frame (only fitted models / session state) returns NULL and is reported as a data-sharing recommendation. An .rds file that was made with save() instead of saveRDS() (so it is really a workspace, which readRDS() refuses) is opened with the workspace reader, with an “R workspace named .rds” warning that makes the light at least yellow.
Columns that need an R package: an .rds or .rda file whose columns hold values of a package’s own type, such as lubridate’s periods and durations, cannot be used without that package. Each such column is read as its plain stored numbers, the package is named in the column’s needs_package value in $table, and a “Needs an R package” warning tells the authors that anyone without that package cannot open the file.
Old Excel .xls files are first opened in a separate R process, because some of them (for example an old Mac Excel file) make readxl stop the whole R process. Such a file is reported as unreadable and the rest of the check goes on.
UTF-16 text files, such as E-Prime and other Windows exports, are read in that encoding.
Invalid UTF-8 bytes (a Latin-1/Windows-1252 export) are re-interpreted as Latin-1 rather than failing (.utf8_repair_df()), and the per-column repair count is recorded in a utf8_repaired attribute so the mixed encoding can be reported.
Two kinds of files that once stopped the whole check are now read. An .rds or .RData file with a labelled text column made by the haven package, such as PISA’s three-digit country codes, needs haven to be turned into text; readRDS() and load() do not load it, so the reader now loads haven when a column is labelled. A .tar archive made on a Mac holds binary records of each file’s extended attributes; these records are skipped, and only each file’s path and size are read.
Qualtrics exports get special handling on read (see below).
24.10 What the module holds back or reclassifies after reading
Several file types read as a data frame but are not straightforward datasets. data_check handles each so they are neither silently dropped nor wrongly treated as data:
File manifests (data_is_manifest()): a “table of contents” CSV whose cells name other files in the repository. Detected by content (≥80% of a column’s values resolve to repo files, across ≥2 distinct extensions, so a genuine stimulus column of images does not trip it), then demoted to supplemental.
R workspaces with no reusable data: reported with advice to share the underlying data as CSV + codebook.
Coding worksheets (.tabular_usable()): a human-formatted worksheet (mostly free-text annotation columns, mostly empty cells) reads as a data frame but is not a rectangular dataset. Detected from the already-computed facets using a prose fraction and a missingness fraction with tiered rules (overwhelmingly free text, or moderately free-text and mostly empty). Columns are not extracted and nothing is sent to the LLM, but the file stays classified as data so the spreadsheet-formatting checks (formerly the separate excel_check module) still inspect it. Crucially, high missingness alone never excludes — a branched/planned-missing survey is 90%+ missing but is real numeric data.
.txt reclassification (txt_classify_content()): a .txt is ambiguous by name (an E-Prime export, a task log, a codebook, or prose all ship as .txt), so all .txt files are fetched under download = "data" and reclassified from their content once on disk. Only ever an upgrade to data (an E-Prime header block, or a delimited table with a real header row); an unrecognised .txt keeps whatever its name implied, so prose is never mistaken for data. A name-based README/codebook verdict is authoritative and never overridden.
Trial-level files (.bh_is_trial_level_file()): E-Prime, Inquisit, jsPsych, and native Behaverse files publish one file per participant per block. Treating each as its own dataset would produce hundreds of fragmented “datasets” for one instrument, so they are held out of the tabular extractor, recorded, and are designed to be later merged per instrument into paradata/<instrument>.json by convert_psychds() (not yet available), following the Behaverse Data Modeltrial standard (a source-agnostic, tidy, trial-level schema). Nothing is deleted. To recognise an Inquisit, jsPsych or Behaverse file, data_check reads only the file’s first line (the column names), so the check stays fast even on a table with thousands of columns; an E-Prime export is recognised from the header block in its first lines.
24.11 Qualtrics survey exports
Qualtrics CSV/TSV exports have a fixed, recognisable shape: a set of reserved response-metadata columns (StartDate, Duration (in seconds), Finished, ResponseId, …) and, for the “use choice text” export, extra header rows (question text and an ImportId JSON row) as the first data rows, which force every column to text. data_check detects these files with data_check_is_qualtrics() (matching ≥4 reserved metadata names and/or the ImportId row), strips the extra header rows with data_strip_qualtrics_header() so the columns type correctly, and tags each metadata column with its semantic role via .qualtrics_tag_cols(). The reserved-name → tag map is the schema .qualtrics_meta_cols (startdate → qualtrics_start, durationinseconds → qualtrics_duration, finished → qualtrics_finished, …), matched case-insensitively after stripping non-alphanumerics so Duration (in seconds) and the R-mangled Duration..in.seconds. both hit. The substantive question columns are deliberately not interpreted here — that is the job of the Qualtrics metadata summary and the scale-block detection in codebook_check.
24.12 Keys in the repository
A repository sometimes holds a key: an API key, an access token or a private key. Anyone who downloads the files can then use a paid service at the authors’ cost. A preprint’s OSF project, for example, held a key for a language model service in the survey flow of two Qualtrics surveys (.qsf files), as embedded data, and the Qualtrics export also wrote that key into every response of the data files. data_check therefore searches the files for keys:
The files the authors wrote (code, settings files such as .env, survey definitions, documentation, outputs) are searched whole. Of a local folder, the hidden files are searched too.
Of a tabular data file, whose values come from participants, only the column names and the first 3 rows of the table as read are searched. These hold a key that was set for a whole survey.
Files over 25 MB and binary files are not searched, and neither is encoded data (an unbroken run of 1,000 or more base64 characters, such as an image inside a knitted HTML page).
The search uses the rules of high and medium confidence of the secret scanner betterleaks, with its filters against placeholders and examples. A rule made for some files only, such as a Terraform password for .tf files, is tried on those files only.
A key found turns the traffic light red. The report gives, for each key, the files and lines (or, for a data table, the rows) it is on, its type, and only its first characters, and says what to do: revoke the key with its provider, make a new one, and then remove the key from the files. Deleting the file or the line alone does not remove the key, because GitHub keeps every earlier version of a repository, and the OSF keeps earlier versions of a file. The same information is in the $keys table, and key_n in $summary_table counts the different keys found for each paper. A key is never tested with its provider. Every module’s result, not only that of data_check, has any key it holds replaced by “[key removed]” (see Creating Modules), so Metacheck never copies a key into its own reports, tables or saved results.
24.13 What you get back
Element
What it contains
$traffic_light
the worse of two separately computed lights: whether files were actually read ("green"/"yellow" when some tabular files had no local copy/"na" when there is nothing to check), and the data-quality checks on the columns that were read ("green" with no issues, "yellow" below 25% of columns flagged — or any careless-responding, spreadsheet-formatting or space-separated .csv finding at all — "red" at 25% or more). A clean extraction never hides a real data-quality problem, and a quality problem on the columns that loaded never gets hidden by an otherwise-successful read. A key found in the files (see Keys in the repository) always makes the light red
$summary_text
plain-text bulleted summary: keys found, files classified, columns extracted, study groups, and any held-back files
$summary_table
per-paper counts (data_file_n, column_n, empty_col_n, key_n, plus wide per-representation counts)
$keys
one row per place a key was found: file, line (or row for a data table, 0 for its column names), key (a number for each different key), rule (the type of key) and shown (its first characters)
$table
one row per column, with the facets, sample_values, utf8_repaired, needs_package, and summary statistics
$structure
one row per file, with data_type, data_format, group, tabular_usable / non_tabular_reason, and source_file (see below)
$previews
the full data frames that were read
$gated_repos
repositories found but not listable, taken from repo_check (for example a private OSF component, an OSF link during an outage, or a GitHub repository too large for GitHub to list)
$manifest_path
path to a written JSON manifest, when one was requested
Files with the same name in different folders (for example a log.csv in each of 22 scan folders) are all read and checked. Each file has a source_file name that is unique within the run: the file name when no other file has it, so nothing changes for most repositories; otherwise its path in the repository (scan1/log.csv); and where even that repeats, the repository’s name in front. A spreadsheet with several tables has a source_file for each sheet, such as harvest.xlsx [data_sheet], so columns with the same name on two sheets stay apart, and the sheet column names the sheet; a spreadsheet with one table keeps its own name. The tables in $previews, the column table, the findings and codebook_check’s labels use this name.
The findings of the data-quality checks are in $findings, described in Data Quality Checks. When no file could be read as a table (a repository with only code, documents or images, or with data files that are not usable tables), data_check still returns its summary, report and traffic light; $table then has no rows, and any spreadsheet-formatting or space-separated .csv findings are in $findings.
The $structure and $previews elements are what the downstream modules consume, which is why you rarely need to run data_check on its own beforehand — running codebook_check or psychds_check triggers it automatically and reuses the result, and convert_psychds() (not yet available) is designed to do the same.
24.14 Downloading files
When a paper links to an online repository, data_check downloads the files it needs. Within one R session the files are shared with the other modules, so a file is fetched only once; with cache = TRUE they are also kept for later sessions (see Caching downloads below and Caching and Reuse). The download argument controls what is fetched:
download = "data" (the default) fetches only the machine-readable files the checks analyse — tabular data plus codebook/README files, and every .txt (which is reclassified from content afterwards).
download = "all" fetches every file in the repository (code, materials, PDFs, assets, …). This is the right choice when building a complete data archive.
download = "none" (or FALSE) downloads nothing — files are only classified by name. TRUE is accepted as a synonym for "data".
# default: fetch only the readable data + codebook/README filesmodule_run(paper, "data_check")# fetch everything, for building a full data archivemodule_run(paper, "data_check", download ="all")
24.14.1 Caching downloads
By default (cache = FALSE) downloads go to a temporary directory discarded when the R session ends, so nothing accumulates on disk. Pass cache = TRUE to keep files in a persistent on-disk cache (repo_cache_dir()) reused on later runs — the right choice when repeatedly checking the same repositories or building an archive across sessions. Clear it with repo_cache_clear().
24.14.2 Size caps
Two caps bound how much is downloaded per repository:
max_file_size (default 100 MB): a file larger than this is skipped; the rest of the repository is still downloaded.
max_download_size (default 500 MB): a budget for the repository’s total. Files already in the cache count towards it, and the smallest missing files are added until the next one would exceed the budget, so a larger repository is downloaded in part, and the same files every run.
Set either to Inf for no cap. A file whose size neither the repository’s listing nor the host gives is downloaded with a limit: max_file_size, or what is left of the repository’s max_download_size budget when that is less. The transfer stops once the limit is passed, and the file is then reported as too large, like a file whose listed size is over the limit. Every skip is reported in the summary text, with the argument to raise, and recorded in the manifest (see below), so you can raise the relevant cap and re-run.
Some hosts sit behind an Amazon web firewall that, after many requests in a short time, answers with a check meant for a web browser instead of the file, so the file’s size cannot be read and it cannot be downloaded. Metacheck then gives a warning, once per host in each R session, naming the host and saying to try again later. Such files used to be left out without any message.
These caps govern downloading, and they apply to every platform alike. Listing a repository is never capped: since listing transfers no file contents, GitHub repositories are now listed in full whatever their size, as described in the repo_check chapter.
24.14.3 Skipping asset types, and peeking inside zips
When building an archive you often want the data but not the stimuli or media — those are usually better linked to than mirrored. Two options handle this:
skip_types — a vector of data_types never to download, even under download = "all". Most usefully skip_types = "materials" leaves out stimuli/media and other non-analytic files. When left at its default (NULL), data_check behaves as if you had passed c("materials", "unknown"), so some filtering always happens unless you explicitly pass skip_types = character(0). Skipped files are still listed (with the reason) in the manifest.
peek_zips = TRUE (the default) — before downloading a .zip, look inside it via an HTTP range request (fetching only the zip’s tail, not the whole file — zip_peek() / zip_decision()) and only download zips that actually contain data or a codebook. Downloaded data-zips are unpacked (.expand_zip()) and their inner data files added to the archive, with the zip itself demoted to a container; inner materials are dropped. Only an archive Metacheck downloaded itself is unpacked next to itself; an archive in a folder of your own is unpacked into R’s temporary folder, so data_check never writes into your folder. With cache = TRUE, a file that was taken out of a zip or tar archive before is reused instead of being unpacked or downloaded again.
Pass manifest = "some/dir" to write a per-paper JSON manifest (.data_check_write_manifest()) listing every repository file with its name and path, its repository and file URL, its size, its data type and format, and whether it was downloaded (and if not, why). It records this provenance in a self-describing block whose field names map onto DDI-Codebook 2.5 elements (provenance$ddi_mapping), the metacheck/R version and platform, and — when the LLM was used — the model. Files not downloaded are split into intentional (a policy decision — download mode, skip_types, the size caps — where re-running changes nothing) and unintentional (wanted but not fetched: a transient failure or a missing URL — the re-run signal, flagged rerun_recommended). For a file inside an archive that was not taken out, the manifest gives the real reason, such as a size over max_file_size or the error met while reading the archive (“unsupported compression method (9)”). A file a Dataverse depositor restricted is not requested and is listed as skipped, “access restricted by the depositor”. Unsized files are resolved with a lightweight HEAD request so the manifest carries real sizes — useful for auditing a corpus or choosing size caps before a large archive build.
With llm_use(TRUE), data_check adds things the rules cannot do reliably. (On Metacheck’s website and in the Docker installation, only the power check uses a language model, so this only happens when you run data_check in R.) The LLM path is layered on top of the rules — it only fills gaps, never overrides a confident rule:
File types for files the rules left unknown (.llm_classify_batched() classifies into documentation/materials/output/unknown).
Concepts for cryptically-named columns — the rules only tag a concept when the name gives it away, so a reaction time called q3 stays blank; the LLM fills it in.
Refined measurement levels for ambiguous integer columns (nominal vs ordinal vs ratio).
Study groups for files no deterministic pass could place (see above).
All LLM classification runs through .llm_classify_batched(), which sends numbered listings and expects index-mapped {index, value} results back, so a dropped or reordered answer never misaligns the others. A header-signature dedup means that in a repository of pp1.csv … pp30.csv (all sharing a schema), an ambiguous column is classified once and broadcast to the identical-header files, turning 30 repeated questions into one.
llm_use(TRUE)llm_model("groq/openai/gpt-oss-20b")mo<-module_run(test_paper(), "data_check", local_path =repo, local_only =TRUE)mo$table$concept# filled in for cryptic namesmo$structure$group# study grouping
See the LLMs chapter for how to configure a model.
24.16 Building a Psych-DS archive: convert_psychds()
Note
convert_psychds() is not yet available in the released version or in dev. This section describes what data_check’s output is for. For a folder on your own computer, FAIR_repo() already writes a copy in the Psych-DS layout; see Checking Local Files.
data_check’s classification, facets, study groups, and read data frames are what make it possible to rebuild a repository as a standard-compliant dataset. That will be the job of convert_psychds(), which is planned to write the Psych-DS, OpenScales OSD, and Behaverse schemas. As planned: run data_check with download = "all" (so every file is on disk), then convert_psychds(paper) copies each file to its standard Psych-DS location, generates a dataset_description.json (with variableMeasured built from these facets and the codebook labels), splits multi-study repositories into study-<group>/ datasets under a collection.json, writes identified scales as scales/<code>.osd files, and normalises trial-level data into paradata/<instrument>.json.
24.17 Related
Data Quality Checks describes the automated data-quality checks that data_check runs on the columns it extracts, including spreadsheet-formatting checks (colour coding, merged cells, empty rows/columns) for .xlsx/.xls/.ods/.fods files.
Codebook Check checks whether the columns are documented.