Many papers share their data, code, or materials in an online repository — a separate web page, hosted on a site like OSF or GitHub, that holds a folder of files. repo_check finds the links to these repositories in a paper and looks at what is actually inside them.
Here is the module running on a demo paper (this needs an internet connection, because the module has to visit the repository online to see what it contains):
#>
#> - We could not check 2 repositories: https://osf.io/5tbm9 (private); https://github.com/Lak- (links to a GitHub account, not to a specific repository).
#> - We found 73 files in 5 repositories.
#> - We found 6 README files and 2 repositories without READMEs.
#> - We could not classify 1 file.
#> - We found 7 file naming problems to fix.
#> - We left out 5 files that do not belong in a repository: operating system (5). Remove them from the repository.
Running a module always works the same way: module_run(paper, "module_name"). Saving the result to a variable, here called mo, lets you look at specific pieces of it instead of only the printed summary. mo$table is a table — rows and columns, like a spreadsheet — listing every file found across all the repositories the paper links to:
The traffic light here is about file organisation, not about the data or code itself. A green light means every linked repository has a README (a short text file explaining what is in the folder), no files are hidden inside a zipped archive, and every file has a recognisable name. A yellow light means something about that organisation could be improved — for example a missing README or an archive file — which is worth a look but is not necessarily a sign anything is wrong with the actual research. Always check mo$summary_text to see exactly what was flagged before deciding whether it matters.
22.2 What it checks
The repo_check module finds links to research repositories in a paper, retrieves the list of files in each, and reports what is shared. It is the foundation for every other module that reads repository files: code_check and data_check both build on the file list repo_check produces. See Repositories it searches below for the full list of platforms it recognises.
Beyond listing files, repo_check runs a set of checks on what it finds:
whether each repository has a README;
whether files are bundled into archives rather than shared individually;
whether any files are unclassifiable by name or extension;
whether the studies the manuscript names match how the repository’s files are grouped;
whether proprietary E-Prime binary files (.edat/.emrg) have a matching plain-text export;
whether any files sit behind restricted or embargoed access;
whether an OSF registration points at a closed or inaccessible source project; and
whether the repository holds files that do not belong in it, such as the __MACOSX folder a Mac adds to a zip (see below).
The most recently added platforms are Mendeley Data, DataONE (Arctic Data Center, KNB, and other member repositories) and LDbase; the table below lists every platform.
Note
This module makes live network calls to the linked repositories. You need an internet connection to run the code below.
Run Metacheck on a Repository: choose a folder or a .zip file, or paste a link to an online repository. The website runs repo_check, code_check, and data_check on it, and can also run the R and Python code (Reproducibility Check).
Run Metacheck on a Paper: tick the module under Checks to run it on the repositories the paper links to, or tick Also run the code in the paper’s repository.
A folder or .zip file you choose is uploaded to the server. To keep it on your own computer, use the Docker installation, which offers the same website. The website runs the modules with their default settings; to change them, run them in R as described in this chapter.
#>
#> - We could not check 2 repositories: https://osf.io/5tbm9 (private); https://github.com/Lak- (links to a GitHub account, not to a specific repository).
#> - We found 73 files in 5 repositories.
#> - We found 6 README files and 2 repositories without READMEs.
#> - We could not classify 1 file.
#> - We found 7 file naming problems to fix.
#> - We left out 5 files that do not belong in a repository: operating system (5). Remove them from the repository.
The table lists every file found across all linked repositories:
repo_check looks for links to the following repository platforms, each via its own link-finding helper and its own file-listing call. Some are a single hosted service (Zenodo, ResearchBox); others are open-source software that many independent organisations each run on their own address, in which case repo_check recognises any address on a maintained list rather than one fixed host — see each row’s note for how many addresses that currently covers.
Platform
Link finder
How files are listed
Coverage
OSF
osf_links()
recursive osf_info(), including registrations (see below)
the Open Science Framework (single service)
GitHub
github_links()
github_tree_files() — one repo-metadata + one full-tree API call, listing the repository in full
github.com (single service)
GitLab
gitlab_links()
gitlab_tree_files()
gitlab.com (single service)
ResearchBox
rbox_links()
rbox_file_download() (downloads and unzips the box)
researchbox.org (single service)
DSpace (legacy)
dspace_links()
psycharchives_file_download() (DSpace’s older REST API; lists without downloading)
PsychArchives and 5 other installations still running this API version
DSpace 7
dspace7_links()
dspace7_file_download() (DSpace 7’s newer REST API; lists without downloading)
66 installations, including Cornell’s eCommons, Georgia Tech’s Digital Repository, and ETH Zürich’s Research Collection
Zenodo
zenodo_links()
zenodo_info()
zenodo.org (single service)
Dataverse
dataverse_links()
dataverse_info() / dataverse_file_download()
121 independent installations, including Harvard Dataverse, DataverseNL, and the DANS Data Stations
Figshare
figshare_links()
figshare_info() / figshare_file_download()
figshare.com (any branded subdomain) plus 24 addresses on institutions’ own domains that use the same service, e.g. the University of Leicester and DTU Data
data.4tu.nl (single service; runs Djehuty, a documented copy of Figshare’s own API)
Dryad
dryad_links()
dryad_info()
datadryad.org (single service)
ReShare
reshare_links()
reshare_info()
UK Data Service’s self-deposit repository (single service)
Mendeley Data
mendeley_links()
mendeley_info()
data.mendeley.com (single service)
DataONE
dataone_links()
dataone_info()
a federation of independently-run repositories; 5 member nodes recognised, including the Arctic Data Center and KNB
LDbase
ldbase_links()
ldbase_info() (reads the public web pages; see below)
ldbase.org (single service), including LDbase DOIs (prefix 10.33009)
DSpace, Dataverse, and Figshare each recognise more than one address because the same software (or, for Figshare, the same underlying service under a branded address) is run by many organisations. That list is maintained, not exhaustive — an installation not yet on it simply goes unrecognised rather than misidentified, so it is always safe to ask for one to be added.
A full stop, comma, semicolon or colon at the end of a repository address is dropped, since it usually ends the sentence the address stands in: “https://doi.org/10.5063/f1mc8x92.” is the same repository as “https://doi.org/10.5063/f1mc8x92”, and its files are listed once.
22.5.1 OSF registrations follow their source project
An OSF registration is a locked snapshot. Many “Open-Ended Registration” entries (common in older schemas) only describe what was shared, with the real files living in the project the registration was made from (registered_from). Since the manuscript usually links only the registration, repo_check follows registered_from automatically:
if the source project is public, its files are listed under the source project’s own URL (not the registration’s);
if the source project is closed or inaccessible, the report flags this explicitly, distinguishing two cases: the registration itself still holds a mirrored copy of the files (nothing is lost, but the permanent home is unreachable), or no copy exists anywhere (the files are genuinely unreachable).
22.5.2 GitHub repositories are listed in full
Every GitHub repository is listed in full, whatever its size, exactly as OSF, Zenodo, ResearchBox, DSpace and local directories have always been treated. Listing a repository is cheap — it costs two API calls and transfers no file contents — so there is nothing to configure and no size threshold to set.
There remain two reasons a GitHub repository can fail to be listed, and neither is one a setting can fix. Either the repository is invalid or inaccessible (it has been renamed, deleted, or made private), or it exceeds GitHub’s own limit of 100,000 items per tree request, in which case GitHub returns a listing it marks as truncated and metacheck reports it as too large to list rather than silently using a partial one. Repositories in either state are recorded in $gated_repos (see below) with the reason, so a report can explain why a paper produced no GitHub files instead of simply reporting none found.
Note that this concerns listing only. How much of a repository is ever downloaded is still capped, by the per-file and per-repository size limits in data_check described in the data_check chapter.
NoteA change from earlier versions
Earlier versions of repo_check took three arguments — github_gate, github_max_repo_size_mb and github_max_files — that skipped listing a repository over a size threshold. These have been removed, so code passing them will now fail. Delete them from any script that used them; no replacement is needed, because listing is no longer gated.
22.5.3 LDbase is read from its web pages
LDbase is the data repository for educational and developmental sciences at Florida State University. It organises a deposit as a project that holds datasets, code, documents, and sometimes further projects inside it. ldbase_links() finds links to LDbase projects, datasets, code and documents, and LDbase DOIs (prefix 10.33009). It does not count LDbase’s own DOI (10.33009/ldbase), which authors cite for the repository itself rather than for a deposit.
LDbase offers no API, so ldbase_info() reads the same public web pages a visitor sees, one page per project or item. For a project it lists every dataset, code file and document in the project’s hierarchy, including those in projects inside it, each in a folder named after the item; for a single dataset, code file or document it lists that item’s current file. Older versions of a dataset are not listed. No account or token is needed, and the files download without one.
LDbase’s firewall answers a quick series of requests with a check meant for a web browser instead of the page, so metacheck requests LDbase pages at most once per second. A link that still cannot be listed is kept, with the reason, in $gated_repos.
A repository that is linked but cannot be retrieved is kept, as a repository that could not be checked, and is never counted as an empty repository or as one without a README. Its reason is recorded in $gated_repos. For the OSF, the reasons say whether the OSF did not answer (HTTP 503, for example during an outage), the project was not found (HTTP 404), was deleted or withdrawn (HTTP 410), or the rate limit was reached (HTTP 429). A failed OSF lookup is not stored in the cache, so a later run asks again. The summary and report name each repository that could not be checked, the summary table counts them in repo_unchecked, and the traffic light is not green while any could not be checked.
22.5.5 Private OSF projects linked through a view-only link
A view-only link (an OSF address ending in ?view_only= and a key) lets anyone with the link see the files of a private OSF project. Such links are made for peer review, and a paper that keeps them after publication often links the same project twice: once plainly, and once with the view-only key. repo_check reads the files through the view-only link and checks the project once. The plain link to the same project is left out, so it is not reported as a repository that could not be checked (“private”), and not counted as a second repository. The summary and report name the project instead: “The OSF project … is private. Metacheck examined the files through the view_only link. As the preprint is public, and the OSF Files are accessible to anyone, consider making the OSF project public.” The traffic light is then not green. The summary table counts these projects in repo_view_only, and the result’s $view_only table lists, for each paper, the view-only link (repo_url) and the project it leads to (project).
Some links are recognised but lead to no single repository:
a link to a GitHub account or organisation (https://github.com/OneZoom/) instead of to one repository is reported as “links to a GitHub account, not to a specific repository”; a GitLab address with one path segment is reported the same way;
a Dryad sharing link (datadryad.org/stash/share/...), made for reviewers before publication, is followed to the dataset it leads to, and dryad_links() finds such links in the text. A sharing link that leads nowhere is reported as “private or expired Dryad sharing link”;
an OSF registration’s licence is looked up for the project it was made from, since that is where its files are listed from; a project that is closed or has no licence takes the registration’s licence.
22.6 Study grouping
repo_check runs a preliminary classification and grouping pass over every file it finds:
data_classify_files() labels each file’s data_type (data, code, documentation, materials, output, or unknown) from its name/path/extension;
.data_doc_role() further labels documentation-type files (readme, license, codebook, …);
data_group_llm() assigns each file to a study group, using deterministic path/repo-split rules first and falling back to an LLM (llm_use(TRUE)) only when those are ambiguous — see the LLMs chapter. Root-level files that are collection-wide by nature (a top-level README, ro-crate-metadata.json, or LICENSE) are excluded from grouping, since they don’t belong to any one study.
This pass is preliminary: repo_check never downloads file contents (except where a platform’s own listing call requires it, like ResearchBox), so it classifies from names and paths only. data_check reclassifies afterwards using file contents too, and its classification is authoritative for where a file is placed in a paper’s structure. repo_check’s own pass exists to power its own report — the file-naming check, the classification dropdown, and the study/repository mismatch check below.
22.6.1 Study/repository mismatch
If the manuscript explicitly names studies (e.g. “Study 1”, “Study 2”), repo_check compares that roster against the studies the files actually separate into. A mismatch — a named study with no matching files, or a file-derived group not named in the manuscript — is flagged in the report. A manuscript that never names studies explicitly has no roster to compare against, so this check is silently skipped rather than treated as a mismatch.
22.7 File naming conventions
check_file_naming() checks every file name for machine-parseability. Issues are split into two severities:
bad — a real naming problem (should be fixed);
suggestion — not required, but a good habit.
The bad rules:
spaces — the name contains a space;
special characters — a character other than a letter, digit, dot, underscore, or dash;
diacritics — a non-ASCII character (e.g. an accented letter);
unclassifiable — the file could not be classified by name or extension at all (data_type == "unknown");
path length over 255 characters — the full relative path exceeds the hard limit most filesystems and tools enforce.
The suggestion rules:
zero-padding — within a family of files sharing the same prefix/suffix around a number (e.g. data-2.csv, data-11.csv), the numbers aren’t zero-padded to a consistent width, so the files don’t sort in numeric order;
path length over 228 characters — the full relative path, accounting for a typical Downloads-folder prefix;
directory length over 100 characters — the directory portion of the path alone;
filename length over 50 characters — the base filename alone.
could not be classified by name or extension (data_type is ‘unknown’); add a recognisable keyword (data, code, materials, …) or a known extension
to_err_is_human
Study 1.r
spaces
bad
contains a space
to_err_is_human
Study 1 - csv - dataset 1.csv___CODEBOOK.csv
spaces
bad
contains a space
to_err_is_human
Study 1.csv___CODEBOOK.csv
spaces
bad
contains a space
to_err_is_human
Study 1 - csv - dataset 1.csv
spaces
bad
contains a space
to_err_is_human
22.8 Special file types
22.8.1 Proprietary E-Prime files
E-Prime’s .edat/.edat2 (and .emrg/.emrg2 merge files) are proprietary binary formats that only the E-Prime software can open — metacheck neither downloads nor reads them. The analysable data lives in E-Prime’s plain-text export instead. repo_check flags any .edat/.emrg file that has no matching .txt export alongside it (matched by file stem), so authors know to also share the readable export (File → Export in E-Prime).
22.8.2 Restricted-access files
A DSpace (legacy) item flagged restrictedAccess/embargoedAccess — most often seen on PsychArchives — hides its files behind institutional (SSO) login — they cannot be fetched programmatically. repo_check flags any repository with restricted files, since metacheck cannot examine them and other researchers cannot reuse them without requesting access.
Dataverse has two ways to hold files back. A file the depositor restricted is served only on request, so Metacheck does not ask for it, and the data manifest lists it as skipped, “access restricted by the depositor”. A file behind a guestbook, a form the depositor wants filled in before a download, is reported as such; Metacheck does not fill in the form.
22.8.3 Files Dataverse converted on upload
Dataverse converts an uploaded SPSS .sav, Stata .dta or .csv file into its own tab-separated .tab format. Metacheck lists and downloads such a file as it was uploaded, under the listed name with the original’s extension (such as Study1_June2017manuscript.sav). That is the name the authors’ code reads, the file then matches the checksum Dataverse lists, and an SPSS file keeps its value labels for data_check.
22.8.4 File names that cannot be written on every computer
A file or folder name that some operating system does not allow (with a colon, one of the characters * ? " < > |, a control character, a final dot or space, or a name such as CON or NUL) is changed in the same way on every computer wherever Metacheck saves a file, so a check does not stop on it, and the result does not depend on the computer it ran on.
22.8.5 Archives, and looking inside a .zip
An archive shared as a single file hides what is in it. By default repo_check therefore reads each .zip’s own listing and reports the files inside it, in place of the archive as one opaque row. This costs one HTTP range request per archive and downloads none of its contents: a .zip stores a directory of what it holds at the end of the file, and a server that honours range requests will return just that stretch of bytes. The helper that does this is zip_peek(), and you can call it yourself on any archive URL.
Turn it off to leave archives as single rows and make no per-archive request at all:
module_run(paper, "repo_check", peek_zips =FALSE)
A plain (uncompressed) .tar has no such directory either, but its files can be listed from one download: a .tar whose size the repository gives, up to 50 MB, is downloaded once, and its file names and sizes are read from that copy. The copy is kept for the session, so data_check takes the data and documentation files from it without downloading it again. A larger .tar, or one of unknown size, stays one row.
.7z, .rar and .tar.gz keep no equivalent directory that can be read from the end of the file, so they would have to be downloaded whole, and some formats metacheck cannot read at all. repo_check warns when an archive is in one of these formats, because that limits how discoverable and reusable its contents are. An archive whose listing cannot be read for any other reason — a server that ignores range requests, for instance — is simply left as a single row, and data_check opens it after downloading, as before.
22.9 Files that do not belong in a repository
Some files end up in a repository because software creates them by itself, not because the authors meant to share them, and they hold nothing a reader can use. repo_check leaves these files out of its file list, after looking inside any zips, so they are not counted, not classified, and not checked for naming. data_check leaves them out too, so it no longer tries to read, for example, the ._ copy of a .csv file that a Mac adds to a zip as if it were data. In one LDbase zip, such files made up 993 of the 1,826 files listed.
The files left out are of two kinds. The first kind should be removed:
Kind
Examples
operating system
macOS’s __MACOSX/ folder and the ._ copy of every file a Mac puts in a zip, .DS_Store, .TemporaryItems; Windows’s Thumbs.db and desktop.ini; recycle-bin and trash folders
RStudio session files
the .Rproj.user/ folder, which holds open tabs and unsaved edits
R history
.Rhistory, the commands typed in an R session, which can hold file paths and passwords
Jupyter checkpoints
the .ipynb_checkpoints/ folder of automatic notebook copies
compiled Python files
the __pycache__/ folder and .pyc files, which Python recreates when the code runs
open-document lock files
~$report.docx and .~lock.report.odt#, left while Word, Excel, PowerPoint or LibreOffice has a document open
editor backups
file~, .bak, and Vim’s .swp and .swo files
temporary files
.tmp and .temp files a program did not remove
The second kind should only be kept on purpose:
Kind
Examples
version control
.git/, .svn/ and .hg/ folders, which hold the project’s history, including earlier versions of files and data that was later removed
editor settings
.idea/ and .vscode/ folders, which can hold paths on the author’s own computer
The list follows GitHub’s templates for files to keep out of a repository and the rules TU/e’s data managers check before a data package is archived, and each kind was found in real repositories. Files that may still matter to someone, such as .RData files and knitr caches, are kept. When repo_check lists a local folder, local_files() includes hidden files of these kinds (such as .DS_Store, .Rhistory and .git/), so they are found there too; other hidden files stay out.
When any such files are found, the summary text says how many were left out and of which kinds, and the report adds a section, “Files that do not belong in the repository”, listing each file and explaining how to keep them out (for example, creating zips on a Mac with zip -r -X, or adding a .gitignore file based on GitHub’s templates). Files of the first kind make the traffic light red; files of the second kind make it at least yellow. They are returned in $useless_files, one row per file, with the kind and a severity of "problem" (first kind) or "suggestion" (second kind), so other modules can use them.
22.10 A clean example and one with problems
The demo paper links to several repositories, and running repo_check on it produces a red light: the repositories hold files that should be removed. Because the module visits the repositories online, its result changes when a repository changes (for example when a project is made private). The summary text reports exactly what it found when this chapter was built:
#>
#> - We could not check 2 repositories: https://osf.io/5tbm9 (private); https://github.com/Lak- (links to a GitHub account, not to a specific repository).
#> - We found 73 files in 5 repositories.
#> - We found 6 README files and 2 repositories without READMEs.
#> - We could not classify 1 file.
#> - We found 7 file naming problems to fix.
#> - We left out 5 files that do not belong in a repository: operating system (5). Remove them from the repository.
The light is red because of the files that do not belong in a repository: here the __MACOSX folder and the ._ files that a Mac adds when it creates a zip file. The summary also names the other problems, each of which alone would give a yellow light: links that could not be checked, repositories without a README, a file that could not be classified, and file names to fix. The per-paper summary table counts each of these (repo_unchecked, files_readme, files_unknown, naming_issues):
links to a GitHub account, not to a specific repository
A clean result, by contrast, would include a README in every linked repository (files_readme ≥ 1 for each), share files individually rather than as a zip (files_zip of 0), have no unclassifiable or badly-named files, agree with the manuscript’s study roster, and have no closed registrations — producing a green light with no items to address.
22.11 Options
repo_check accepts arguments for working with local files instead of (or in addition to) online repositories:
# also include files from a local foldermodule_run(paper, "repo_check", local_path ="path/to/downloaded/files")# only check a local folder, skip all online lookupsmodule_run(paper, "repo_check", local_path ="path/to/files", local_only =TRUE)
local_only = TRUE is useful when you cannot or do not want to contact external services — for example when checking a reviewer submission you downloaded as a zip. See the Local Files chapter for details.
If you have no manuscript at all — just a folder to check — report_repository("path/to/folder") wraps this same local_path/local_only = TRUE pattern (and the three modules below it) into a one-line report; see Getting Started and Creating a Report.
model and params are passed through to data_group_llm() for study grouping, and only take effect when llm_use(TRUE) and the deterministic grouping rules cannot place a file:
module_run(paper, "repo_check", model ="gpt-4o-mini", params =list(temperature =0))
22.12 The traffic light and what you get back
Light
Meaning
green
repositories were found and every one could be checked; every one has a README; no archive files, unclassifiable files, bad file names, or files that do not belong in a repository; no study/repository mismatch; and no closed OSF registrations
yellow
at least one repository could not be checked or is missing a README, an archive file was found, files could not be classified, real naming problems were found, the manuscript’s study roster disagrees with the repository’s file grouping, a registration points at a closed source project of which no copy of the files exists, or version-control folders or editor settings were found
red
the repositories hold files that do not belong in a repository and should be removed (see above)
na
no repository links were found in the paper at all
Element
What it contains
$table
one row per file across all repositories: repo_url, repo_name, file_name, file_path, file_url, file_location, file_size, file_type, data_type, doc_role, group, paper_id
$summary_table
per-paper aggregates: repo_n, repo_unchecked (repositories linked but not retrieved, so “no repository linked” and “a repository could not be checked” can be told apart), files_n, files_data, files_code, files_readme, files_zip, files_unknown, naming_issues, roster_mismatch, repo_view_only (private OSF projects checked through a view-only link)
$naming_issues
full output of check_file_naming() — one row per issue, with severity
$roster_check
the manuscript-vs-file study roster comparison (roster, missing, extra), or NULL if not applicable
$gated_repos
repositories found but not listed (e.g. a GitHub repository too large for GitHub to list in one request, a private OSF component, an OSF link during an OSF outage, or an LDbase link that could not be read), with the reason
$view_only
private OSF projects checked through a view-only link in the paper: paper_id, repo_url (the view-only link), project
$useless_files
files that do not belong in a repository: paper_id, repo_url, file_path, kind, severity
$repo_metadata
one row per repository (not per file): paper_id, repo_url, doi, license
$group_no_evidence
TRUE if study grouping had no evidence to work from
$report
formatted report sections: repositories, files, README, archives, E-Prime, restricted access, private projects checked through a view-only link, closed registrations, unclassified files, roster mismatch, naming, files that do not belong in the repository, and a full classification breakdown
$summary_text
plain-text bulleted summary of each check
22.13 Related
The companion code_check module analyses the actual code files that repo_check discovers. data_check re-downloads and reclassifies the data files using their contents (not just names and paths), and runs the data-quality checks on the columns it finds.