Skip to contents

Convert a research folder into a clean, FAIR copy

Source: R/fair-repo.R
FAIR_repo.Rd

Writes a reorganised copy of a folder of research files (data, code, materials, documentation) on your own computer. Nothing is downloaded and no link is followed: only the folder you name is read, and it is not changed. The folder is first copied to a temporary folder, and the checks run on that copy, so nothing they write (such as code recovered from an output file) can reach your folder; the copy is removed at the end. The checks run the way report_repository() runs them: repo_check takes an inventory of the folder, and code_check, data_check, codebook_check and reproducibility_check (with execute = FALSE, so no code is run, unless you set execute) build on it. Whether a language model is used is decided by llm_use().

Usage

FAIR_repo(path, output_dir, paper = NULL, overwrite = FALSE, execute = FALSE)

Arguments

path

the folder to convert

output_dir

the folder to write into; created if needed. There is no default, because CRAN does not allow writing into the working directory unless you ask for it.

paper

optionally, the paper that uses these files (see read()), for its title, authors, DOI, CRediT roles and funding. Its links are not followed. Without it, these are listed in the README as still to be filled in.

overwrite

whether to replace an existing output_dir. When FALSE (the default) and output_dir exists, nothing is written and the returned list has existed = TRUE.

execute

whether to run the R and Python scripts, as reproducibility_check(execute = TRUE) does: in a copy, with each script in its own R or Python process (see the reproducibility_check module for what that does and does not protect). Only then is a script found that uses a function of a package it does not load; the library() calls reproducibility_check adds for it are added at the top of the script in the copy. Default FALSE: no code is run.

Value

(invisibly) a list with output_dir, n_files_copied, n_studies, plan (the file plan, with a written column: the path each file was written to, NA when it was not), scripts (one row per changed script: target_path, paths_changed, setwd_removed, setwd_paths, libraries_added), skipped (files that could not be read), copy_failed and renamed (a file whose target another file already took), empty_plan (TRUE when the folder had no files), and the paths of the metadata written: descriptions, rocrate, requirements, logs and readme_changes. When an existing output_dir was kept, the list has only output_dir, the counts, and existed = TRUE.

Details

Every file is placed where the Psych-DS layout puts it (data under data/, code under analysis/, materials under materials/, documentation under documentation/, one study-<group>/ folder per study when there are several; see psychds_file_plan()), keeping its own subfolders. A file a script writes and no script reads, and an R data file holding a fitted model, is a result and goes to outputs/. A data file in a format that needs particular software (.xlsx, .sav, .dta, ...) is also written as a real CSV file, with the original kept next to it; an Excel file with several sheets gets one CSV per sheet, named after the sheet; an R data file only when it holds a table and no CSV of the same name is next to it. A table in a text file (.txt, .tsv) is kept as it is. A data CSV file is copied without a byte order mark. The files of an archive package (BagIt), which list the files as they were, go to logs/original-package/. A file from an archive in the folder that is identical to a file next to it, and a README identical to another, are left out.

R and Python scripts (.R, .Rmd, .qmd, .py, .ipynb) are changed so they still find their files in the new layout, run from the top folder: each file path that repro_rewrite_paths() can match to one file of the folder is replaced by that file's new path, as is the path a script writes a file to that is in the folder or goes to outputs/; and each setwd()/os.chdir() line that sets a folder written out in the script, no folder, or the script's own folder is replaced by a statement that does nothing, with the original line kept in a comment (one that sets a folder worked out when the code runs, such as setwd(tempdir()), is left). Every change is reported, and listed in CHANGES.md. Scripts in other languages (Stata, SPSS, MATLAB, ...) are copied unchanged, so their file paths may no longer work.

Code that code_check recovers from an output file (a rendered R Markdown or Quarto page, an SPSS .spv, a Stata .smcl or an Mplus .out file) is kept as a script in analysis/, unless the folder has the script it was made from. In a README that the folder has, a link to a file of the folder (such as an image) is changed to that file's new place.

Metadata is written next to the files:

  • dataset_description.json in each study folder with data (Psych-DS), describing every data column, with its label from the codebook when codebook_check found one;

  • ro-crate-metadata.json at the top (RO-Crate 1.3): every file, the paper, the authors with their CRediT roles when the paper states them, the contact, funding (the paper's funding statement, or what funding_check finds in its text), the licence, the languages and packages the code uses, the software proprietary formats need, which files each script reads and writes, the missing-value codes, and how each study was run (DDI-Lifecycle terms). "unknown" is written only where the authors are asked for something: the licence, an author's CRediT roles, and when and how the data were collected and processed;

  • README.md, a readable version of the RO-Crate, in the top folder and in each study folder, unless the folder has its own README there. Without a paper, the title is taken from the folder's own README;

  • requirements.txt, the Python packages the code imports, for pip install -r requirements.txt, unless the folder has its own requirements.txt, environment.yml or pyproject.toml. R packages are listed in the README and the RO-Crate;

  • logs/, the list of files and the results of the checks, without the places on this computer the files are in;

  • CHANGES.md, and a CHANGES.md in each study folder that has none.

Examples

# convert the demo repository bundled with metacheck, with its paper for
# the title, authors and funding; the copy is written to a temporary folder
# \donttest{
out <- file.path(tempdir(), "stroop_effect_paper_FAIR")
FAIR_repo(demofile("repo"), output_dir = out,
          paper = demopaper("stroop_effect"))
#> Wrote 10 files to the new layout.
#> File paths were changed in 2 scripts, so they find the files in the new layout.
# }