Download all Zenodo Project Files

Creates a directory for the Zenodo ID and downloads all of the files using a folder structure from the Zenodo project nodes and file storage structure. Returns (invisibly) a data frame with file info.

Usage

zenodo_file_download(
  zenodo_id,
  download_to,
  max_file_size = 10,
  max_download_size = 100,
  unzip_types = NULL,
  pb = NULL
)

Arguments

zenodo_id

an Zenodo ID or URL

download_to

path to download to; there is no default, because CRAN does not allow writing into the working directory unless you ask for it

max_file_size

maximum file size to download (in MB) - set to NULL or Inf for no restrictions

max_download_size

maximum total size to download - set to NULL of Inf for no restrictions

unzip_types

file categories to extract from .zip files in the record rather than downloading the zip whole, named as data_classify_files() names them: "data", "code", "materials", "documentation", "output", "unknown". More than one may be given. NULL (the default) downloads zips whole and leaves them zipped, as before. max_file_size applies to each extracted member rather than to the archive.

pb

a progress bar passed from another function

Value

data frame of file info. When members are extracted from a zip, its row reports the zip with extracted giving the number of members written; size_on_disk and checksum_ok are then NA, because what is on disk is the members rather than the archive Zenodo published a size and MD5 for.

Details

You can limit downloads to only files under a specific size (defaults to 10MB) and only a maximum download size (largest files will be omitted until total size is under the limit). Omitted files will be listed as messages in verbose mode, and included in the returned data frame with the downloaded column value set to FALSE.

A .zip in the record is normally fetched whole and left as a zip. Set unzip_types to pull only the files you want out of it instead: Zenodo serves per-file URLs with byte-range support and each member of a zip is compressed separately, so the wanted members can be read out of the archive while the rest is never transferred. A zip of stimuli and three data files then costs the three data files. When the archive cannot be read that way (the listing fails, or the host refuses ranges) it is downloaded whole as before, so this never loses a file – it only avoids transferring one.

Examples

# \donttest{
if (online("zenodo.org")) {
  dl <- zenodo_file_download("2591593",
                             download_to = file.path(tempdir(), "zenodo"),
                             max_download_size = 10)
}
# }