Check downloaded Dataverse files against the file system

Mirrors .zenodo_verify_downloads(): confirms every file the download planned to save is present, is the size Dataverse reported, and (when Dataverse published an MD5 or SHA-1) matches the checksum.

Usage

.dataverse_verify_downloads(files, download_to)

Arguments

files

the file table being built, with path, size, checksum, checksum_type, and downloaded columns, and optionally extracted

download_to

the folder the dataset was saved in

Value

files with downloaded corrected against the file system, plus size_on_disk and checksum_ok columns

Details

A row whose extracted count is set is exempt from both checks, for the same reason as the Zenodo case: Dataverse's size and checksum describe the zip as published, and that zip was deliberately never downloaded – only chosen members were, each verified against the CRC32 in the archive's own central directory as it is extracted.