Skip to contents

Flag a free-text column that may contain incidental personal information

Source: R/data_check_helpers.R
data_check_pii_freetext.Rd

Open-ended typed responses (comments, explanations, descriptions) can contain names, places, or other identifying detail, so they warrant a "review before sharing" prompt. The aim is to flag genuine typed prose only – not any long, varied string. Long values that are not prose (numeric matrices with blank headers, IDs, hashes, URLs, file paths, base64) are common in research data and previously produced false positives, so a column is flagged only when its typical value actually reads like written language:

Usage

data_check_pii_freetext(
  x,
  min_median_chars = 40,
  min_unique_frac = 0.8,
  min_multiword_frac = 0.6,
  min_alpha_frac = 0.5
)

Arguments

x

a character or factor vector

min_median_chars

typical (median) length above which a column may be free text

min_unique_frac

minimum fraction of distinct values (prose is rarely repeated; a coded category is)

min_multiword_frac

minimum fraction of values that contain more than one word (a space between word characters)

min_alpha_frac

minimum share of alphabetic characters in the typical value (screens out numeric/ID/hash/URL columns)

Value

list(problem, message, values)

Details

  • long enough (min_median_chars),

  • varied enough to be responses rather than a repeated category (min_unique_frac),

  • multi-word – most values contain whitespace between words, and

  • predominantly alphabetic – letters, not mostly digits/punctuation.

Examples

x <- c("I found the task long and I lost focus near the end of it",
       "My little brother was ill so I did this at night after work",
       "The second block felt much harder than the first one to me",
       "I live close to the lab and walked over in the morning sun",
       "Nothing to add, everything went fine during the whole study")
data_check_pii_freetext(x)$problem
#> [1] TRUE
data_check_pii_freetext(c("yes", "no", "yes"))$problem
#> [1] FALSE