Skip to contents

Compare a Preregistration with a Paper via RegCheck

Source: R/db-regcheck.R
regcheck_compare.Rd

Sends the full text of a paper and the text of its (pre)registration to a RegCheck server, which uses an LLM to compare them dimension by dimension (e.g., sample size, hypotheses, exclusion criteria) and judge for each dimension whether the paper deviates from the registration.

Usage

regcheck_compare(
  paper_text,
  prereg_text = NULL,
  registration_id = NULL,
  client = c("ollama", "groq", "openai", "deepseek"),
  base_url = NULL,
  api_token = NULL,
  dimensions = NULL,
  reasoning_effort = "medium",
  poll_interval = 5,
  timeout = 3600
)

Arguments

paper_text

the full text of the paper (single string)

prereg_text

the text of the preregistration (single string); provide exactly one of prereg_text or registration_id

registration_id

a clinical trial registration ID (NCT number) to compare against, as an alternative to prereg_text

client

the LLM client the RegCheck server should use: "ollama" (local), "groq", "openai", or "deepseek"

base_url

the base URL of the RegCheck server; defaults to http://localhost:8000 for ollama and the hosted RegCheck app otherwise (can also be set with the REGCHECK_BASE_URL environment variable)

api_token

the RegCheck API token

dimensions

optional data frame with columns dimension and definition to override the server's default comparison dimensions

reasoning_effort

reasoning effort for OpenAI models ("low", "medium", "high"); ignored by other clients

poll_interval

seconds between polls of the task status

timeout

maximum seconds to wait for the comparison to complete

Value

a data frame with one row per compared dimension, with columns dimension, deviation_judgement ("yes", "no", or "missing"), paper_summary, prereg_summary, deviation_information, paper_quotes, and prereg_quotes. The raw server result is attached as the "regcheck_result" attribute.

Details

By default this uses the locally running RegCheck server bundled in metacheck with Ollama (client = "ollama", http://localhost:8000), so no text leaves your machine. Set client to "groq", "openai" or "deepseek" to use the hosted RegCheck app instead; note that this sends the full paper and registration text to that server and the corresponding LLM provider.

All RegCheck servers (including a local one) require an API token. Set it in your .Renviron (use usethis::edit_r_environ()) as:

REGCHECK_API_TOKEN="your_token"

For a local server this must match the REGCHECK_API_TOKEN you configured when starting the server.

A comparison makes one LLM call per dimension and can take several minutes. The function polls the server until the comparison completes.

Examples

if (FALSE) { # \dontrun{
paper <- demopaper()
paper_text <- paste(paper$text$text, collapse = " ")
prereg_text <- "We will collect 100 participants..."
rc <- regcheck_compare(paper_text, prereg_text)
} # }