Skip to contents

Extract the statistical tests a paper reports

Source: R/extract-tests.R
extract_tests.Rd

Builds the paper-side counterpart to the analysis output that reproducibility_check extracts: one row per reported TEST, with its components kept together and the sentence it was reported in retained, so a matched result can be traced back to the claim it supports.

Usage

extract_tests(paper)

Arguments

paper

a paper object (uses its $eq and $text)

Value

a data.frame, one row per reported test, with paper_id, test_no (sequential within the paper), text_id, paragraph_id, section_id, sentence (the reporting sentence), anchor (the test statistic the report is built around, e.g. "t"), n_components, components (a list-column of name/comp/value/df), and reported (the test rendered back as text, e.g. "t(23) = 3.77, p = .001, d = 0.77").

Details

This is a purpose-built alternative to extract_eq() for matching. It reuses extract_eq()'s parsing of name <op> value fragments, then does three things that parsing does not:

  • isolates separate tests within one sentence. extract_eq()'s grp_id is the sentence number, so a sentence reporting two t-tests yields a single ten-component "test" that no analysis can produce; here a repeated anchor (a second t, F, r, ...) starts a new test;

  • keeps the sentence, its text_id, paragraph_id and section_id, so a match links a result_id to the sentence making the claim;

  • filters non-statistics, so a figure's height = 4 or a sample's range = 18-29 is not offered as evidence. A run of components with no recognised test statistic in it is dropped entirely.

Examples

tests <- extract_tests(demopaper("stroop_effect"))
tests[, c("anchor", "reported")]
#>   anchor                                        reported
#> 1      m                              M = 15.1, SD = 4.1
#> 2      m                               M = 23, SD = 4.78
#> 3      r                                        r = 0.37
#> 4      t                        t(129) = 4.53, p < 0.001
#> 5      t    t(130) = 18.04, p < 0.001, Cohen's dz = 1.58
#> 6      m                                        M = 7.77
#> 7      m                                        M = 8.13
#> 8      t t(120.37) = -0.42, p = 0.673, Cohen's d = -0.07