Metacheck provides extendable and integrated tools for automated checks of research outputs for best practices. (formerly named papercheck)
Inspired by practices in software development, where automated checks (e.g., CRAN checks for R packages) are used to identify issues before release, metacheck screens scientific manuscripts to identify potential issues or areas for improvement and guide researchers in adopting best practices. It can also assist with processing large numbers of papers for metascientific enquiry.
These check are meant to assist, but not replace, human expertise. For example, metacheck can identify incorrect citations, add missing DOIs, and cross-reference citations against Retraction Watch, PubPeer, and the FLoRA replication database. It can also identify commonly misreported aspects, such as power analyses, and provide general advice. However, actioning this advice is always up to a human who can interpret the advice in context.
See Get Started for the basics, Articles for more detailed examples of what you can do, or our introductory blog post for a more conceptual overview.
Our Values
The values below guide our priorities and decision-making for all Scienceverse projects.
Quality Control: The knowledge, data, code and software we create will be verified for accuracy, using publicly available methods and measures, such that users have enough context to interpret the results.
Transparency: We are committed to open source software and open access datasets for validation.
AI Optional: The use of large language models (LLMs) will be restricted to classification of existing text, not evaluation of the quality of practice. The use of LLMs will always be opt-in and transparently declared. We will prioritise non-LLM functions where possible, and limit use to cases where it provides substantial benefits that cannot be realised with other methods, such as regular expressions.
No Automated Evaluation: While our software is designed to assist the evaluation of research by highlighting aspects where practice may be improved, we will never support automated quality assessments, rankings or scoring.
Data Privacy: We will be fully transparent about what data we have access to and how those data will be used, as well as what will be shared with external services. Wherever possible, we will prioritise functions that share minimal data and services that have robust data protection. We will meet all EU data privacy regulations.
Accessibility: We will provide our knowledge, data, code and software to all potential users with minimal barriers to access. We will prioritise access that is not unduly limited by wealth, disability or technical knowledge.
Community Engagement: Our work will reflect and respect the needs of a diverse range of research communities, and benefit their research practices. We are open to critique from the research community.
Sustainability: We are committed to the long-term viability of our work. We will actively protect and future-proof our work through technical maintenance, governance structures and strategic planning to make it a durable and relevant public good.
Fair Recognition and Reward: Contributions to knowledge, data, code, and software will be uniquely and persistently identified, and benefit a diverse range of contributors.
The Team
Metacheck is developed by a collaborative team of researchers, consisting of Lisa DeBruine (developer and maintainer, UofG), Daniël Lakens (developer, TU/e), Cristian Mesquida (postdoctoral researcher, TU/e), Jakub Werner (RA, TUE), Lavinia Ion (RA, TU/e), Hadeel Khawatmy (RA, TU/e), Levi Baruch (RA, TU/e), Mink Veltman (RA, TU/e) René Bekkers (collaborator and PI of Transparency Check, VUE), and Max Littel (RA, VUE).
Metacheck was initially developed by Lisa and Daniël in 2024 during Lisa’s visiting professorship at the Eindhoven Artificial Intelligence Systems Institute (EAISI).
