The notion of reproducibility is fundamental to good science.
If a finding cannot be reproduced by independent research groups its relevance is very limited, regardless of its validity. It is therefore crucial for scientists to describe their experiments in sufficient detail so they can be reproduced, challenged, and built upon.
However, due to recent technological advances in the biological and computational sciences, experimental protocols, data analysis and interpretation have become increasingly complex. This has made reproducing research findings more challenging.
As the number of high-profile cancer studies that can't be reproduced continues to grow (see
eLife Reproducibility Project), some researchers have suggested the biomedical sciences are experiencing a "reproducibility crisis."
To address this important issue, the US Food and Drug Administration (FDA) led the
MicroArray and SEquencing Quality Control (MAQC/SEQC) projects to assess the reproducibility of the new high-throughput biotechnologies used in cancer research for diagnosis and prognosis.
Recognizing new technologies and analytical approaches are developing at a rapid pace, members of the MAQC/SEQC consortium decided to form a new international society, the
Massive Analysis and Quality Control (MAQC) Society, whose mission is to promote the best research practices for enhanced reproducibility.
As described in a letter published in
Nature Biotechnology, the initial focus of the new society will be on data analysis.
"While it is obvious that assays used in biomedical studies must be robust, the reproducibility of computational analyses is a relatively new concept that is yet to be fully adopted by the scientific community," says Dr.
Benjamin Haibe-Kains, Scientist at Princess Margaret Cancer Centre and co-founder of the MAQC Society.
Given the complexity and diversity of the computational analyses used in biomedical research, the society will select examples of fully reproducible studies, which will be collectively analyzed to draft practical guidelines for future studies.
"This is an ambitious initiative that will only be possible with the support of the society members who will directly benefit from the resulting guidelines," adds Dr. Haibe-Kains.
To foster collaborations and the active involvement of the community, Carl Virtanen, Director and Research Lead of UHN Digital and head of the Bioinformatics and HPC Core, created the
MAQC Society website.
"I followed with great interest the previous efforts led by the FDA to assess the reproducibility of cutting-edge technologies used in cancer research and was involved in similar initiatives back in the era of microarrays," Carl says.
"Because the Bioinformatics Core also runs clinical software pipelines for genetic testing at UHN which are, by law, mandated to be fully reproducible, I was eager to get involved and help build a web-resource for the new MAQC Society."
"Data analysis is a critical part of all cancer studies, and we must ensure the studies' findings are fully reproducible," Carl adds. "This initiative is vital and will help to ensure the massive investments in cancer research deliver benefits to patients and the health system in the long-term."
This initiative was supported by The Princess Margaret Cancer Foundation.
