High-quality medical research requires that equipment has been validated for accuracy

Abstract

It is not always appreciated that medical equipment may be cleared by regulatory authorities to sell within a country, without ever having been tested for accuracy performance according to scientific validation standards. Instead, manufacturers can undertake in-house accuracy testing, using variable methods and without any requirement for test results to be made publicly available. This lack of full transparency together with potential for industry bias can place doubt over the quality of validation results provided to regulatory authorities. Currently, this situation affects the field of hypertension research, where most blood pressure devices have not been independently validated for accuracy according to international scientific standards, nor as expected in clinical practice guidelines. More attention should be paid to such practices in order to improve the quality of research and to optimize further translation of scientific findings to clinical practice. The clinical implications of inaccurate measurements in research can be far-reaching, ultimately impacting on a patient’s health. Well-planned validating studies should be more widely considered for new devices that are candidates to be used in research protocols. The awareness of the lack or uncertain validation of equipment used for verifying research hypotheses should prompt all investigators to revisit the idea of conducting the study or, at least, to acknowledge this issue as a relevant study limitation. One of the ways in which authors submitting research findings for publication can add to the quality of the reporting of their work is to ensure reference to the accuracy validation of their research equipment.

Key words: medical devices, equipment and supplies, standards, performance

The international community and funders of research expect the study to be conducted ethically and with high quality, such that it is rigorous, transparent, reproducible, and avoids waste.¹ These expectations are not only essential for the translation of research knowledge to the benefit of the wider community, but also help to promote trust in research findings and deliver the highest value for the money that is invested in research.² The failure of many research discoveries to be reproduced by independent investigators is a symptom of rigorous research methods not always being applied, and indicates that more training to improve the understanding of high-quality research methods, particularly for research apprentices, is needed.^{1, 3}

Resources to aid the rigor and accountability of research practice, from the design to conduct and reporting of research outcomes, have been developed for researchers (e.g., Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network, Cochrane Collaboration).^{4, 5} Working towards similar intent to improve the quality of research publications, the International Committee of Medical Journal Editors (ICMJE) publishes “Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals”.⁶ These recommendations align with best practice standards to assist all those involved in publishing medical research to present unbiased findings clearly and accurately. The journal “Advances in Clinical and Experimental Medicine” is an ICMJE member journal that requires authors of all submitted papers to adhere to the ICMJE recommendations.

The ICMJE Recommendations rightly state that, in order to allow others to reproduce the results of research, there should be sufficient detail provided on the methods, equipment and procedures of the research.⁶ However, a factor that is not considered within the ICMJE recommendations, but can have a marked influence on the results of research, is providing confirmation that the research equipment is accurate, preferably through testing it in accordance with established scientific validation standards. Without such confirmation, there is little to assure that the equipment can measure what it purports to measure, nor whether its use may critically undermine the quality of research outputs. This concept of confirming the quality of all data to increase the confidence in research results is familiar to researchers across disciplines.^{7, 8, 9}

It is not always appreciated that medical equipment may be cleared by regulatory authorities to sell within a country, without ever having been tested for accuracy performance according to scientific validation standards. Instead, manufacturers can undertake in-house accuracy testing, using variable methods and without any requirement for test results to be made publicly available. This lack of full transparency together with potential for industry bias can place doubt over the quality of validation results provided to regulatory authorities. Currently, this situation affects the field of hypertension research,^{10, 11} where most blood pressure devices have not been independently validated for accuracy according to international scientific standards, nor as expected in clinical practice guidelines.^{12, 13, 14} Global estimates indicate that less than 20% of blood pressure devices from among thousands of separate models available for purchase have been independently validated.^{15, 16}

As may be anticipated, the untested, non-validated devices are more likely to be inaccurate when compared with established reference standards,^{17, 18, 19} which in addition to the impact on research quality has obvious implications for appropriate clinical diagnosis and management. Compared with validated devices, those whose accuracy has not been tested also have a greater dispersion of measurement values (higher variance),²⁰ which will influence estimation of sample size for research projects. Higher variance in the measurement of a primary outcome variable means that more research study participants will be needed to answer the research question and demonstrate statistical between-group differences.²¹ This places extra burden on research resources and also has ethical implications in higher-risk study protocols by needlessly exposing more research participants to otherwise avoidable risk.

Regulatory loopholes that enable widespread availability of blood pressure devices with uncertain quality assurance¹⁰ may also apply to other medical equipment used in research, such as blood glucose monitors and pulse oximeters,^{22, 23, 24} although the breadth of the problem more generally across equipment being used in research is not known. This being the case, the onus of responsibility to determine the worthiness of medical equipment to be used in research falls to the shoulders of individual researchers in the planning and conduct of their research projects. What can researchers do in relation to this?

Researchers can check that all research equipment has been assessed, and passed validation testing, or the performance can be verified (including where participants of the study use personal monitoring equipment to collect research data).¹⁷ Ideally, the values derived from the equipment have been compared using an established, standardized protocol with a recognized reference standard, for example, blood glucose measured with a point-of-care device compared with an automated procedure from an accredited laboratory with known accuracy and precision. Researchers can also seek information from relevant professional societies,²⁵ web-based resources,⁵ other sources of published guidance,²⁶ or ask the equipment distributor to provide published material as proof of accuracy. Where available, they should cite the published validation material within the methods of research manuscripts when referencing the equipment used. It is important to note that validation testing is only relevant to the device under scrutiny and the results cannot be extrapolated to other models or devices made by different manufacturers.

In the hypertension field, international efforts are being undertaken to redress the problem of non-validated equipment used in research, clinical practice and population level interventions.^{27, 28, 29, 30} This requires ongoing, multisectoral lobbying for regulatory change, as well as widespread education and advocacy for exclusive use of validated equipment. Whether such activities are needed in other research areas is something for the wider research community to be alert for.

Indeed, the scale of the problem being discussed is increasing, and more attention should be paid to it in order to improve the quality of research and to optimize further translation of scientific findings to clinical practice. The clinical implications of inaccurate measurements in research can be far-reaching, ultimately impacting on a patient’s health. Well-planned validating studies should be more widely considered for new devices that are candidates to be used in research protocols. The awareness of the lack or uncertain validation of equipment used for verifying research hypotheses should prompt all investigators to revisit the idea of conducting the study or, at least, to acknowledge this issue as a relevant study limitation. One of the ways that authors submitting research findings for publication can add to the quality of the reporting of their work is to ensure reference to the accuracy validation of their research equipment.