Critical barriers to safety assurance and regulation of autonomous medical systems

M Sujan, Dominic Furniss, D Embrey, M Elliott, D Nelson, S White, I Habli, N Reynolds
in Proceedings of the 29th European Safety and Reliability Conference, ESREL 2019, Conference paper (text)


© 2019 European Safety and Reliability Association. Published by Research Publishing, Singapore. The use of automation and artificial intelligence is transforming medicine and the way health care is accessed and delivered. The introduction of these new technologies introduces challenges for safety assurance and regulation. The paper describes the aims and first results of a project looking at safety assurance of highly automated and autonomous systems for the management of intravenous drug infusions. The paper introduces a taxonomy of levels of automation, and describes two reference scenarios. These are used to analyse the safety assurance challenges. The identified challenges relate to the handover of control from the autonomous system to the clinician, the problems associated with data quality, and the difficulty of assuring sufficiency of the intended functionality of the autonomous system. The pace of innovation and adoption of automated and autonomous systems in health care is now overtaking other industries, but the research evidence is lagging behind. Regulatory approaches in health care are not yet set up to deal with these kinds of systems.