Independent mixed-methods evaluation of the NHS AI Lab HTML SummaryThe aim of this evaluation is to understand and assess the processes, impact, and value for money of the programme as a whole, now and in the future, capture key learning and demonstrate value for money and accountability to its stakeholders, and establish specific and generalisable learning to inform future AI Lab activity and other similar programmes.The objectives of this evaluation are to:Gather and present objective and robust evidence of the processes, impacts and value for money of the AI LabIdentify lessons learned from the AI Lab to support the design and future delivery of the remaining ongoing AI Lab projectsUse the evidence and conclusions to provide real learning to future AI programmes in public sector healthcare as well as the wider public sectorWe will conduct a retrospective and prospective mixed-methods formative and summative evaluation, working with commissioners in evaluating the conception, progress, and evolution of the AI Lab in England. This will be delivered through the following complementary lines of enquiry organised around five Work Packages (WP0-WP4): Image Diagram showing Work packages: WP0 Establish liaison mechanisms and define key stakeholders | WP1 Assessing the processess surrounding the implementation of the AI Lab | WP2 Assessing the imapct of the AI Lab | WP3 Assessing the short-term and long-term value for money of the AI Lab across health and care | WP4 Identifying the lesson learned, impacts and value for money of the AI Lab programme overall Key PeopleExpand allCollapse allKathrin Cresswell: Project Manager and Work Package 4 LeadCresswell is a social scientist with extensive experience in conducting formative evaluations of digitally enabled change and improvement programmes in healthcare with over 140 publications.She is an expert in conducting evaluations of digital applications, having applied formative methods tin a variety of health and care settings (hospitals, primary care, and home). Bringing together stakeholders with varying perspectives is key to her ongoing work, where she examines how political, commercial, organisational, patient and healthcare workers' interests need to be aligned to transform health and care through safe and scalable technologies.Expertise: Formative evaluationKathrin Cresswell's profileEmail: kathrin.cresswell@ed.ac.ukRobin Williams: Work Package 1 LeadWilliams is Professor of Social Research on Technology. Since his appointment to lead UoE's successful bid under the ESRC Programme on Information and Communications Technology, Williams has acquired 30+ years' experience, through 60+ external awards, in leading large interdisciplinary research collaborations and conducting qualitative studies of the social shaping of technology. His personal research trajectory, in five books and 100+ papers, focuses on the technical and organisational dimensions of the design, implementation and use of IT-based systems, their usability/dependability and organisational outcomes.Expertise: Science and technology studiesRobin Williams' profileEmail: r.williams@ed.ac.ukMiguel Bernabeu: Work Package 2 LeadBernabeu is Professor of Computational Medicine. His research concerns the development of computational approaches capable of answering open questions in biomedicine and healthcare. He works closely with biologists, clinicians, and industry in problems of high relevance in both basic and clinical research, including@ a) the development of automated methods for eye and systemic disease diagnosis in retinal scans, b) the evaluation of AI systems supporting healthcare delivery in real clinical settings, c) the study of how the structural properties of tumour vasculature affect oxygen / drug transport, and d) the investigation of vascular remodelling during angiogenesis.Expertise: Health Data Science and Artificial IntelligenceMiguel Bernabeu's profileSheena Dungey: Work Package 3 Lead, NHS Arden and Greater East Midlands Commissioning Support UnitDungey is an experienced health economist with an extensive background in evaluating service redesign, resource allocation and cost-benefit and effectiveness evaluation of interventions including digital health and quality improvement innovations. Her work spans NHSE, large acute trusts, NIHR, Commissioning Support Units and independent consultancy undertaking real-world health data research and analysis with application to strategic finance, economics, evaluation and data strategy from the point of data being recorded to its use for operational and strategic insight and research.Expertise: Health economicsEmail: sheena.dungey2@nhs.netSally Eason: Support for Work Package 3, NHS Arden and Greater East Midlands Commissioning Support UnitEason's experience in evaluation and change has been gained through a career spanning the NHS, social care and the third sector. She has designed and led cross sector change programmes, ensuring robust stakeholder engagement, and embedded methods for continuous improvement. She has extensive experience of evaluation, working on the GDE programme and more recently a national evaluation of a new model of care for vulnerable young people. She currently leads on Improvement and Evaluation for AGEM's consultancy arm.Expertise: evaluation and change managementEmail: sally.eason1@nhs.netStuart Anderson: Support for Work Package 1 and Work Package 4Anderson is a professor of Dependable Systems at Edinburgh. He is particularly interested in socio-technical systems, resilience of such systems and how Social Science and Informatics provide a unique perspective on the conception, design, deployment and operation of computer-based systems.Expertise: Ethics and RegulationEmail: s.anderson@ed.ac.ukHajar Mozaffar: Support for Work Package 1 and Work Package 4Mozaffar is a senior lecturer in innovation with a focus on digital transformation, building on her industrial experience and prior mixed-methods research with this team. She has an interdisciplinary reserach interest in digital transformation of health sector and its intersection with Innovation, Science and Technology, and Organisational Studies.Expertise: Innovation, communities of practice, learning ecosystemsHajar Mozaffar's profileEmail: hajar.mozaffar@ed.ac.ukContactsDr Kathrin Cresswell: Kathrin.cresswell@ed.ac.ukProfessor Robin Williams: R.Williams@ed.ac.ukKey Collaborations Image NHS Arden and Greater East Midlands Commissioning Support UnitTimelineStart Date: April 2024End Date: March 2025Duration: 12 MonthsScientific themesArtificial Intelligence, Public HealthMethodology key wordsMixed-methods, Evaluation This article was published on 2024-09-24
HTML SummaryThe aim of this evaluation is to understand and assess the processes, impact, and value for money of the programme as a whole, now and in the future, capture key learning and demonstrate value for money and accountability to its stakeholders, and establish specific and generalisable learning to inform future AI Lab activity and other similar programmes.The objectives of this evaluation are to:Gather and present objective and robust evidence of the processes, impacts and value for money of the AI LabIdentify lessons learned from the AI Lab to support the design and future delivery of the remaining ongoing AI Lab projectsUse the evidence and conclusions to provide real learning to future AI programmes in public sector healthcare as well as the wider public sectorWe will conduct a retrospective and prospective mixed-methods formative and summative evaluation, working with commissioners in evaluating the conception, progress, and evolution of the AI Lab in England. This will be delivered through the following complementary lines of enquiry organised around five Work Packages (WP0-WP4): Image Diagram showing Work packages: WP0 Establish liaison mechanisms and define key stakeholders | WP1 Assessing the processess surrounding the implementation of the AI Lab | WP2 Assessing the imapct of the AI Lab | WP3 Assessing the short-term and long-term value for money of the AI Lab across health and care | WP4 Identifying the lesson learned, impacts and value for money of the AI Lab programme overall Key PeopleExpand allCollapse allKathrin Cresswell: Project Manager and Work Package 4 LeadCresswell is a social scientist with extensive experience in conducting formative evaluations of digitally enabled change and improvement programmes in healthcare with over 140 publications.She is an expert in conducting evaluations of digital applications, having applied formative methods tin a variety of health and care settings (hospitals, primary care, and home). Bringing together stakeholders with varying perspectives is key to her ongoing work, where she examines how political, commercial, organisational, patient and healthcare workers' interests need to be aligned to transform health and care through safe and scalable technologies.Expertise: Formative evaluationKathrin Cresswell's profileEmail: kathrin.cresswell@ed.ac.ukRobin Williams: Work Package 1 LeadWilliams is Professor of Social Research on Technology. Since his appointment to lead UoE's successful bid under the ESRC Programme on Information and Communications Technology, Williams has acquired 30+ years' experience, through 60+ external awards, in leading large interdisciplinary research collaborations and conducting qualitative studies of the social shaping of technology. His personal research trajectory, in five books and 100+ papers, focuses on the technical and organisational dimensions of the design, implementation and use of IT-based systems, their usability/dependability and organisational outcomes.Expertise: Science and technology studiesRobin Williams' profileEmail: r.williams@ed.ac.ukMiguel Bernabeu: Work Package 2 LeadBernabeu is Professor of Computational Medicine. His research concerns the development of computational approaches capable of answering open questions in biomedicine and healthcare. He works closely with biologists, clinicians, and industry in problems of high relevance in both basic and clinical research, including@ a) the development of automated methods for eye and systemic disease diagnosis in retinal scans, b) the evaluation of AI systems supporting healthcare delivery in real clinical settings, c) the study of how the structural properties of tumour vasculature affect oxygen / drug transport, and d) the investigation of vascular remodelling during angiogenesis.Expertise: Health Data Science and Artificial IntelligenceMiguel Bernabeu's profileSheena Dungey: Work Package 3 Lead, NHS Arden and Greater East Midlands Commissioning Support UnitDungey is an experienced health economist with an extensive background in evaluating service redesign, resource allocation and cost-benefit and effectiveness evaluation of interventions including digital health and quality improvement innovations. Her work spans NHSE, large acute trusts, NIHR, Commissioning Support Units and independent consultancy undertaking real-world health data research and analysis with application to strategic finance, economics, evaluation and data strategy from the point of data being recorded to its use for operational and strategic insight and research.Expertise: Health economicsEmail: sheena.dungey2@nhs.netSally Eason: Support for Work Package 3, NHS Arden and Greater East Midlands Commissioning Support UnitEason's experience in evaluation and change has been gained through a career spanning the NHS, social care and the third sector. She has designed and led cross sector change programmes, ensuring robust stakeholder engagement, and embedded methods for continuous improvement. She has extensive experience of evaluation, working on the GDE programme and more recently a national evaluation of a new model of care for vulnerable young people. She currently leads on Improvement and Evaluation for AGEM's consultancy arm.Expertise: evaluation and change managementEmail: sally.eason1@nhs.netStuart Anderson: Support for Work Package 1 and Work Package 4Anderson is a professor of Dependable Systems at Edinburgh. He is particularly interested in socio-technical systems, resilience of such systems and how Social Science and Informatics provide a unique perspective on the conception, design, deployment and operation of computer-based systems.Expertise: Ethics and RegulationEmail: s.anderson@ed.ac.ukHajar Mozaffar: Support for Work Package 1 and Work Package 4Mozaffar is a senior lecturer in innovation with a focus on digital transformation, building on her industrial experience and prior mixed-methods research with this team. She has an interdisciplinary reserach interest in digital transformation of health sector and its intersection with Innovation, Science and Technology, and Organisational Studies.Expertise: Innovation, communities of practice, learning ecosystemsHajar Mozaffar's profileEmail: hajar.mozaffar@ed.ac.ukContactsDr Kathrin Cresswell: Kathrin.cresswell@ed.ac.ukProfessor Robin Williams: R.Williams@ed.ac.ukKey Collaborations Image NHS Arden and Greater East Midlands Commissioning Support UnitTimelineStart Date: April 2024End Date: March 2025Duration: 12 MonthsScientific themesArtificial Intelligence, Public HealthMethodology key wordsMixed-methods, Evaluation
