Abstract
This paper presents an overview of the usability engineering process for the development of a personalised clinical decision support system for the management of type 1 diabetes. The tool uses artificial intelligence (AI) techniques to provide insulin bolus dose advice and carbohydrate recommendations that adapt to the individual. We describe the role of human factors and user-centred design in the creation of medical systems that must adhere to international standards. We focus specifically on the formative evaluation stage of this process. The preliminary analysis of data shows promising results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aamodt, A., Plaza, E.: Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Commun. 7(1), 39–59 (1994)
Abrial, J.R., Abrial, J.R.: The B-book: Assigning Programs to Meanings. Cambridge University Press, Cambridge (2005)
Årsand, E., et al.: Mobile health applications to assist patients with diabetes: lessons learned and design implications. J. Diab. Sci. Technol. 6(5), 1197–1206 (2012)
Bellazzi, R.: Telemedicine and diabetes management: current challenges and future research directions. J. Diab. Sci. Technol. 2(1), 98–104 (2008)
Bertini, E., Catarci, T., Dix, A., Gabrielli, S., Kimani, S., Santucci, G.: Appropriating heuristic evaluation methods for mobile computing. In: Handbook of Research on User Interface Design and Evaluation for Mobile Technology, pp. 780–801. IGI Global (2008)
Brooke, J., et al.: SUS-A quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)
Brown, D.: Reflections on the role and design of visualizations to support patients with type 1 diabetes (2016, in preparation)
Cefai, J.J., Shapley, J.L.: Micro-valve. US Patent 8,048,041, 1 November 2011
Cellnovo: Cellnovo system (2007). https://www.cellnovo.com/en/homepage
Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13, 319–340 (1989)
Tufte, E.: The Visual Display of Quantitative Information. Graphics Press, Cheshire (2001). vol. 4, no. 5, p. 6
Fast, E., Horvitz, E.: Long-term trends in the public perception of artificial intelligence. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 963–969. International Joint Conferences on Artificial Intelligence, Menlo Park, Calif (2017)
Georgsson, M., Staggers, N., Weir, C.: A modified user-oriented heuristic evaluation of a mobile health system for diabetes self-management support. Comput. Inf. Nurs. 34(2), 77 (2016)
GOV.UK: Medicines and healthcare products regulatory agency. Alerts and recalls for drugs and medical devices (2016). https://www.gov.uk/drug-device-alerts
Hengstler, M., Enkel, E., Duelli, S.: Applied artificial intelligence and trust–the case of autonomous vehicles and medical assistance devices. Technol. Forecast. Soc. Chang. 105, 105–120 (2016)
Herrero, P., López, B., Martin, C.: PEPPER: patient empowerment through predictive personalised decision support. In: ECAI Workshop on Artificial Intelligence for Diabetes, pp. 8–9 (2016)
Herrero, P., et al.: Safety layer for an insulin delivery system. In: Abstracts from ATTD 2017 10th International Conference on Advanced Technologies & Treatments for Diabetes Paris, France, 15-18 February 2017, p. A-62 (2017). https://doi.org/10.1089/dia.2017.2525
IEC: International electrotechnical commission (2018). http://www.electropedia.org/
International Standard IEC: 62366-1: 2015 medical devices part 1: application of usability engineering to medical devices. International Organization for Standardization, Geneva (2015)
International Standard IEC: 62366-2: 2016 medical devices part 2: Guidance on the application of usability engineering to medical devices. International Organization for Standardization, Geneva (2016)
ISMP: Institute for safe medication practices, fluorouracil incident root cause analysis report (2007). https://www.ismp-canada.org
ISO: International organization for standardization online browsing platform (2018). https://www.iso.org/obp/ui/
Kirwan, B., Ainsworth, L.K.: A Guide to Task Analysis: The Task Analysis Working Group. CRC Press, Boca Raton (1992)
Klonof, D.C.: The current status of bolus calculator decision-support software. J. Diab. Sci. Technol. 6(5), 990–994 (2012)
Klonoff, D.C.: Telemedicine for diabetes: current and future trends. J. Diab. Sci. Technol. 10(1), 3–5 (2016)
Munzner, T.: A nested process model for visualization design and validation. IEEE Trans. Vis. Comput. Graph. 6, 921–928 (2009)
Nielsen, J.: Usability Engineering. Morgan Kauffmann, San Francisco (1994)
Nielsen, J.: Severity ratings for usability problems. Pap. Essays 54, 1–2 (1995)
O’Kane, A.: Individual differences and contextual factors influence the experience and practice of self-care with type 1 diabetes technologies. Ph.D. thesis, UCL (University College London) (2016)
Rice, A.D., Lartigue, J.W.: Touch-level model (TLM): evolving KLM-GOMS for touchscreen and mobile devices. In: Proceedings of the 2014 ACM Southeast Regional Conference, p. 53. ACM (2014)
Russell, S., Dewey, D., Tegmark, M.: Research priorities for robust and beneficial artificial intelligence. AI Mag. 36(4), 105–114 (2015)
Thimbleby, H.: Feature ignorance of interaction programming is killing people. Interactions 15(5), 52–57 (2008)
USFDA: MAUDE - manufacturer and user facility device experience (2018). http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/TextSearch.cfm
USFDA, et al.: Applying human factors and usability engineering to optimize medical device design. FDA Center for Devices and Radiological Health: Silver Spring, MD, USA (2011)
Zhang, J., Johnson, T.R., Patel, V.L., Paige, D.L., Kubose, T.: Using usability heuristics to evaluate patient safety of medical devices. J. Biomed. Inform. 36(1–2), 23–30 (2003)
Acknowledgement
This work has received funding from the EU Horizon 2020 research and innovation programme under grant agreement No. 689810. We thank all partners of the PEPPER consortium, in particular clinical teams in UK and Spain who conducted the Phase 2 of the formative usability evaluation. Ethical approval has been obtained from the relevant authorities for all elements involving users.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Appendix A
Appendix A
The following section of this report document the notable and agreed issues and recommendations against each heuristic during a debrief with the evaluators. In some cases, issues listed by the evaluator for a particular heuristic have been relocated to a more suitable heuristic following the debrief.
-
Z6 - Informative Feedback
-
Average score: 0.33
-
Standard deviation: 0.29
-
Result: No usability issues
-
Issue: Activity monitor connection is not informative.
-
Recommendation: Provide feedback and information on the connection status of the activity monitor.
-
B1 - Visibility of System Status and Losability/Findability of the Device
-
Average score: 1.83
-
Standard deviation: 0.76
-
Result: Minor usability issue
-
Issue: Network, activity monitor and CGM status are not clearly indicated.
-
Recommendation: Include clear indications of connection status for network, activity monitor and CGM on the status bar.
-
B2 - Match Between System and Real World
-
Average score: 1.22
-
Standard deviation: 0.51
-
Result: Cosmetic problem only
-
B2.1 Issue: Ambient temperature location on the Get Bolus Advice interface is not intuitive.
-
B2.1 Recommendation: Relocate ambient temperature to the other category.
-
B2.2 Issue: The location of obtaining a capillary blood glucose reading is not intuitive.
-
B2.2 Recommendation: Include the ability to obtain a capillary blood glucose reading from the main menu and calibration interfaces.
-
B2.3 Issue: The application interface is always in portrait orientation.
-
B2.3 Recommendation: Appropriate landscape interfaces should be included. For example, changing the visualizations on the dashboard when in landscape orientation.
-
B2.4 Issue: The Android back button (bar at the bottom) interfaces with the interface due to full screen mode.
-
B2.4 Recommendation: Change PEPPER to a non-full screen application for MDI users on the handset or ensure that this bar does not interfere with the PEPPER interface.
-
B3 - Consistency and Mapping
-
Average score: 0.33
-
Standard deviation: 0.58
-
Result: No usability issues
-
Issue: The target blood glucose thresholds do not match what actually happens, stating that the lowest/highest possible value is invalid.
-
Recommendation: Investigate and correct the target blood glucose thresholds to ensure the valid range displayed to the user is correct or that valid inputs are accepted.
-
B4 - Good Ergonomics and Minimalist Design
-
Average score: 0.33
-
Standard deviation: 0.29
-
Result: No usability issues
-
Issue: The application has a mixture of light and dark interfaces.
-
Recommendation: Use a consistent colour palette throughout the interfaces.
-
B5 - Ease of Input, Screen Readability and Glanceability
-
Average score: 0.53
-
Standard deviation: 0.23
-
Result: Cosmetic problem only
-
B5.1 Issue: The abbreviation IU needs clarifying.
-
B5.1 Recommendation: Change the abbreviation IU to Units (based on clinician advice).
-
B5.2 Issue: The menu can only be accessed from the dashboard interface.
-
B5.2 Recommendation: Replace the home button with the menu option on all interfaces.
-
B5.3 Issue: The CGM cannot be automatically calibrated from the capillary blood glucose meter.
-
B5.3 Recommendation: Add a calibration option which obtains the blood glucose reading from the capillary blood glucose meter and automatically calibrates the CGM.
-
B6 - Flexibility, Efficiency of Use and Personalisation
-
Average score: 1.22
-
Standard deviation: 0.51
-
Result: Cosmetic problem only
-
B6.1 Issue: No ability to customise the application.
-
B6.1 Recommendation: Add some degree of customisation. For example, quick links and custom colour palettes.
-
B6.2 Issue: The keyboard overlaps input fields on the Get Bolus Advice interface.
-
B6.2 Recommendation: Remove the next option from the keyboard when inputting on the Get Bolus Advice interface, instead include the done button to close the keyboard.
-
B6.3 Issue: Some interfaces use number pickers rather than a keyboard for data entry. For consistency and speed these should all be keyboard inputs.
-
B6.3 Recommendation: Replace number pickers with appropriate keyboards, limiting the characters/digits to only valid inputs.
-
B7 - Aesthetic, Privacy and Social Conventions
-
Average score: 1.22
-
Standard deviation: 0.51
-
Result: Cosmetic problem only
-
B7.1 Issue: There are some harsh edges on the interface.
-
B7.1 Recommendation: Use the Android material design features to soften the edges, for example z-index.
-
B7.2 Issue: The applications lock screen does not provide security in its present state.
-
B7.2 Recommendation: The lock screen is not needed as the phone has its own lock screen. On the CSII version, the tap the Xs should be replaced by a personalised PIN to prevent unauthorized use.
-
B7.3 Issue: The application does not indicate that data has been transmitted successfully.
-
B7.3 Recommendation: Include an interface of data sent to the PEPPER server application, perhaps on the Events interface.
-
B8 - Realistic Error Management
-
Average score: 1.83
-
Standard deviation: 0.76
-
Result: Minor usability issues
-
Issue: The application does not provide any undo functionality for adding boluses/meals.
-
Recommendation: Include the ability to undo the previous bolus/meal input.
-
D1 - Safe and Efficient Numerical Data Entry
-
Average score: 1.75
-
Standard deviation: 0.25
-
Result: Minor usability issues
-
D1.1 Issue: Not all inputs are validated to ensure safe data entry.
-
D1.1 Recommendation: Clinically approved thresholds should be introduced for all inputs. For example, carbohydrates on the Get Bolus Advice interface.
-
D1.2 Issue: There is no ability to obtain a blood glucose reading from the capillary blood glucose meter on the Get Bolus Advice interface in the event of CGM failure.
-
D1.2 Recommendation: Add the option to obtain a capillary blood glucose reading via Bluetooth on the Get Bolus Advice interface. Performing this task should be mandatory if the application has not received data from the CGM recently.
-
D2 - Clear and Meaningful Numerical Data Visualization
-
Average score: 0.33
-
Standard deviation: 0.33
-
Result: No usability issues
-
D2.1 Issue: Some of the text is small and difficult to read.
-
D2.1 Recommendation: Introduce a minimum text size to ensure the information is clear. Additionally display more information when clicking on the carbohydrate/bolus/activity on the dashboard with a large text size and precise underlying data.
-
D2.2 Issue: Bolus and carbohydrate values on the dashboard can overlap if input within a small time interval.
-
D2.2 Recommendation: Merge overlapping data on the dashboard visualization (carbs and bolus) and indicate that this is multiple inputs and provide the user the ability to tap to view details.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Martin, C., Aldea, A., Duce, D., Harrison, R., Alshaigy, B. (2019). The Role of Usability Engineering in the Development of an Intelligent Decision Support System. In: Koch, F., et al. Artificial Intelligence in Health. AIH 2018. Lecture Notes in Computer Science(), vol 11326. Springer, Cham. https://doi.org/10.1007/978-3-030-12738-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-12738-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-12737-4
Online ISBN: 978-3-030-12738-1
eBook Packages: Computer ScienceComputer Science (R0)