© Marcos Casiano/Shutterstock.com

1 User's Needs

The world is witnessing a growing number of elderly and disabled individuals, they live at home, in care centres or are mobile. These individuals require usable, accessible, and flexible healthcare to receive the treatment and support they need in an integrated manner, regardless of their abilities or living context. The rapid advancements in healthcare and information technology over the past few decades led to a significant amount of research on providing integrated healthcare for everyone. A wide range of technologies are enabling the implementation of new services towards integrated healthcare, where diagnosis and treatments are tailored to individual patients. Health analytics and informatics, using massive data sets, Internet of Things (IoT) solutions, machine learning, and high-performance computing, are indeed enabling optimized ways of disease diagnosis, management, prediction, as well as smart ways of care delivery.

2 Involvement of End Users

Involvement of real end users right from the beginning of an IT project is understood nowadays as good practice particularly in research projects where new and innovative concepts are to be investigated [1]. It is considered as part of user empowerment. The goal is to ensure that developments are useful and easy to use by the intended target user group. For this purpose, user requirements are collected at the beginning of a research project which can be achieved by various methods among those are focus groups, cocreation workshops, interviews with users representing the target user groups and ethnographic methods such as participatory observation of the domain context. Such field studies help to understand the context-of-use and typical usage scenarios of the envisioned application can be deduced out of these studies. The results of field studies build then the base for elicitation of user requirements. User requirements are descriptions of how a system is expected to perform from the user’s perspective. They are intended to serve as primary source of information for developers when determining system architecture and specifications as well as required software functionalities and features [1]. To communicate user requirements in a suitable format they provide at its core a short but precise description of the user’s need, a rationale for that and most important a fit criterion which states in a quantified manner when a user need is met, e.g. by providing a specified feature [2]. However, complexity of integrated eHealth projects has been rapidly increasing, so user requirements of the end application are not fully understood even after the phase of user requirements gathering has been completed resulting in high volatility of defined requirements. Beyond this, user needs, and fit criteria cannot all be defined as precise as needed to serve as encompassing source for derivation of software requirements which is a precondition for creating an application that suits the domain context and is characterised by good usability and accessibility. It is rather to expect that user requirements will need substantial refinement and enhancement by new ones requiring further user involvement to ensure suitability to users’ needs. Testing a system by its end users requires robust systems that have been technically validated by carrying out functional, system and integration tests. After reaching a specific maturity of the integrated eHealth platform and its user interfaces (Apps), the next step, usability evaluation, can be conducted. Usability study is concerned with effectiveness, efficiency, and user satisfaction of eHealth apps, which are key concepts of usability (see ISO 9241-11:2018 “Suitability for use: terms and concepts”). For this purpose, feedback from selected group of users representing all target user groups shall be used. All test subjects receive a mock-up version of the application relevant to their user group and will be asked to go through typical workflows. Single-ease questions (SEQs) and the standardized questionnaires SUS and UEQS as well as some open questions are then used to determine how user-friendly the test subjects found the applications. The focus of a usability study is on the execution of standard tasks by test participants that are typically carried out by the target group with this application. During the execution, the test participants are observed in order to uncover usability problems, e.g. the design of the user interface does not adequately support the completion of the task, as the naming of menu items or the grouping of information under menu items does not meet the expectations of the test participants, so that, for example, a task could only be partially solved or not solved at all. In some situations, the involvement of real end users is not feasible though, because this would cause unreasonable burden on target users, e.g., in case of chronically ill older patients or it is simply not possible, because administrative hurdles like permission by ethical committees need to be overcome. Therefore, it is in such cases necessary to install substituting methods, achieved by implementing group-based expert walkthroughs where experts with different work-domain expert knowledge, in this case from usability, accessibility, clinical practice and software development, walked through an application by following typical usage scenarios target users are supposed to achieve. In contrast to the original method outcomes of walkthroughs shall be documented in form of software requirements, i.e. essential functionalities and features of the envisioned application instead of usability and accessibility issues. This way a reference document for developers evolved that closes the gap in bringing user requirements to application development [1].

3 Combined Usability and Accessibility Evaluation

The World Wide Web Consortium (W3C) published the first version of the Web Content Accessibility Guidelines (WCAG) in 1999 as a recommendation to provide accessible websites [4]. The World Health Organization recognizes that digitalization of health care can facilitate universal health coverage and points out that eHealth should be developed with accessibility in mind and designed to be inclusive [5]. The European Accessibility Act (EAA) mandates that a range of products and services, including many types of online experiences such as eHealth apps, are accessible to people with disabilities. The law, which applies to those doing business within the EU, aims to resolve discrepancies in accessibility requirements between different European countries. Originally encoded into law in 2019, the EAA was adopted by individual EU nations starting in 2022. Providers of products and services covered by the EAA have until June 2025 to become compliant. A complaint process and surveillance monitoring will be established, and organizations that fail to meet these requirements may face penalties, including fines. Conducting accessibility testing according to WCAG guidelines in health applications is crucial to ensure that all users, regardless of their abilities, can readily access and use related apps. Combined accessibility evaluation consisting of automated and expert evaluation would really unveil the accessibility problem of an application, especially when it provides different and distributed user interfaces. In CAREPATH project case these are web applications and Android apps targeting various stakeholder. Additionally we think that a combined accessibility and usability evaluation is required to ensure good user experience of an application, though there certainly some overlap between them, even though there might be different rationales for that, e.g., for visually impaired users who have limited or no access to visual information, it is crucial that adequate text alternatives are provided for non-text content like images and that link texts clearly describe the target otherwise they will not be able to navigate in an application. However, same issues will also pose a usability issue because self-descriptiveness and conformity to user expectations are understood in usability as important features to support users in achieving their tasks and facilitate learnability of an application [3].

4 Technical Testing and Usability Evaluation in CAREPATH

In CAREPATH we Followed a user-centred design approach including implementation of a user requirements engineering process, follow up on prototype development by group-based expert walkthroughs in laboratory tests. Furthermore, a usability study is planned as part of a Technical Validation and Usability (TVU) study to capture and fix usability issues before the clinical investigation starts. For the elicitation of additional input to common practice and obstacles when implementing guidelines in clinical day-to-day practice workshops with clinicians at each pilot site were conducted. At the beginning a workshop in each pilot site was conducted with clinicians to elicit guidelines, constraints, and conflicts.
Clinicians’ needs and constrains were gathered via user and co-production workshops, the results lead to use cases that have been developed as a basis for the integrated adaptive care planner and the accompanying decision support services. The prototypes resulting from the first development phase were validated by expert walkthroughs. Technical validation was conducted to test proper integration of all CAREPATH components. Next step the usability evaluation will start with target end users. Feedback from users on usability issues will be provided to technical partners as discovered to support agile software development. The focus of the usability evaluation is on the apps for the clinicians and patients, so the test design and then the test materials were first developed. This includes the creation of a test protocol, checklist for preparing the usability evaluation, standard tasks with the possibility of documenting observations by the tester as well as comments from test participants. The evaluation of the questionnaires will follow a standardized procedure. This allows to easily interpret and compare results with the existing measurement scales. The SEQ indicates how easy the test participants found the individual tasks to be. The UEQS determines the user experience by asking how helpful (pragmatic quality) and interesting (hedonic quality) the application will be for the test participants. The SUS, in turn, measures the usability of an application. Legal requirements on ethics, data protection and secure data processing will be complied to according to EU, national and local regulations.