Measuring Telestroke Usability with the System Usability Scale

Powerful Clinical Decision Support Systems (CDSS) and digital health tools are entering acute stroke care rapidly, and unevenly. Across our four national stroke networks, MDRinAIS has a front-row view of this systemic transformation. As stressed in multiple project outputs, the human factor remains crucial to successful integration: structured training, workflow alignment, and continuous follow-up are core pillars of real-world implementation. The true differentiator is rarely the underlying algorithm alone; it is the healthcare professionals, the care pathways, and the everyday usability of the systems clinicians are asked to trust in the first, precious minutes after a large-vessel occlusion.

Across European stroke networks, significant variability exists in door-to-groin and door-to-needle times, even among hospitals operating within the same Drip-and-Ship framework. Usability sits directly upstream of this operational variability. A tool that clinicians find awkward, slow, or untrustworthy under intense time pressure could not be used as intended, no matter its technical accuracy on paper.

To quantify these workflow dynamics, MDRinAIS is conducting a targeted investigation into the usability of telestroke and AI-assisted systems deployed along our reperfusion pathways. While running alongside our primary work package streams, this effort directly feeds into our broader project mission: identifying adoption friction early in stakeholder ecosystems (WP3) and optimizing clinical and technical workflows (WP4/WP5) before delay translates into lost minutes on the reperfusion clock.

What the System Usability Scale Measures, and Why We Chose It

The System Usability Scale (SUS) is a ten-item questionnaire developed by John Brooke in 1986 that yields a single 0–100 score reflecting perceived system usability (Brooke, 1996). Its appeal lies in its simplicity: it is quick to complete, technology‑agnostic, and backed by decades of comparative benchmark data. A widely accepted standard places the average SUS score at 68 (SD 12.5), and a meta-analysis evaluating digital health applications confirmed that this benchmark holds up robustly for healthcare technologies, providing a defensible yardstick rather than an arbitrary target (Hyzy et al., 2022).

The SUS has become one of the most widely applied instruments for evaluating healthcare innovations. Numerous validated translations now exist across Europe, a vital feature for a multi-country initiative like MDRinAIS spanning diverse regional healthcare systems. It has already been successfully applied to tools directly parallel to our focus, including emergency clinical decision support tools, mobile safety applications, and AI chatbots evaluated by frontline healthcare staff (Wohlgemut et al., 2023).

At the same time, we remain clear-eyed about the limitations of the SUS. A single composite score can flatten nuanced workflow challenges; the tool was not originally tailored for complex clinical software; and unvalidated adaptations can compromise internal reliability. Purpose-built healthcare instruments, such as the Healthcare Systems Usability Scale (HSUS), were created specifically to capture workflow integration and patient‑safety dimensions that generic scores might overlook (Ghorayeb et al., 2023). Furthermore, recent health technology assessment (HTA) literature highlights that while validated usability instruments exist, variation in methodological rigor can limit their portability into formal HTA frameworks. Within MDRinAIS, our goal is not to treat the SUS as a final verdict, but as an efficient, standardized initial signal that highlights where clinical pathways require deeper qualitative investigation.

Workforce Readiness and the Propensity to Adopt

Adoption is ultimately a human challenge. In studies evaluating healthcare workers’ interaction with AI platforms, the strongest predictors of high usability scores were not merely technical features, but user attributes: belief in the tool’s clinical benefit, self‑reported familiarity, and digital literacy confidence. Feedback from these surveys can and should also inform industry providers (which apply similar scales in the development phases) on pitfalls and targets for improvement.

This reframes usability as a readiness challenge as much as an interface design problem. If confidence and perceived utility drive uptake, then structured training, transparent governance, and trust‑building are not optional add‑ons; they are the core levers determining whether an AI‑assisted stroke pathway is embraced. Ideally, healthcare institutions and policy makers could apply these to evaluate implementations in CDSS or health technologies in general.

This perspective lies at the heart of the MDRinAIS vision: the future of European stroke care will not be decided solely by algorithmic accuracy, but by collective capacity to render these tools usable, trusted, and seamlessly integrated into emergency workflows, keeping clinical judgment firmly at the center.

Share Your Experience: MDR in AIS Telestroke Usability Survey

Are you a clinician, radiologist, or stroke team member using telestroke or AI decision-support tools in acute care?

Help us map usability and workflow integration across European stroke networks. Your frontline perspective directly informs our stakeholder analysis and policy recommendations.

Take here the MDR in AIS Usability Questionnaire 

OR

scan the QR code below.