webapp, NHS, GOV.UK Design System
The calculator has been produced to provide doctors and the public with more nuanced information about the risk of serious illness due to COVID-19 and to help patients and doctors reach a shared understanding of risk within the context of individual circumstances, risk attitude and the sorts of preventative measures people can take in their daily lives.
COVID-19
Risk Assessment
Risk Assessment
Task
As soon as possible, develop a web-based calculator interface. The uniqueness of the product was that the key users were the NHS employees who had not had time to explore new interfaces and products at that period.
The calculator was created based on GDS, taking into account their requirements and recommendations.
The calculator was created based on GDS, taking into account their requirements and recommendations.
Solution
The calculator consists of only three pages: Personal and medical information, result page.
The most simple interface and familiar patterns and elements allow users to fill out the questionnaire with high speed and accuracy.
The most simple interface and familiar patterns and elements allow users to fill out the questionnaire with high speed and accuracy.
Test
Purpose
The purpose of the usability tests was to gain an understanding of how end-users would use the platform in their role, identify any usability problems and determine users' satisfaction with the platform.
The purpose of the usability tests was to gain an understanding of how end-users would use the platform in their role, identify any usability problems and determine users' satisfaction with the platform.
Methodology
We conducted 7 remote usability tests. The sessions consisted of me as a moderator who asked the user questions and guided them through the tasks, a notetaker who recorded key insights and quotes and an end-user who answered the questions and completed the tasks. The sessions lasted for around 30 minutes.
To start, the users were asked a series of contextual questions about their role, device information and experience using the software. Following the contextual questions, we asked the user to share their screen and open the website (after we provided them with the URL) so we could see how they accessed the site and to capture their initial impression. Users were asked to think aloud as they explored the website and completed the tasks. We then asked them follow-up questions to gain more insight into their thinking and behaviour when interacting with the website.
To start, the users were asked a series of contextual questions about their role, device information and experience using the software. Following the contextual questions, we asked the user to share their screen and open the website (after we provided them with the URL) so we could see how they accessed the site and to capture their initial impression. Users were asked to think aloud as they explored the website and completed the tasks. We then asked them follow-up questions to gain more insight into their thinking and behaviour when interacting with the website.
Participant Profiles
Occupational Health, medical consultants. Use the calculator to predict staff risks, to inform a personalised discussion between an employee and their employer regarding their occupational duties.
Academics. Use for academic research or for the purpose of peer review.
Clinicians, GPs, NHS national deployment. Use the calculator in clinical trials to recruit people into a study. To predict patients’ risks.
Academics. Use for academic research or for the purpose of peer review.
Clinicians, GPs, NHS national deployment. Use the calculator in clinical trials to recruit people into a study. To predict patients’ risks.
Tasks and Scenarios
We created two types of tasks – calculating risk and finding information – for the scenarios below. All the calculation tasks involved the user entering personal and clinical information for persona cases, so no real information of the user or patients was used.
Clinical use scenario
Clinical use scenario
- Calculation task: calculate a patient’s risks and interpret the result information
- Finding information task: identify and show an understanding of information on the FAQ page and privacy policy page.
- Calculation task: Calculating and comparing risks for two patients
- Finding information task: Understanding of the License and the Algorithm information.
- Calculation task: Calculating “own” risk using persona details
- Finding information task: identify and show an understanding of information on the FAQ page and privacy policy page.
Limitations
Desktop/tablet testing: most of the users accessed the website using their desktop, laptop or tablet devices. We only had one user accessing the website using a mobile device.
Familiarity with the website: half of the users had seen the website before meaning that the findings are unlikely to reflect an accurate initial first impression.
Lack of ‘public use’ users: we didn’t have any users that fell into the ‘public use’ category for the moderated usability test. However, we used a proto persona for ‘public use’ and was able to test the corresponding task and scenario. We had 1 ‘public use’ user in the unmoderated usability test.
Familiarity with the website: half of the users had seen the website before meaning that the findings are unlikely to reflect an accurate initial first impression.
Lack of ‘public use’ users: we didn’t have any users that fell into the ‘public use’ category for the moderated usability test. However, we used a proto persona for ‘public use’ and was able to test the corresponding task and scenario. We had 1 ‘public use’ user in the unmoderated usability test.
Findings
Overall, the feedback from users was positive. The findings show that the website has a high level of user satisfaction and usability and that users were able to navigate through the website, although ‘Calculation’ didn’t appear to be an intuitive header name.
The main comment that all users seemed to agree on is the simplicity and clarity of use.
Generally, the users found the calculator straightforward and easy to use. Users were able to understand the information on the ‘Risk assessment results’ screen however there was some confusion and feedback relating to specific parts.
Where there were issues, we’ve attached a severity level based on Nielsen’s (1993) categorisation.
The main comment that all users seemed to agree on is the simplicity and clarity of use.
Generally, the users found the calculator straightforward and easy to use. Users were able to understand the information on the ‘Risk assessment results’ screen however there was some confusion and feedback relating to specific parts.
Where there were issues, we’ve attached a severity level based on Nielsen’s (1993) categorisation.
Approximately 2.2 million people with pre-existing medical conditions thought to be the most serious were deemed 'clinically extremely vulnerable' during the first wave of the COVID-19 pandemic in England. These individuals were advised to shield themselves from the virus by staying at home.
www.qcovid.org
www.qcovid.org