In this post, standard 11 is being used as an example. All points raised in this post have been made to all 14 standard points where relevant. In addition, there were other minor content changes made to the standards tested in this round of research, that haven’t been highlighted in this post.
What we did
We conducted user research sessions to test users’ comprehension and the content of service standards 9, 11, 12, 13 , 14. Additionally, we measured users’ confidence of applying the standards before and after they read the content during research.
Users included civil servants and contractors in architecture, developer, delivery management and product management roles in DfE.
Please note, we also conducted our fourth round of testing in parallel so some similar findings were made in both rounds of research.
What we found
Initially users struggled to see how this related or was different to the Government Service Standard on the homepage until they went into the standards content.
In addition, users recognised that only some professional manuals were linked out to on the homepage and suggested the architecture and tech manuals to be added here too.
The colour of the homepage was felt to be too bright and striking to some users.
What we’ve done as a result
We’ve changed the hero content, ‘Guidance and information for how to apply the Service Standard to design and build accessible, usable and understandable services,’ to ‘Guidance and information for how to apply the Government Service Standard to design and build accessible, usable and understandable services.’
All current profession manuals in DfE have been added to the homepage.
The tone and colour of the homepage has been lightened.
What we found
The profession tags indicating who the content is most relevant to were received positively by users, however, users highlighted that there was an inconsistency in listing the roles, as some were personified, whereas others were a broad category, for example, ‘technical’ or ‘architecture’.
What we’ve done as a result
We have personified all the role tags to ensure consistency and so it’s clearer for users to understand which specific roles the content is relevant to.
What we found
The ‘tips for assessors’ box caused confusion as many users read it as the audience of the box being for assessors, whereas the rest of the content on the page is for service teams.
In addition to this, users were unsure of its relation to the ‘things to consider’ points and questioned whether they would still meet the standard if they only actioned the ‘things to consider’ points, as there were further considerations in the ‘assessor tips’ box.
What we’ve done as a result
We’ve removed the ‘think about box’ and added the relevant content into the ‘things to consider’ bullet points, as this was the place that users found the most value.
What we found
Users felt the practical examples and links out to how to action the points in the things to consider sections were the most valuable elements of the content to enable them to meet the standard.
What we’ve done as a result
We’ve worked with subject matter experts in the department to add DfE specific guidance and examples of what good looks like in the areas users raised in the research.
Measuring users’ confidence survey
We measured users’ confidence of applying the standards before and after they read the content during research. We found that the average rating of confidence on standard 9 remained the same after users read the content, whereas for standards 11 and 12 confidence increased, however confidence for standards 13 and 14 slightly decreased.
| Service Standard point | Before | After |
|---|---|---|
| 9 | 3.75 | 3.75 |
| 11 | 3.63 | 4 |
| 12 | 4 | 4.13 |
| 13 | 4 | 3.75 |
| 14 | 3.63 | 3.50 |
What we’ll do next
We have now tested all standard points. Due to the quantity of changes and the mixed confidence ratings after users had seen the content, we will conduct another round of research testing on standards 9,11, 12, 13, 14.