In this post, standard 2 is being used as an example. The changes listed have been made across all 14 standard points.
What we did
We conducted 8 user research sessions to test users’ comprehension and the content of service standards 1 to 5. Additionally, we measured users’ confidence of applying the standards before and after they read the content during research.
Users included user researchers, designers, delivery, and product managers in DfE.
User needs
Service teams need to know how to apply standards to their work So that they are confident they’re doing the right thing in their profession
Service teams need to know how standards apply at each phase, So that they know what to focus on
Service teams need standards in the DfE to be clear, measurable and understandable So that they can consistently apply them to each service
Service teams need to understand the rationale behind standards So that they can understand why they are applying them
We are using point 2, solve a whole problem for users, as an example of design changes made across points 1 to 14.
What we found
Users were immediately drawn to the orange box and recognised the importance of the message. However, some users felt it was a call to action or a warning.
What we’ve done as a result
Removed the orange box with the ‘You’ll be assessed on this’ heading and added box content underneath heading 2, Why it’s important.
What we found
Users liked the think about box as it brought together the standard across the phases.
What we’ve done as a result
We've now incorporated this across all other 14 standard points using data from assessment reports to write relevant content.
What we found
Users appreciated the soft language of 'what assessors will look for', however, felt it was too focused on assessments.
What we’ve done as a result
Changed ‘What assessors will look for’ to ‘Things to consider’ to reduce focus on assessments and to be more in line with ‘Things to avoid’.
What we found
To make it more applicable and actionable, users expected to see more text linked out to enable them to do what the content was highlighting.
What we’ve done as a result
Added relevant links to content to provide context where needed, guidance and internal DfE resources and references.
What we found
Users had to remind themselves which phase each 'things to avoid box' referred to. By not having the phase outlined on the box, meant users were scrolling up and down.
What we’ve done as a result
Added each phase to box heading, for example, Things to avoid in discovery, so that the boxes can be read out of context and it is clear which phase the points relate to.
What we found
The related content links were seen by all users, however users felt it was more appropriate to use the links within the content where it is most relevant.
What we’ve done as a result
Removed related content from the end of each standard and embedded relevant link content within content where it is more relevant.
What we found
It wasn't clear to all users, who in the team was responsible for points listed in the content, and felt it would help. However, others recognised that they were the entire team's responsibility.
What we’ve done as a result
Added tags to each standard to show which disciplines in the team the standard point is most relevant to.
Measuring users’ confidence survey
What we found
We measured users’ confidence of applying the standards before and after they read the content during research. We found that the average rating of confidence on all standards increased once users had read the standards content for 1 to 5. Users told us they felt more knowledgeable and they could practically apply Service Standard points 1 to 5.
| Service standard point | Before | After |
|---|---|---|
| 1 | 2.6 | 4 |
| 2 | 2.5 | 3.8 |
| 3 | 2 | 3.5 |
| 4 | 2.9 | 4.1 |
| 5 | 3.1 | 4.3 |
What's next?
We will test standards 6 to 8 in the next round of research with delivery and product managers.