User research round 5 - standards 9, 11 to 14

All points raised in this post have been made to all 14 standard points where relevant. In addition, there were other minor content changes made to the standards tested in this round of research, that haven’t been highlighted in this post.

What we did

We asked user research participants to conduct unmoderated highlighter testing on the standard points 9, 11, 12, 13 and content of what was clear and unclear to them. We then had a follow up call to discuss their highlights and comments. Additionally, we measured users’ confidence of applying the standards before and after they read the content during research. This is the second round of testing these standard points.

Users included civil servants and contractors in architecture and developer roles across DfE.

What we found

Users told us that on the standard points, it jumped straight into the content with no introduction to what it was, making it difficult to read and understand.

What we’ve done as a result

We’ve added a H2 title ‘summary’ at the beginning of each standard, followed by the content outlining what is required for each standard point.

What we found

Users noticed that some standards said ‘what to do at each phase’ whilst other standards said ‘what to do at every phase’. This was due to some standards being broken down into discovery, alpha, beta and live whereas others the points raised were relevant to all phases. Despite this, the inconsistency caused confusion.

What we’ve done as a result

We have changed each standard point to say ‘what to do at every phase’ to ensure consistency and it's still relevant whether it’s broken down by specific phases or not.

What we found

In all of the rounds of research, participants told us to find what they need to do for each phase, they would need to go through each standard and copy and paste the points elsewhere to have a complete list of what is needed at discovery, alpha, beta and live.

What we’ve done as a result

We have included a spreadsheet file in the service that breaks down all the service standard points requirements at each phase. Users will be able to download and use this to track their progress.

Measuring users’ confidence survey

We measured users’ confidence of applying the standards before and after they read the content during research. We found that the average rating of confidence on standards 9, 11 to 14 had increased once users had read the content.