User research round 4 - standard 10

All points raised in this post have been made to all 14 standard points where relevant. In addition, there were other minor content changes made to the standards tested in this round of research, that haven’t been highlighted in this post.

What we did

We asked user research participants to conduct unmoderated highlighter testing on the standard 10 content of what was clear and unclear to them. We then had a follow up call to discuss their highlights and comments. Additionally, we measured users’ confidence of applying the standards before and after they read the content during research.

Users included civil servants and contractors in business analysis, performance analysis, delivery management and product management roles in DfE.

Please note, we conducted this round of testing in parallel with the third round of research so some similar findings were made in both rounds.

What we found

The ‘DfE assessor tips’ box caused confusion, as many users read it as the content being specifically for assessors, whereas the rest of the content on the page is for service teams.

In addition to this, users were unsure of its relation to the ‘things to consider’ points and questioned whether they would still meet the standard if they only actioned ‘things to consider', as there were further considerations in the ‘DfE assessor tips’ box.

What we’ve done as a result

We’ve removed the ‘DfE assessor tips’ box and the content within it, as users felt the points in the box were addressed elsewhere in the content or were not relevant to standard 10.

What we found

Listing all the profession manuals at the end of each standard raised questions from users, as they felt only professions that had relevance to the standard should be listed. This was further amplified as the standard 10 content is aimed at business and performance analysts, however their community spaces are not linked out to.

What we’ve done as a result

Only the relevant profession manuals are linked out to each standard point. In addition, we are working with the business analyst community to add their community space to the manual.

Measuring users’ confidence survey

We measured users’ confidence of applying the standards before and after they read the content during research. We found that the average rating of confidence on standard 10 had increased once users had read the content.