Questions
Iteration 1:
The use of technical terms in questions on broadband, back-up broadband, network cabling, network switching and wireless network did not test well with users. The terms were found to be confusing and prevented users from answering the questions quickly, or even at all, in some cases.
Iteration 2:
Hint text was introduced to help uses understand the terminology in the question. While this helped with some questions, it did not solve the confusion completely. In some cases, the hint text was missed.
There was also concern over questions that asked schools to make a judgement. For example, ‘Is your wifi slow?’. It was pointed out that ‘slow’ is too subjective and what one person experiences as slow, could be an acceptable speed for another.
Iteration 3:
The questions were reworded to remove technical terms.
Hint text was included only where relevant.
Questions asking schools to make a judgement were reworded or changed to focus on factual outcomes
Questions that did not give additional insight into a school’s maturity level were removed.
To keep questions as short as possible, some answers were expanded to include more information.
Recommendations format – learnings taken forward from alpha
During alpha we established a user need for recommendations and specific actions to support schools with the planning process.
Users were given 3 options:
- Recommendation header, plus approximately 300 words giving actions to take in paragraph format.
- Recommendation header, plus:
- Several sub-headed paragraphs explaining dependencies, why the recommendation is important and what it can help schools to achieve.
- Stepped list of actions to take.
- Recommendation header, plus:
- Graphic showing cost of project (low, medium, high),.
- Graphic showing difficulty of project,
- Several sub-headed paragraphs explaining dependencis, why the recommendation is important and what it can help schools to achieve.
- Stepped list of actions to take.
Users were initially shown option 1:
This was received well as many users were not expecting to get any recommendations at all.
Users were then shown option 2:
Most users preferred the additional information. They valued the fact that they were told about other considerations, and what it could help them to achieve.
Users were then show option 3:
This did not compare favourably. The main issue was that the graphic was not in context. For example, users stated that installing broadband was judged to be high cost and difficult. What they wanted was more specific information about what would be considered a high cost for installing broadband.
For the reasons given, option 2 was considered the optimum approach for recommendations.
Creating recommendations
To create the recommendations, the content design (CD) team consulted with subject matter experts (SMEs). At least 2 scoping sessions took place for each category, namely:
- broadband connection
- back-up broadband
- network cabling
- network switching
- wireless network
- leadership and governance
The purpose of the scoping sessions, for each category, was to establish what schools look like that are:
- currently thriving and fit for the future – deemed high maturity
- getting by but struggling in some areas or unable to scale behaviours and likely to encounter future problems – deemed medium (or mid) maturity
- struggling in most or all areas, with technology hindering teaching and learning – deemed low maturity
This enabled a judgement to be made as to where each school was in terms of maturity and to determine actions needed for schools to become more digitally mature.