After releasing changes to support January course start dates in Apply, we carried out user research to test whether applicants understood the new designs.
The original work introduced separate recruitment timelines for September and January courses. While this addressed a service limitation that added administrative burdens for providers, we wanted to understand whether users could confidently interpret the new information and whether any content created confusion.
What we wanted to learn
We wanted to understand whether applicants could:
- identify which deadlines applied to their applications
- understand the difference between September and January recruitment timelines
- understand what happens to applications after September
- understand how application limits work when applying across different start dates
- complete key tasks without needing support from providers
How we carried out the research
We tested the content and designs using unmoderated research in Qualtrics. Participants completed a series of scenario-based tasks using screenshots of the application dashboard and supporting content.
The scenarios followed an applicant's journey across the end of one recruitment cycle and the start of the next. Participants were shown content and dashboard states that reflected different points in the recruitment timeline, including changes to application statuses and deadlines for September and January courses.
Participants were asked to:
- identify content and dashboard elements they found clear or confusing
- explain why they found particular content confusing
- answer questions to test their understanding of application deadlines, recruitment timelines and application limits
- complete tasks using information shown on the dashboard
We analysed responses using Qualtrics’ internal analytics dashboard to identify areas of confusion and assess how well participants understood the new recruitment timelines. We reviewed both quantitative ratings and qualitative feedback to understand the reasons behind any misunderstandings.
We also analysed responses across different applicant groups. Understanding was broadly consistent across demographics, with only limited variation between groups.
Finally, we combined findings across related areas of the journey to identify which content and dashboard elements caused the most confusion. We analysed results across different applicant groups and found few differences in understanding. Comprehension was higher for one question among UK-based applicants and those further along in their application journey. This helped us prioritise changes to deadline messaging, application limit guidance and navigation to previous applications.
What we learned
January and September recruitment timelines were generally understood
While participants generally understood that January courses followed a different timeline, comprehension levels were lower in certain areas.
Some participants incorrectly believed that:
- all applications would automatically be rejected in September
- they could continue applying for January courses while Apply was closed
- January applications followed the same rules as September applications
The research showed that users were not always able to identify which deadlines applied to which applications. In several cases, participants interpreted messaging about September deadlines as applying to all applications regardless of start date.
Course start date headings were unclear
Participants did not consistently understand headings such as 'Courses starting by January' and 'Courses starting by September'.
Some participants interpreted these headings differently from how they were intended. The phrasing created uncertainty about whether courses started during those months, before those months, or by a particular deadline.
Application deadlines needed more context
Participants were often unclear whether the deadline referred to:
- submitting an application
- completing an application
- having an application reviewed by a provider
Some users assumed activity would stop completely after the deadline passed. They did not always understand that submitted applications could remain active and continue progressing through the recruitment process.
Application limits were difficult to understand
Some users found the application limit messaging confusing.
Participants were unsure:
- what period the limit of 4 applications applied to
- how this worked when applying for courses with different start dates
- why information about a maximum of 15 applications was shown
The additional information about the overall maximum limit often distracted from the more important information users needed to complete their task.
Previous applications were difficult to find
Only a small number of participants successfully found the "View your previous applications" link.
The link's location under support content meant many participants overlooked it completely during testing.
Changes we made
Clarified course start date headings
Based on research findings, we updated section headings so they more clearly reflected course start dates and reduced ambiguity.
Instead of using phrasing that could be interpreted in multiple ways, we explored headings that aligned more closely with how users understood course timings.
Made deadline messaging more explicit
We revised content relating to application deadlines to explain that deadlines refer to the latest point at which users can submit a new application.
We also made it clearer that submitted applications can continue progressing after these deadlines and do not automatically stop being considered.
Simplified application limit content
We simplified content about application limits by reducing unnecessary detail and focusing on the information users needed most.
We reviewed whether information about the overall maximum number of applications was helping users make decisions and removed or reduced emphasis on content that created confusion.
Improved consistency across deadlines and statuses
Research showed that users sometimes struggled to understand how deadlines related to different application statuses.
We reviewed the language used across dashboard messages, status information and supporting guidance to make terminology more consistent. This helped reduce the amount of interpretation required from applicants.
Made previous applications more visible
We moved the "View your previous applications" link to help users locate it more easily.
This change was intended to improve discoverability without introducing additional complexity to the dashboard.
What we'll do next
We will continue monitoring how applicants interact with recruitment timeline content and assess whether further support is needed for courses that do not follow the standard September recruitment cycle.
We will also continue exploring whether future design changes should support additional course start dates and recruitment timelines beyond January.