Building on the proof of concept
In the previous phase of this project, we produced a technical proof of concept for showing calculation information to providers in a data-driven allocation statement. We concluded that calculations can be data-driven. We also decided to focus on ‘factors’ (the values involved in the calculation) before ‘operations’ (the mathematical process being applied to the value) as this data was more readily available.
Our aim for this phase of work was to design, test and build a working minimum valuable product (MVP) that would:
- display data-driven calculation factors
- be scalable to funding streams outside of our adult funding pilot group
- meet provider needs for calculation information to complete their budgeting, forecasting and reconciliation tasks
- reduce effort for internal operators in production of separate calculation tools
- minimise development and operational effort for both implementation and maintenance
Focusing on factors
Our findings confirmed the technical feasibility of showing calculation factors in the allocation statement. However, we knew that more technical understanding would be needed of the data structures to make sure they could be displayed in a data-driven and user-friendly way.
Whilst our tech team and business analyst (BA) began investigations, the research and design teams set 2 objectives for testing:
- Understand what kind of layout would best support users to interpret and review calculation factors
- Understand the importance of ordering and grouping the calculation factors
We did this by using a round of user research to test 4 different design concepts, using existing GDS components. The concepts also had varying levels of ‘grouping’ or structure:
- Concept 1 presented all factors as immediately visible on the allocation statement page – this was also the least ‘structured’ version, presenting all factors in a long list, in the same order they would be received from internal databases
- Concept 2 used the accordion component to allow users to interact with calculation factors on a funding line by funding line basis
- Concept 3 used the details component to enable users to show and hide all calculations with one interaction
- Concept 4 explored using tabs to separate out the standard allocation information from the calculation information
Concepts 2, 3 and 4 were the more ‘structured’ designs – showing calculations grouped by funding line and then ordered in the same way they would be carried out. We knew these would be the most technically complex to implement.
We also needed to learn about which factors were most important to users. Internal databases contain a significant amount of data, so we needed to know what to prioritise. As a starting point, we showed data based on the current adult skills fund calculator. This is the bespoke tool that providers receive following their allocation to help them understand it in more detail.
Validating our assumptions
We were able to validate the following assumptions:
- Users are not interested in seeing all calculation factors
- Most user roles might be interested in calculation factors, but they generally do not need to dig into them unless there is a disparity between their forecast and their allocation
- Users want to download/export calculation factor data for their own processes
- Users typically investigate calculations when conducting long term forecasting and budgeting, or when their allocation does not match their forecast
We also found that although users found value in the least structured concept, they found calculation factors information most useful when grouped by funding line. They also found them most useful when presented in an order that they are familiar with.
Our findings led us to initially move forward with a grouped approach using the details component to show and hide calculation information. This allowed for users to look into the detail of how their allocation had been calculated, funding line by funding line as and when needed. Many users mentioned that they would need further explanation of what some of the factors were and would rely on the guidance for this.
For our next round of testing, we moved the guidance link above the calculation factors details component so that it would be more obvious to the user and not be lost when the calculations were expanded. We also further explored users’ interpretation of calculation factors and how they might benefit from them in their own internal processes.
Discovering constraints
Following our second round of research, our technical team had more capacity to work alongside us on the calculation factor implementation. This meant we were able to learn more about the data structures, solutions and potential blockers to achieving our preferred design.
We learned that existing data structures were not consistent enough to lend themself to grouped and ordered calculation factors. To achieve our initial design, a new technical solution would be needed, or there would need to be change to internal operator ways of working. Neither of which would be achievable for MVP.
The design and tech team worked with our BA to iterate the designs as investigations continued. We worked in a cycle of updating designs, referencing the user research findings, looking into technical and operational feasibility, and then iterating again as needed. We looked at hybrid approaches with varying levels and types of structure, but further tech investigations showed that these also would not be achievable for MVP. We therefore moved towards an ‘unstructured’ viable MVP solution, whilst ensuring we understood the value we could bring to users without grouping and specific ordering.
Our team agreed our MVP approach with the following rules: Calculation factors will only be displayed in the data driven pattern if they have an external calculation factor name and do not have a value of null or £0.
These rules mean that with the data-driven pattern, internal operators can determine which calculation factors are surfaced on allocation statements, and they appear in an unstructured list, with the order and user-friendly name decided by Funding Operations. (This is similar to the display of funding lines in the original beta data-driven statement.)
We planned to test a final time with our completely unstructured MVP design approach, but we were unable to complete the round due to low participant turnout. However, our findings from testing the less structured concept in round one gave us enough confidence to continue.
Next steps
In the next phase, our plan is to build on the calculation factor foundation by introducing calculation operations to the statement. This will be underpinned by reaching a technical solution for grouping and ordering calculations – which will allow us to address the lack of grouping in the factors MVP, and also some accessibility concerns around the density of information on the page and length of some of the factor names.
We will also test the pattern with different funding streams outside of our adult funding pilot group to further understand scalability.