The AI response page is a key part of the Find User Insights (FUI) service. It helps users quickly understand research findings, explore the underlying documents and confidently reuse insights in their own work.
The page has 3 main elements:
- a disclaimer that introduces and frames the AI-generated summary
- an AI-generated summary
- the source documents and metadata
The challenge
Research highlighted 2 main issues.
First, users found some responses difficult to work through. Long summaries and metadata created effort, particularly in longer conversations. This was especially true for neurodivergent users:
"The wall of text feels long and overwhelming."
Second, users were sometimes unsure about how to interpret and use the responses. They had questions about the role of AI, the limits of the summaries and their own responsibility when using the research:
"Can I use the research or not?"
"I'm surprised that we're not declaring use of AI."
A subject matter expert also noted:
"The information is presented as if it's the whole truth, when in fact a lot is missing."
Together, these findings showed the whole response experience needed improvement.
We explored improvements to AI summaries
We started by reviewing and revising the prompts used by the AI with the aim of aligning the summaries with the GOV.UK style.
To understand the impact of the changes, we first trialled them in ChatGPT. We compared existing summaries with versions generated using revised prompts and assessed readability using Hemingway.
The revised versions were clearer and scored better for readability.
This first example shows an original AI-generated summary from FUI:
This second example shows same summary with the new prompts applied using ChatGPT:
We then tested the same prompts in the live FUI service. The summaries became slightly easier to read, but the improvements were much smaller than we had seen in ChatGPT.
We later investigated why.
Unlike ChatGPT, FUI is not simply rewriting text. It first has to find relevant research before generating a summary. This means that FUI has to balance several tasks at once, including retrieving relevant information, following instructions and linking the summary back to the source documents. This reduced the impact of the writing-style prompts we introduced.
This reinforced that prompt design is only one part of the solution. The quality of an AI-generated summary depends on the whole AI system, including the research it finds, the instructions it follows and how those requirements are balanced.
This third example shows the AI-generated summary in the FUI live service after the prompt updates:
We reduced cognitive load
Improving the AI-generated summaries was only part of the solution. Research showed that the wider response page also created effort for users. Users felt fatigued when scrolling through long conversations with multiple AI responses.
The metadata section under each summary contributed to this. We redesigned it to reduce cognitive load by:
- turning document names into clickable links
- removing the separate “Open document” button
- simplifying the layout to use less space
These changes created a cleaner page and helped users reach source material faster.
Users needed more confidence in the responses
Research also showed that users were unsure about how much they could trust the responses.
Much of this uncertainty stemmed from the disclaimer that introduced the summary, so we focused on improving how it explained the role of AI and how users should use the summary.
We made it clear that:
- summaries are generated using AI
- summaries are a starting point for exploration, not a final answer
- summaries come from underlying research
- source documents include important context and limitations
- users should check original research before making decisions
We also strengthened supporting content across the service by:
- making research standards clearer and more visible on the homepage
- creating a dedicated “About the research in this service” page explaining standards, processes and checks
- adding explanations about the use of AI
What we learned
Improving the AI response page involved more than adjusting a single element.
Research showed that users experienced the page as a whole. Long summaries, lengthy metadata sections, uncertainty about AI use and questions about how to use the research all affected how people felt about the experience.
We therefore made changes across the response page, focusing on clearer summaries, simpler supporting information and a more transparent explanation of the AI.
Together, these changes made the response page easier to use and helped users understand how to use AI-generated summaries appropriately. More importantly, they showed that the quality of an AI experience depends on both the underlying AI system and the surrounding user experience. Both contribute to making responses useful and trustworthy.