Pilots are small-scale pieces of research with a limited number of a service’s users. They are a great way to test and iterate elements of your service. They let you test out processes for their feasibility in practice, interact with users and get their feedback before these processes are scaled.
This agile way of working can save lots of time and money in making changes later. It can also lead to better outcomes for users.
Over the past 2 years GDS has helped with a dozen assisted digital pilots to test support to government digital services. Amongst others, we’ve worked with the Carer’s Allowance team at the Department for Work and Pensions. And the Student Loans Company on the Apply Online for Student Finance service.
Here is some advice on how to run an assisted digital support pilot.
Research first
Before starting you should research the sort of support you will offer. Ideally, this research will be part of the discovery into the service, as you should consider Assisted Digital support right at the start. Then you can run the pilots as part of the alpha.
Things to consider:
- should the support be walk-in or appointment based?
- will the support provider need training in advance of the pilot?
- what kind of preparation should participants be given before they arrive?
- what format should this take?
- in which part of the country would it be best to run the pilot?
- who should be providing the support?
- what questions should we ask participants to get the best findings?
Find users (the right users)
User recruitment is key to a pilot. Without users there is nothing to research. But finding the right users to run your pilot with can be tricky.
It is hard enough recruiting people who need to be supported through digital services. That’s before you get to other issues such as pilot timing, specific locations and getting enough people through to make it worthwhile.
But there are ways to address this. You can use recruitment agencies, although there are obvious cost issues and you may need to work closely with the recruitment agency for a successful result. Our previous blog post covers issues with recruitment agencies.
Partner organisations can also help, but it’s important to give them lots of notice. We’ve found that some can help in 3-4 weeks, but others need longer.
You can also go to the government user research community for ideas, or look for referrals from within the service or from other public services.
Get the right location
If the pilot is going to be face-to-face, you’ll need the right location. This should be convenient for all concerned.
Room hire costs tend to be relatively low, compared to the overall cost of the pilot, but be prepared to arrange payment in good time to keep partners on side.
Make sure you have user researchers
Another key thing to consider is the availability of user researchers. It is possible to run parts of pilot research without user researchers (user research is a team sport after all!). Richer findings, though, come when user researchers are involved throughout.
Have a back-up plan
A key part of running a pilot is to have a reliable service to test with. It is definitely worth arranging for a backup URL, someone to phone, or an offline user journey mock-up to draw on as a last resort.
Arriving at the venue and testing the service before the start is the bare minimum. In our experiences we’ve been quite lucky. We’ve lost a few hours to services being down at critical moments, but never days, thanks to a wary approach. Bearing in mind the expense of a pilot, it’s worth having a back up plan.
Keep your partners in the loop
Pilots usually have multiple partners. All of these partners need to know what’s going on, when, and what’s expected of them. It’s better to communicate too much than too little, so regular telephone conferences – and ideally face-to-face meetings – will prevent an ‘us and them’ situation.
If you don’t have a joint Trello board or similar agile planning tools, it’s a good idea to prompt with brief notes on actions from these meetings to keep everyone clear on what’s required.
It’s also important to note that building relationships with partners is a key part of the process. Relationship building also often leads to further work together later. In our case this has included further pilots!
Be agile
Along with communication, the ability to adapt to changing circumstances has also been essential.
We’ve been lucky enough to have worked with some fantastically helpful and agile partners. An example of this is when user recruitment became an issue on one of these pilots. This was discussed at an early stage, and all of our partners came up with innovative ways to boost recruitment, which ultimately saved the day.
Another example of agility is when a new theme around touchscreens emerged during research at our first Walk-in pilot venue. Our partner pulled out the stops and large touchscreens were in place in record time to research this further at the second research venue.
Share your experiences
Piloting service support with users is hugely useful in planning the full-scale support required and there is still so much to learn. Please keep sharing your experiences. We'll keep collating these and updating the service manual in light of the findings.
Follow Katie Regan and Richard Palfery on Twitter and sign up for our email alerts to read our future blogs about pilots and other assisted digital content.