Richard Palfery, analyst in the GDS Assisted Digital team, reflects on his experience of recruiting research participants with assisted digital support needs.
One of the things members of our team have heard repeatedly over the last two years of digital transformation across government is that finding research participants with assisted digital (AD) needs is difficult. Digital services are for everyone, so it’s important that the needs of people with low online skills or those lacking access are represented in research and in the design of end-to-end services, including any support needs. We were running a pilot (see Pilot White Paper here) and wanted to recruit several participants with AD needs. The pilot was providing support for a universal service so we thought to save time we’d use a recruitment agency with experience of recruiting participants with AD needs. Sounds simple, right?
Doubtful agency responses
Bad news came quickly, when the first two agencies replied within a few hours. One said they’d like us to pay for them to research if it would even be possible to recruit this type of user in the area we wanted to run the pilot, before they’d be prepared to submit a bid to take it on. The other said they could recruit participants with low skills who were online but not those who were completely offline. They indicated that their database wouldn’t contain this type of person (presumably because it was compiled through online approaches such as email, which wouldn’t reach participants with the greatest AD needs) and they didn’t have good coverage in the areas we were aiming for (nobody local to do the face to face recruitment that would be required).
It was a demoralising moment. Doubt crossed my mind. Perhaps the brief was too ambitious? Perhaps the research was in the wrong area? Perhaps we should change the research design or expect to pay more? We had heard of services who had received poor service from recruitment agencies for this type of research and now we were getting it ourselves.
Don’t give up
Fortunately, John Waterworth (joint Head of User Research at GDS) was on hand, with other user researchers, to encourage us to hold our nerve. Our message to others in this situation would also be ‘Don’t give up!’. The good news came just a day later. Two agencies had submitted quotes for the full recruitment at prices within our expectations. We interrogated the cheapest one and the responses were positive. The methods they would use to recruit were appropriate. Face to face recruitment would be used in the local area of the venues where we needed participants to arrive. The recruiter had experience in recruiting participants with low online skills and an understanding of how to achieve a wide participant demographic (for example, not just recruiting older participants).
Was there a happy ending?
This wasn’t the end of the story. Some bad news came at the venues. The first four participants who arrived clearly had basic digital skills. One was an IT trainer and another ran an online business. Despite complaining and asking for participants with lower skills, at the second venue we had one person who had been submitting their tax online for years and most participants were already online, albeit with variable levels of skill and confidence.
This was disappointing, however, the good news was that we still had enough participants with AD needs to do our research with. Further good news came when the recruitment agency agreed to only charge for those that met the original brief. Thanks to a clear recruitment brief (and a bit of complaining), we got some really useful findings and saved money that we could use on further research in this area.
Is recruiting your own better?
Our key learning point was that it might have been better to do the recruitment ourselves. We discovered that the agency’s recruiter had gone for the obvious options, which we could potentially have covered more effectively ourselves as well as searching further afield. Some services have found this to be more effective as well as better value for money.
Follow Richard Palfery and John Waterworth on Twitter and sign up for our email alerts to read our future blogs about pilots and other assisted digital content.
4 comments
Comment by Peter Farrell posted on
Hi Richard
Next time you want to make contact with people who have assisted digital needs get in touch with Remploy. Peter Farrell or Mark Burrett can help.
Comment by Huw Pritchard posted on
It often is better to do it yourself. No matter how clearly you explain the project to another, no-one will understand it better than yourself.
The reasons to go external are if you lack the skill or resource.
Even then, going to an agency is a bit like getting someone to do your teeth for you. Just never quite as good
Comment by Simon Hurst posted on
Hi, we've had problems with this in the past. I now work with a superb lab who generally get me the right type of user. I wrote a blog about how we recruited people here: https://userresearch.blog.gov.uk/2015/02/13/how-we-recruited-people-with-lowno-digital-skills-on-carers-allowance/
I think a few people have adopted and adapted this approach and it seems to work fairly well.
It isn't an exact science, people always seem to say 'I'm no good on a computer', their definition of 'good' seems to be a technical architect level of skill!
Comment by Rory Macdonald posted on
We had the same issue with recruited AD users turning up on their iphones after going through TNS recruitment. We talked it through with the agency and rejigged the recruitment spec to make sure they screened users along certain criteria (not using Facebook, smartphones, etc). Second time round they came through with some really good low-confidence users (1-3 on the DI scale).
We've also had some really useful sessions with AD users by going into Jobcentres for user testing.