The main reasons we built the user research studio was to increase the amount of research with participants and reduce the amount of spending on commercial research studios.
The main benefits of building our own research studio are:
-
Having our own lab space will give projects more opportunity to test more iterations with users. This leads to delivering more user focused digital services.
-
Reduce the load on the GDS user research studios. It is a wonderful, but finite resource for all of government.
-
Projects which have no funding for hiring commercial studios or can't access the GDS user research studio will be able to use the lab to test their services, helping to ensure that we can improve all services.
-
Having labs on Home Office buildings will reduce lots of travel to labs.
-
Building our own lab from scratch will give us precise control over the specifications and won’t leave us dependent on whatever facilities are at existing sites.
-
If there are times when the lab is not being used, it could be offered to other departments which would increase x-gov community collaboration.
We contacted and visited a number of government and commercial research studios. These ranged from small ad-hoc labs to academic labs to high-end expensive studios in Internet companies.
We learned a lot from all of these labs and are very grateful to the people who shared their expertise with us.
After carrying out this research, we learned a lot of things. We had 5 main options, ranging in facilities, and cost.
This would mean continuing to use external research studios for our usability testing sessions - a mix of commercial studio and GDS research studio time.
This would have avoided spending on any equipment and building space. This could be seen as the main benefit.
However the main drawback would be, it would have meant continuing to pay for commercial space, and trying to get time in the very busy GDS facility. The cost for this would increase as more research with participants was being carried out.
Another considerable drawback would be projects would still need to find budget for ongoing research. Providing an internal studio would mean providing facilities at lower cost.
Another would be at busy times it would have meant possibly not finding availability in the already busy studios.
This would mean building our own lab, probably in a Home Office building somewhere, or maybe on another departments buildings, if we explore the collaboration option.
From research into modern commercial off the shelf equipment, the cost here is under £10,000. This would heavily reduce our dependence on external labs, allow us to carry out more research and save on expenses.
The philosophy of this approach would be the researchers doing the research would manage the lab themselves. This may not be possible for all, and so may need someone to manage the lab.
This would allow us to avoid most, or possibly our entire annual commercial lab spends.
- Ongoing commercial lab costs avoided
- More lab access
- Having the studio located in a Home Office space would reduce costs further.
- Lower equipment costs means less risks
- If it was in a Home Office building it would reduce the need for travel
- Support costs
- Have to find initial capital budget
- Must arrange space for the lab
- Concerns over the technical use and stability
- Concerns that the spec may not be adequate to replace external labs.
This level of lab would be a more sophisticated setup, incorporating dedicated hardware audio and video mixers, a number of cameras and screens and supporting ancillary equipment.
While not as high spec as some of the very expensive labs, this option should do most of what researchers need, most of the time and should allow for the use of external, commercial labs to be all but eliminated.
However, it would have higher associated setup and running costs and may be technically more difficult to use than the simpler solutions or using existing lab space.
From carrying out research with commercial studios and other government studios this approach could cost approx. £40,000. It would also require ongoing support.
-
Ongoing commercial lab costs avoided
-
More lab access
-
Better specification will give researchers better data and reduce external need
-
Located in Home Office buildings so less travel required
-
This level of spec would start to become more attractive to other departments and hence has more reuse and recharging capacity, should there be spare time available in the lab.
-
Support costs
-
Have to find initial capital budget, relatively large budget
-
Must arrange space for the lab
-
Concerns over the technical use and stability
-
Concern over difficulty of use
A commercially built professional studio can cost more than £100,000 (as of may 2016) http://rikwilliams.net/ux/user-research/user-research-labs-designing-building-managing/
https://gds.blog.gov.uk/2014/12/03/our-gds-user-research-lab-is-6-months-old/
This solution represents the most highly specified lab, it will have multiple tilt and pan cameras,
HD footage, large viewers, different devices, and likely storage and editing facilities. At the
lower end is the Google style bespoke solution, still built around dedicated hardware mixers and
HD equipment, ranging up to the DMS full solution, which is on G-Cloud. Although this is likely
to be as much lab as anyone in government could ever want, there must be some debate about
whether it is too feature rich for the cost. In addition, setting up streaming and support is likely to
be very expensive for these options. The actual cost will likely range from between £80,000 to
£250,000 but realistically we’d probably expect to be paying c. £160,000 plus support and
streaming costs for this.
Please note, on the financial modelling tables, I have divided this into two categories, an
advanced lab, which is approximately equal to what Google have, i.e. mostly a hardware
solution, and a de luxe lab, which is essentially the latest DMS offer as at DVLA / Tesco and
The Guardian. We can see that this affects how quickly the investment can repay itself.
- Ongoing commercial lab costs avoided
- More lab access
- Better specification will give researchers better data and reduce external need
- Having labs on Home Office buildings will reduce lots of travel to labs
- Very high interest in reuse by potentially rechargeable other agencies.
- Support costs
- Have to find initial capital budget, very high capital costs
- Must arrange space for the lab
- Concerns over the economic justification
- Concern over difficulty of use without dedicated lab manager.
This works with any option between 2 and 4, but is obviously particularly well suited to options 3 and 4, it would reduce many of the risks, give us more accommodation options and allow us to share the costs.
On the downside, if we collaborate, we reduce our dedicated lab time.
However, it may just be too long and slow to go down this route if we need the lab within the relevant time.
Concerning the two viable options there are a few remarks to be made about recommending one or the other. In essence, it is a conflict between features and affordability.
Option 2 will give us the minimum specification, it will fulfill the lowest specification requirements and because it is fairly cheap, it will give some security over the financial envelope should, for whatever unforeseen reason, the amount of costs being avoided by in-sourcing lab tests fail to materialise in the real world.
It therefore poses the least risk and as an additional benefit, it will be less complex and more user friendly than the other do something options and may obviate the need to recruit a full-time studio manager.
On the downside, the level of facilities that this can offer is barely above what we might be able to get from just setting up a couple of laptops and cameras on an extemporary basis and has considerable technical risks around drivers and stability issues. Also, this would leave little budget for important other features like large screens in the observation area which is so critical for engaging stakeholders, leadership teams and project workers. Moreover, the quality of the footage captured is likely to be lower than other options and run the risk of not providing the best quality research to support developing and improving services. There must be some concerns too, over whether the quality produced by this solution would be sufficient to allow projects to avoid using external labs. If it is not then we lost a key justification for pursuing this option.
Option 3 will give us good quality feeds and the ability to install more cameras and sources. It should provide adequate quality for most researcher needs and yet still be highly affordable and pay for itself within the likely lifetime of the equipment. It would also allow us to have some of the other useful equipment in the observation area like large screen TVs, storage and other facilities as well as being more generous in terms of allowing for other spending on tools like accessibility solutions.
As a more highly specified lab, this would also be more capable of appealing to external users, such as other Government departments and agencies. This sort of arrangement might be amenable to recharging, so this configuration could also generate a better income stream, in principle. It also gives us a chance to prove that a lower specified lab can provide most of what user researchers require at a lower price point, which could showcase a way of saving money for other agencies.
There are drawbacks, it clearly won’t pay for itself as quickly as the cheaper option and represents a higher initial cost of investment. It’s not as much of a one stop shop solution as the larger lab specs provide and this still might introduce the risks of having to go external on some occasions, though this risk is much smaller than with the previous option.
As a more sophisticated lab it is likely that it would require more technical support at greater expense and also that it may require more personnel intervention, for instance, if no lab supervisor, we might need some volunteer(s) to support the lab in a local expert role.