As I have noted in previous posts, there is comparatively little information about the value and best practice of user testing in back-office systems, where the users are not ‘customers’, but employees using the software as part of their work.
In this post I share some of the things I have learned through my own experimentation in this area.
Why user testing?
Awareness of the importance of user testing is growing. Testing early versions of your product, your prototype, even your design or sketches of your ideas, has proven to be invaluable for designing products that people actually want and can use. There’s plenty of resources out there discussing best practice in this area and even 3rd party services you can use to perform the user tests remotely (such as usertesting.com). The earlier this testing is started, the quicker your design and even concept gets on the right track to success. Iterating through a cycle of design, test, design, test can save a great deal of development time by getting these things right in the beginning.
However, most resources focus on websites, apps etc that are aimed at customers or public users. There are far fewer resources discussing best practice for user testing on back office systems. But these systems often represent a substantial capital investment for companies in terms of development, and getting it right from the beginning can mean less money spent on the project and also better results, such as improved productivity, increased sales and higher staff morale.
User testing on back office systems
I am currently carrying out the role of Product Owner on a project for one of 2 Aardvarks’ travel industry clients. Alongside other 3rd parties, we are designing and building a holiday quoting tool that will be used by our client’s sales teams across the world. The legacy system is neither intuitive nor particularly usable, and requires a great deal of training to get new users up-to-speed. Our aim is for the new system to be intuitive, usable and fairly self-explanatory, to keep training need to a minimum.
Since being involved with project, I have worked to learn more about the area of usability and user testing, through books, articles, blogs, webinars, UX events and even gaining a usability qualification from Nielson Norman Group. The UX community is very generous in sharing knowledge and best practice.
It has been interesting try to apply what I have learned, in the context of back-office systems. Here are a few of the things that I have discovered so far…
1. Remember your goals are different
Normally in user testing, you give a novice user a task to perform and they have to work out what to do. You are testing how intuitive the interface and process are for someone coming in cold with no previous experience, and they normally have to work it out on their own. With a back office system, users will normally be using the software on a daily basis and will likely receive some sort of initial training, so novices are not always the right users to test on. Remember that the usability you are aiming at may be that a user can enter a lot of data quickly, or use the system effectively whilst on the phone to a customer, rather than the system being intuitive on first use.
2. Involving real users too early can be a risk
When it comes to implementing a new computer system in an organisation, I always say that at least 50% of the effort is on managing the organisational change itself – stakeholder and user expectations, concerns over change, training etc. The change has to be ‘sold in’ and carefully implemented to ensure the change is accepted positively by the users.
There can be great benefit in testing early concepts on real users for the design team, but consider the effect on the reputation of the project. These users will be the first to see the system, and if you are testing early concepts, their experiences could be quite confusing or negative. Most users aren’t familiar with how software is developed, so may come away with idea that the system is badly thought-out or you don’t really understand their needs. They will of course share these first impressions with colleagues, and it could prejudice an implementation before it has even started.
3. Real users are not needed at the start
The truth is, in the early stages of designing functionality it probably doesn’t matter who you test on. I have found that as long as you can explain the context of what a page or process if supposed to do in a concise way, testing on anyone will very quickly yield some significant observations. Things like unclear navigation, missing functions, the possibility of getting stuck in a cul-de-sac etc come up very quickly. I used people from my client’s office who are not intended users nor directly involved in the development of the system at this stage and it allowed me to make significant improvements.
4. Use moderated tests – you are allowed to show users what to do
When you do start testing on real users or potential users, use moderated tests and don’t be afraid to show them what to do as you go through. I recently moderated some testing on whether novice users could find the menu button on a specific page. As a rule they couldn’t (hamburger menu!), so I showed them where it was during the test and we moved on to the next step. Once I had showed them where the menu was, in all cases they continued to use it without a problem. If this had been an unmoderated user test, most of them would have ended at this point and we would not have got to the more useful parts later in the tests. Its not necessarily a priority for the system to be 100% intuitive, so help users in the test if they get really stuck as they would receive help and training during the real implementation anyway.
5. Use expert users from the project team
With back office systems, it is important to test new features on experienced users. The only way of doing this before the system has been implemented is to use people within the project team who use the software (but best not to use people involved with designing it as they know too much!). I have found it hugely beneficial to discuss ideas/wireframes and try out new developments on trainers, business analysts and project managers etc. This can be in the context of a formal user test or just a discussion or watching them use the software informally. Even watching as the developments are tested by your QA can be quite insightful.
6. Observe people who are already using the software
#
Once the first version of your back office system has been implemented, you can now access a trained, experienced user base. Instead of carrying out user testing, I find it much more beneficial to evaluate current usability through field research. Visit users at their own desks, and watch them using the software for real. This is one of the most enlightening things you can do, and it never fails to amaze me how far users go to avoid operating the system in the way I intended! Both ingenius and daft workarounds, use cases you did not anticipate, and real opportunities for improvement come to light in these sessions. Be sure to observe several users as behaviour usually varies significantly between staff members carrying out the same task, even when they work in close proximity!
7. User testing back-office systems is inexpensive and very beneficial
Overall, I believe observing users and testing the software in these ways is hugely beneficial. Remember, you are not asking what users think of the idea or development, or asking for their ideas and suggestions (although this is sometimes useful). You are observing how they interact with the software itself to help inform design improvements that will make their day-to-day working easier and more efficient.
This research does not have to be expensive. In the image above I have used a couple of laptops and Gotomeeting to record the sessions. You could even do this remotely as long as you can confidently manage expectations and understanding of the users. The sessions pay for themselves by helping to direct the design and development early on to ‘get it right first time’.
And finally… you can even power the whole thing with a squirrel on a wheel, as in Steve Krug’s ‘Lost-our-lease usability lab’ below, from his book Don’t Make Me Think!
