Monday, January 29, 2007

Five ways to identify intranet usability issues

Many intranets are under-used. Intranet managers lament the low use and discuss how to get staff to 'use the intranet more', resulting in marketing and promotions activities to increase use.

Marketing and promotion activities are wasted however, if staff return to the intranet and find that it doesn't help them to do what they need. Before telling staff why the intranet is useful for them, make sure that it is both useful and usable.

Intranet teams should assign part of their workload and budget to improving the usability of the intranet and actively investigate intranet usability issues.

This article provides five techniques to identify likely usability problems in your intranet. Some techniques provide indications about where the main problems lie, others provide concrete evidence. Each technique can be used alone, or in combination to give you a rich picture of usability issues.

1. Stakeholder interviews

Stakeholder interviews involve one-on-one discussions with staff to identify key knowledge needs, gain an understanding of issues, and uncover problems. They are generally used in the early stages of projects when gathering requirements. For an intranet project, they help to identify how an intranet may assist staff by discussing information use, need and sharing.

Stakeholder interviews are generally not used to identify usability issues, but the technique can be varied to do so.

To identify usability issues using stakeholder interviews, the interview should be conducted in the environment where the intranet is used – usually at the staff member's desk. The interview should discuss what the staff member's job involves and how they currently use the intranet (or don't use it) to support their work.

The key question to ask during the interviews is "show me". Get staff to show you how they look up contact details in the staff directory, how they find important policies, how they check the news and other tasks that they frequently use the intranet for. If staff tell you that they had difficulty finding information or doing a task, ask them to show you again.

You will be surprised at the way people use the intranet. In particular, you may learn about:

  • work-arounds for tasks
  • different paths staff follow to information
  • parts of the screen or information that staff miss
  • areas of the intranet that staff don't understand
  • sections of the intranet that staff don't know exist

You will also see how staff have their computer set up and how they use technology (including any assistive technology).

An advantage of using this technique over others in this article is that you are able to explore a wide range of tasks that staff attempt. Another advantage is that, if you do not know what staff are attempting with the intranet, you gain a better understanding of staff information needs.

A disadvantage is that each staff member may show you different tasks, so you may not see any consistency in the usability issues.

2. Walking through scenarios

This technique involves preparing an extensive set of scenarios, and then attempting to complete them yourself (or with your team). The goal is to gain an understanding of what staff experience when using the intranet.

For this technique, a scenario is a short story describing a task that staff attempt on your intranet. It may be something short that staff undertake frequently or something longer involving the intranet as just one part of the overall task. For example:

  • Find the phone number of someone in personnel to contact about leave arrangements.
  • Joe needs to travel to Perth next week for work. He hasn't travelled to Perth before, but has travelled to other capital cities. He knows that he first needs to fill in the travel approval form and have it signed. He also needs to book a flight to get him there for a 9am meeting, book a hotel close to the office and pick up cabcharge vouchers.

A pre-requisite for this technique is a good set of scenarios. If you don't have a good understanding of what staff need to do on the intranet, you can run some stakeholder interviews (discussed above) or undertake some task analyses. Task analysis involves walking through individual tasks with staff and identifying all of the components and activities that make up the task (intranet and non-intranet).

When you are writing scenarios, make sure that they fully describe a task that staff attempt, not just the part that involves the intranet. It is helpful to think about why someone would be trying to find information, and what else they are interacting with. It is also useful to write scenarios without looking at the intranet – this helps to write them neutrally without details of the current implementation. Ask someone to read through the scenarios to make sure you cover the task well.

When you have the list of scenarios, walk through them yourself or with a team. While doing so, pay attention to the decisions you make and the reasons for your decisions. For example, if you choose a particular navigation item because you already know that the information you need is there, think about what the item is and whether you would choose it without that knowledge. Look at the information available at each point and see if it supports the scenario and helps you to achieve it.

If the scenarios contain complex tasks, you may find that for some you need to visit a range of places in the intranet – this may help you to identify areas of content that would be better grouped together.

It is useful to see whether you can achieve each scenario in a number of ways. For example, by navigating, using the search engine or using an A-Z list.

The main advantage of this technique is that it allows you to see how staff may be experiencing the intranet. At the end of the process, you will also have a solid list of scenarios describing use of the intranet.

The disadvantage of this technique is that you may have too much existing knowledge about the intranet for this to work or that you use the intranet very differently to staff.

3. Review existing data

You may already have data that can provide insight into usability issues. For example, you may have:

  • search logs
  • feedback
  • help desk requests

Search logs

Search logs provide data on the terms users are typing into the search engine on the intranet. Search logs are always interesting – they clearly show the main items that staff are looking for on the intranet and there are usually a set of queries that are consistently popular.

Search log data needs to be interpreted carefully as staff use search facilities for many reasons (not just for information they can't find). For example, staff may be using the search engine because they can't determine where to go using the navigation or because they know exactly what they are looking for, and expect the search engine to get them to it quickly.

You can use the information from the search logs:

  • as input when writing scenarios for a walk-through (technique 2) or a usability evaluation (technique 4)
  • to improve results displayed (by identifying the 'best' pages and ensuring they are high in the result list)
  • to create a list of 'quicklinks' on a home page
  • to identify information that staff are looking for that does not exist and arrange for content to be created


If you collect feedback from staff in a structured way, perhaps by the use of a feedback form on the intranet, you may have a rich source of information about usability issues. Take some time to analyse this information and look for consistent patterns in questions asked or comments made.

Help desk requests

Help desks can be an amazing source of information about the intranet and other business systems. Help desk operators have a good understanding of tasks that staff find difficult.

Arrange to discuss the intranet with help desk staff regularly. Talk about the types of issues that are occurring and what impact changes have had on the types of questions asked. Don't only talk about the existing intranet, but also about how the intranet could help staff in the future.

For example, you may be able to create some quick reference pages for common queries. The help desk officers can send staff to the quick reference pages, increasing use of the intranet and helping staff see that it is a useful source of information.

Most help desks have a job tracking system with a database of queries. Analyse this in the same way as feedback, looking for patterns in staff questions.

Language and labelling

All of these information sources provide details on the language that staff use and the way they describe what they are looking for. For example many staff talk about 'personnel', 'HR' and 'training', even when the organisation no longer uses these labels.

If the language that staff use does not match how you describe content on the intranet, chances are that they are not finding the information they need. You can use this information to improve your labelling and content.

4. Usability evaluation

The best way to identify usability issues is by undertaking a usability evaluation. This involves observing staff attempting tasks using the intranet. The main elements include:
  • determine what you need to achieve out of an evaluation
  • prepare a set of scenarios to evaluate
  • recruit participants
  • observe the participants attempting the scenarios
  • note what happens
  • analyse what happens and do something about it

The key differences between a usability evaluation and stakeholder interviews as described in technique 1 is that the usability evaluation:

  • has all participants working through the same set of scenarios
  • provides more consistency in issues identified
  • can provide measures of success and other metrics

For a successful usability evaluation:

  • scenarios need to be realistic and represent core tasks that users undertake (scenarios can be developed in the same way as in technique 2)
  • a wide range of scenarios may be necessary to evaluate core tasks undertaken on the intranet
  • participants in the evaluation should be representative of your user group
  • you may need to videotape the usability evaluation for thorough analysis

If your team has not conducted usability evaluations in the past, it is worthwhile undertaking the first evaluation with a usability professional or attending usability evaluation training. The benefit of this is not only to learn how to run the evaluation, but also how to determine what a usability issue is.

Some usability issues that may occur include:

  • staff cannot find the information they are looking for
  • staff do not see something that is on the screen that would help them
  • staff are 'lost' – not sure how to get back to a previous step or do not know where they are located in the intranet
  • staff cannot determine whether information is accurate
  • staff cannot use the elements of the screen (e.g. staff may have difficulty using navigation elements, clicking the 'search' button, or getting the cursor into the search box)

It is important to look for repeated behaviour – if a problem occurs for one person it may not be worth following up, but if 3 or 4 people have the same problem, it is worth investigating further. During the evaluation, look for 'a- ha!' moments, but also think about underlying issues and what has caused the problem

5. Expert review

An expert review is an inexpensive technique involving an assessment of your intranet against a set of heuristics to identify likely usability issues. Heuristics are 'rules of thumb', guidelines or best practice that the reviewer uses.

The 'expert' should have experience in both human-computer interaction and intranets (although it is difficult to find people who have significant experience in both of these areas). An expert reviews will not identify all usability issues, but will provide suggestions for further exploration via a usability evaluation.

As it is difficult to do an expert review of your own intranet without bias, the expert involved may be:

  • a consultant with experience in intranets
  • a colleague on your internet team (if there is a separate team)
  • an intranet manager from another organisation