Measurements and Metrics

It’s funny how these things come together sometimes, when two separate discussions, one here in the office and the other in the Author-it Forums, nicely lead me to a conclusion on something I’ve been pondering recently. How to measure what we produce?

The first discussion was with a new guy in our team, who was voicing concerns about the amount of information he was producing. He stated that, when describing some of the concepts our product uses, he would spend a lot of time figuring them out, talking to people about them and understanding them, but that usually translated into “not a lot being produced”.

I pointed out that, as far as I’m concerned, the more concise and effective the information, the better. Some things do require a lot of content, others don’t. There are additional benefits when you consider the single source aspect as well, it’s much easier to re-use a tightly focussed topic than one which tries to cover too much information.

The second discussion, in the Author-it forums, was someone asking if there was a way to track the number of words each writer was producing, apparently as a way to track productivity.

Don’t worry, plenty of people pointed out the fallacy of that line of thinking; it’s very easy to pad out a document or topic with additional words even though they might not add any value and may lead to ambiguity.

However I’m not really thinking along the lines of productivity, nor measuring the individual, I’m more concerned with measuring what we produce.

But how?

The obvious answer is to engage with our audience and get their feedback about the documentation. There are various ways of doing this, and depending on your audience some might not be available.

Arranging time to sit down with the people who use the product, and your documentation, is best and can be run as a product focussed session. If your company runs customer forums or workshops then it should be easy enough to schedule time into the agenda (your company understands how important documentation is, right?), but even if you can’t get direct access you could try a questionnaire, allowing customers to ‘score’ the documentation.

Ultimately you need to get feedback from the people who use your documentation, find out whether they can find the information they need, once they’ve found it do they understand it, is it clear, accurate, unambiguous? It’s not easy to quantify what we do at every level but using a questionnaire which includes an indication of a score can give you a way to start further discussions. The score itself isn’t the important bit, it’s what you do with the feedback that matters.

I’d love to hear if you’ve tried other ways of measuring your documentation, and I’m not alone. In the current economic climate there is more pressure to justify what we do, so making sure we have some good weapons up our sleeves will benefit us all.

Comments

  1. Measurement is tricky but definitely worth the effort, in my experience. We follow several metrics, tied to the purpose of documentation in our organisation – but they are so high level that they can do little more than point up the need for more
    investigation or let us know that we’re generally doing okay.

    On the qualitative feedback, though, we’ve had some great feedback via a single question in the monthly survey that gets sent to anyone’s who’s dealt with the support team in the previous month.

    The survey also asks people to leave an email address if they don’t mind us contacting them to find out more. Which gives us the email address of people who are happy to talk about their experience with the documentation – more efficient than sitting in general product feedback sessions in the hope that someone has had some experience of the documentation.

    I hope that helps. Feel free to get in touch if you’d like to know more!

    Rachel

Comments are closed.