Why your deadline will destroy your metrics (but only if you let it)

It’s a situation that everyone working in digital has experienced. After many workshops and much thinking, your team had agreed some beautifully user-centric, value-driven measurements for the work you’re  going to deliver. You talked about Lean Analytics, and discounted any vanity metric that wouldn’t provide an actionable insight. 

Everything was going great… until the Terrible Deadline approaches, and your team gets bombarded by requests for exactly those same vanity metrics you’d decided to ignore. Then, before you know it, you’re scrambling to put together reports for your stakeholders, using measurements you know won’t reflect the real value of the work you’re doing. 

The Terrible Deadline has managed to destroy your beautiful lean metrics - again! Sound familiar? 

If this is happening to you, what can you do to stop it? Unfortunately, I haven’t found the perfect silver bullet. However, there are some things worth trying, which could help put you back on the right path (and eventually take your metrics out of harm’s way).

1. Notice it’s happening

Having the self-awareness to notice that you’re moving away from the narrow path of value-driven, lean metrics and into the wilderness of vanity ones is half the battle. If you can pick up on this as a team, and if you can do so as soon as possible, you can fix things and stop the Terrible Deadline before it does too much damage. Of course, to notice things, you need the time and space for self-reflection. Which brings me to...

2. Meet as a team and talk about it

Team retrospectives, which come from the agile framework, are simply regular meetings that offer your team a safe space to reflect on how you’re working together (you can read more about retros here). Unfortunately, when deadlines approach and time seems tight, retros tend to be one of the first things to get sacrificed, in favour of emails and PowerPoints for the stakeholders. 

So now you’re in a vicious circle: your team is under pressure and not working well, and you don’t have the chance to talk about how to fix it. What to do? Very simply: make sure your retros stay in the diary, re-schedule them if someone can’t attend, or go ahead with a smaller group and share the outcomes later. But  don’t  let your team’s balance get sacrificed to the Terrible Deadline.

3. Show the problem to your stakeholders

Once you’ve met as a team and understood the impact of the Terrible Deadline on your metrics and on your ability to stay focused, bring these insights to your stakeholders. After all, they may have no idea of what’s happening in your group! Instead of simply pushing back on a certain metric or trying to justify why you’re not using it, explain how it’s disrupting and distracting your team. 

Make sure to bring specific examples of what this means in practice, so you can show rather than tell. And if you haven’t had this conversation before, be clear on what else you want to use to measure your work’s value, and why.

4. If you fail, try again, and fail better.

Perhaps once you’ve done this, the Terrible Deadline and its vanity metrics will go away - success! Perhaps they’ll become a bit quieter, but not disappear. Or perhaps you won’t be able to escape them. 

If that’s the case, don’t let this affect your commitment to being value-driven: instead, use this as a learning opportunity. What could you do differently next time to avoid being in the same situation? What conversations could you have had before the Terrible Deadline got in sight that would have helped you later down the line? Don’t forget to have an honest, post-delivery retrospective as a team and to capture these lessons somewhere visible for both yourselves and your stakeholders. Next time, you can all use these learnings as starting point for a team charter.

These are just some ideas of things to try out - we don’t have all the answers, but at Cancer Research UK, we’ve been trying to slay the Terrible Deadline monster and to empower teams to choose, and most importantly stick to, the metrics that help them deliver value, rather than the ones that make for nice PowerPoints. If you’ve ever faced similar battles, we would love to hear your thoughts!

Giulia Merlo
Lead Proposition Manager

10 top tips for awesome remote usability testing

Usability testing is fundamental to how we do digital projects at Cancer Research UK, helping us get to our ambitious target that by 2034, 3 in 4 by people diagnosed with cancer will survive. Testing our ideas out early and often on the people who use our products and services means we save a lot of time and money and give our users what they need in the most intuitive way possible.

We do all sorts of different types of testing (A/b testing, guerilla testing and lab testing to name a few!). But what about if you don’t have the time or budget to meet people face-to-face? That’s where remote usability testing comes in. With remote usability testing, you can share what you're testing with people who are in different locations to you and ask them questions about it. It’s a great way to talk to people who may not have the time to come in to talk to you. And it means you can speak to people that don’t just live near where you work. As a national charity, this is especially important to us at Cancer Research UK.

We recently did some remote usability testing on a new version of our publications website, which allows health professionals to order leaflets and other resources to help educate the public about cancer.

Here are 10 top tips that helped us run a great session:

1. Assemble an ace team

User testing is a team sport so it’s useful to get a small team of people together to help organise all the tasks you’ll need to do. Our team included me (UX designer), Hayley (Product manager) and Becky (Digital producer). We were all quite flexible in the tasks we picked up and all very willing to get stuck in!

2. Know your research goals

Before you start, it’s important to work out what you want to get out of your testing. For this session we wanted to know more about our users, and see if they could complete specific tasks when ordering resources. From these research goals we created a list of questions about the participant’s day-to-day working lives and some more specific task-based questions where we asked participants to use a prototype of the new website and tested out whether they could use it. 

3. Recruit the right people

This is really important for getting meaningful results. Luckily we had a list of active users from the old website who had already signed up to hear from us. So we emailed them to ask if they’d  be willing to help us improve the new site. A few people responded, so we were able to get 6 willing participants (6 is usually a good number to aim for, enough to start seeing patterns without getting too many repetitions). There are lots of other ways to find people though. In other projects we have asked our work colleagues and friends if they know the kinds of people we’re looking for. We’ve also put shout-outs on social media.

4. Write the script

Based on your research goals, you’ll know what you want to get out of the session. Now you can write the questions and tasks that will help you get the answers you need. For each task, think of a goal, the question you will ask and what a successful result looks like. Here’s an example from our session:

5. Do a dry run

Sometimes the questions you’ve written will make total sense on paper, but don’t work out when you ask them on the day. To minimize the risks of this, practice all your questions and tasks out on a member of your team who isn’t too close to the project. This is a great way to find out whether the questions make sense to someone else, and an opportunity to get your timings right.

6. Prepare the tech in advance

This bit is super-important. It’s very easy for a perfectly planned testing session to be cruelly foiled by a dodgy microphone or an out of date screen sharing subscription. For this test we used join.me to share our screens with participants. We made sure we tested both this and our own hardware (computers, microphones etc) out the day before we did the real thing.  

7. Meet with the team to make sure everyone knows their roles

Once you’ve assembled your team, make sure everyone understands their roles, both in preparing for the tests and what they’ll do on the day. You’ll need to work out who will be facilitating (asking the questions) and who will be doing the note taking (Pro tip: If possible, the facilitator shouldn't be someone who's been too involved in the design, as this can bias the testing). We used google sheets for note taking, as it’s the easiest way for the whole team to share and add their notes.

8. Invite the whole team along

Invite anyone involved in the project, including your development team and any stakeholders you have. This is incredibly useful as the project progresses so you will have a shared understanding of your users and their challenges.

9. During the session – Don't forget the notes

Now’s the interesting part where you actually get to hear from your users! Make sure the note taker is taking qualitative notes (describing how the person is doing the task) and quantitative (usually a score for each task of between 1-3, based on how easily someone completed the task). Both will be very useful after the session for understanding what the pain points were.

10. After the session – get into the detail

Well done, you made it through all those interviews in one piece. Take a well deserved break, but not for too long. Now’s your opportunity to take stock of what happened. Review the session with your team. Take a look through all your qualitative and quantitative notes and make a list of what can be improved. Don’t leave it too long or you may start to forget what happened in the session.

Remote usability testing – one piece of the testing puzzle

Remote usability testing is a great way of testing out your ideas early and in this project it helped us discover insights we would have never have uncovered otherwise.  But it’s just one of the many tools we use at Cancer Research UK to understand our supporters better, making them happier and helping us beat cancer sooner.

Why should charities use Jira and Confluence to get things done?

I recently spoke at an Open Charity meetup event in London about how charities can save thousands, reduce waste and create awesome digital products using tools made by a company called Atlassian. Here are some of the main takeaways from that presentation. Powerpoint slidedeck can be downloaded here. 

What are JIRA and Confluence?

Jira

Jira is used to track tasks and issues for your project and helps you deliver more work, faster. It is popular with software teams but is by no means limited to them. A team using Jira can easily track many types of work, from simple tasks to bugs and user stories common to agile teams. We use multiple management styles at Cancer Research UK, and Jira is well-suited to many situations. The main benefit of Jira over good old white board and post-its is its reporting capability: all the work your team has ever done is kept in Jira so you can set up powerful dashboards and reports which empower managers to better forecast future delivery.

Confluence

Confluence is essentially a wiki for your organisation. it is a great tool for team collaboration and document management. Have you ever spent half an hour searching for that really important information buried somewhere in your email inbox? Confluence allows your teams to have 'one source of truth' for collaborative documents and meeting notes. Powerful macros allow you to dynamically display content to keep landing pages fresh. If integrated with Jira, you can dynamically link to lists of work items and dashboard widgets (See 'examples of good user stories' Jira widget below).

Why should I care?

If you are a registered charity and you have the capability to install and configure these programs in-house, they are completely free for you. This would include many add-ons which normally cost thousands.

These tools allow teams to be on the same page with high visibility and transparency of their work (see project dashboard below). They are highly configurable such that they can scale from a tiny startup to a massive enterprise-grade company. Dynamic reporting tools and dashboards eliminate what might otherwise be manual effort. 

What about other tools?

At Cancer Research UK, we use a combination of tools for Digital delivery.  We want delivery teams to be free to choose the tool that is right for them. There are competing products but lately we've found Slack, Trello, JIRA and Confluence to be the most useful. But it is a crowded field. Competing in this space are Sharepoint, Yammer, Skype, Hipchat, VersionOne, MS Office and Google Drive to name but a few. For a charity which needs to watch their budget, I recommend Trello, the Atlassian tools, Slack and Google Drive.  Pivotal Tracker is also worth a mention, as they have a significant free plan for non-profits.

Conclusion

Whether you’re a charity of 5 people or 500, the tools described here will help you deliver high quality work, quicker, across multiple projects varying in degree of complexity. See below for further reading and video tutorials. 

• JIRA Software information
• Confluence information
• Atlassian's Youtube channel

Open Charity Meetup

Open Charity is a public meetup based in London. For several years it was a rag tag group of charities and their partners meeting privately, but as of late 2016, their public events feature guest speakers, lightning talks and networking. Their focus is on bringing charities and partners together to collaborate and share open solutions to create value in the digital space. If you are interested in sponsoring, speaking or providing a venue, please get in touch via one of these channels. 

• Twitter: @opencharityorg
• Meetup: meetup.com/Open-Charity
• Web: opencharityuk.org
• Facebook: facebook.com/opencharityorg