What is Quality?

The phrase ‘The team is responsible for quality’ seems to get thrown around quite often, especially when something goes wrong. It holds the team accountable rather than an individual and tries to avoid scenarios like this:

Fictional Manager: “Should we blame the person who has the word quality in their role title?”

Other Fictional Manager: “Seems legit, they’re the ones we made quality explicit with.”  

In previous roles, I have been responsible for essentially, asking questions and providing information. In no way was I ever responsible for sign off or for stopping anything going into production. Project managers were held accountable for final release but it was my job to ensure that they were making an informed decision and knew the risks, the likelihood and impact. My responsibility was to know as much as I could about how the system worked, how a user would interact with it, how other systems would interact with it. Testing was designed to provide information about how the system didn’t work as expected, how customers couldn’t interact with it as expected and how other systems couldn’t interact with it as expected, in addition to identifying other things we found out along the way because as we know more, we know more.

For the purpose of this blog, I want to focus on two phases, Analysis and Design and Build because they are most relevant to my day to day experience. Post Release is also a topic in itself!

Analysis and Design

During the analysis and design phase my focus would be on reducing ambiguity, “What do you mean by ‘most of the time’? When would there be a time when that didn’t happen?” and thinking about functions and interactions, “What happens if I refresh the page? Are any of the fields mandatory?”

The reason for asking these questions is so I can understand how the software behaves and how it will be used once it is complete. The questions I ask will be centered around what’s important to me based in my own experience and knowledge so it’s highly likely that a different person will ask different questions to me.

This is where the power of the team comes in. By having different perspectives and areas of importance being addressed in a group conversation, others can help to answer questions. Additional questions that you may not of thought of with your own line of thinking may come up and you’d be surprised at what others know or care about. Asking questions to understand is not a role based responsibility but rather an individual responsibility to give ourselves the best possible chance of success.  

This part of software creation is often glossed over in favour of something more tangible however time and time again I’ve seen projects derailed and deadlines blown out because a simple question wasn’t asked earlier on. Asking questions is easy. Asking the right questions is hard so it’s important to practice forward thinking and ask ourselves, ‘What’s really going to hurt me later on if I don’t have clarity on it now?’ Sometimes this is a best guess however past experience should be able to help guide us. Reducing ambiguity increases quality in this phase.


‘The team is responsible for quality’ is heard quite often during this phase but what does it mean? Previously, Analysts analysed, Developers developed, Testers tested and Project Managers managed. Everyone had their own area of focus and expertise, they had mastery or were interested in getting better at what they did (hopefully). Did this mean that those individuals didn’t care about quality? No (most of the time).

Quality means different things to different people and it’s not because of their role or title. Beliefs about quality is directly related to the beliefs that you hold and what your values are. Quality can be defined at an individual level as well as a team level. One metric often used by managers is the bug count. But, does a high number of bugs indicate low quality? Without getting too far into this argument I want to say, not necessarily. It really depends on the context of the bugs, the severity, the likelihood of them occurring and the impact they have. I’ve been in meetings where a CEO would decide which bugs we could live with and which needed to be fixed. Decisions I didn’t agree with I had to explain and justify why I thought differently. Sometimes he changed his mind, sometimes he didn’t but at the end of the day he was accountable for the success of that product so the decision was his. In Agile, we push that decision down but we still need someone to be accountable and make those hard decisions when differing opinions are on the table.

Individual responsibility for quality within a team should exist. Regardless of your role, individuals should think about producing something of quality. My own litmus test for this is ‘Am I proud of what I’ve just produced?’ If I am, that’s as good as it gets at that moment for me. If I’m not, then why not, what could I have done better or differently? This is the space in which I learn and grow.  Other things I think of are:

  • ‘Is what I produced worthy of a high five?’
  • ‘Do I feel good having done what I just did?’
  • ‘Could I have done more?’
  • ‘Have I helped someone else out today?’
  • ‘Have I shared information with my team today?’

These are my own metrics for quality which are role or title agnostic. Anyone could ask these same questions and apply them to their own role. Individuals can and should have their own questions they ask themselves to define quality, a way for people to recognise their own contribution to a team. If everyone does their best and tries to get better, team quality can only go up. Mastery is important to individuals as it provides a feeling of achievement and value and many working parts make the engine run smoothly.

All too often, our focus on quality is also on the reduction of bugs but I would argue that a team responsibility for quality is much more than that. This is where I come to team quality and why I think some ways of thinking are diluting quality rather than increasing it.

By measuring team quality with a single metric, this then becomes the team focus. If this becomes the team focus, this becomes what people care about. “We measure what we care about and we care about what we measure” Having a single focus or quality measurement can be dangerous because this becomes the thing that matters and individual quality, individuals doing the best job that they can do can fall to the wayside.

If individuals get better at what they do and are given the time to focus on it, team quality can improve. We need to give people the space and time to understand what they are expected to do and also provide an environment for people to do their work to the best of their ability.

Leading without authority

At the recent WeTest 2016 conference (http://www.wetest.co.nz/), I had the opportunity to talk about leading without authority. Many companies are ditching the more traditional, hierarchical team structure in favour of autonomous, flat structures where big sticks of authority no longer exist. This can be a dramatic change for a lot of people in positions of leadership or management. Where previously people often did things because they were told to, in this new structure teams are empowered to say ‘no’ and think for themselves.

Leading without authority can be a massive hurdle for all of us, especially if we’ve never really had to think about who we are as leaders, because our titles gave us everything we needed. I had come from a place where I was familiar with telling people what to do rather than listening to what people needed. Still being a leader but having all the authority taken away was a very steep learning curve for me. Having spent the past 15 years in all types of leadership positions, I wanted to share some of the myths I had busted along the way.

Myth: People are born <leaders>, <successful>, <lucky>

Some of the time we can look at people who have ‘gotten ahead’ of us in some way and think that they don’t deserve it, that they may have gotten lucky in some way. It’s all too easy to sit back and feel sorry for ourselves, thinking that success isn’t something that will happen to us. I’ve been fortunate to have met a lot of successful people in all types of disciplines and they all have something in common. They’ve all worked for it. I feel it’s important to make a distinction here in that I’m referring to the people that are respected in their fields as people in a community as well as an organisation.

When we’re introduced to people who have already achieved success, we’re often only seeing the tip of the iceberg. What we haven’t seen is the hours of dedication, training, self reflection, learning and growing that person has been doing for years before. I can guarantee that they’ve had a few failures of their own but they made the choice to keep trying and move forward. It’s also important to recognise that these people do these things for themselves, not for other people.

People who perform well, usually do so because they have a supportive environment which allows them the time to work at getting better.

They also have the right tools, resources and opportunities to continue moving forward. It’s important to keep this in mind when building teams by asking ourselves if we’re creating the right environment for people to grow and improve. Change the environment and the people will also change.  

Myth: To be a leader, I have to know more and do more than everyone else

Like many people I’ve suffered from imposter syndrome. The thought that I was going to be found out in some way, that I didn’t know as much as people thought I knew. In my early 20’s I was given advice from an older colleague, ‘Once you accept that you’ll never know everything and that it’s ok, you’ll be much happier’. Turns out they were right. Being able to say ‘I don’t know but let’s find out’ is rather empowering. This is why we work in teams. To be able to share knowledge and collectively come to better decisions together.

A lot of my time spent in a position of leadership is listening and asking questions. Sometimes people just need a sounding board or a way to gain a different perspective of the issue or problem at hand. The other side of this is that if I know everything and do everything then I don’t give others the opportunity to learn and to grow. I am looking after myself at the expense of other people’s development.

Myth: Extroverts make the best leaders

Over the past few years there has been a shift towards classifying people as being either Introverted or Extroverted. On the face of it, we tend to classify people who speak up at meetings or at conferences, organise events, are good at networking and considered fun and loud as extroverts. Looking into this deeper, it appears this is just another way for us to classify or stereotype people. I believe that everyone is an individual and there are things that every person brings to what they are doing regardless of being introverted, extroverted or ambiverted.

No personality type makes a leader better or worse, instead we should be looking for the skills, qualities, values and behaviours a person exhibits when identifying leaders.

Human’s have an innate ability to do this. Leaders are the people we follow regardless of title. However, when it comes to job descriptions, we sometimes forget this and hire for an ‘outgoing, energetic, fun, enthusiastic person’ to lead our teams. These qualities, while inherently positive, are not always the things that make leaders great.  

Myth: If I fix all the problems people will respect me

This is a thought I believed for quite a while until I had the realisation that I needed to respect myself and my time first. I’ve worked in many offices where 10 hour days were expected and out of hours phone calls or email demanded an immediate response. I once got out of bed at 11 o’clock at night to head down to the office and check something for a senior. Talking about this to other people, I began to realise that this wasn’t healthy behaviour and that I wasn’t doing it because I cared, I did it because I was scared. Reflecting on this event, there was no reason why he couldn’t do it himself, or it could have waited until the following day.

Something happens when you become the person that fixes all the problems. You become the person that fixes all the problems. More and more often, people will continue to push boundaries and ask more of you until something breaks. The biggest lesson I learnt was to say ‘no’ and that I needed to respect myself first.

Myth: The team should do things my way

This is something I see people do quite often after a promotion or joining a new team and I am guilty of it myself. After being in a position where I was unable to make decisions and action my ideas, the excitement of being in a new role where I could make my mark was intoxicating. When joining a new team, I’ve since found the opposite to be more useful. Instead of jumping in, changing things and making a statement, it is better to relax, listen to others, gain trust and then help create an environment where people feel inspired to change.

When someone takes charge and makes decisions in isolation, teams can become apathetic, feeling as though their contribution or opinion isn’t valued.

Coming to the realisation that I am one person with my own thoughts, biases and fears, I can only do so much. Clones of myself really isn’t the best solution to anything. Utilising the creativity of the team by allowing people to do things their own way even if that includes making a mistake will foster a more harmonious team environment. Creating an environment where people feel safe to ask for help and share ideas so the team isn’t limited by their leader starts with giving trust and support to the team.  

Myth: If I had a different <title>, <team>, <workplace> things would be better

When things get hard, it’s easy to pack it all in and walk away. Unfortunately, if we walk away for the wrong reasons and don’t do the work to identify our own contribution to the situation, we can take our problems with us. It can be hard to be honest with ourselves, especially if we have been unkind or difficult in a situation when we had a choice to be kind and compassionate.

There are definitely reasons to want a new title, team or workplace like seeking a new challenge or opportunity to grow. I’ve rarely had success when I’ve made a change to get away from something. If stressful situations that are eerily familiar keep presenting themselves, the best thing to do is reflect and look at what was a common factor. Self reflection is not an easy thing to do and most people avoid it, choosing to deflect or blame something else outside of their control but it is the most valuable tool in my tool box. Being aware of my own contribution to an outcome, both positive and negative, allows me the space to grow and change.

Over the coming years, I’m sure other myths will be busted and I will learn new things about who I am as a person and a leader. I expect that I will change, evolve and grow but for now, my own truths about leadership are:

“It’s not about me”  

“Lead with empathy”

Creating a Team Experience

When helping to create and deliver software, the user experience (UX) is at the core of my daily activities. It helps guide my questioning and decision making while helping to identify the value a product will bring when it’s being used.

I recently had the opportunity to speak at UXNZ 2016 (http://www.uxnewzealand.com/) on the topic of applying the things I’ve learnt in UX to teams. I believe that the people behind the software, the creators, designers and analysts, are more interesting and more complex than what they produce.

“Great teams that are respectful to each other and work in a safe and supportive environment are more capable of building great things.”


Like most things, my story starts with the people. In UX, we use Personas to help create empathy and connect with who will be using our product. There is a trap that we can fall into when we start to classify people and that is the stereotype. In our workplaces, a lot of us have a title, something that helps others identify what our role is or what we are responsible for. When a group of people have the same title, it can be easy to assume that all of those people are alike. We assume that the people with the same title must all be the same because we can make sense of that. In short, it’s easier to think this way.

When we do this, comments like ‘Developers don’t like change’ or ‘Testers like breaking things’ or ‘Product Owners only care about deadlines’ start to be heard. This reinforces stereotypes and we lose the identity, skill and unique experience that the individuals in our teams bring. I don’t work with anyone called ‘Developer’ or ‘Tester’, there are people with names that do those types of activities and they should be recognised as the individuals they are. I like to think that addressing people by their titles is a way in which we distance ourselves from unkind remarks because we’re not frustrated with the person, we’re frustrated with the role that person plays. Unfortunately, in reality, the person being talked about can feel attacked, boxed in and restricted by their title or role. We give persona’s a name because it helps us to connect.


The creation of prototypes enables us to gather feedback without the need to be production ready. Having something we can interact with makes it easier to identify any usability gaps and helps develop a clearer understanding of the vision for the product.

Words only ever give us part of the picture and words can conjure up different images for different people based on their own experience or knowledge in a particular area. If I ask you to think of a vegetable, because that’s a broad category, it’s likely that we are both thinking of different vegetables. While your mental prototype might produce a carrot, mine could produce a cucumber because that was something I encountered earlier in the day. We are both thinking of a vegetable and are both technically correct but we have still failed to have a shared understanding of the same vegetable.

Being explicit in our communication by using pictures as well as clarifying words, a shared understanding is easier to reach.

When we are being implicit, gaps and ambiguity can creep in. We tend to jump to our own conclusions and understanding based on many of our own biases and beliefs that we hold. If unsure about the shared understanding of a word or idea, don’t be shy to ask ‘What does that mean to you?’.

Measuring Success

In more recent times, improvements through iterations has become critical to a product’s success. Many companies have the ability to release code to production ‘when ready’ and users can have immediate access to those updates. When we first start to measure success, there are a lot of unknowns and sometimes a few nasty surprises when we find out we’re not doing quite as well as we thought we were. The natural reaction to something we don’t want to see is to make it go away. I call this the ‘box factor’. When opening a present on christmas morning we are usually either pleased because the gift in the box meets our expectations or we are disappointed because it doesn’t. One of these gifts will be opened while the other will probably remain in the closed box, left to gather dust.

When measuring success, it’s important to be transparent and accepting of both favourable and unfavourable possibilities.

At all times we expect the outcome to indicate either success or that there is still some more work to be done, that improvements are required. This approach means that there are no nasty surprises hiding in the box because we have been transparent about the possible outcomes. Reducing the impact of the surprise of an unfavourable outcome allows us to respond from a more measured and less reactive place.


Underpinning everything we do should be empathy, an understanding for our fellow humans. Empathy is the ability to connect with another person by understanding their point of view and recognising emotional responses. During my time in product development, I’ve picked up on three regularly said anti-empathic statements:

“At least they can…”

“There’s a workaround for…”

“Don’t worry, it’s just an edge case”

Hearing any of these in a review or product meeting immediately raises red flags for me and makes me re look at the solution in front of us. It’s a great indicator that there is still more work to be done. These statements show that we have failed to understand not only our customer and their user goals, but also that throughout development, the team has failed to communicate or understand a shared vision. Creating gaps in the product gives users the space to become frustrated or to experience an unhandled error. We have failed to map out their entire journey and have left metaphorical rivers and no bridge to cross them with.

By finding new ways to connect with the people around us in our work places, we can help create environments that foster innovation and products people love to build and use.

Interesting User Stories

I recently held a coaching session with one of my colleagues around the topic of User Stories. There are a lot of articles and books written on the subject, each with the basic structure of:

As a…

I want…


User stories are something that I’ve seen myself and others struggle with. In the past I’ve often put ‘get better at writing user stories’ on my personal development board as I’ve always felt they could be better. I’ve also read a lot on the subject in an attempt to get better but for one reason or another it hasn’t stuck with me.

During this coaching session, I asked for an example of a user story that we could potentially improve on. The story followed a familiar format:

As a user

I want a search field

So I can find a report

At this point I realised we didn’t have enough context or information to jump straight into improving this story. We had to take a step back and look at what we were trying to achieve with the story first. I’ve recently been thinking a lot about the ‘why’. Why we do things, why we believe, why we change and many more.

I often like to take the ideas of software and apply it to something more tangible. For this session we came up with the idea of ‘Clothes’. This became the context or domain in which our character (person, user) was set. This became our product.

Our next step was to identify the ‘Why’. Why might this character need our product? For example:

  • To not be naked
  • To keep warm
  • To be fashionable
  • To feel good

Once we had this information, we then thought about the type of person our character might be. Who are they? What is their context? We then began experimenting with how we might structure a user story using this information:

As a person who wants to leave the house

I want something to cover my body

So I won’t be naked in public


As a person who lives in a cold climate

I want something that will prevent me from becoming too cold

So I can keep warm

These were ok, but we could do better. By changing the person’s context and their why, our stories could become more specific..

As a person who is travelling to Antarctica

I want protection from the cold

So that I can survive -20 degree Celsius temperatures

None of these stories gave a solution. The ‘How’ could come later. By structuring our stories in this way, we allowed room for multiple solutions to a problem rather than narrowing our thoughts into one area. If we take the last user story above and provide a solution, all of a sudden our options for how is significantly narrowed:

As a person who is travelling to Antarctica

I want a survival coat

So that I can survive – 20 degree Celsius temperatures

This seemingly innocent structure is limiting our ability to innovate.

Going back to our original story, we thought about how we could make it better by following a new structure

(Who) As a <character> who…

(What) I want <need>

(Why) So <my need can be satisfied>

Our original story rewritten looked like:

As an accountant who wants to file my client’s GST report at the end of the financial year

I want to find clients who have outstanding GST reports

So I don’t miss the deadline for GST returns and I avoid any penalties

This poses the question, if user stories are supposed to tell a story, do you want to tell a boring story with ambiguity or an interesting story with meaning?

The Evolution of UAT

User Acceptance Testing. It’s a phase of testing that’s bothered me for a while. Not because I think it’s a waste of time, in fact, quite the opposite. It’s bothered me because I think it hasn’t been given the care, attention and expertise it deserves.

The intent of UAT has always been a noble one. We wanted our users to accept our product and accept what we had built. The way we went about proving that acceptance though was deeply flawed. The process looked a little like this:

Step 1: Find some users or, if users aren’t available, you can do it yourself

Step 2: Write a script for the user which outlines exactly how they should test the system step by step in predefined workflows

Step 3: Get the user to complete each step putting a ‘Pass’ mark next to each one

Step 4: Write any questions or confusion from the users down as ‘training issues’ (If you’re testing it yourself, you probably won’t have any questions because you’ve already tested it in the system and integration test phases and you know it works. Besides, you’re not the one who has to use it on a daily basis anyway)

The problem with this approach is that it seeks to validate the product even if it invalidates the experience of the person using it. At this point it’s not really about your user accepting your product, it’s about your product functionally working when put in front of an end user.

How do we solve this problem?

Firstly we need to understand our users. We need to understand their needs, their activities and the context in which they are performing those activities. By doing this we change the meaning of UAT into a list of things our users will accept, things that will delight our customers once we’re done. Once the feature goes live, User Confirmation Testing can commence. Confirmation testing allows us to measure the degree to which we were successful against our initial assumptions, hypotheses and insights. Both qualitative measures such as customer feedback and quantitative measures through the use of data analytics can help us to identify what our users have accepted and what they have rejected. This gives us the opportunity to research, build, monitor, iterate and improve.  

UAT done in accordance with this new meaning is hard. It involves a lot of effort, time, understanding and empathy. Not all of the “defects” we find during the first UAT phase will be ironed out so confirmation testing shows us where we’ve been successful and where we need to iterate and improve. This pushes testing to the start of the development lifecycle and helps to prevent usability flaws in our finished product. Perhaps, most importantly, it provides the opportunity to test before the code is written, as the code is being written and after the code has been deployed adding value at each of these stages.

At a time where people have so much choice and greater expectations that software should be engaging and intuitive, understanding who we are building software for can help us to meet those expectations. It is no longer good enough to be happy with “work arounds” or “at least they can still <insert anything unrelated to the thing they can’t, but really want to do>” attitudes towards software development. Focusing on what our customers value and delivering the things that will really make a difference give us a head start on any competitors in our market.

Every time a new idea for a feature comes into a team we can all strive to understand who we are building it for and what problems this change will solve. If we can’t answer these two questions, it’s time to put the deerstalker on and seek out the answers!