Goodbye Testing Columns, Part 1: A New Hope

We’re getting rid of the In Testing and Ready For Testing columns from Jira, woo! It’s all very exciting…

This is Part 1 (of 3, if all goes according to plan), which will cover the decision making and planning leading up to the event.

First off, some background on why we’re doing this.

We have a lot of queueing time on our board, from tickets and bugs waiting around to be picked up because people have moved on to the next piece of work. It’s super annoying for everyone, and we hope that removing the queue columns and moving to more direct handoffs will help improve our overall speed of delivery. It might also have the effect of making devs more quality minded, and spreading the responsibility for quality across the team… Who knows!

A note on the format of this post: I decided to write this as a diary of sorts, so that I wouldn’t forget any of the challenges faced during discussions etc. Because of this, it’s pretty verbose, bit of a chonker of a post. Well done if you get through it though!

Before time

So I’ve been talking in 1-1s for a while now about wanting to try out getting rid of the Ready for Testing and In Testing columns. It often feels that stuff gets stuck in ‘my’ columns when I’m not even touching them because they’re waiting on bug fixes that just don’t seem to be a priority. At the time of writing this, there is a ticket In Testing that has been there for two weeks waiting on a subtask bug to be picked up. I’ve poked and prodded, brought it up in standups but no game. New work just keeps coming on top of it in terms of priority.

There’s this unconscious thinking that once something goes into Ready For Testing, it’s my responsibility and my problem as the team’s tester. Most of the time, dev and test work really well together and we build on features iteratively and try our best to keep good comms. I’m part of an engineering team with the developers and share a manager with them. Yet there’s still this weird thing with the jira board where it gets past a certain point and then it’s on me.

I don’t want to say that the current state of affairs is somehow the fault of the developers or management on the team – it’s not. I’m guilty as anyone of propogating the idea that testing and releasing are my responsibility. When I first became the primary (read: only) tester on this team, I was happy to take responsibility for testing and deploying to production. It’s part of my job, and it’s way better than having someone externally tell me when we have to deploy, while I scramble to get any meaningful testing done in time. We deploy as and when we need to according to my judgment, and usually between 2 and 8 times a week. I was happy about this for a while. I wanted to be the gatekeeper at the ‘end’ of the development process, because it was an easy and direct way to be useful.

But it’s not the most useful I can be. It’s the most important I can feel, and I should do better.

So here we finally are. We’ve come to a group decision that we’re going to get rid of the two testing columns, and I am beside myself with both excitement and anxiety. I know this is going to be great in the long run, but I also know that I hate feeling out of control and so losing control of the process is going to be a struggle for me.

November 1st

We’ve got a meeting set up for Monday and the team has been asked to put forward some talking topics ahead of time. I’ve written up a few notes, but I really want to get wider perspectives on this from people who already work that way (or don’t).

Here are a few of the great responses I got:

Think about how you are going to test a feature before building it. And add that info to the story during refinement. This will help understand intended behaviour, find missing information and improve testability.

— Marit van Dijk (@MaritvanDijk77) November 1, 2019

One thing I worry about is changing the board but people behave the same way.

It could be less easy to spot potential bottlenecks but I suppose it depends how things are visualised (or not).

— Vernon McRichards (@TesterFromLeic) November 1, 2019

I personally like splitting up the in dev / in test columns. I’ve found it helps show where bottle necks are. If you’re following Kanban with 2 columns to limit the work in progress to the two columns combined…. ie can’t have more than 5 tickets in dev and test total.

— Butch Mayhew (@ButchMayhew) November 1, 2019

There were loads of comments and I got ideas from all of them. Thanks to everyone who replied with opinions and advice! Super useful. 😀

November 4th – pre meeting

We’re using a webapp called Candor, which removes some bias by allowing you only to see the cards you wrote until the leader reveals them all. This means you can’t be influenced by others, and you’re more likely to engage by adding your own points rather than afk agree with what other people have written.

I wrote ten cards, but could easily have gotten more out if not for the fact that the meeting is only an hour long.

• OPINION: Yes we should do the thing
• RISK: Are we all clear on why we want to do this and what the benefits are?
• RISK: Needs buy-in from everyone to commit to changing process and associated behaviours, or it will only make things worse by keeping current state but with less clarity
• RISK: Beware the ‘as a tester I want to assure this feature works’ story breakout
• CHANGE: Need to start writing test plans for things properly so that testing is documented and anyone can do it with confidence?
• CHANGE: Decide how a feature will be tested before it is built, write it down in refinement as part of acceptability criteria.
• CHANGE: Limit on n of cards in dev+test TOTAL – if the limit is reached and someone wants to pick up new work, they gotta help get what’s there through the board first – prevents bottlenecking
• CHANGE: More working collaboratively eg when a ticket is finished the dev and tester have a 2-5min handoff call or smth to make sure all is chilli
• CHANGE: normalisation of adding useful comments as to status changes on tickets eg “first round dev done, pairing with Bruce today on checking xyz”

November 4th – post meeting

Thumbs up! We were all on the same page about wanting to do this and why, and there was good representation of views from different parts of the team which is awesome. There were 23 cards over all. A few of the things that came up that I hadn’t thought to write are listed below:

• To what extent do we need to clarify roles/responsibilities up front?
• We don’t have enough automated tests to not manually check most stories
• Having a single person testing the output of 5 developers doesn’t scale
• Multiple comments on how this will hopefully help the team become more quality-oriented
• Switching from a batch process for development (chucking over the fence) to a direct handoff will seem inefficient at first, however would potentially increase the speed of delivery of individual tickets

The only hiccup was a misunderstanding over the testing aspect, ie that getting rid of the columns would mean that we don’t test things, because there isn’t a tester at the end of the line checking everything that goes past.

I think I finally understand why so many people object to the title ‘tester’, because it felt like a blocker to communication on this issue. I don’t want to be the tester who tests things the developers made, I want to be an engineer specialising in quality who helps the other engineers make great stuff.

Although we all generally agree that we wanna try this out, we need a better definition of how the new process would actually work. Dom suggested we make some straw man proposals – I had never heard of these before, but I think they’re great so here’s the wikipedia link! It’s a simple proposal outline that you keep coming back to and refining – through various stages like stone man, tin man, iron man – until you end up with the final document.

I’ve written a chunky straw man (why use one word when you could use a thousand…) which details how I envisage the new process will look. I wanted to copy this into the post but it’s pretty long, so I’ve made the Notion page public instead. Anyone can read and comment on the first version of it here.

My proposal focuses on working collaboratively, with a shared responsibility for quality. Instead of handing off to a tester after the development work has been done, the software engineer has been checking in with others at every stage of work before then. It would be a drastic change to both the team as a whole, and to my role. I’m not super sure I’m prepared for it, but it could be totally awesome for all of us so I want to try it.

We also decided in the meeting that the engineer picking up a new piece of work should be responsible for writing the test plan, as it’s the best way to make sure that they’re thinking of quality from the start. We’ll still make sure there are clear acceptance criteria during refinement though, which will be a more general guide. The test plan can then be more specific to the implementation details decided by the engineer, giving them the freedom they need to experiment with implementation. Experimentation is one of our core values as a team, so we don’t want to restrict that at all.

We’ll see where this goes from here.

November 6th

Today I had my 1-1 with Dom (ACE team lead, my manager) and we spent an hour discussing my proposal. It was awesome sharing ideas and finding common ground, as well as talking out the points we differ on. I’m quite cautious and risk-minded, while he likes to go and get stuff done so it was a good combination.

I revised the stawman and made a v2 here. There were two main changes, and then a bit of reordering things.

Instead of describing the person in charge of a piece of work as the ‘primary engineer’, we switched it to ‘caretaker’ because the work going across our board isn’t all done by engineers, and because it allows for more freedom of delegation.

I also just really like that word, like you’re the person looking after a lil egg and helping it hatch and take its first steps out into the world. Fly, fly tiny ticket-bird thing!

I also added a concerns section at the bottom of the strawman, just so we all know that the risks and feelings of people affected by the change have been taken into account even if there isn’t much we can do about them.

As well as changes to the straw man, we talked about how we think the change is gunna go.

In the pub yesterday, the manager of another team brought up that they’d stopped assigning tickets, meaning that anyone can pick up bugs or issues with a ticket without thinking “Well ABC worked on that ticket so they should pick up the bugs”, leading to wait times until that developer becomes available to pick up new work again. We’re not going to implement at the same time as our other changes, but we like the idea in general.

We also decided on some actions we can both take to make the process easier. Dom is going to write a skeleton job spec for me to make sure that my new role is clear to the entire team, and I’m going to talk to some of the other engineers to find out where they stand outside of a meeting setting, and try to find a consensus beforehand. This way we could spend the next meeting making decicions instead of arguing over details.

I’ve been thinking about methods of communication, and how effective comms are different depending on whether someone agrees with you or not.

I know that getting out my fully-fleshed-out proposal with someone who usually has similar views to my own would generate new ideas and insights, bring up flaws in a constructive way and help us both to understand the process change better. But if I did that with someone who often has different views, then we could get caught up discussing every little point for an hour without getting anywhere towards agreement.

To that end, I have made a cunning plan… Instead of spewing my idea all over them, I will ask them for their proposal, and then find points of commonality with mine to build from. That way we can bring our ideas together by making a core of common understanding, and build on top of that with the best actions and worst concerns from all of our views.

OK so not so cunning, just common sense really. This goes into the realm of Leadershippy Things and Influencing though, pretty fancy stuff.

November 12th – pre meeting

Ok so we had a lot of holidays in the engineering team, plus a weekend between the two meetings so I have talked to precisely 0 engineers about it. I dreamed big, implemented nothing. Now I am very nervous. Luckily Dom had all the 1-1s in the known universe, and probably talked to everyone! I’m sure it’ll be fine…

It also seems like I’m the only person who has put forward a straw man, which worries me. If engagement is low now while it’s all idea generation, then I hardly think it’s going to be any higher later when it comes to actually doing the things.

The best laid schemes o’ mice an’ men
Gang aft a-gley

Robert Burns

Look, a poetry quote. I’m cultured now.

November 12th – post meeting

Phew. That was super stressful, lemme tell you. Lots of questions and concerns I should have been prepared for. I definitely didn’t have the answers, and that sucks – BUT!

The important thing is that we came to a common understanding. We have decided to move forward with the plan, and remove the two columns soon.

Here are some questions that came up, and the answers we agreed on:

Where is UAT?!

We debated whether or not UAT should move into the new In Progress, and decided against it in the short term. The Client Services team often has a long waiting time (sometimes days) before they can pick things up (we deploy to production before this step though), depending on their incredibly busy schedules. It would take a big change in their priorities to make it feasible to incorporate that into the new In Progress, so we’re leaving it out for now – but hopefully new behaviours around communication will mean that we can suck that in one day.

What about Ready For Release?

This one took a bit more debating, because on the one hand we want the Caretaker of the ticket to see that ticket through to Done, which is post deployment and some testing in prod. On the other hand, if we keep the Ready For Release column then we retain clarity over releases that are blocked either by another ticket’s bugs in staging or by cross-team work that needs doing. We decided to keep it in the short term, and if we end up not using the column much because we release quickly anyway, then we can just delete it later.

Will we use subtask bugs?

This question was raised because we have sometimes used the number of subtask bugs raised, or the time they take, as part of assessing flow – and having bugs written down somewhere means that theoretically we could check new bugs against ones that have existed in the past to know how to easily fix them a second time. On the other hand, bugs found at any point in the process should be turned into cases in the test plan, which then documents them without need for ‘documentation’ on the board. We’ll see what feels natural as we go along, I suppose. Some people might find it useful to have bug tickets during a transition period, so that they don’t forget to fix things.

What happens to all the tickets currently on the board when we switch over?

FREE FOR ALL. CHAOS AND FLAMES.

No, really. Whoever is currently working on a ticket is the new caretaker of that ticket (so instead of testing 7 things, I get to be the caretaker of 7 things hahaha… hah… It’ll be fine, I’m organised). We can sort out any weirdness during standup.

How will we communicate that we need help or extra eyes?

We talked about this for all of a minute before deciding that everyone’s going to have their own preferred ways of talking to each other and that’s fine. If someone needs an extra pair of eyes and they’d rather pop over to a colleague’s desk then that’s all good. If they want to put a call-out on slack, or message a particular person then that’s cool too. In-person? Nice. Remote? Nice. Carrier pigeon? Awesome! Email? Uhh, well maybe not that.

We just want to make sure that everyone has the easiest time possible of acclimatising and getting used to the change, using whatever methods of communication works for them and for their purpose.

We then delved pretty deep into defining what quality looks like, and how we know if we’re improving product quality by introducing this change. This was one of those times where I really really remember that I’ve only been a tester for a year and a half, with no senior or mentor at the company for guidance and no training during that time. I had not a single clue how to answer these questions properly, and wasn’t prepared for this branch of discussion at all. I got pretty stressed out worrying that I was suddenly expected to have a full-blown test strategy in place (which I admit would be advantageous ahah…) and a working, concise definition of what quality means to our team. Fewer bugs in prod? Fewer issues raised in UAT? Better client satisfaction? More confident changes to code due to improved test coverage?

I suppose I could have said any of those things, but I’ve had half an hour to think about it before writing this and I felt pretty put on the spot so my answer was more like “uhhhhhhhhhh”.

It then transpired that there’d been a misunderstanding about why we were making the change in the first place, probably owing to a few things I wrote in the strawman, and said while explaining it. The purpose of the change is to reduce handover queues and improve overall speed of release – not to improve quality, though hopefully that will be a side effect of developers and me working more closely together. I’m definitely going to write up a team testing strategy though.

November 13th – Day 1

We destroyed the columns yesterday afternoon. I hadn’t expected it to happen so soon. I’d just spent twenty minutes lying in a dark room, staring at the ceiling and waiting for my post-meeting stress/anxiety to fade when I got back to my desk and saw that they were GONE. It was pretty crazy, terrifying, exciting. I thought oh God what have I done I’m not ready for this.

And it’s true, I’m not ready – but if I wait until I feel ready to do things then I’ll never get anything done!

Okay this is the end of Part 1, or it’ll be way too long to read in one go. I’m writing this paragraph from the future, and I can tell you now that Part 2 is going to be as depressing and self-deprecating as anything I’ve written before. I have never before felt this useless and unprepared every single day of the week, which is saying something because I always feel useless and unprepared. I’m gunna stick with Star Wars and name the next part Empire Strikes Back. There will be crying in the bathroom, banging my head against a wall, poor foresight, poor oversight, poor undersight and sidesight, and in hindsight it might have all been a terrible idea. I’m hoping that in Part 3: Return of the Jedi, things will start looking up again.

This has been the boring set-up episode full of meetings and thinking without getting anything done, but you’d better get ready for an emotional journey of, like, self discovery and learning and stuff… Cos I’ll prolly have a lot to say later about what we should have done differently.