Goodbye Testing Columns, Part 2: The Empire Strikes Back

In Part 1 I spoke about the lead-up to changing our team’s process and behaviours by deleting the testing columns from our kanban board. This was mainly a success because we talked about it and then we did it. Classic getting-things-done action right there.

In this second installment of the trilogy, I’m gunna talk about the aftermath of the change, the problems we faced as a team and the ones I faced alone while staring down the unlit road of maybe this is what my job is now but I’m not really sure and neither is anyone else.

Part 3 will be include the grand overview, a study of what worked and what led to the results we got.

November 13th – Day 1 continued

I’ll be honest, I’ve spent all day in this weird kind of limbo. I cleared out things that were assigned to me, and now there are no tasks on me and no one has asked for any guidance. I’ve done a few bits and pieces of automation, mostly improving what exists and adding a couple more test cases. The developers are working with each other a lot more, which is great to see, but I definitely thought that this change would make me feel more a part of the team, not less. I have a catchup with my manager tomorrow afternoon so I’m just going to write up documentation this afternoon and prepare some points to discuss with him.

Places I have cried today: the bathroom at work; at my desk.

November 15th – Day 3

I thought that I had wasted most of yesterday on documentation, but it turns out that it can be useful sometimes! I never knew because I suppose I didn’t have the time for it before…

I basically took a bunch of our 300+ ad configurations and put them into ‘product sets’ of similar configs, which eradicates the need during new client rollouts for slack conversations along the line of “which configs do I need to enable for X type of ad, again?” -> “who knows, I think we might have been drunk when we named them”. We’d like to use tags or something in the future so that it’s easier to version and enable new product types.

I’ve also created the first draft of a proper team testing strategy so that anyone can pick up testing on a piece of work, knowing that they can do something mostly good. Our team has a few different types of work that we do including localisation, client rollouts for ads, tool/UI creation and the development of APIs and cloud functions for various things. It’s pretty diverse, but it’s been good for me to write down some notes on how I test different types of work.

I still don’t have a rough spec for what my job actually is now, which makes it difficult for me to reliably say what I should be doing. I mean, I know roughly what I should do in the grand scheme of things, but in the right-now tiny scheme of things, I have no idea. I could just be using this as an excuse to keep feeling lost, thus avoiding responsibility for having to learn how to do my job all over again. That seems like something I would do.

It feels a bit like my first month with the company again, where there was no onboarding and I had to push and push and push to learn the most basic things. I guess that’s really bad, but at the same time I did fine then so at least this time I know that pushing and pushing and pushing actually works. I have the vague belief that at some point I will turn around and realise that I’m doing ok.

My manager and I did decide on some actions that I’m going to follow up on in the mean time though.

• Introduce team to the test strategy I wrote up, get feedback
• Annoy people into pairing with me on what they’re doing
• Bribes?? Bake another walnut banana loaf, that’ll do it
• Improve test automation in the areas that have some
• Write proposal for better config management

Places I have cried today: nonewhere, yaaas

November 18th – Day 6

Tomorrow the columns will have been gone for a week. This morning I spent an hour crying onto my keyboard because I still have no idea what I’m doing. I keep having to remind myself that I knew from the start this would be a massive learning curve, both in terms of my testing abilities but also when it comes to working methods. Since I’m quite a social being, I hadn’t really noticed before how much I was doing in isolation, and so I hadn’t taken into account how difficult I would find it to just rock up to someone and ask to pair with them on whatever they’re doing. It feels like an unwanted imposition, and I have doubts over what I have to offer in that situation.

Another part of it is the loss of control. I like things that are organised, and I like to be the person organising them, so it’s stressful being in this limbo where anything could belong to anyone, and I can do whatever I want. I told one of the developers at lunchtime that I was feeling lost, and he asked “Well, what do you want to do?”

I don’t know. I do not know, for the life of me. I’m so preoccupied with being lost that I’ve lost sight of my goals.

This has been going on for days, and I have no idea who I can talk to about it. My manager already seems pretty stressed about everything, and there have been some less than courteous words exchanged in the team over slack, which was a massive surprise. Lack of clarity over the process has everyone on edge, I think – even though we did agree on a process, and it’s vaguely written down, it might be that everyone was expecting more time to work out the details and think on the changes.

I’ve spent the afternoon trying to add that clarity with mixed to poor results by encouraging the developers to pair with me and by wading into tense conversations and keeping things chill.

Luckily we have the retrospective tomorrow, so there will be an opportunity to air out issues and reinforce positive behaviours.

Things I need to do:

• Check in with others to make sure they’re doing ok with the change
• Prepare a retrospective for tomorrow afternoon
• Ask in the devs channel if anyone wants to pair, instead of wallowing
• Prep for the two meetup talks I’m doing this week and next, c’mon Bruce jeez

Places I have cried today: in bed; on the loo; in the kitchen; on the sofa. Aaaaaaaaall over my flat, basically. Tears up to the knees in every room, I’m telling you.

19th November – Day 7

Most of the issues that came up during the retro were the usual things, and I was surprised at first that no one else was bringing up the process change. I can see now that although the change to my role was massive and difficult, everyone else has just been carrying on with their jobs and not really struggling at all other than feeling that the defined process was pretty vague. I’d also assumed that it was the main contributor to the stress the team was feeling, but in reality it was bigger in my own head than it was in anyone else’s.

We listed our villains (risks) of the next iteration and then placed them on a Threat Level board, and confusion around the new process was placed in Tiger which is the second-lowest category. This really put it into perspective for me – the issues I’ve been facing are just teething problems, and I need to communicate about them and be proactive in ensuring that everyone knows what I’m here to help with.

Actions that came from the retro:

• Bruce to create an ace quality channel in slack
• Everyone to make threads in there when they have need of Quality (ahah)
• Bruce to arrange Destroy Columns Checkin for next week

Even in the few hours left of the day after retro, I was able to sit with three separate developers to pair on testing. It seems that people just weren’t sure exactly how/where they were supposed to ask for guidance, and at which stages of work. Having a dedicated channel means that the discussions are concentrated and kept from cluttering other channels, and information isn’t lost in DMs.

All in all, I’m pretty happy right now.

Places I have cried today: during the walk to work.

26th November – Day 14

Things have been going a lot more smoothly since the retro. With a new slack channel for seeking help, we’ve dedicated a place to have conversations. We’re finally feeling the benefits of reducing our queue time, which is satisfying for everyone.

The simple act of checking for new bugs used to involve so many queues… Tickets went through development, then got dumped into a queue waiting for testing, then bugs were found in testing so the work entered a queue for bug fixing until a dev was available to pick them up. Then the bugs got fixed and it went back into the testing queue again. Looking back, it was pretty ridiculous, and I wouldn’t be suprised if many tickets spent more time queueing than they did actively being worked on.

Our current process is more like this:

It’s basically a direct conversation, with other bits thrown in for funsies. Turning this common test-dev interaction into a conversation helps ease the way for having other quality-related conversations as well.

I mean, I’m not going to say that this is perfect, and there are aspects we need to improve on. For example, we’re still doing most of the quality assurance activities at the end, but devs are actually checking their stuff and solving the really obvious problems before-hand. We also have fast feedback within the engineering team, and I feel pretty confident that things will be ok while I’m on holidays next week. We won’t get a repeat of the last time I went on hols, where I came back on the Monday to a mountain of testing. xD

We had the Destroy Columns Checkin today, which was a chance to air out any problems and find out how everyone’s feeling about the change.

The checkin was split into three parts: General Feelings; Comments; and Actions.

We quickly ascertained that we’re all on the good side of indifference, and that no one regrets destroying the columns. Winner.

The Comments section was one big box, with a ‘Gooder’ and ‘Badder’ side. I quite liked having one box, instead of two sections for ‘Things that went well’ and ‘Things that went badly’, which is what we usually do. It sped things up, anyway.

I won’t pop a screenshot in here because it’s too crowded, but the main points that arose were:

• Without Bruce being the person who is aware of everything that has been merged and gatekeeping releases, deployments felt a bit hairy at times
• We need to better define what quality looks like for a piece of work early in the process, ie during refinement
• Lost clarity of progress on the board
• No metrics yet to know if our throughput has improved
• Some testing work felt too simple for pairing on
• Fewer bugs raised
• Direct handoffs feel much faster
• Rubber ducking via handover notes catching issues before they became issues
• Developers being more thorough and thinking about quality
• More tests written
• Problems sorted way faster

What’s super cool is that I wrote none of those things. I’m the only QA and I facilitated this discussion, but these thoughts about how we ensure quality as an engineering team, came from the whole team. That’s super crazy awesome.

Places I have cried today: my desk, but it was happy emotional tears so it’s all good.

10th December – Day 28

It has now been four weeks since we destroyed the testing columns. I was on holiday all last week, and I can safely say that when I returned, there was no mountain of tickets built up in ‘Ready For Testing’. There also was no mountain of unanswered requests in the slack channel, so that’s all good. Peeps were checkin’ each others’ work and being generally awesome.

We realised that we don’t use the ‘Ready For Release’ or ‘Subtask Bug’ columns at all since removing other queues, so they’re also gone. We just have things that are in progress, then we deploy them to production where they get UAT’d (this is our new bottleneck) and then moved to Done.

More interestingly, I was talking to a couple of people from another engineering team about the change and found out that while I was on hols last week, they also decided to delete their testing columns. That’s a good sign, because it means that others in the company have spoken to us and seen what we’re doing, and they think it’s pretty cool. We must be doing something right.

It’s still early days, but it seems to have been a bit more of a struggle for them from what I could gather. When we made the change, we talked about it first. We made sure that everyone affected by the change knew it was coming. We gathered insights, acknowledged risks and agreed on new behaviours. This is because we’re a team of planners. Sometimes it seems like we spend too long planning and not enough time doing, but our twin team is the total opposite. They embrace the make-fast-fix-fast style, and it works well for them I think.

When they deleted the columns, there was at least one person on holiday, and more members of the team who were just not brought into the discussion at all. These people were very surprised and confused indeed to discover that everything had changed while they were away from their keyboards…

I’m going to talk in more depth with that team, since this provides great extra data for the process change, and then for the third and last part of this blog series I will give an overview of what worked or didn’t, and why.

For those who recently attended my Diversity and Inclusion talk at the MOTLeeds meet-up, you might remember the ‘Actually Useful Slides’ that had tips on them instead of Dr Who gifs. The next post will be the equivalent of that…

The Actually Useful Blog Post, if you will.

I’ll try to keep it concise, intelligent and informative. Unlike my usual ramblings (oh, who am I kidding, it’s going to be rambly and winding as a mountain goat trail xD).