Friday, May 26, 2017

Incremental steps to continuous releases

The last eight months for me have had one theme in particular that I consistently drive forward, in small steps that sometimes feel small enough that others don't realize how things are changing.

There's an overall vision in mind for me: I want to take us through the transformation to daily releases for the windows client + management backend product I'm working with.

Where I started from

As I joined 8 months ago, the team I joined that been working for several months on a major architectural type of change - no releases but a build that could be played with internally. We had "8 epics" to drive through the architectural changes, and none of those were done. There was a lot of dependencies all around and making a release someone would use wasn't a straightforward task.

I started in September. The first release went out November 23rd.

There's more than a decade of history on making continuous releases of the detection and cleanup functionalities within the product, but the frame of the product has been released annually or quarterly for production use, and monthly or biweekly for beta - something I was introducing here a decade ago.

When I started talking of daily releases, I was told it was impossible. It took me 4 months to get rid of the "it cannot be done" comments.

The pain of regularity is necessary


I had a firm belief (which I still hold) that when things are deemed hard, you just need to do more of them to learn how to make them less hard. So I struggled with my team through the discussions of "releasing takes too much time and is away from real work", with the support from our manager setting it a team goal tied to bonuses that we would turn our 4 day release to a 4 hour release.

Each release would see a little more automation. Each release would see a little more streamlining. We would find things that would be difficult (not impossible) to change and postpone those from focusing first on the low hanging fruit, never giving up on the ultimate goal: a touch of a button releasing to various environments.

A month ago, I could happily confirm that the 1st goal as it ended up being written down was achieved.
[Team Capability] Turn 4 day release to 4 hour release
We believe that ability to make our client releases with shorter duration will result in saved time in making multiple releases. We will know we have succeeded when team does not feel need to escalate release-making as a threat to features.
We also worked on another capability:
[Team Capability] Min 2 people can make client releases
We believe that having at least two people with skills, knowledge and accesses to make client releases will result in being able to make releases while one is sick. We will know we have succeeded when release happens without 1st key person present at office within same / similar timeframe.  
What next?

We have come to a point of bi-weekly releases, which is only taking us to the level I introduced decade ago. But building on that, the next things would be to figure out ways of not breaking the builds within the 2 week intervals, and that change takes me far away from just my own team, including changing the ways test automation supports our development.

There's still work on making the four hours into four minutes of work, and I look forward to stepping through that challenge.

Our very first production environment release was just done. With more environments in play, each 4 hours can easily grow into five times this, so that would be a next step to work on too.

So the vision I'm working for:
[Team Capability] Four-minute release throughout the environments
We believe that having a push-of-a-button release will result in us focusing more on valuable features and improvement for the user and our organization. We will know we have succeeded when releases happen on a daily basis as features / changes get introduced. 
Why would I, the tester, care for this?

I have people every now and then telling me this is not testing. But this fundamentally changes the testing I do. It enables me to test each change, isolate it and see its impacts all the way through production. It supports small, human-sized discussions on changes together in the teams and gives us an ultimate definition of done - production value over task completion.

It makes developers care about the feedback I give, and enabled the feedback to be more timely. And it makes way for the necessary amount of thinking and manual work to happen in both coding and testing so that what we deliver is top-notch without exerting too much effort into it.


Pair Testing with a 15-year-old

A few months back, I had the pleasure of working with a trainee at F-Secure. As usual in schools in Finland, there was a week of practice at work with the options of taking a job your school assigns you (I did mine at age of 15 in an elderly people home) or you can find one of your own. This young fellow found one of his own through parents, and I jumped on the opportunity to pair test with him.

At first, he did not have a working computer so it was natural for us to  get started with strong style pairing:
With an idea from my head to keyboard, it must go through someone else's hands (Llewellyn Falco)
He was my hands, as I was testing firewall. And in many ways he was a natural in this style of work. He would let me decide where to go and what to do, but speak back to me about his observations and ideas, extending what I could see and do all by myself. Most of the things we did together were things I would have done by myself. Only difference was the times of going to the whiteboard to model what we knew and had learned, where I guided him to navigate me in the ideas to document very much in the same strong style pairing. As the driver drawing, I would ask questions based on our shared testing experience when he would seem to miss a concept.

His ability to test grew fast. He learned to use the application. He learned to extend his exploration with test automation that existed and play with it to create the type of data we wanted.

My reward was to see him enjoy the work I love so much. His words on the end of our joint experience without me prompting still make me smile: "I now understand what testing is and would love to do more of it".

He joins us for a full month in June. I can't wait to pair up with him again.

Wednesday, May 24, 2017

Impact of Test Automation in my Everyday Worklife

I'm not particularly convinced of the testing our teams test automation does for us. The scenarios is automation are somewhat simple, yet take extensive time to run. They are *system tests* and I would very much prefer seeing more things around components the team is responsible for. System tests fail often for dependencies outside the team control.

I've been actively postponing the time of doing really something about it, and today I stopped to think about what existence of the minimal automation has meant for me.

The better test automation around here seem to find random crashes (with logs and dumps that enable fixing), but that is really not the case with what I'm seeing close.

The impact existence of test automation has had for my everyday work life is that I can see with a glimpse if the test systems are down so that I don't need to pay attention to installing regularly just to know it still installs.

So I stopped to think: has this really changed something for me, personally. It has. I feel a little less rushed with my routines. And I can appreciate that.

Tuesday, May 9, 2017

Bias for action


'Bias for Action'. That's a phrase I picked up ages ago, yet one that has been keenly on my mind for some time now.

It means (to me) that if I can choose planning and speculating vs. doing something, I should rather be doing something. It's in the work we do that we discover the work that needs doing.

There are things I feel need doing, and I notice myself trying to convince others in doing those over being alone in doing those. I notice being afraid of going in and starting the restructure of our test automation to a shape that would make more sense.

Without bias for action, I procrastinate. I plan. I try to figure out a way of communicating. I don't get anything done.

With bias for action, I make mistakes and learn. I make myself more vulnerable and work with my fears of inadequacy.

It's been such an important thing to remember: things don't change without changing them. And I can be a person to change things I feel strongly for.

Thursday, April 20, 2017

Dear Developer

Dear Developer,

I'm not sure if I should write you to thank you for how enthusiastically you welcome feedback on what you've been working on and how our system behaves, or if I should write you to ask you to understand that is what I do: provide you actionable feedback so that we can be more awesome together.

But at least I want to reach out to ask for you to make my job of helping you easier. Keep me posted on what you're doing and thinking, and I can help you crystallize what threats there might be to the value you're providing and find ways to work with you to have the information available when it is the most useful. What I do isn't magic (just as what you do isn't magic) but it's different. I'm happy to show you how I think well around a software system whenever you want to. Let's pair, just give me a hint and I make the time for you.

You've probably heard of unit tests, and you know how to get your own hands on the software you've just generated. You tested it yourself, you say. So why should you care about a second pair of eyes?

You might think of testing as confirming what ever you think you already know. But there's other information too: there are things you think you knew but were wrong. And there are things you just did not know to know, and spending time with what you've implemented will reveal that information. It could be revealed to you too, but having someone else there, a second pair of eyes, widens the perspectives available to you and can make the two of you together more productive.

Us tester tend to have this skill of hearing the software speak to us, and hinting on problems. We are also often equipped with an analytic mind to identify things you can change that might make a difference, and a patience to try various angles to seeing if things are as they should be.  We focus our energies a little differently.
 
When the software works and provides the value it is supposed to, you will be praised. And when it doesn't work, you'll be the one working late nights and stressing on  the fixes. Let us help you get to praise and avoid the stress of long nights. 

You'll rather know and prepare. That's what we're here for. To help you consider perspectives that are hard to keep track of when you're focused on getting the implementation right.

Thank you for being awesome. And being more awesome together with me.

     Maaret - a tester

Time bombs in products

My desk is covered with post-it notes of things that I'm processing, and today, I seem to have taken a liking to doodling pictures of little bombs. My artistic talent did not allow me to post one here, but just speaking about it lets you know what I think of. I think of things that could be considered time bombs in our products, and ways to better speak of them.

There's one easy and obvious category of time bombs while working in a security company, and that is vulnerabilities. These typically have a few different parts in their life. There's the time when no one knows of them (that we know of). Then there's the time when we know of them but other don't (that we know of). Then there's the time when someone other than us knows of them and we know they know. When that time arrives, it really no longer matters much if we knew before or not, but fixing commences, stopping everything else. And there's times when we know, and let others know as there is an external mitigation / monitoring that people could do to keep themselves safe. We work hard to fix things we know of, before others know of them because working without an external schedule pressure is just so much nicer. And it is really the right  thing to do. The right thing isn't always easy and I love the intensity of analysis and discussions vulnerability related information causes here. It reminds me of the other places where the vulnerabilities were time bombs we just closed eyes on, and even publishing them wouldn't make assessing them a priority without a customer escalation.

Security issues, however, are not the only time bombs we have. Other relevant bugs are the same too. And with other relevant bugs, the question of timing sometimes becomes harder. For things that are just as easy to fix while in production and while developing an increment, timing can become irrelevant. This is what a lot of the continuous deployment approaches rely on - fast fixing. Some of these bugs though, when found have already caused a significant damage. Half of a database is corrupted. Communication between client and server has become irrecoverable. Computer fails to start unless you know how to go in through bios and hack registries so that starting up is again possible. So bugs with impacts other than inconvenience are ones that can bring a business down or slow it to a halt.

There's also the time bombs of bugs that are just hard to fix. At some point, someone gets annoyed enough with a slow website, and you've known for years it's a major architectural change to fix that one.

A thing that seems common with time bombs is that they are missing good conversations. The good conversations tends to lead to the right direction on deciding which ones we really need to invest on, right now. And for those not now, what is the time for them?

And all of this after we've done all we can to avoid having any in the first place. 


Wednesday, April 19, 2017

Test Communication Grumpiness

I've been having the time of my life exploratory testing a new feature, one that I won't be writing details on. I have the time of my life because I feel this is what I'm meant to do as a tester. The product (and people doing it) are better because I exist.

It's not all fun and happy though. I really don't like the fact that yet again, the feedback that I'm delivering happens later than it could. Then again, as per ability, interest and knowledge to react to it, it feels very timely.

There's three main things on the "life of this feature". First it was programmed (and unit tested, and tested extensively by the developer). Then some system test automation was added to it. I'm involved in the third part of its life, exploring it to find out what it is and should be from another perspective.

As first and second parts were done, people were quick to communicate it was "done". And if the system test automation was more extensive than it is, it could actually be done. But it isn't.

The third part has revealed functionalities we seem to have but don't. Some we forgot to implement, as there was still an open question regarding them. It has revealed inconsistencies and dependencies. And in particular, it has revealed cases where the software as we implemented isn't just complicated enough for the problem it is supposed to be helping with.

I appreciate how openly people welcome the feedback, and how actively things get changed as the feedback emerges. But all of this still leaves me a little grumpy on how hard communication can be.

There are tasks that we know of, like knowing we need to implement a feature for it to work.
There are tasks that we know will tell us of the tasks we don't know of, like testing of feature.
And there are the tasks that we don't know of yet but they will  be there.

And we won't be done before we've addressed also the work we just can't plan for.