AAA – Australian Adventure at Atlassian

Tomorrow I start my new job at Atlassian here in beautiful Sydney, Australia. Everything about this seems like it was meant to be. Jira, the defect tracking tool was to be the next in my series of essential QA tools. I found it to be everything a defect tracking tool _should_ be.  Simply designed, robust, reliable, scalable. As I’m typing out my draft and browse the Atlassian website for more information, I find that they are hiring QA Engineers. In San Francisco! What an experience that would be for me, I thought. I clicked the link and applied. Right below that link was one that listed a QA Engineer position in Sydney!!! As in Australia?!?!? I read up on everything I can about Atlassian. I really resonate with their company values:

  • Open company, no bullshit
  • Build with heart and balance
  • Don’t #@!% the customer
  • Play, as a team
  • Be the change you seek

Atlassian Careers website: https://www.atlassian.com/company/careers

As I proceed through the interview process, I find a blog by the same people that interviewed me. Lo and behold, the Atlassian QA process greatly resembles the ideals I’m trying to accomplish with my QA teams. For reference, you can find their blog here. As I read through their blog and talk to them further, I get increasingly happy about the possibility of working for this company. There is so much I can learn from their process. I’m happy to know that everything I’ve been imagining QA could be is actually in practice here and it’s working! To my surprise and excitement, I was offered a job on the Jira team! The product I’d come to respect and admire, is the one I’ll spend night and day dreaming about how to make better (IN AUSTRALIA).

So, wish me luck as I embark on this new chapter in my career and my life!

Personal Budgeting – A fun topic

 

This post won’t be test-related. It’s “just for fun”. Budgeting. Yep, this is how I have fun. Testing and budgeting…

So recently, I was asked how I manage my budget, to which I answered… I have a complicated and long-term spreadsheet outlining all my expenses from now until 2018 and warning me if I’m going to be in trouble anywhere… I’m a nerd, what can I say? So, I decided to share my budget (a fake copy of it anyway) for those who were interested.  I’ve created copies of my budget for people who are paid weekly, bi-weekly, monthly, or paid at random unpredictable times. If you’re an excel whiz, this will be obvious to you, but if you’re not, maybe this budget will come in handy.

The pay dates are across the top. The checking beginning balance of each pay period is below that. It indicates how much money you plan to have in your checking account on pay-day. Then you will inevitably (hopefully) earn income. This is below that. Below that is the “Total Positive” which is the amount in your checking account after you get paid. From that, you will take out and spend money on all your expenses. Below that is a sum of expenses for the pay period and below that is how much will be in your checking account after the expenses are taken out. This number is used as the starting amount for the next pay period.

The way I use this budget is heavily in the first column. I like to monitor as expenses are taken out and make sure they are in line with what I expect. I remove them after they’ve been paid, then add new ones if anything comes up. I live my financial life in this budget and it keeps me planning for the future. For example… If I have a big vacation coming up, I can look at the date of the vacation to see if I have money available to spend. I can budget in advance for what date I believe I can buy the tickets, book the hotel, etc. This budget has served me well and I figured I’d share it for those interested. Happy budgeting!

 

BudgetExample

AgileFall

AgileFall

Are you stuck in the “Agile-Fall”? In software development, there are some key things that have to be done:

  1. Decide what we’re going to make.
  2. Make what we’re going to make.
  3. Deliver what we made to the customer.

The key reason for Agile software development is to ensure the “process” doesn’t get in the way of those activities. From the agile manifesto:

Manifesto for Agile Software Development We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. Kent Beck Mike Beedle Arie van Bennekum Alistair Cockburn Ward Cunningham Martin Fowler James Grenning Jim Highsmith Andrew Hunt Ron Jeffries Jon Kern Brian Marick Robert C. Martin Steve Mellor Ken Schwaber Jeff Sutherland Dave Thomas © 2001, the above authors this declaration may be freely copied in any form, but only in its entirety through this notice.

This all sounds good in theory, but when you are dealing with people who have been doing traditional waterfall for years, it can be tough to focus on what’s important ‘now’. There are many aspects of Waterfall methodology that are different and that deviate from the goals in the above list. Here is a (admittedly exaggerated) view of a waterfall project at a typical company:

  1. Talk about what we might build.
  2. Talk about the impact and the schedule of what we might build.
  3. Write down some plans about what we might build.
  4. Have several iterations of review on the plans about what we might build.
  5. Finally approve what we decided we are going to build.
  6. Start building what we want to build… But wait, something’s wrong. Go back to step 1 and start over. Repeat all this several times.
  7. Finally, we’ve built the thing.  Throw it over the wall to “test” or “QA”
  8. Uh-oh, test found a bug, go back to step 6.

… and so on.

People are so used to this and find safety in the predictability of the process. It requires quite a bit of trust in the people around you and in yourself. That last part can often be the hardest part. Imagine you are planning a vacation to a new place. You’ve never been here, so you’ll definitely want to:

  1. Talk about where we might go.
  2. Talk about the schedule about the trip.
  3. Write down the plans of what you are going to do on the trip.
  4. Make several calls to travel agents to book and revise bookings of your plan.
  5. Finalize the plans.
  6. Get ready for the trip.

…and so on.

But, I’m asking you to suddenly start doing this:

  1. Decide where you want to go.
  2. Go there.
  3. Have fun.

Would you “just do it” (Nike (c)) or would you still do all the steps that make you feel comfortable about making the trip? With the 1st list above, it’s less likely you’ll go on the trip at all. It’s more likely that you will talk yourself out of the trip. Using the 2nd list, you’d be more likely to go on the trip. It may not be 100% as you imagine, but, you’ve got the experience of the trip.  So, you understand all the benefits and agree to do my 3-step process… In deciding, you will likely rely on your experiences. Back to the trusty old first-list.

This is where a lot of companies are. They know agile is ‘better’ for them and makes more sense for them, but they really only trust themselves and their judgment if they’ve done all the “steps”. Otherwise, who knows, your trip may not be perfect?

 

Is your team using an Agile-Fall method of development?

 

 

Recognize, See, and Fuel Performance

Recognize, See, and Fuel Performance

Performance management is one of those topics that everybody dreads discussing. Managers don’t like it. Employees don’t really like it either. It’s cumbersome, it’s frustrating for everyone involved and it just may not be effective. At least that’s what Deloitte is finding with their reinvention of performance management. We all know the drill. At the end of the year, managers scramble to get their employees’ performance reviews put together and employees swallow hard and brace for impact. A final judgment on their performance over the past year.  Deloitte has some new and innovative solutions to help achieve the actual goals of performance management:

  • Improving Performance in the future
  • Rewarding top-performers
  • Retaining top-performers
  • Intervening with low-non performers

According to Deloitte, asking managers 4 questions about THEIR OWN ACTIONS regarding the employee will yield better data than asking about the employee’s performance directly. These questions are asked at the end of every quarter or project. These questions resonate with me. I already think about these things with my employees, then have to turn around and put that intuitive knowledge into a spreadsheet with numbers in it. The 4 questions asked are:

  • “Given what I know of this person’s performance, and if it were my money, I would award this person the highest possible compensation increase and bonus”
  • “Given what I know of this person’s performance, I would always want him or her on my team.”
  • “This person is at risk for low performance”
  • “This person is ready for promotion”

Deloitte’s goals were to “recognize, see, and fuel performance”. They recognized performance by offering variable compensation. They saw performance by using the 4 questions above and continuous, rather than ONLY annual performance evaluations. They fueled performance with weekly check-ins to improve performance in even the best employees.

I hope you will take the time to read this article and internalize some of the information provided. It may just change how you think about performance reviews for your team!

Reinventing Performance Management – HBR.

In Memory of Rosie

In Memory of Rosie

Rosie the Riveter is an iconic Norman Rockwell painting pictured above with its original model Mary Doyle Keefe. Keefe was paid $10 ($150 in today’s money) for posing for this image, not knowing what an icon of feminine strength this image would become. April 21st, Mary passed away in Simsbury, Connecticut where she lived.

Rosie the Riveter, often confused with the picture below, stands as a figure for the resolve of women and their growing importance in society. As a professional woman, it is amazing to take a look back at the real impact women had on the war effort, our economy, and our country. So, I want to say a thank you to “Rosie” and a thank you to the women who have worked so hard to ensure the contributions and strengths that women have to offer are not overlooked. We Can Do It, Rosie!

 

We Can Do It. J. Howard Miller

We Can Do It J. Howard Miller, often confused with Rockwell’s Rosie the Riveter

 

 

Skynet – Is it coming for the executives?

Skynet – Is it coming for the executives?

iCEO on iPad

iCEO on iPad

I’m always on the lookout for automating my life. I found something today on HBR that really piqued my curiosity. “iCEO”. iCEO offers executives the chance to “face the option of helping build these solutions now – or watch as their roles are automated out of existence.” A lot of what managers do is just splitting up tasks, assigning to the right people, then ensuring the tasks are completed correctly.   Could these things really be automated away? According to Harvard, yes, they can. iCEO promises that users can essentially drag and drop in an assembly line of tasks and the projects can be automated into completion. On initial tests, it sounds like it works to automate my job away. This tool automates job postings on Uber, oDesk, and other popular freelance sites. It hires the people to do the task, for example researching and drafting a proposal, then hires people who are professional editors or writers to review the work. Specific functions mentioned by this article that could be automated were “QA and HR”. Ouch! That’s my bread and butter.  (Of course) I feel that this work is creative, comes from my mind, and cannot be easily replicated by a machine… But, if we watch history, as David Fidler points out, the people working on detailed craft work felt the same way.

During the dawn of the Industrial Revolution, a similar argument was made about detailed craft work. However, by breaking such work down into discrete steps, automated craftsmanship quickly became possible. Assembly lines transformed the world in 50 years.Devin Fidler

According to this article, the automation technology in iCEO surpasses human workers in speed, quality, and simplicity. It sounds like as a tester, I need to focus my energy on facing the option of helping build these solutions now… You know… Before skynet takes over.

Terminator Eyes

Terminator

To automate or not to automate…

To automate or not to automate…

A lot of companies are jumping on the “automation train” for good reason… Automated testing has several benefits including:

  • Speed*
  • Predictability and Repeat-ability*
  • Freedom from time constraints*
  • Lower cost*
  • Increased Code Coverage*
  • Coolness factor

I know the tester in you is asking me why I put the asterisk by (most of) the items above. Well, because, in some cases, you don’t get to reap these benefits after your huge investment in automation. I’m going to talk today about some of the pros and cons of automating testing ‘stuff’ and why you would choose to automate or not to automate a certain “thing”. The only thing without an asterisk is Coolness factor because it is always “cool” to automate 😉

Automation Pros and Cons

As I mentioned above, there are many really cool benefits of automating testing. I have made a career out of finding ways to automate what used to be mundane tasks for testers. So let me list some of the benefits I’ve seen from automation:

Speed

Tests can be run quickly. This is true. But, how long will it take you to write the tests? If your deadline to ship is in 1 month and it will take you 3 weeks to write automated tests, automation may not be your best option. While the tests can be run quickly, how close are you to having them ready to roll? If the answer isn’t close enough, you won’t save any time at all.  The main point is… Consider the following questions when determining whether automation will help you test more quickly:

  • How quickly can you get the tests written?
  • How soon do you need to know the results of the test?
  • How often will you be using these tests? Can you re-use them next time?
  • What is the _real_ time savings? Is this a test a human being could do in less than 1 minute?

Predictability And Repeat-Ability

Automated tests always test the same things every time. Mostly true. Consider the nature of your application. If it changes quite a bit, your test results will be very unpredictable. If you can write data driven tests for a data-driven application, this is usually pretty predictable, but if you are coding to a UI that changes often, you will find the tests are very unpredictable. But, for consistent interfaces, automation does do the same thing every time. You also need to consider whether this meets your needs. One simple example I can think of is if you are a tester of a product that does voice recognition.  If you have somehow recorded speech and automated it into your test process, it’s true you will get predictable results… However, you will get ONLY those results. If your goal is to test the recognition, you need a variety of voices, a variety of background noises, etc. Automation can’t fill that need for you.  When thinking about predictability of automated tests, think about the following questions:

  • Does your application lend itself to repeating the same exact test over and over? Should there be any variation in your test input or your test approach?
  • Does your application change a lot such that automation is difficult?
  • Are there timing issues that make a difference with how the results of the test will turn out?
  • Have you covered testing the “unpredictable” portions of the product or feature as well?

Freedom From Time Constraints

Automated tests can be run overnight, 24-hours a day, 7 days a week. This is a huge benefit. The automated tester can spend 8 hours in a day working on automation and the productivity gains may be tremendous. The automation written in 1 day could be run an infinite number of times with multiple configurations.  If done correctly, automation can free up the tester to do other tasks while the automation is running. As long as the tester has made the work truly automated. Some questions to think about here:

  • Have we written our automated tests such that they can run independently and without human interaction?
  • Do we fully trust our automation to run without us there? (i.e. did we test the test?)
  • Do we have the dedicated infrastructure to maintain the automated set-up?
  • Is there a way for a human to review the results?

Cost Savings

Execution of automated tests isn’t always cheaper. There are costs involved including hardware and software on which to execute your automation, set-up costs, etc. I’ve seen cases where automation of our tests required a purchase of a huge server farm. Was it worth it? Does automation pay off? Think about the long term costs of maintaining the automation PLUS the infrastructure of the automation and compare that to the costs of testing the same things manually. Some things to think about:

  • What is the time estimate (read cost estimate) to get the tests planned, written, coded, data creation, etc?
  • What is the infrastructure cost? What software or hardware do you need?
  • What is the maintenance cost of both items above?
  • What is the cost savings if we decide to implement automated tests here?

Increased Code Coverage

People believe that if you have automation, you will automatically also get increased code coverage. This is not necessarily true. The automation must be implemented strategically to ensure you are maximizing the value. As a tester, you know you have limited time to get the testing done. If you spend a huge chunk of it automating a small part of the software, you will have lost out on testing the other areas. Be sure you have analyzed this carefully to determine if it does in fact give you any additional testing over the course of the project.

Coolness factor

If all of this makes you change your mind about automation, just think… You could automate stuff in your spare time if you want because it’s just damn cool!

Happy Automating!

 

 

Essential QA Tools – Test Case Management

TestRail

TestRail

Essential QA Tools Series

This series “Essential QA Tools” will cover my thoughts on tools that testers find indispensable in their daily lives. This is not a comprehensive guide by any means, but will cover some of my favorites.

Why Test Case Management Software?

One of the most important software tools in a QA tester’s toolbox is test case management software. I’ve seen testers using a variety of tools for managing their plans and ideas for how to successfully test software. From enterprise-level tools to excel spreadsheets, word documents, and even just “in the tester’s head”, every team has their own unique way of managing the test approach.

For those unfamiliar with the purpose and benefits of using a test case management, let me give you a quick run-down. Testers make plans to test a piece of software, conduct peer review of these tests, estimate test scheduling, execute tests, and track test results. All of these items can be done with a test case management tool. Software projects can have anywhere from a couple hundred tests to thousands of tests. With that kind of volume it’s clear that the just keeping it together ‘in your head’ may not be the most error-free option. For this reason, professional testers rely on test case management software.

Featured Tool – TestRail

By far, I have found TestRail to stand out above their competiros. TestRail meets the needs of the QA user without over-complicating like other tools tend to do. Gurock has focused on what they do best. Make simple, easy to use test case management software. The modern UI is simple to learn. Within days of trying TestRail (the free trial is a bonus!), I was able to navigate the entire system easily.

My most-used features

  • Flexibility – I’m able to make and use custom fields for individual test cases. I can create individual test steps for more traditional step-based testing, but by default, it’s set to pass/fail at the test case level. This is great for those who are using the simpler approach.
  • Baseline Support – With TestRail 4, Baseline support was added so you can create multiple versions of your test repository when you have multiple versions of a product active at once.
  • Bulk Edit – TestRail 4 has a new Bulk-Edit feature for test cases that was by far the most requested update. With this new feature, you can update several tests at once quickly and easily.
  • Test Estimation – When planning test cycles, it’s important to understand how long it will take to get through a “single run” of the tests. For this reason, TestRail has an estimate feature that is amazing. As testers create test estimates and then add test runs of those tests, TestRail automatically calculates the total estimate. This is something testers have had to do manually in the past and TestRail makes it quick and easy.
  • Configurations – With configurations, you can easily execute the same test suite on multiple operating systems or platforms.
  • Tracking – As tests are run, it is easy for everyone on the team to look at a single page at a glance to see the progress of the software.
  • Speed – This isn’t really a “feature” but a benefit. With some enterprise-level systems, there is quite a large amount of overhead and setup required. With TestRail, once it’s running, it’s just quick. Quick to customize, quick to use, and just plain responsive.

There are a multitude of features available in TestRail that may make it right for your team. They have a pretty impressive list of customers including big names like Apple, Microsoft, eBay, and Amazon.
Is TestRail right for your team? What test case management tool does your team use and why?

5 Ways the QA team can help create a culture of shared accountability

4 Men in sinking boat, only 2 pouring water out

Shared Accountability

So, you’re on the QA team at your company. It’s your job to assure quality, right?

Or… As the saying goes… “Quality is everyone’s job”. So, it’s everyone’s job, right?

Accountability is something all teams should have despite who is responsible for the quality of the product. Everyone in the entire SDLC should have some quality-related performance metrics in their job description. People in official “Test” roles tend to be more likely to have “quality on the brain” all the time, whereas others may tend to treat it as an after-thought. I’ve seen various levels of quality from team members who produce high-quality work almost every time to those who didn’t ensure that requirement #1 was working properly before delivering to test.

One of my favorite books is Nudge – Richard Thaler which describes decision architects and their role in shaping the decisions of others. I argue that part of the responsibility of any good quality assurance organization is to nudge the rest of the group in the direction of building quality into the product from the start. Here I will describe 5 things I’ve done to do just that.

1. ‘Stuff’ Rolls Downhill

Sometimes the decision-makers in your organization are the ones who are causing quality and accountability problems. Many of the times, they don’t even realize it (Think managers who enable poor performers by constantly covering for them) If you can align with major decision-makers at your company and get them on the same page with you, this will go a long way in making changes.

2. Money Talks

Since testers are generally not in control of earlier phases of the development cycle, it’s harder to improve bug avoidance during those phases. To nudge those who do control that area to improve, point out the costs of finding bugs in various times of the cycle. The earlier in the cycle you find bugs (essentially avoid them), the cheaper it is for the company to correct. Here is an account of relative bug costs by phase:

  • Definition Phase – $1
  • High Level Design – $2
  • Low-Level Design – $5
  • Code – $10
  • Unit Test – $15
  • Integration Test – $22
  • System Test – $50
  • Post-Delivery – $100

-The Art of Software Security Testing: Identifying Software Security Flaws

Based on these numbers, you can see how crucial it is for system testers to find bugs pre-release but there is still significant cost savings if everyone in the process finds and avoids bugs as well.

3. Do Root Cause Analysis

Analyzing the cause of each defect found later than expected will help shine a light on areas that need improvement. Perhaps our requirement review process can be improved. Perhaps unit testing could be more robust. There are many causes of bugs and figuring out the earliest time we could have caught each bug helps us find (and hopefully fix) bugs in our process. Keep in mind this process is not intended to blame people for mistakes, but rather to determine if and where our process needs to be improved. Postmortems are a good non-threatening way to talk about the process and make improvements.

4. Build Relationships

It is crucial that testers build relationships with those creating the product. It helps you in many ways. It helps you understand their philosophies about product development and also helps them understand your philosophies. It builds a sense of teamwork and reminds us that we are all working towards the same goals, even if we don’t agree 100% on the methods. Remind the others in your SDLC that you aren’t here JUST to complain about the product, but rather you are here to provide a sense of clarity to our work. When you write a bug, don’t blow a horn or anything to prove your greatness as a tester and their poor performance as an engineer… Instead use this as an opportunity for an open dialog about product quality.

5. Walk the Walk

As someone who is trying to drive accountability in other areas of my company, it is imperative that I hold myself and my team to a high standard. It’s important that the test organization has a sense of accountability and product ownership. This goes quite a long way. If you make a mistake, own up to it. If your team misses a bug, include your processes in the root cause analysis. Don’t assume quality is everyone else’s job. It’s yours.

QA Is my Life!

InvisibleVex

Invisible Vex

So, anyone who knows me will know that I test everything all the time, even outside of work. Friends, family, my toaster, my TV, and maybe most importantly: Video Games. I am an avid Destiny player and I play almost every day if I get the chance. Like I said, I find bugs in everything… Destiny is not bug-free by any means, so I took a video of one that I found the other day while playing. Why am I talking about this? Well… Because I’m a tester and I get obsessed about bugs I find in my every day life. Bugs that annoy me. I want to share with people what software bugs look like and how totally cool my job is where I get to find things like this in software and point them out.

Here’s the story of this video. I was just walking around on Mars trying to get my 9000 Experience Points without dying bounty. Anyone who plays Destiny will know how annoying it is to be killed unfairly while completing this bounty. I spot some Cabal Legionaries just shooting at what appears to be a rock. Instead of continuing the game, I get distracted to go investigate why they are shooting a rock. What do I find? This invisible Vex is there. I dig deeper. Why is only his red dot showing… What kind of vex is this? Why isn’t he walking around? Can I at least kill him? What does he do when I move around him? These are the types of questions I ask at work every day… Except I’m asking about the company’s software I’m testing.

I guess the real reason I’m writing about this is to give people some insight into what “Software Testing” is for those who have never seen it in action. This is me “Software Testing” my video games!