What do you try when you want to find bugs before your customers do?

How do you know you are ready to present your software? Do you attempt to find any showstoppers or critical issues before a release goes out? Is shipping a fearful thing or is it second nature all day every day? How do you find things you might have missed?

Did you find this post useful? Show some love!
DISCUSSION (7)

How do you know you are ready to present your software? When it works. :)

Do you attempt to find any showstoppers or critical issues before a release goes out? Yes, always. Take the requirements documentation / minimal functionality docs and ensure the functionality behaves as expected.

Is shipping a fearful thing or is it second nature all day every day? If you want to be and stay completive, you should be able to deploy every second of every day with minimal fear.

How do you find things you might have missed? Automated Testing, feedback, end user interaction.

To catch errors in software before being deployed is an entire industry unto itself. I would start with automated testing. Unit, Functional, Integration, etc. Test,test,test,test,test,test.

Next, automate the deployment process. Deploy to multiple non-production environments. Create an 'alpha' group from the client/s to sign off on the release before being deployed to production. This gets the clients buy in (and thus shared responsibility) on the release.

I could, and sometimes do, go into this topic in depth. Feel free to reach out to me.

The first question coming to my mind is: »What is a bug?«

Depending on how you define bug, the answer varies.

I find "bug" is the most overused techical term within our industry - and unfortunately even outside on the customer's side. Everything is a »bug« nowadays. And everyone thinks "bugs" are "critical" and so on.

I try to avoid the term most of the time - sometimes it slips over my lips which I regret immediately.

There are several classs of things which are described with the label »Bug«

  • simple typos: 10 taks completed-message instead of 10 tasks completed
  • simple mistakes: Choose between »a« and »a« instead of »a« and »b«
  • Display logic shows unexpected results (which are in itself correct but not expected to be seen)
  • results of calculations are just wrong
  • your webapp comes up with a HTTP 404, HTTP 400 or even HTTP 500.

...

The list goes on and on.

And for each of these classes of things which could go in the wrong direction, there are tools and methods to help detect these.

But I think the most important thing for me is to understand that there are ways to detect problems but not to prevent them.

Depending on what kind of software you build you have to make a cost-effort matrix to find the steps to take. There should be differences in developing software, where peoples lives depend on literally or only metaphorically, i.e. building e.g. medical products vs. the new productivity team communication app.

Working in a branch which is similar to the latter, I have not so hard requirements I can tell how my last project went with this:
It was software for the public sector (local scale, small userbase, small data).

  • How do you know you are ready to present your software?
    I constantly developed preview versions, deployed and the customer reviewed changes. So I presented my software in a pre-beta-phase. But I released when we both agreed for a release. Timeline was flexible here.

  • Do you attempt to find any showstoppers or critical issues before a release goes out?
    Showstoppers were in the category of HTTP 500 so yes, I clicked the paths.
    The customer did the same.

  • Is shipping a fearful thing or is it second nature all day every day?
    As I released on a near daily basis: it wasn't fearful.
    But "going" live is always a bit "thrilling".

  • How do you find things you might have missed?
    The interesting thing is that working closely with the customer dealing with "missing" things is less painful. There is no "blame mode", where a division between customer and provider of a service gets visible.

Since we both worked on the app closely, missing is more of an oopsi, we both haven't thought about that and makes fixes pleasant. As soon as we noticed, I fixed and both were good with it.

That doesn't work for every product and every customer but it was the mode I liked to work in the most so far.

Fast releases. ASAP fixes. No blame culture. Shared responsibility.

Customers are the best QA testers out there. They'll always find the bugs you never expected and no automated tests will find. I unit test as much of the app as possible, then let the customers do the rest for me. But I'm a 1 person dev team on preventalemon.com.

In my day job we have more devs, and we do a round of internal testing with a handful of people, and have the clients do their own QA too. But still, we'll always find new bugs in production.

I think the best way to find bugs that customers would experience, is to ACT like a customer would. Complete tasks and goals that a CUSTOMER themselves would complete. i.e, drop the developer role for a minute and act like you're a user who's using the software without any knowledge of the way it works!

I'm in charge of UAT for our applications and honestly, you fight a losing battle no matter what.

I write a script for each individual aspect of our application, used how it's supposed to be used. Each interaction needs to be screenshotted or something else to prove it was successful. It is the signed off by the tester, who will be a random employee. I require two different testers and have all scripts pass, which at least one full round of testing passing.

If possible, I let other employees play around and try to break the application before it goes out.

I think the mantra is "It'll never be ready, so just ship it". As long as you are confident in the tests and, imo, have proof other people agree with you then it's the best you can do.

Checkout cypress.io for that initial acceptance level testing.

Testing your own software is always tricky. After all, you wrote it so you know how it is supposed to work and when you use it will likely do the "correct" things.

So you have to force yourself to try stuff that might be unusual to you.

If it turns out your product is something you actually can use yourself then you get to "dogfood" it for a while which can help find issues.

Getting trusted people to help (perhaps a small Beta group) can be a great way to get early feedback before you ship to everyone.

Regardless of what you do, your users will still find problems. So you should have a way to deal with them and provide rapid updates. I've found that customers can tolerate issues more when they know that you respond with updates quickly.

Classic DEV Post from Jun 11

🤔 THOUGHTS ON GITHUB

Microsoft acquires Github? Github open source contributions? Do you need Github projects to get hired? Amy Codes shares some thoughts about Github.

Alan Barr
Software Testing Architect passionate about getting people into homes. Learning new things about technology and software that helps people.

How well do you know your own code?

Sign up (for free)