DEV Community: Ben Dowen

How to make a start with Test Automation

Ben Dowen — Sun, 18 Sep 2022 20:49:29 +0000

Sometimes, people ask me how to make a start in Test Automation, or as you might correctly hear it called automated checks. While there is no one answer, because of course it depends, I can pose some questions for you to ask yourself and point you to some great resources, so lets goooo!

Do I need to learn to code to build automation?

No, but yes, well it depends!

There are many tools that exist that do not require explicit programming language knowledge to build automation. Most of these tools do benefit from knowledge of how control flow and conditions work, even if they don't need you to understand the syntax for a language.

Concepts like if and else statements and loops are pretty much unavoidable, and if you want to automate web pages you're going to need some idea about locators, and it wouldn't hurt to understand the Document Object Model (DOM).

Learning to code can also bring a bunch of other benefits, like better understanding the software you're testing and being able to have a different level of conversation with developers. So while learning to code is not a requirement for being a Tester or automating checks, I highly recommend giving it a go.

Transition from manual to automation

If you're already working in Software Testing, but haven't yet written any automation and want to make a start, there are a few questions to ask yourself:

Are you looking to skill up to add automation as part of your current role at your current employer?
Are you looking to take on a new role, at your current employer or elsewhere?
Do you already know how to code, and are simply looking to gain experience in new tool or framework to start your journey?

Of course, as well as your desire, you will need the opportunities, if you don't already know what is available with your current employer it might be worth finding out by asking around.

If you've never written code, then instead of learning test automation right off the bat, you might want to pick a language and learn the fundamentals first. There are many free, and paid ways to learn how to code.

Some great places to start:

Finally, if you have the opportunity to pair with a developer or existing test automation engineer, take it! If you don't already have this opportunity, seek it out. Learning by doing, with the help of others is super valuable and will speed up your learning no end.

Use automation to aid testing

There is no need to limit your use of automation and tools to a suite of automated checks that you might typically consider test automation. In fact if you do, you are probably doing automation and testing a disservice.

There are multiple proponents of using code and tools to support exploratory testing in different ways, one of them who I am lucky enough to work with presently is James Thomas. You can learn more about how James Explores with Automation.

A champion on using code in testing is Maaret Pyhäjärvi, just one fine example of her work is Exploratory Testing on Computer Interfaces (APIs), a Keynote from API Conference in 2019.

And of course, there is Mark Winteringham and Richard Bradshaw who coined the term Automation in Testing, and have produced fantastic courses, workshops and tools for learning how to get the best out of Automation, well beyond automated checks.

Best language to learn for automation

So, where to start? Java, Python, JavaScript, C#? Tell me damnit!

While I can totally understand why you would want a general answer as to what language to use, there isn't one definitive answer.

So, I will ask you to consider the following:

What do you want to learn first? If you haven't got any experience in any language, go spend some time looking at what is available and if any of them make good sense to you. If you already know a bit of something , start there!
Are you looking to grow yourself within your current role in your current job? If so, speak to your team, see what would be useful for you to learn next
If you're looking to grow skills to pick up a new role, look at what is popular in your target area of the world. The popularity of one language over another might be different for the places you are considering working for. Try and find people working in the geographical location you're targeting, recruiters or hiring managers, and ask them.
Careers are long, if you spend time learning the fundamentals of coding, testing, and automating checks using code then you can apply much of those skills across multiple tools and languages. Yes, there is a learning curve to picking up a new thing, but it gets easier with every new one you try.

Resources for learning

One of the absolute best resources for learning Test Automation in my experience is the Test Automation University. Not only is it free, but the content is written by and for testers, so it's way more relevant then the odd testing focused chapter in a code course otherwise written for developers. The are even some introductions into coding on there, for example this course on Java Programming by Angie Jones and this Introduction to JavaScript by Mark Thompson. There is even a whole learning path for Codeless automation.

If you don't have a background in computer science, you may also get a kick out of the Base.cs podcast. In each episode the co-hosts Vaidehi Joshi and Saron Yitbarek introduce and discuss a topic in an approachable way, with great anecdotes, many of them food related and you know I dig that!

Practice apps and challenges

The Restful Booker is a great practice app for exploration or automation. It's a REST API and a frontend for a functional Bed & Breakfast, Shady Meadows.

The Evil Tester, Alan Richardson has created a series of API Challenges that help you explore many of the aspects ot APIs that you might want to test, and therefore automate. While this isn't strictly an automation challenge, you could turn it into one by writing code to beat the challenge! I've started to give this a go myself, but it's work in progress.

Want more? There is a great *post on The Club *, where loads of people have shared products and sites they have used to practice testing on. You can even go add your own!

Just start

Finally, It's way better to start, then spend too much time worrying about where to start. You can also change your mind and try something else later if you don't like where you end up! I'm looking forward to hearing how your automation journey progresses, make sure to hit me up on Twitter, @FullsnackTester, and let me know!

Originally published on my blog:
https://www.dowen.me.uk/how-to-make-a-start-with-test-automation/

Cover Photo by Tara Winstead: https://www.pexels.com/photo/robot-pointing-on-a-wall-8386440/

How to test anything in 5 steps

Ben Dowen — Mon, 15 Aug 2022 18:27:14 +0000

Originally posted on my bog: https://www.dowen.me.uk/how-to-test-anything-in-5-steps/

How to test anything:

Capture what you know so far and build a model
Review the model with others, identify what is important
Explore by running experiments to learn what you don't know
Update the model based on what you know now
Repeat this cycle until you stop finding useful information

Capture

We all build models of the things that we test, these might be mental models we keep in our head, informal models such as mind-maps or more formal models such as state transition diagrams.

Build models that help describe the behaviour of the system you are testing, and capture the information in a way you can easily share and review with others.

Review

Reviewing work with others is important for a number of reasons, mainly because by updating it with input from multiple people we can reduce bias from what we know ourselves and get closer to a truth that is useful.

You can use questions get useful feedback on your model, such as:

What is missing?
What isn't correct?
What is important?
Does this feel right?
Does this sound right?
Does it add up?

Explore

Now you've got an iteration of your model, you have some ideas about how the system should behave so it's time to test those ideas against the current reality of the system.

The goal here is to explore to learn useful information, where to focus is going to depend on your context. Here are some things you might want to learn:

If reviewers didn't agree, find out how the system behaves now?
You identified behaviours that are important, do they indeed behave as you expect?

Your model will have gaps, open questions, fill those knowledge gaps with exploration. You can run your exploratory testing in many ways, use your current model as a guide about what is important, and how you think the system should behave.

It's very important you take good notes as you explore, so you can discuss your findings with your team and use the outcome of that conversation to update your model.

Update

Reflect on what you've learned during this iteration and make sure to update your model based on what you know now. Maybe you found surprises during your exploration, and now you know more, put it in the model.

You will need to make choices as a team if a surprise you find is a bug, or intended behaviour that you haven't modelled yet, so keep them involved as you go.

Repeat

This process is a cycle, that you can continue to repeat as you continue to test the system.

On each iteration, you will learn more, if you stop learning useful information that your team can use to make better choices, you're done for now.

When the next change comes along, you can build on existing models and create new ones as appropriate. It need not be a single model that you build, use whatever is appropriate for this iteration of the cycle.

How others test anything

My esteemed colleague James Thomas as his own take on how to test anything, these are complementary to what I've said above and less cyclical.

You can read find out more about James's approach:

https://qahiccupps.blogspot.com/2020/05/how-to-test-anything.html

https://youtu.be/Lj6RjWA-arI [In 3 minutes]

Graphical Test Planning

My cyclical approach to testing is based on the Capture, Review Update from Graphical Test Planning (GTP), developed by Hardeep Sharma from Citrix Systems, many years ago.

You can learn a little more about GTP here:

https://sites.google.com/site/gtpfortest/

Graphical Test Planning describes a way to model system behaviour in Structured Relationship Diagrams, and how to write Test Case Diagrams that model how to test single observable behaviours.

Cover Photo by Ravi Kant: https://www.pexels.com/photo/low-angle-grayscale-photo-of-empty-brick-stairs-1917328/

Building software is a team sport

Ben Dowen — Wed, 01 Jun 2022 10:56:02 +0000

As I recently posted on Twitter, I really believe that building great software products, services and solutions is a team sport. In some team sports, teams are made up of a fixed number of people, playing fixed roles. In others, squads of multidisciplinary players swap in and out for each game, even mid game.

Then there are the supporting roles, the people who coach the team, the physio who keep them well. How many people does it take to allow two formula one drivers to race a single race, for one team? Hundreds at least!

Building software based products is a similar story, one of teams of people coming together all playing important roles. And like sports, how many people, in what roles, is dependent on a number or key things. Two of these factors are scale and budget.

At the most basic, ten people could out together two five-a-side football teams, hire a pitch and play a game of football. They have no coach, or physio or any of the many other supporting roles. How many people do Manchester United have? Scale, budget, context.

Teams building software are no different. One developer can make software. Budget is low, but there is only one mouth to feed (and maybe their Family). At this scale, any bug fix interrupts development. If multiple customers have competing priorities, one has to wait.

Anything new must be learnt before they can use it. If they go on vacation everything stops. There is no one else to support customers. Some customers will avoid using products from a single developer, because in their context they need uninterrupted support. So... We scale.

As we scale companies, we might have goals we want to achieve with this new capacity. Being able to attract larger customers with deeper pockets and commit for longer. Scope for taking holidays. Building new features faster. Build and operate reliable products.

This is where we start to introduce roles. We no longer have only Developers. Now we have people who look after Product, Sales, Support, Operations, Legal, HR, Quality and more. Depending on context, and scale, these maybe people or services from other organisations.

Somewhere along the way, as the company grows we might run into problems. We grew our budget, added roles but we still haven't met our goals and some of us are miss the days a small team could ship their own product without help.

One way we might try and fix this, is by shaping our teams into multidisciplinary powerhouses. With Devs, Ops, Product and Quality all together, maybe more too! This power team can play their part of the game. But did we solve our problem? Meet our goals? Are we tracking them?

Over time, sports teams and hope they are supported have evolved. Formula One today is light years ahead of where it was in the 1980s. And so too software teams and organisational structures change.

What is important is to understand your goals, identify your problems and measure your progress. Don't get distracted by how others are playing the game, focus on what works for your team! This is a journey. I've heard again and again "No such thing as a DevOps engineer"

I've also heard, that there should be no Testers. Just Developers who are coached and taught how to test. This might work great for some teams That doesn't mean you should feel bad for having your DevOps and QEs in your success team. In your current context, budget and scale.

Originally posted on my blog: https://www.dowen.me.uk/creating-software-is-a-team-sport/

Cover Photo by NEOSiAM 2021: https://www.pexels.com/photo/selective-focus-photography-of-foosball-table-595229/

Building your toolbox and sharpening your tools

Ben Dowen — Sat, 28 May 2022 10:24:44 +0000

Originally posted: https://www.dowen.me.uk/building-your-toolbox-and-sharpening-your-tools/

Too long, didn't read:

Tools can include mental models, methods or software
Get inspired by following people on social media, read blogs and attend conferences
Learn new tools from resources such as presentations, blogs. books, tutorials and courses
Create regular opportunities to practice using tools, so you can be comfortable using them and even teaching others

Quality Engineering

Tester? Quality Engineer? QA? There are many named roles in Software Testing, that can imply different work and scope, but often involve some of the same core skills.

What is your bag? People, Process, or Product? This is the question that has led Meg to have a Job role existential crisis!

In response I gleefully welcome Meg to my world, where I attempt to blend all of these things into my current role as a Quality Engineer.

https://twitter.com/FullSnackTester/status/1529206878793871361

Think about this a bit more, what did I mean and how can it be helpful for others?

Build your toolbox

As I have grown in software testing over my career, I like to think I've made progress becoming a better Tester. A large part of what that means to me, is that I have expanded my toolbox. I know more ways to solve problems, so not every problem looks like a nail and I reach for my hammer less and less.

What are tools?

In this context, as well as software tools such as Postman, I also mean thinking tools like MindMaps, and techniques like Exploratory Testing.

If you want a toolbox that covers People, Process and Product, you will need to collect a broad range of tools.

Learn from resources

The primary way to learn new things, is to start by consuming resources. You may already be familiar with learning from Books, Courses, Blogs and other resources.

Resources I've learned from include:

[Book] Explore It! - Elisabeth Hendrickson
[Course] Test Automation University
[Blog] QA Hiccupps - James Thomas
[Blog] A Tester's Journey - Lisi Hocke
[Newsletter] Software Testing Weekly

Learn by experimentation

Another excellent way of learning new tools, techniques and methods I've used is by finding opportunities to try things out.

By running short experiments, we can quickly find out if something is worth further effort to learn deeper. While I have tried out many tools and run many short experiments, I don't have a useful example to share.

Sharpen your tools

Now you know a bunch of new things, you have new tools in your toolbox! This is a great start, because it expands your knowledge base and helps you choose more specific tools for the problems you encounter.

What I find, is that however much knowledge I consume, however much theory I ingest, I don't feel comfortable using tools until I've spent a good amount of time being uncomfortable and practice using them.

Around the start of the year I had done some good learning Exploratory Testing form resources, but I was not comfortable using it regularly.

In order to sharpen my Exploratory Testing skills I setup regular sessions to practice, every week and on the software my team is working on. This worked better then I hoped, and I am now comfortable Exploring and working with others introduce them to the method.

In summary: Create regular opportunities to practice using tools.

Anatomy of test automation

Ben Dowen — Mon, 01 Feb 2021 23:24:11 +0000

With the rise of low-code and no-code solutions, I've been thinking about the problem they are trying to solve.

This lead me to think wider about the parts of Testing that can be supported by tools and automation.

System under test

In the begining, you need the correct version of the System Under Test.

This maybe as simple as taking the latest build of a single isolated service.

More likely, this will involve getting matching versions of multiple services that work together to form a system.

This task alone may not be trivial, more then likely it can be automated with help of a Continuous Integration (CI) platform.

Something like Jenkins that can be configured to kick off builds when you commit changes to source control.

All CI systems I know of require at least minimal code, or at least a configuration file of some sort that gives instructions on how to build the Software.

Likely this is a wider concern then Test Automation, you won't be the only one who wants software built.

But I've worked in consultancies who still build the odd thing by hand and FTP it to a server.

So if you don't already have a slick answer for this, it's a barrier to automation straight away.

The supporting environment

The SUT will seldom live on it's own, unsupported by dependencies. This might e mean having the right Node or Java versions available.

It could also mean setting up:

Databases
Storage
Queues
Proxies
Load balancers
Third party API or appropriate mocks

Each of these may also need special configuration or base data seeding that isn't specific to any given scenario.

If you are lucky you can get some or all of your dependencies deployed using some type of automation. Maybe Docker images that can be Deployed by your CI System.

Living the dream. More automation, definitely some code, hopefully some Infrastructure as code.

Failing that, you might have some static servers. hopefully managed by a friendly Ops engineer. Otherwise another job for you.

Again, if this isn't already slick and automated, another barrier to entry.

Test scenario

Okay, the big one. Let's assume we overcame all the hurdles above. Now let's break down our Test cases.

Prerequisites

In addition to all the setup you've done to get this far you might still need a big more. Maybe some users with appropriate access.

Test data

If your inputs and outputs are static, this might be trivial.

For me, I often work with non trivial test data that requires some level of templating.

I often need to make sure dates represent today, even if I wrote the test and captured the data months ago.

In some cases test data can be generated automatically using libraries like faker. I know some tools, like Mockoon include Faker out the box.

System state (Given)

Now your going to want to get your system into the starting state.

Some examples:

User is logged in
A the right page is loaded
An entry doesn't already exist for new TODO item you are about to create

This is something your automation framework, code or otherwise really should handle.

Finally, something we can recognise as Test Automation.

Action (When)

This is the core of the test. Stimulating the users action or a sequence of multiple actions that we expect to make some change to the system state.

Examples:

User clicks a button
API call is made
File is changed

Assertion of expected result (Then)

This is where most of the debate comes in about Automated Checks and Human testing. Ignoring any Ai for a moment.

This is where we have a coded assertion. I don't mean we need to be using a programming language, but we need to have an unambiguous way to choose if the actual results we got are what we expect.

If they match our coded expectations the test passed, otherwise it falls.

While we can make test smart to an extent and we can look for shapes and ranges, ultimately we can only assert on what we can expect.

This can definitely be done by any automation framework, whatever the language, low-code or no-code.

If we got this far we are winning, provided of course we did our Test Analysis right and we are checking for useful things.

Logging and reporting

Analysing logs and making reports can definitely be fully or partially Automated.

This might take the form of some Console output that can be captured by your CI System and be linked to a build. It could also take the form of a HTML report, complete with graphs, screenshots, API responses to even video recording.

What you will probably miss is the contextual logging. The logs from the dependencies and more distant parts of your SUT. This can also be captured and logged, but takes a fair amount more thought.

Tools and supporting script's

Of course this maximalist description describes pretty much an end to end or system integration test.

As I'm sure expert's like Mark Winteringham, Richard Bradshaw and Alan Richardson would tell you, you can get great value from Automation and Tools to support your testing way short of an end to end test.

I've done this myself on plenty of occasions by creating tools to capture or generate test data, of by mocking APIs to support my Exploratory Testing.

Conclusion final thoughts

If you were not convinced already that to succeed, an automation strategy need support and buy in from many people in various roles, I hope you are now a convert.

In my opinion, and I am making some assumptions, low-code or no code only attempts to solve.

In fact, I would go further and say Test Automation Frameworks only try and solve part of the problem. And that's OK.

Building software is a team game, and Software Quality even more so.

Hosting a Twitter bot for free on Heroku

Ben Dowen — Sat, 30 Jan 2021 21:20:56 +0000

Cover Photo by Kaboompics.com from Pexels

Originally posted on my learning journal: https://journal.fullsnacktester.com/learning/2021/01/30/twitter-bot.html

!! Please note, Heroku are removing their free dynos, so you can no longer run a Twitter bot on Heroku for free. See https://blog.heroku.com/next-chapter

Following a tutorial

I started writing the @TesterOfTheDay Twitter bot by following a tutorial. I am not ashamed to admit it, I wanted to make a Twitter bot, not learn how to make a Twitter bot. So I used the final state of the tutorial as my starting point.

You can find the tutorial I followed here:

How to create a Twitter Bot using Node.js and Heroku

I have no intention of repeating anything from that tutorial, if you want to follow it, go read it. Of course I changed it enough so that it searched for #TesterOfTheDay.

If you want to start from my final state, go right ahead and grab the code from GitHub:

testeroftheday-twitterbot

Using dotenv to protect credentials

The tutorial used a config.js method for providing the app with Twitter API keys, with a vague note not to leak your credentials. As I wasn't familiar with how to use a config.js in a secure why while hosting my repo openly on GitHub, I went with what I know and used dotenv.

For local development I have a .env file that I do not check into GitHub. The way dotenv works, if a .env file is present it loads the content into environment variables, accessible in NODE via process.env.

require('dotenv').config();
const twit = require('twit');


const T = new twit({
    consumer_key: process.env.CONSUMER_KEY,
    consumer_secret: process.env.CONSUMER_SECRET,
    access_token: process.env.ACCESS_TOKEN,
    access_token_secret: process.env.ACCESS_TOKEN_SECRET
})

I then configured the environment variables (my Twitter credentials) in Heroku. For this I followed the Heroku documentation: Configuration and Config Vars

Switching to 10 minute schedule for huge savings

The problem

By default Sumedh Patkar Twitter Bot uses setInterval to trigger once every minute. This gives a very fast response, and is infrequent enough to comply with the Twitter API rules, as far as I can tell at least. This method has one big consequence, it was burning thorough my free dynos, aka credit on Heroku.

The solution

I changed from using an infinite loop to a one off execution that could be triggered by the Heroku Scheduler.

Note to install the free Heroku Scheduler plugin, I did need to add a Credit Card to my account. This had an added benefit of doubling my free dynos.

Now the scheduler in the free plugin only allows a trigger once every 10 minutes, but for the Tester of The Day bot this was still plenty. Realistically there are only a small handful of Tweets on that hashtag every day and no where near enough to justify a 1 minute repetition.

You can see for yourself the minor code change on GitHub in this commit:

https://github.com/dowenb/testeroftheday-twitterbot/commit/ea095b8572f29555e548c2b61afae386406fea3f

It also required following the docs and setting up the Scheduler plugin. The job simply runs node bot.js every 10 minutes and was trivial to setup once I had added my credit card.

Make sure you also turn off the web and worker Dyno formation. You don't need that running, because execution now comes only from the scheduler.

The savings $$$

On a free Heroku account you get 550 "dyno hours", raising to 1000 when you register a credit card.

In a month with 31 days, running the Twitter bot 24 hours a day in a loop would consume 24 * 31 hours, that is 744! Even when adding my credit card, this wouldn't of left me with much credit to host anything else for free.

With the change to running one every 10 minutes, in January my Twitter bot consumed only 51 hours. What is more, that includes over one full day wroth of hours a the start of the month before I made the change!

So I've one from using up all my credit, to gently sipping away at it. I expect in February I'll see the full benefit with even further reduction in usage.

In conclusion

If you don't need a process to respond to web requests, and you don't need it to be running 24/7 consider using the Heroku Scheduler. It could allow you to run for free what might otherwise cost your more, or reduce your usage if your are already on a paid tier.

While I found this change valuable on Heroku with my Twitter bot, the principle is potentially transferable to other situations. Let me know how you get on!

Tester of The Day

While I am here, let me tell you, Tester of The Day is a daily award that celebrates great people who contribute to the Software Testing community. Winners are induced into the Tester of The Day: Hall of Fame. Check it out!

Testing HTML5 Upload

Ben Dowen — Mon, 23 Sep 2019 14:48:31 +0000

Background

A current project I am testing involves a HTML5 file upload. I had thought that the "Browser" a file prompt would prevent me from using a Selenium test framework to automate this process. However it turns out, as with many things that this is a "Solved problem".

The short answer is, you can identify the "Browser" button input element and use "Send Keys" function to provide the path to the file you want to upload. You then trigger the submit butting using the "Click" command as normal.

Setup a virtual environment and install SeleniumBase

I entered the following commands in a PowerShell prompt, I already have Python 3, pip and virtualenv installed.

cd e:\dev
virtualenv uploadtestenv
.\uploadtestenv\Scripts\activate.ps1
pip install seleniumbase
seleniumbase install chromedriver
seleniumbase mkdir file_upload_test
cd file_upload_test

SeleniumBase Help docs are available: https://github.com/seleniumbase/SeleniumBase/tree/master/help_docs

file_upload_test.py

To run this example - copy the following into a file called file_upload_test.py

from seleniumbase import BaseCase
from time import  time
from os import remove

class MyTestClass(BaseCase):

    def test_basic(self):
        # setup testdata to be uploaded

        # time based filename generated for this run only
        filename = str(int(round(time() * 1000))) + ".txt"

        # path with filename
        path = "E:\\dev\\file_upload_test\\" + filename

        f= open(filename,"w+")
        for i in range(10):
            f.write("This is line %d\r\n" % (i+1))
        f.close()

        # SelinumBase test
        self.open("https://the-internet.herokuapp.com/upload")
        self.send_keys('#file-upload', path)
        self.click('#file-submit')
        self.assert_exact_text(filename, '#uploaded-files')

        # Clean up testdata
        remove(path)

To run the test (again, PowerShell, same folder):

pytest .\file_upload_test.py --demo_mode

Thank you

This prototype example makes use of the HTML5 Uploader on the wonderful "The Internet" resource, by Dave Haeffner. A very helpful resource for helping to test out Selenium tests. Check it out: https://the-internet.herokuapp.com

The test data creation example is based on a Guru99 article: Python File Handling: Create, Open, Append, Read, Write

Cover Photo "Python Book" by Christina Morillo

Works on my machine

Ben Dowen — Fri, 19 Jul 2019 16:22:45 +0000

Originally posted on my blog:
https://www.dowen.me.uk/it-works-on-my-machine/

So, you just found this great bug and it's returned "cannot reproduce", or you have been told "it works on my machine", classic right?

As a tester, in this situation, it can be easy to get defensive. Am I being told I am making it up? Am I being disrespected, why should I raise bugs if they are not important enough to fix?

My suggestion, turn this on its head and consider this for a moment. Time to get technical! What details can be added to help identify the root cause of the issue? Can you do more investigation and help understand what is different between the environment you tested on, and the one the developer tried to reproduce it in?

Ways you could add value in response to "it works on my machine" in include:

Identifying the exact build or deployment you observed the issue on, and check is this the same build the developer is checking against? Can you check the build they are using, maybe the issue is already fixed, but you haven't been given the latest code?
Identify what platform are you both using. If it's web, make sure to check not only the Browser but the version of that browser. If it's mobile, what version of Android or iOS is installed?
Is there any additional logging you can get to help add context? If you can find an exception being thrown that is captured in a log, this can go a long way towards helping identify the issue.
If you can continue to reproduce the issue, can you give the developer access to your machine or environment? If you are remote, can you set up a screen sharing session? At a minimum, it should be easy enough to capture a video of the steps. for this, I use Open Broadcast Software Studio https://obsproject.com/download, but many alternatives exist.
If any of these have helped get to the root of the problem and get the bug fixes, try and use them more often in your usual bug reports, and see if you can reduce your "Works on my machine" count.

I hope my thoughts on this are useful, I am really interested to know if you agree, disagree, or even better have additional hints and tips! if you do, please let me know on twitter: https://twitter.com/dowenb

Two simple tests you should be using with Postman

Ben Dowen — Fri, 12 Apr 2019 14:02:21 +0000

Add some tests

Postman is a fantastic part of your testing toolbox. I've recently had cause to use it to help track down a performance issue.
One thing I found pretty quickly, is that reading a wall of "PASS" results in the test runner, and trying to pick out slow responses is painful.

So how do you get a bit more out of Postman, without investing time writing detailed tests? Simple, use the built-in examples!

Capture the request you want to test

First you need to capture the request you want to test. If you don't already know what this is, you can capture it pretty easily.

Load Dev tools in a browser such as Chrome, visit the page you want to test, identify the network request and right-click copy request (cURL Bash).

Now use the "Import" option in Postman, and import as "Raw text", past in the copied request.

Tip: Note if your request is authenticated, this will only work while your bearer token is valid.

Open the "Tests" tab

Open the "Tests" tab on your request, this is where you will see, on the right, a list of built-in examples.

Check HTTP status code

At a very minimum, you should have an idea what return code your expecting. If you expect the request to be successful, your probably looking for "200" OK.

For other HTTP codes see: HTTP Status Cats or HTTP Status Dogs

pm.test("Status code is 200", function () {
pm.response.to.have.status(200);
});

Check Response time

And here is the fun bit, setting an expected response time. This is another built-in example, make good use of it!

pm.test("Response time is less than 200ms", function () {
pm.expect(pm.response.responseTime).to.be.below(200);
});

Use the Collection Runner to run it lots of times

Now if you want to check running the request many times - save the request to a "Collection" and use the Test Runner to run it lots of times!

For extra danger, try injecting faults or creating a "worse case scenario" while the test is running to see how your service responses to fault conditions.

Learn more

I learned a lot of what I know about Postman from the excellent free course by Amber Race on the Test Automation University: Exploring Service APIs through Test Automation.

What do you do, when you are stuck?

Ben Dowen — Thu, 28 Mar 2019 15:05:07 +0000

I've been struggling with bug I've been trying to investigate now for over 24 hours. At this point, I need a break, so much so I'm considering going to the Gym, if you know me, you will know that is extreme.

This leads me to my semi-serious question, what do you do when you get stuck?

If I get stuck for a short amount of time on something, I make a cup of tea. Maybe watch a few minutes of a show on Netflix and let my mind wonder.

But now I'm stuck for a little longer - what should I do? Of course part of the answer is ask for help, and I am. But I'm thinking more about tactics other people use to break down problems I might be missing.

Your input is most welcome! Please comment below.

Show off your Test Automation Skills!

Ben Dowen — Wed, 20 Mar 2019 09:09:53 +0000

I recently posted about the Test Automation University:

Path to Test Automation Success

Ben Dowen ・ Mar 6 '19

#testing #automation #learning #qa

They have just introduced a shiny new feature, where you get a "Certificate" for completing a course, that can easily be shared on Twitter, LinkedIn or put on your CV!

They have also introduced a few new courses since my last post, including:

Introduction to TestNG with Rex Jones II
Automating your API tests with REST Assured with Bas Dijkstra

If you have already completed a course, you can
find your certificate here.

For example, here is one of mine:

Got any other, creative ways, to show off your learning and skills? Maybe you have a great technique for building a portfolio, or you do all your learning in open GitHub repos so you can show perspective employers?

Let me know! I'm looking for new ways to record and show off my learning - while also opening myself up so others can more easily critique my work and give me feedback. Discuss in comments.

Testing IS for Beginners

Ben Dowen — Thu, 14 Mar 2019 17:16:14 +0000

Jon Calhoun recently posted a very detailed explanation of why "Testing is Not for Beginners"

Testing is Not for Beginners

Jon Calhoun ・ Mar 13 '19

#go #beginners #testing

I can relate and agree to much of the content, but reject the main premise.

Health warning

I am writing this to express myself and understand for myself why I think test analysis skills are important to developers. If you find I am waffling on, feel free to move on! honestly, I won't judge or be offended.

Also I probably have lot's more to say on the topic I haven't managed to get down yet - so I may even make further posts. Deal with it.

Bias of a professional tester

First of all, please know that I am biased. I have been working professionally as a Tester, and not a Dev since 2012. For longer then that I have been an armature developer, but I have never done it professionally.

I reject your reality, and replace it with my own

I honestly believe learning some of the skills of a Test Analyst can make even a new Dev a better Dev. But how?

OK so I hope you will agree with me that:

Finding bugs early saves time and sanity
Bugs in Design or Requirements can be very expensive
Learning from failure is good and important, besides when does it ever work first time?

Where I totally agree whit Jon, if you try and learn how assertions and unit tests work on day 1, before you learn concepts and syntax, you're going to fail. In that context, "Testing is NOT for beginners", sure.

But how can Test Analysis help with any of this?

Fundamentally Test Analysis is not about writing tests. It is about learning, exploring and asking questions.

What kind of questions? I won't cover that again in detail, as I already made a post about that.

Questions you should be asking before you test a new feature

Ben Dowen ・ Mar 12 '19

#testing #testanalysis #qa #beginners

My premise here is:

The more you understand about the problem you're intending to solve (Requirement) the easier it is to break it down into smaller problems and solve them
Thinking time invested early is time well spent and should save you time later

But what about spikes and prototyping and LEARNING how to develop?

Of course, this doesn't mean you shouldn't being experimenting and learning! To be clear when I talk about a New Dev, I really don't mean some day 0 of your journey.
Anyway this is still part of exploring and exploring is important!

When you get to the point when you're starting a job, or contributing to open source projects you're no longer doing so on your own. This is where it starts to become more relevant to think beyond the code.

How do you know you are done / it's working?

There is no need to use formal TDD or BDD while you are doing early learning. Instead spending time to undetand what functionality is important to get working, even for a demo or learning project, is useful.

It's useful because it helps you be done. It's useful because it helps you break things down into smaller problems.

Also it will be even more useful when you're working in a team, because you may only be responsable for a part of the functionality. And you will need to understand where "your bit" ends and some one else begins.

It can be useful to express some of this in some form of Test Case, or expected behaviour, so you can exercise your code and see if it does what you expect, or if you need to keep going!

Why "working" code isn't enough

Guess what - once you get as far as a job as a Developer, or as a Tester in part of an Agile team. Working code isn't good enough.

You have to remember the context for why code is being written in the first place. And usually this will be to solve a Business need of some kind, assuming your intending to make money (or make some one else money). Or at very least your solving a problem.

Often the problem being solved is part of a bussiness process that could be solved without technology at all, or historically relied on analogue equipment. Think shop till and paper ledgers, mail order to modern e-commerce.

This is where applying some of the fundamental analysis skills from testing come in:

Remember to ask "Why?" - a lot. If you understand the problem you can identify better solutions
If your lucky enough to have direct access to your customer/client, maintain a dialogue. Try and "Speak the same language", and this won't be code. But it might be models, mind maps, UML, Flow Diagrams, white boards, conversations. If your really unlucky it will be written statements
Remember or working on a solution to a problem, and not making software for the sake of it

Enough already! let me get coding

OK, I'm done. I genuinely hope this was useful for you, but honest I mostly wrote it to be able to express my opinion.

Do make your own opinions and experience known, comment, write your own post, share. I am interested to learn from you so feel free to let me know!