Thank you for this article Jess! Testing vocabulary is indeed unsettled and sometimes very confusing. I found this article on Guru 99 has been helpful in the past, and now I will be adding yours to my resource list!
A couple points I would like to share.
"...Acceptance testing is usually a set of manual tests...". Acceptance does not have to be a manual process. Ideally the entire test/s process should be automated as to facilitate rapid deployment. Tools like uilicious.com/ , cypress.io/, and selenium to name a few.
"...There isn't necessarily pass/fail criteria associated with a performance test...". Failing would be 'application did not load' :) . Honestly though when application reaches the point of implementing performance testing a 'known good' level of operation is know. That becomes the baseline performance. Change should decrease hardware usage per request, decrease wait time to respond, or both. Rarely should changes increase the error rate.
Thank you again for the article. I really enjoy seeing and reading other people care about testing and knowing more about it as it applies to software.
Ah. I think I was using a slightly more literal definition of Acceptance Testing - literally checking that the specs are met on a quick pass through - but those resources for more robust acceptance testing could be very helpful to someone setting up a new environment. Thanks!
Also, for performance testing, I'm assuming if you get to this point, then the application will load. I'd hope you have failing tests well before this stage if things are that broken! :) If you have any good resources on "known good" performance levels I'd love to add that to my personal resource list! It can be tricky because so often when we test things as developers, we're testing on high speed ethernet on fairly good computers, and that is not an accurate depiction of all our users. Having some accepted benchmarks that work for a wider variety of environments would be very helpful!
Re: Acceptance Testing, your view is correct. Those 'specs' can be provided in a machine format (image, HTMLDom, etc) to be compared to the output from the test run. :). Reading some of the other comments I agree 'Acceptance Testing' covers a wide swath. Maybe break out Security, Compliance, etc. I have no idea what they would be called though. :S
Re: Perf. Testing, Unit tests can pass, integrations pass, but the running application does not respond before a timeout; but maybe does in 120 seconds. A trick I learned was that Chrome can throttle the network speed of a request. This means headless version can as well :). The other side of Perf. Testing is x successful responses in Y seconds under Z load. The measure of these can be benchmarked using APM tools (Application Process Monitors). New Relic, Sentry, and AppDynamics are three of the more well known vendors.
"Application Testing": as wide as the horizon and as deep as a gravity well of a black hole.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Thank you for this article Jess! Testing vocabulary is indeed unsettled and sometimes very confusing. I found this article on Guru 99 has been helpful in the past, and now I will be adding yours to my resource list!
A couple points I would like to share.
"...Acceptance testing is usually a set of manual tests...". Acceptance does not have to be a manual process. Ideally the entire test/s process should be automated as to facilitate rapid deployment. Tools like uilicious.com/ , cypress.io/, and selenium to name a few.
"...There isn't necessarily pass/fail criteria associated with a performance test...". Failing would be 'application did not load' :) . Honestly though when application reaches the point of implementing performance testing a 'known good' level of operation is know. That becomes the baseline performance. Change should decrease hardware usage per request, decrease wait time to respond, or both. Rarely should changes increase the error rate.
Thank you again for the article. I really enjoy seeing and reading other people care about testing and knowing more about it as it applies to software.
Ah. I think I was using a slightly more literal definition of Acceptance Testing - literally checking that the specs are met on a quick pass through - but those resources for more robust acceptance testing could be very helpful to someone setting up a new environment. Thanks!
Also, for performance testing, I'm assuming if you get to this point, then the application will load. I'd hope you have failing tests well before this stage if things are that broken! :) If you have any good resources on "known good" performance levels I'd love to add that to my personal resource list! It can be tricky because so often when we test things as developers, we're testing on high speed ethernet on fairly good computers, and that is not an accurate depiction of all our users. Having some accepted benchmarks that work for a wider variety of environments would be very helpful!
Re: Acceptance Testing, your view is correct. Those 'specs' can be provided in a machine format (image, HTMLDom, etc) to be compared to the output from the test run. :). Reading some of the other comments I agree 'Acceptance Testing' covers a wide swath. Maybe break out Security, Compliance, etc. I have no idea what they would be called though. :S
Re: Perf. Testing, Unit tests can pass, integrations pass, but the running application does not respond before a timeout; but maybe does in 120 seconds. A trick I learned was that Chrome can throttle the network speed of a request. This means headless version can as well :). The other side of Perf. Testing is x successful responses in Y seconds under Z load. The measure of these can be benchmarked using APM tools (Application Process Monitors). New Relic, Sentry, and AppDynamics are three of the more well known vendors.
"Application Testing": as wide as the horizon and as deep as a gravity well of a black hole.