DEV Community: Vivek Mannotra

How does Cloud Testing help Product Designers?

Vivek Mannotra — Tue, 05 Jul 2022 04:40:54 +0000

Designers play an important role in converting meaningful business opportunities into billion-dollar tech companies. They create the design of the software application, which includes the user interface, layout, and overall look and feel. In addition, designers are also responsible for creating branding and marketing materials.
According to a study by the Software Usability Research Laboratory, designers can improve the usability of a software application by up to 58%. Furthermore, another study by IBM found that good design can lead to a decrease in development costs by up to 50%.

In addition, designers also collaborate with other teams to ensure that the final product meets the user's expectations. Using cloud-based tools for testing purposes has allowed teams to work together in a real-time collaborative mode, let's explore further.

Stages of application design & development

Designing an application entails understanding the purpose of the application and the needs of the users. The design process begins with understanding the problem that the app will solve and the users that will be using it. Only once the design is finalized, the development process begins. This includes coding the app and testing it on various devices to ensure it works as intended.
In a conventional team setting, you will have a business analysis team which works closely with the business leadership in identifying relevant business opportunities. The business analysis team documents the expectations and goals of the business leadership and hands them over to the design team.
The process of business analysis leads to design requirements by identifying the needs of the business and determining how those needs can be met. It includes identifying the goals of the business, determining what the business needs to do to meet those goals, and identifying the resources that the business will need to meet those goals.
Business requirements are communicated to designers through a variety of methods, including business analysis, stakeholder interviews, and document analysis.
The overall process can be understood by breaking it down into the following steps:

Planning: this starts at the level of business leadership, then through business analysts, documented requirements are passed on to the designers. Designers work with technical leadership to create a plan for the project. This includes defining the scope of the project, setting timelines, and identifying the resources that will be required. This is also the start of a long-term documentation and maintenance process. This can be done through brainstorming sessions, user research, and competitor analysis. Popular tools for this stage include Excel, Trello, and Asana.
UX Design: This is the stage where designers create the initial designs for the software application. This includes creating wireframes, prototypes, and mockups. Many powerful third-party tools are available which can help. The way it is applied in the context of websites and web applications is different from the way it works on mobile devices, wearables, smart TVs etc. This involves creating a style guide for the app that includes things like colors, fonts, and other design elements. This can be done using a tool like Adobe Photoshop, Balsamiq, Sketch. If you are creating a prototype of the app, it can be done using a tool like Adobe XD, InVision, MS PowerApps etc.

Development: During this stage developers turn the designs into a working software application. This step starts with designers handing over assets and design specifications documents with page layouts, user flows and other important details like color codes, illustrations, icons, fonts etc. This stage involves providing support to the developers who are working on the app and resolving any queries they have. This can be done through communication tools like Slack and email.
Testing: Once the software application is developed, designers work with the QA team to ensure that it meets all the necessary requirements. Any new feedback which requires updates to the design is incorporated. This stage involves regularly updating the app and testing it to make sure that it is working correctly. This can be done through tools like BrowserStack Live, TestFlight and Crashlytics.
Deployment: Finally the deployment of the software application, including creating installation and user guides, as well as providing support to users during the rollout.
Maintenance: It involves creating and maintaining documentation and logs at various levels, testing new features and bug fixes, and providing feedback to developers. Also working with user groups to gather analytics and present valuable feedback. This can be done through tools like Google Analytics, SpeedLab, AppScan and Veracode. Output formats for this stage include performance reports and security audits. The number of steps and the task related to each step could vary for different business use cases but the overall theme of bringing design thinking methodology into the software development cycle is widely adopted as the way to go forward There are a variety of product management methodologies and paradigms that are used by web and mobile application teams. Some of the most popular ones include:
Agile: The Agile methodology is a popular choice for web and mobile application development projects. It is an iterative and incremental approach that emphasizes collaboration, customer feedback, and rapid delivery.
Waterfall: The Waterfall methodology is a traditional approach to software development. It is a linear and sequential process that progresses from one stage to the next in a linear fashion.
Lean: The Lean methodology is a newer approach that focuses on efficiency and waste reduction. It is based on the principles of the Toyota Production System and has been adapted for use in software development. The role of designers is multifaceted and involves constant back and forth between business and development teams. Among all these activities, it is natural to encounter a few challenges and obstacles.

Common challenges

There are a variety of challenges and problems that are commonly reported by designers while working with various teams and verticals:

Validation across platforms: Even with a QA team performing manual or automated functional and visual testing of the application, designers still have to sometimes help validate the implementation of the user experience on a variety of platforms. The more platforms you have access to the better and more efficient the debugging process is going to be. It is a very common scenario to find developers and designers using high-end platforms where their user base in reality will be running the application on a range of budget devices. So having the ability to test features like responsiveness, speed and accessibility across a wide range of devices is a must have for all tech teams. BrowserStack live and app live platforms provide access to more than 3000+ devices and browsers so that you can easily make a list of the relevant devices and start testing your application. You can even automate the process with the automate and app automate features. We understand that implementing a complete test strategy takes time and it is usually implemented by developers and quality assurance engineers. For a designer, a task could involve testing the application behavior on a specific test environment for various business applications. Because of this we recommend you try these tools first:
Responsive: Just go to the URL, enter the web address of the application you want to test and run the check to see how the responsive layout functions on a range of platforms as shown in the illustration below.
SpeedLab: Apart from the functional aspects of an application you need to have strong metadata and performance optimization-related validations run on the application before deploying it to production. Speed lab gives you an in-depth report which contains vital metrics and recommendations for improvements to help you deliver a fast and efficient user experience without compromising on speed, reliability and SEO friendliness. You can change the location, network conditions, device type and browser. What makes this exceptional is the fact that the report contains visualization, tables, comparisons and suggestions which can be evaluated by different stakeholders from different perspectives, helping you optimize your application with an uncompromising technical depth. Also, this is a free tool which allows you to run parallel evaluations across a selection of devices.
Screenshots: This is a powerful tool which allows users to generate screenshots on a range of device browser combinations and share it with other stakeholders. You can test your local host websites, share results over email, test on different resolutions, change device orientation etc.
Lack of communication: One of the most common problems reported by teams is a lack of communication between the design and development teams. This can lead to misunderstandings and errors in the final product. This can be resolved by effectively using communication tools and ensuring that all the necessary documentation is being shared across the teams.
Misaligned goals: Another common problem is that the goals of the design team and the development team are often misaligned. This can lead to frustration and conflict between the two teams. This is a very bad situation to encounter as it means that two of your most important teams are working towards slightly or completely different goals. It can lead to resource bleed and unnecessarily delayed timelines. To ensure this never happens the leadership has to make sure that all the challenges faced by design and development teams are taken into consideration so that any conflicting scenarios can be spotted and avoided. Once development has started a very simple and practical advantage of using BrowserStack platforms is that without having to interact or wait for signals from the development or QA teams, others can see all new changes that have been pushed and the effect it is having on the existing workflow.
Lack of feedback & consistency: A third common issue is that designers often do not receive enough feedback from stakeholders during the development process. This can lead to designs that do not meet the needs of the users. Especially working under the agile model it is very common to see updates coming in from time to time and both the designers and developers having to adjust on the fly. Business and technical leadership have to be very careful with long development cycles, because in the last decade, short feedback loops have proven to be of critical importance even if the projects are very broad. Once the development has started on one set of agreed upon documents, any new change has to be a critical improvement and adding new low impact functionality should be avoided.
Unrealistic deadlines: A lack of judgment on the time required and the kind of work needed to finish a task is a very common scenario both for designers and developers. It is commonly observed both sides underestimate the work required on the other side. This can lead to rushed and poor-quality work. Because designing and coding are very different kinds of disciplines and they involve very different working methodologies, it is difficult for designers to gauge the implementation side complexity of anything they are proposing. Hence it is suggested that technical leadership points out all such instances of complexity and adjust for them in the sprint planning. It is also important to make sure that the task allocation within the team is happening with the consideration for the skill set and experience level of each team member. Especially while dealing with micro services and Apps with multiple interfaces, the success of development efforts is directly dependent on the way sub-teams are structured in a semi-decentralized work environment.
Lack of resources: A fifth problem that is commonly reported by designers is that they often do not have enough resources to do their job properly. This can include things like access to the right software or enough time to do the job properly. In the context of a rapidly growing startup, it is a common occurrence to find leadership trying to make the most of a limited set of resources. Because design is a creative endeavour, sometimes under limited timelines and resources it gets difficult to work and the outcome might not be optimal. This is only a human problem, with the advent of low-code/no-code application development management and testing paradigms in the coming years you will see the amount of resources needed to complete the software development cycle is going to shrink drastically. Although there are more than one ways to solve the problem but making the right decision will help you be future proof. Any rapidly evolving landscape of technologies requires a sophisticated technical judgment from the leadership to ensure that the teams are doing the best work that they can.

Synchronize Business Analysis, DevOps, and QA with Cloud Testing

Vivek Mannotra — Tue, 05 Jul 2022 04:34:20 +0000

As software development and testing continue to grow and evolve, so too must the team architecture and management practices of leadership. With a growing team comes new challenges and common issues that must be detected and addressed on time.
In this article, we will explore the various phases of team growth and the common challenges faced by tech leadership during each stage and also learn about tools and techniques to avoid pitfalls, because according to this report, disharmony among teams and investors is among the top reasons why tech companies fail.

Tech start-ups and teams

The historical background of tech startups can be traced back to the early days of the internet and the dot-com boom. This is when a new breed of entrepreneur emerged, one that was more comfortable with technology and its potential to create new businesses and disrupt existing ones. These entrepreneurs were quick to seize on new opportunities and were willing to take risks to achieve their goals.

Today, the tech startup landscape has changed dramatically. There are now many more verticals, each with its own distinct set of challenges and opportunities. However, the core ethos of the tech startup remains the same: to identify and capitalize on new opportunities through the use of technology. The most common pattern for team management in today's tech startups is to divide the process into 3 parts:

Business Analysis (BA): The business analysis process is a set of activities conducted to identify business needs and determine solutions to business problems. Business analysts use a variety of techniques to perform these activities, including requirements gathering, process modelling, data analysis, and business process improvement. In the context of a tech company, BA will determine the design specifications and requirements for a tech product.

Development Operations (DevOps): Development operations or DevOps is a set of practices that aim to automate and improve the process of software delivery. DevOps is a response to the intersection of two major trends in the IT industry: the increasing complexity of software delivery and the rising expectations of customers and businesses for faster and more reliable software releases. In a modern tech startup, the technical leadership/architects plus coders from the DevOps workflow. They are responsible for actually developing a software product.

A sample DevOps workflow using Docker

Quality Assurance (QA/QAOps): At a tech startup, the quality assurance or QA team is responsible for making sure that the products and services that the company produces are of high quality and meet customer expectations. This involves developing and executing test plans, testing new and existing products, and reporting any issues that are found. The QA team works closely with the engineering team to ensure that products are designed and built to meet quality standards. Also, they are responsible for validating product features, versions and approving releases.

It is likely that a tech company will have dedicated teams for all 3 activities, but in a rapidly evolving market and ever-increasing degree of automation, most-optimal team structure will depend mostly on the specifics of the business use case.
The 3 processes are interdependent because they all need to be in sync with each other in order to produce a high quality product. Business analysis is responsible for understanding the requirements of the product, development operations is responsible for ensuring that the product is developed according to those requirements, and quality assurance is responsible for ensuring that the product meets the quality standards set by the business.
There are several common points of validation or work that these processes need to be in sync with. For example, the business analyst needs to make sure that the requirements are clear and achievable, the development team needs to make sure that they are building the product according to those requirements, and the quality assurance team needs to make sure that the product meets the quality standards set by the business.
If any one of these processes is not in sync with the others, it can lead to problems. For example, if the business analyst does not understand the requirements, the development team may build the wrong product. Or, if the development team does not build the product according to the requirements, the quality assurance team may find defects that could have been avoided.

Common trends and challenges

As a startup grows, so too must its structure and management practices. There are generally three phases of team growth:
Startup Phase:

The startup phase is characterized by a small team size and a high degree of uncertainty. The biggest challenge during this phase is ensuring that everyone is on the same page regarding the project goals and objectives.
Another common challenge during this phase is managing expectations. It is important to set realistic goals and expectations for what can be accomplished given the limited resources available. Expansion Phase:
The expansion phase is characterized by a larger team size and a greater degree of specialization. The biggest challenge during this phase is coordinating all of the different moving parts.
Another common challenge during this phase is maintaining quality control. With more people working on the project, there is an increased risk of errors or defects slipping through the cracks. To combat this, it is important to implement rigorous quality assurance procedures early on in the expansion phase. Maturing Phase:
The maturing phase is characterized by stable team size and well-defined processes and procedures. The biggest challenge during this phase is sustaining momentum as everyone settles into their roles within the organization. This can be accomplished through continuous improvement initiatives that encourage everyone to identify areas for improvement within their own workflows.
Another common challenge during this phase is managing change requests effectively. As new features or functionality are added to the project, it's important to have a clear process for managing change requests so that they don't fall through the cracks. In the startup phase, the primary goal is to build a working product and capture a decent market to ensure business sustainability and profitability. After a working version of the product is released to the customers, there is an immediate need for a quality assurance process to be set in place. But considering the lack of resources and rapid pace of change during that particular period, it is hard to come up with a testing strategy and plan that can scale up with the product and company smoothly.

As the team grows, the structure of the team will change. The most common challenge is making sure that BA, DevOps and QA are in sync. In order to do this, the tech leadership needs to be aware of the following:

Establishing a consistent and repeatable testing process.
Managing testing resources as the company grows.
Keeping up with the pace of change as the company expands.
Dealing with legacy code and outdated systems.
Ensuring the quality of software releases.
Managing customer expectations around software quality.
Establishing clear and concise communication channels between the three workflows.
Ensuring that each team is aware of the work that the other teams are doing and how it may impact their own work.
Creating processes and procedures that allow for smooth and efficient collaboration between the three teams.
Managing expectations and ensuring that each team understands the role that they play in the overall success of the company. The field of software testing is constantly changing, and new challenges are always emerging. As a result, tech leadership must be constantly adaptable and ready to face new challenges. New technologies and approaches are constantly being developed, and it can be difficult to keep up with the latest trends. However, it is important to stay up-to-date on the latest developments in order to be able to provide the best possible service to your clients. The software testing industry is constantly evolving and changing. Here are some recent trends and developments in the industry:
Automation: Automation is becoming increasingly popular in the software testing industry. Automation can help to speed up the testing process and improve efficiency.
Cloud Testing: Cloud testing is another trend that is gaining popularity. Cloud testing allows testers to test applications and systems in a real-world environment.
AI/ML Based Tools: AI and machine learning are being used more and more in the software testing industry. AI and machine learning can help to automate the testing process to greater degrees and improve accuracy.
QAOps: QAOps is a new trend in the software testing industry that focuses on grouping quality assurance teams, techniques and operations.
Real-Device Cloud: Real-device cloud is a new trend in the software testing industry that allows testers to test applications and systems on real devices in the cloud.

Importance of cloud-based testing

With the ever-changing landscape of technology, the field of software testing has had to adapt in order to keep up. In the past, testing was done primarily on physical devices and platforms. However, with the rise of cloud-based services, testing is now often done in virtualized environments.
Cloud-based automated visual testing can be used to bridge gaps between business analysis, DevOps and quality assurance workflows by providing a way for all three teams to collaborate on testing and development projects. By using a cloud-based platform, all three teams can access the same tools and resources and can share information and results easily. This can help to streamline the development process and ensure that all teams are working towards the same goal.
Cloud-based testing platforms like BrowserStack provide a number of important benefits for software testers and tech leaders:

Increased flexibility and scalability: With cloud-based testing, you can quickly scale up or down your testing efforts as needed, without having to invest in expensive hardware or infrastructure. This is especially helpful when you need to do large-scale testing for a new release or when unexpected issues arise.
Reduced costs: Cloud-based testing can save you money on hardware, infrastructure, and maintenance costs. You only pay for what you use, and there are no upfront costs or long-term commitments.
Increased speed and efficiency: Cloud-based testing platforms offer pre-configured test environments that are always available on demand. This means you can get started with testing immediately, without having to set up and maintain your own test environment.
Improved collaboration: Cloud-based testing platforms make it easy for distributed teams to collaborate on test cases and results in real time. This is essential for efficient and effective team communication during the software development process.
Better insights and analytics: With cloud-based testing, you can collect data from every test run and use it to improve your overall process Cloud testing platforms can help in validating application projects and syncing BA, DevOps and QA with the work being delivered in several ways.
Cloud testing platforms can provide a way to test the product on a variety of devices and browsers. This can be helpful for the business analyst, who needs to make sure that the requirements are clear, and for the quality assurance team, who needs to make sure that the product meets the quality standards set by the business.
Cloud testing platforms can provide a way to automate the testing process. This can be helpful for the development team, who needs to make sure that they are building the product according to the requirements, and for the quality assurance team, who needs to make sure that the product meets the quality standards set by the business. Percy dashboard is a collaborative visual tool for managing differences and approving releases.
Cloud testing platforms can provide a way to share the results of the testing process with the team. This can be helpful for the business analyst, who needs to make sure that the requirements are clear and achievable, and for the development team, who needs to make sure that they are building the product according to the requirements.
Cloud testing platforms can provide a way to track the progress of the testing process. This can be helpful for the business analyst, who needs to make sure that the requirements are clear and achievable, and for the development team, who needs to make sure that they are building the product according to the requirements. Owing to all these practical advantages we can safely say that the future of business-critical software testing is going to be cloud-based and highly automated.

Data Visualization for Better Debugging in Test Automation

Vivek Mannotra — Tue, 05 Jul 2022 04:26:08 +0000

Visualization is to software developers what vision is to the brain. Just as our brains use vision to process and understand the world around us, product managers/technical leaders in startups use visualization to see and understand the applications and data they are working with.

Illustration depicting common components of data visualizations
The role of test suite visualizations in a CI/CD process is becoming increasingly important as the popularity of automation testing grows. There are many benefits to automating your tests, including the ability to run tests faster and more frequently. However, one of the challenges of automation testing is debugging automation test failures for complex websites and mobile apps.
According to this report, 30–40% of mature companies use tools for implementing unit and integration tests with visualization testing, dedicated functionality and resources to help manage the test plans. On the other hand, among less mature companies, only 12–15% use such tools.

The science behind importance of visualizations

Humans can process images in as little as 13 milliseconds, while it can take up to 150 milliseconds to process text. This difference is due to the fact that our brains are wired to process visual information much faster than written information.
This difference in processing speed is significant, and it underlines the importance and benefits of data visualization in information processing. When we are presented with large amounts of data, it can be very difficult to process all of the information if it is presented in textual form. However, if that same data is presented in a visual format with proper format and labelling, our brains can quickly and easily make sense of it.

With increased data generation, visualizations are likely to gain even more importance
Visualizations allow us to take in large amounts of information quickly and easily, which is why it is such an important tool for information processing. It is applicable to all fields where data is generated and has to be analyzed. In the context of a tech startup, visualizations are a critical part of daily work and decision-making.
As a startup grows, the number of automation test scripts they use to manage the quality of their product will increase. This can make it difficult to keep track of all the tests and their results. However, if the test suites and reports are visualized, it will be much easier to see which tests are passing and which are failing. This will help the startup to quickly identify and fix any issues with their product.
Data visualization with visualization testing are powerful tools, that can help startups to manage and debug their automation test scripts. By visualizing the test plan and reports, startups can quickly identify issues and take corrective action.

How to solve common test automation debugging challenges

Automation test plans and results can be visualized using graphs, charts and tables on dashboards and documents. Using dashboards for this purpose is especially popular as they are dynamic applications suitable for collaboration across teams and roles. Some common issues faced by tech startups while debugging complex automation test failures are:

Lack of clear error messages: When an automation test fails, it is often difficult to understand why the failure occurred if the code has improper error handling, or there is no error message. This can make it difficult to debug the failure and identify the root cause. By visualizing the automation test results on a live dashboard, teams can monitor the execution of tests and see a list of all running and scheduled tests. Identifying the failing test by name can help locate relevant modules for fixing. System logs play major role in debugging such scenarios.

A snapshot of popular CI tool Jenkins with TestComplete plugin for unit testing

Inconsistent test results: Another common issue with automation tests is that they often produce inconsistent results. This can be due to a number of factors, such as changes in the environment or code, or bugs in the test itself. This inconsistency can make it difficult to trust the results of the tests and make decisions based on them. Saving and then comparing test reports across time can help with identifying changing outcomes making it easier to see which tests are producing inconsistent results. Also, use an automation tool that can generate consistent test environments. This will ensure that the environment in which the tests are run is always the same, which will eliminate any inconsistencies in the results.

Test results with history on Jenkins Dashboard

Lack of test coverage: If a test plan does not have adequate test coverage, it may lead to missed critical bugs that could impact the product. By recording and visualizing the test coverage, we can quickly identify the areas that are not covered by tests. Tools like SonarQube, Jacoco etc. can be used to visualize the test coverage.

Tests taking too long to run: Automation tests that take a long time to run can be very frustrating for those who have to wait for them to finish. Delays could be due to technical issues with the implementation, or the test suite could be very large with many dependencies resulting in slow execution. This can impact productivity and lead to frustration and even resentment towards the testing process. By scheduling test runs and report we can eliminate the need to manually trigger tests. Also, monitoring the time taken for each run can help track which part of the test takes more time than it should. SoapUI ReadyAPI is a popular tool used for integration testing on APIs

Tests not covering all scenarios: Automation tests should cover all possible scenarios that could occur during the use of the software being tested. However, this is often not the case, and some important scenarios may be missed by the tests. This can lead to critical errors being missed by the tests and cause serious problems for users of the software. This is a tricky situation because if the scenario is not a part of the test plan, there is no way for the visualization to pick up the missing part. Hence, it is recommended to make sure all approved business requirements are properly documented and communicated to the developer and automation tester properly. Also, you can add a layer of white-box tests like an automated visual regression test as the final validation step before releasing a build version.
Inconsistent test environments: If the environment in which the automation tests are run is not consistent, or the environment is not an accurate representation of the real-world use case, it can lead to false positives or false negatives. This can make it difficult to trust the results of the tests and makes debugging more difficult. By using a tool which provides a high-quality and reliable execution environment you can minimize the issues. BrowserStack's Real device cloud provides access to 3000+ devices, OS and browsers.

Unstable dependencies: Automation tests often rely on other software components, such as libraries or frameworks. If these dependencies are unstable, this can lead to test failures. It can also occur due to improper versioning in code or using deprecated or unsupported functionality in the app. Maintaining the versioning of dependencies as part of project specification and documentation can help. Also, by capturing and visualizing the logs at various checkpoints in the CI/CD process, you can ensure that all dependencies are installed correctly and the build is completed with all necessary version requirements resolved.
Race conditions: Race conditions can occur when two or more threads of execution access shared data and one thread modifies the data before another thread has a chance to read it. This can lead to unpredictable results and makes debugging very difficult. A dashboard with live updates and configurable warnings on resource consumption can help monitor and avoid deadlocks. Ideally, the scripting of test runs should be done keeping in mind the available computing capacity and memory.

Testing Multi Experience Apps using Real Device Cloud

Vivek Mannotra — Tue, 05 Jul 2022 04:16:50 +0000

Multi-experience (MX) applications are shaping the future of web and mobile applications by making them more immersive and interactive. With multi-experience applications, users can interact with a brand or business in a variety of ways, using a variety of devices and platforms.
Super-apps like WeChat, Paytm are growing in popularity because of the range of their offerings and the platform-agnostic multi-experience they provide to users. Multi-experience development platforms (MXDP) centralize all the activities involved in putting together a multi-experience application and are becoming increasingly popular.
According to this report, more than 2/3rd of all users are interested in apps with a great multi-experience. Considering the prospect of growth in the area of MX apps and development platforms, real device clouds have a significant role to play in the quality assurance or testing part.

Components of a multi-experience app journey

Visualizing multi-channel, omni-channel & multi-experience
What separates a MX app from a normal web or mobile app, is the range of supported platforms and interaction modalities. Here are some common components or touchpoints in a MX journey:

Mobile apps: Mobile apps are the most common type of multi-experience application. They are designed to run on mobile devices such as smartphones and tablets, which dominate in terms of the number of devices and area coverage.
Web apps: Web apps are designed to run on web browsers which provide access to internet sites through WWW protocols and a host of I/O and accessibility features like speech, camera, geolocation etc.
Desktop apps: Desktop apps are designed to run on desktop computers. The most popular desktop operating system platforms in the world are Windows, MacOS and Linux.
Chatbots: Chatbots are computer programs that simulate human conversation. They can be used to provide customer support or to automate tasks such as booking tickets or ordering products.
Voice apps: Voice apps are designed to be controlled by voice commands. They can be used to perform tasks such as making phone calls or controlling smart home devices. Popular examples include Siri by Apple, Alexa by Amazon etc.
Augmented reality (AR) apps: AR apps are designed to superimpose computer-generated images on the real world. They can be used for gaming, navigation or education. Microsoft Hololens and Google's Glasses are among the most popular AR platforms.
Virtual reality (VR) apps: VR apps are designed to create a realistic, immersive experience. They can be used for gaming, entertainment or training. Quest by Meta , Vive by HTC are some of the commonly adopted VR platforms.
Mixed reality (MR) apps: MR apps are designed to combine the real world with the virtual world. They can be used for gaming, entertainment or education.
Wearables: Wearables are devices that can be worn on the body. They can be used to track fitness data, receive notifications or make payments. This is a rapidly expanding industry category with prices dropping rapidly across a wide range of devices.
The Internet of Things (IoT): The IoT is a network of physical devices that are connected to the internet. These devices can be used to collect data or control devices remotely. Web browsers remain the most popular application touchpoint, though mobile apps are certainly on the rise. Other devices like smartwatches, smart-TVs, voice-driven assistants etc. are also becoming popular with dropping prices.

Rise of MXDPs

There are a number of multi-experience development platforms (MXDP) on the market like Microsoft's PowerApps platform, Google's Firebase, Oracle Mobile Hub etc., each with its own features and capabilities. Here are some reasons for the rise of multi-experience development platforms:

The advent of Low-code/No-code: Low-code/no-code platforms can help accelerate the development of multi-experience app platforms by reducing the amount of coding required to build them. This can help to speed up the development process and make it easier to create complex applications. MXDPs often make use of these technologies to help accelerate development.
Faster deployment of valuable digital experiences: As the world becomes more digitized, businesses need to be able to deploy new experiences quickly and easily in order to keep up with customer demands. Multi-experience development platforms make it possible to rapidly deploy new experiences across a variety of devices and platforms.
The need for easier scalability: With multi-experience development platforms, businesses can easily scale their apps up or down as needed. This is due to the fact that these platforms are cloud-native, meaning they are designed to be run on cloud infrastructure.
The need for better collaboration: Multi-experience development platforms offer opportunities for better collaboration between different teams within an organization. This is because they allow for a more visual approach to software development, which makes it easier for non-programmers to contribute.
The need for faster decision-making: Multi-experience development platforms can help organizations make decisions more quickly, due to the fact that they streamline collaboration. This is because all stakeholders can easily see and contribute to the development process. The future of MXDPs is bright. They are becoming increasingly popular as the need for them grows. As new devices and modalities emerge, so too will new platforms to support them. We can expect to see more platforms that offer low-code and no-code development, as well as platforms that focus on specific modalities or devices. Because of a wide and ever-evolving range of touchpoints, MX apps have to keep on adding and updating support for every new relevant platform that comes to the markets. Also, even though many MXDPs come with a built-in set of testing functionality and support, most business use-cases will require elaborate tool-stacking and complicated workflows for quality assurance, which leads to a new set of challenges. Challenges with testing multi-experience applications The main challenges with quality assurance testing of multi-experience applications are:
Ensuring compatibility across multiple devices and platforms: One of the main challenges with quality assurance testing of multi-experience applications is ensuring compatibility across multiple devices and platforms. With so many different types of devices and operating systems out there, it can be difficult to ensure that your app will work properly on all of them.
Managing different interactions and touchpoints: Another challenge is managing different interactions and touchpoints. With a multi-experience app, users will interact with it in different ways depending on the device they are using. For example, they may use touch gestures on a mobile device, but use voice commands on a smart speaker. As a result, you need to make sure that all of these different interactions are tested and work correctly.
Ensuring a consistent user experience: It is also important to ensure a consistent user experience across all touchpoints. This means that the app should work in the same way regardless of the device or platform that is being used. For example, if a user starts an interaction on a mobile app, they should be able to continue it on a website or chatbot without any issues.
Testing different types of content: Another challenge is testing different types of content. With a multi-experience app, you may need to support different types of content, such as text, images, videos, and audio. This can be a challenge when it comes to testing, as you need to make sure that all of this content is displayed correctly on all devices and platforms.
Managing different versions of the app: Another challenge is managing different versions of the app. As new devices and operating systems are released, you may need to create new versions of your app to support them. This can be a challenge when it comes to quality assurance testing, as you need to make sure that all of the different versions of the app are tested properly.
Ensuring a seamless user journey: Finally, it is important to ensure a seamless user journey. This means that the app should be easy to use and navigate, and that there should be no errors or glitches. This can be a challenge when testing, as you need to make sure that the app is stable and that all of the different interactions work correctly. Testing multi-experience using Real Device Cloud With so many different types of devices and operating systems out there, it can be difficult to test your app on all of them. However, with a real device cloud, you can test your app on a wide range of devices without having to physically own them all. A real device cloud can help to ensure a consistent user experience. This means that the app should work in the same way regardless of the device or platform that is being used. For example, if a user starts an interaction on a mobile app, they should be able to continue it on a website or chatbot without any issues. There are a number of reasons why you should consider real device cloud for testing multi-experience apps:
Ensure compatibility across multiple devices and platforms: One of the main challenges with quality assurance testing of multi-experience applications is ensuring compatibility across multiple devices and platforms. BrowserStack's Live real-device cloud platform allows you to test your app on a wide range of devices and operating systems, you can ensure compatibility and a seamless user experience. With such a wide range of test environments, you get the ability to recreate real user scenarios and device experiences. You can adjust real-world variables like network conditions, geo-location etc. Test accessibility features like navigability and screen readers. You can test notifications on devices. Also, get access to the developer tools section of browsers on a range of operating systems.
Manage different interactions and touchpoints: With a multi-experience app, users will interact with it in different ways depending on the device they are using. For example, they may use touch gestures on a mobile device, but use voice commands on a smart speaker. As a result, you need to make sure that all of these different interactions are tested and work correctly. BrowserStack provides a platform that allows you to use features like mic, camera etc. to replicate a wide-ranging I/O exchange.
Ensure a consistent user experience: This means that the app should work in the same way regardless of the device or platform that is being used. BrowserStack lets you test your app experience on different desktop OS', Mobile devices and web browsers. With the automated visual testing tool Percy, you can validate changes at the level of UI across a range of devices, automatically.
Testing different types of content: Another challenge is testing different types of content. With a multi-experience app, you may need to support different types of content, such as text, images, videos, and audio. This can be a challenge when it comes to testing, as you need to make sure that all of this content is displayed correctly on all devices and platforms. BrowserStack lets you run your app on real devices so that you can be assured that your test environment is as close to production as possible.
Managing different versions of the app: Another challenge is managing different versions of the app. As new devices and operating systems are released, you may need to create new versions of your app to support them. This can be a challenge when it comes to quality assurance testing, as you need to make sure that all of the different versions of the app are tested properly. BrowserStack provides a platform that allows you to easily manage different versions of your app, ensuring that all of them are tested properly.
Ensuring a seamless user journey: Finally, the app should be easy to use and navigate, and there should be no errors or glitches. This can be a challenge when testing manually, as you need to make sure that the app is stable and that all of the different interactions work correctly. BrowserStack's automated visual testing and visual diff engine let you automatically detect changes and help you approve builds with all stakeholders kept in the loop.

Is Visual Comparison Testing the Future?

Vivek Mannotra — Tue, 05 Jul 2022 04:08:44 +0000

Visual comparison testing, as the name implies, is the process of testing software applications by focusing on their visual components. It is part of the test strategy for most quality assurance (QA) teams, but the way it is implemented and the tools used differ widely.
To optimize DevOps and QA workflows, the technical leadership has to deploy an up to date system architecture, which has to fulfil the business goals at a justified cost and pace. In an Agile development scenario, managing outcomes can be tricky due to variabilities in business requirements and the technical landscape.
Using automated visual testing as a primary tool for testing production readiness in software applications is becoming increasingly popular because of its ability to streamline the most crucial phase of product validation.

Major challenges in software testing

Before we move on to discuss visual testing, let's try and understand what are some of the most commonly occurring challenges in the software testing process:

Automated Vs Manual: Any testing strategy which involves a lot of manual effort is surely going to be resource and time-consuming. Plus in a manual scenario, you are relying on inter-personal communication for getting updates regarding important test runs and reports. Automation on the other hand is technically challenging to implement and requires the testing team to have a fine understanding of the framework being deployed. Also, it is difficult to maintain an updated automation test plan in cases where business requirements are changing frequently.
Test environments: A product, depending on its real-world use case, will have a set of relevant devices, browsers, network conditions etc. on which it needs to perform. So, another significant challenge in testing software applications is to provide the test team with access to all relevant run-time environments and scenarios, so that they can replicate real-world usage in a controlled environment.
Turn around time: Developing functionality in a software application takes time, and so does testing. The total time to production is the sum of time spent in both phases. Naturally, you can understand the need to optimize both areas to get the best possible result in the least amount of time. CI/CD has been widely adopted because it automates the application build and test triggers, and also helps track overall status through dashboards. It is common to see a team adopt automated testing for increasing time efficiency and in the process of migrating from manual to automated they fail to bring core business understanding into the test script resulting in faster but incomplete tests.
White-box vs Black-box testing: White-box tests are usually done by developers, while black-box tests are done by testers. This can lead to a disconnect between the two groups, which can make it difficult to sync the tests. Another challenge is that white-box tests tend to be more detailed and specific, while black-box tests are more general. This can make it difficult to ensure that the tests are covering the same areas. White-box tests are typically run on a development or staging environment, while black-box tests are run on a production environment. This can make it difficult to replicate test results and debug issues.
Documentation and reporting: When a bug is detected in an application it is supposed to be documented in a format where all meta-data and steps are recorded for reproducibility purposes and deeper analysis. Documenting application behavior at the level of features/modules with steps to recreate, expected behavior, observed behavior etc. Detected issues are assigned to developers as tasks for resolution. In a manual scenario, a team member uses bug tracking tools like Jira, Trello etc. to mark down all details and share with the team. The problem with this approach is that upon witnessing an abnormality, the tester has to articulate and record the event into the bug tracker, which introduces space for human error. Understanding these broadly laid out challenges will help you grasp the complexity involved in the software development and testing processes. Now to resolve these issues we need to find and work with the right tools.

Features and benefits of automated visual testing

Automated visual testing or visual regression testing is an advanced way to carry out white-box product validation tests on a piece of software. The degree to which the tests are automated is usually an open variable and different teams will end up with different ratios.
A great visual test automation framework should have the following qualities:

Ability to configure different devices, browsers and operating system combinations as test environments for a test suite.
Support for well designed and efficient scripting languages for writing test cases.
Ability to integrate with application development platforms like React, Angular etc. and end-to-end testing frameworks like Ember, Cypress etc. through SDKs.
Ability to work smoothly in sync between local development, testing and production environments.
Ability to record test run sessions with I/O and screen output.
Ability to find and document differences between versions of the application.
Should be lightweight and cost-effective. Here are some key benefits of implementing automated visual tests:
Decoupled technical and business level testing: With a separate automated workflow testing visual differences between versions of an application, you have a clear delineation of responsibilities for teams running unit, integration, system and acceptance tests. integrated into the CI/CD workflow, it delivers a seamless layer of business validation before every release.
Save time and resources: Automation helps you save time and resources by reducing or eliminating the need for manual testing. Automated visual testing can also help ensure that test cases are being run correctly and consistently.
Improve efficiency: Automated visual testing can improve test coverage and accuracy by providing more reliable and consistent results. Advanced image processing features offer various ways to find differences and record issues automatically, using advanced AI/ML techniques to improve accuracy over time. A well-trained image processing network can easily outperform an army of humans.
Minimal human intervention: This is great for multiple reasons, first up it reduces the possibility of human error by automating the core analysis part. Secondly, it lets team members approve and disprove at the end helping retain important control over the process, while at the same time automating the rest. Also, it can help to improve software quality by providing early feedback on potential problems.

Automated visual testing

BrowserStack's Percy is an all-in-one visual testing, analysis and review platform which helps automate the complete process of visual regression testing and provides access to builds, differences, screenshots and a whole host of other features.
Percy is part of the BrowserStack Suite, which along with other tools like Live and Live Automate, provides full coverage for all your visual testing needs. Let's briefly run through the steps you need to follow to implement:

To get started you first need to sign up for Percy.
Create a project on the Percy app dashboard.
Follow the integration guide applicable to your programming framework and use the token provided for linking account and validation.
Install project dependency on your local environment as per the guide and now you can write percy commands into your application execution script and take snapshots.
Build and run your application, and check logs for updates on screenshot events.
After the build is complete, go back to the dashboard and there you should be able to see your build listed.
If you click on any entry you will be taken to the review section where you can see the difference explorer and other features of the percy dashboard. Preview of build approval on Percy dashboard Once your build is showing on the dashboard, you can now tweak the project to add more features like webhook, slack, repository integrations and the ability to change the browser in which the snapshots are taken. You can add your team members and configure it for the production environment, where each build can be evaluated for visual differences and the team can collaborate on approvals. After completing the setup successfully you get:
Full-fledged visual testing & approval workflow: Percy not only helps with automated visual testing but also provides an end-to-end solution for reviewing and approving builds before they are deployed to production.
Easy integration: Percy has easy to follow guides and SDKs for all popular application development frameworks. So you can expand your test suites easily with your growing application.
Real-time results: Percy's live preview feature enables you to see the test run results in real-time, so you can debug and fix issues on the fly.
Cross-browser support: Percy supports all major browsers and devices, so you can be sure that your visual tests cover all relevant run-time environments. New devices and features are constantly being announced, check out the latest releases.
Powerful analysis tools: Percy's image processing capabilities are powered by advanced AI/ML algorithms which help you find and fix visual issues quickly and accurately.
Flexible pricing: Percy's flexible pricing model enables you to pay only for what you use, so you can scale your visual testing efforts without breaking the bank. It also provides real-time system monitoring with historical data on uptime and incidents.

Cloud based testing is the future

The benefits of cloud testing are many and varied, but the most significant are its speed, cost-effectiveness, and ability to provide comprehensive coverage. Cloud testing tools allow businesses to save time and money by providing a real-made testing environment that is identical to the production environment. Testers simply have to sign up, select the real devices they want to start tests on, and start flagging bugs.
In addition to its speed and cost-effectiveness, cloud testing also has the ability to provide comprehensive coverage. By choosing to run testing in the cloud, teams and QA managers can meet their goals faster, with greater accuracy and minimal investment. Cloud Testing is easy, fast, and practical, contributing in every way to technical and business precision.

Overcoming Challenges in End-to-End Microservices Testing

Vivek Mannotra — Tue, 05 Jul 2022 03:58:46 +0000

Web and mobile application developers around the world are using microservices architecture to build and deliver amazing tech products that are fast, responsive and highly integrated within an ecosystem.
Unlocking high potential in the context of a rapidly growing tech startup comes with a set of big rewards, so anything which breaks barriers in the development of better tech is appreciated and adopted by all.

State of microservices by Jetbrains - 2021
Microservice architecture is one such paradigm which is shaping the technical architecture of many popular and upcoming tech products. It is also shaping the way we build teams and assign responsibilities. But like any other powerful area of exploration, this one also comes with a unique set of challenges.
While improving on many aspects of web services architecture, they are also known to introduce some peculiar problems. In this article, we will look at some important challenges in end-to-end testing of applications that are built on microservices architecture.

What are microservices?

Web service or service is a term used to describe a web contract which works over protocols like HTTP and acts as an interface between application UI and database. Breaking down a long and complicated business and user story into very small modules and building separate services for each module is basically the essence of microservices architecture.

Some key benefits offered by microservices architecture are:

Autonomous development: Instead of having one very large team and project, which is hard to manage, microservice architecture lets you break down your team and code into self-sufficient groups. Teams work and deliver autonomously, leading to a clear delineation of responsibilities and independent work priorities.
Functional specialization: When modules are identified based on business needs, and proper team structure is implemented, it leads to each team becoming a specialist in their domain of functionality. This is also helpful during defect analysis.
Dynamic resource allocation: With modern cloud platforms offering on-demand scaling for your application infrastructure, microservices on cloud platforms let you fine-tune for the ultimate resource optimization strategy.
Plug and play: Agile development with microservices enhances the ability to add or remove modules from the application without downtime, which is helpful in scenarios where rapid feature addition is needed in the SDLC.
Application agility: Since the application is not a single monolith, one non-critical failing component generally will not affect all the other working parts, hence rendering the application more agility in failure scenarios. In a CI/CD environment, a properly implemented microservices architecture helps the technical leadership keep track of the overall progress, as all the steps beyond code commit by developers are automated and trackable through dashboards and other communication tools. When a contribution is merged in the master repository, a series of chain events are triggered, including testing at all levels. If tests break, the whole CI/CD pipeline could stop potentially, and fixing the issue could take time. In some cases, microservices architecture tends to increase the complexity of issues. Let us try to understand how.

Unit and Integration testing of microservices

Unit tests are the first amongst various layers in application testing. It means testing individual functions of code, by writing test code. As you move up the layers of testing the number of test cases tends to decrease in number but increase in cost and complexity, so testing as much as possible at the level of unit testing is the goal of every honest and dedicated developer.

Integration tests are basically the next layer where unit tests and app functionality are grouped based on broader business functions and tested in batches. Writing unit tests involves a fair deal of back and forth as it requires the developer to not only write the test case once, but also to prepare mock input data and external service responses as unit testing is not connected to live DB and services, and also updating all this whenever any change to original function occurs.
Here is how things get tricky:

A CI/CD flow with a number of microservices integrated into a few streams is a commonly occurring scenario, consider one module with a breaking set of tests has the potential to cause the CI/CD to come to a halt.
The number of reasons why unit tests fail is so vast, that in an environment with dependencies spanning multiple modules and teams, it could take hours or even days in trying to figure out exactly what broke because of the chain reaction effect.
It is a common scene to find out tests breaking in one module due to recent updates in some other module and because of separation of concerns and internal competition within teams, developers will find it hard to spend time and effort on or co-operate in fixing errors in modules not owned by them.
While mocking live services from things like payment, booking providers or external data sources, it is important to capture all the nuance of the contract and test for edge scenarios.
With an accelerated push to develop newer functionality, developers will find it hard to focus on unit test coverage, the application complexity will grow rapidly and consequentially over time it gets more and more difficult to go back and maintain a high-quality test package. This could very well lead teams to be negligent of unit testing, as with increasing complexity the effort starts sucking in too many resources, and it's not uncommon for the leadership to say that the juice is not worth the squeeze, and cut down heavily on unit and integration tests, which is a short term gain for a potentially big long term cost. Here are some suggestions that will help you avoid this:
One simple way could be to layer code modules and test suites based on the priority of flow, i.e. test suites for core business flows are more important than others, and in order to change and modify code for core business flows, you would need more experience and approvals.
For unit and integration testing, it is advisable that the strategy for writing and executing tests be designed to accommodate the possibility of complexities arising in future. Also, the test reporting system could be configured to mark test severity appropriately such that only very important flows are set to break a delivery/deployment push.
It is the job of technical leadership to define solutions to and manage potentially complicated problems like changing external service contracts, or highly branched dependencies and make sure all developers are up to date with proper documentation.

Team management while developing microservices

One of the biggest challenges associated with building tech products on microservices architecture is the structure and management of teams working on it, and the complexities arising out of that. The main reason for wanting to implement microservices architecture is to supercharge the development process for a rapidly evolving product.
In a fast-growing startup where changes are coming in on a daily basis along with new business and user scenarios, and you are looking to maximize the speed and efficiency of the dev process, that is where microservices are most beneficial. But beneficial only if implemented and executed properly.
If your team is not in sync with your overall technical standards specification, or if there are holes in your specification plan itself, it is easy to find folks struggling to deliver even at average speeds and quality while maintaining an overhead of a bloated delivery line.

Here is how challenges usually come up:

When you have a system set up afresh, the complexity is usually manageable, but once you start building modules on top of that, it goes up exponentially.
As complexity rises, it is the job of technical leadership to observe, understand and control it. Because if the tech leadership can get their heads wrapped around what is going on, only then they can guide the developers to do the right thing. But if the leadership fails to do so, turbulent times are more likely to come.
If leadership fails to contain complexity, it usually leads to a scenario where no one has a 100% picture of what is happening. Resulting in developers having less than ideal command over the application.
In such cases, things break in unpredictable ways. Imagine ghost service calls and unnecessary batch processes running without proper monitoring or syntax errors and memory leaks causing heavy resource consumption for no reason, or some random breakage in non-essential areas causing critical flows to break.
After something actually breaks and causes trouble, finding the actual point of error and assigning responsibility is also trickier. How to avoid:
Although it is desirable to set up the dev sub-teams to work in an asynchronous and independent manner, to maximize productivity. You have to ensure that it does not come at the expense of clear responsibility and task allocation within the team. Clear communication regarding critical changes within teams has to be ensured at all times. Plus a strong culture of writing proper code syntax, with comments and documentation being enforced as a standard within the development team.
It is recommended that the business and tech leadership work in close synchronicity, and ensure minimal confusion regarding current and future needs of the tech product so that they have a chance to clean up and contain issues. More often than not we see an over-optimistic business analysis ends up overwhelming a dev team with requirements which don't even result in any significant improvement or functionality at the user's end.
Using automated black-box tests is a strong way to ensure application functionality on the user's side without having to dive deep into code issues. Tracking and fixing issues If the leadership fails to fix product and team issues early on time, it is inevitable that they will lead to bugs in customer sessions and cause problems. After an issue is detected by QA and the product team is notified, they are expected to revert back with a report on the cause and fix the issue in the least amount of time. Here again, we see the complex nature of microservices architecture and distributed team structure creating some more challenges:
In a scenario where the structure of the application loses proper form and is plagued by bad code, when a functionality breaks QA does its job normally but for the developer, it gets harder to detect the root cause due to a lack of code comments, documentation and/or proper syntax.
Due to dependencies, one breakage may lead to a cascade of breakages, which requires an even deeper debug analysis, generally causing more time. This causes the detection of the error part on dev side to be slightly or significantly delayed.
In a situation where multiple things break, it's easy to see finger-pointing among team members, where developers not having a deep understanding of the code package will further delay the detection process.
When finally the issue is detected and resolved, updating the unit, integration and other test scripts for that functional portion of code still stands as an added task for the responsible developer.
After all the accumulated delays due to back and forth between QA and developers, it is easy for the business team to lose consideration for the complicated nature of the tech process and become increasingly hostile, causing tension within the team at large. This is the worst nightmare for all tech leaders to see their teams descend into dysfunctional modes. And for the business leadership, this could be an even worse disaster when the expensive tech resource pool seems to not be able to command customer confidence and appreciation as per their expectations. You can try to avoid this by:
Put in extra effort during the growth phase to keep the business analysis, experience design, quality assurance and development teams in sync at all times.
Document, log and monitor communication between all systems and participants because evidence serves better than people's claims.
If there is any mistake or miscalculation at the level of system architecture or CI/CD design or implementation, it is better to halt the process and fix the core issue first even if it is detected after writing a fair bit of code on top of it.
Try building a culture where human factors are clearly understood and interpersonal equations running both directions in a hierarchy are managed justly.
Cooperation and helpfulness within a team should be acknowledged and awarded. The challenges mentioned here are not simple and will require dedicated analysis and case-specific study to optimally resolve them in real life. And also the variables of every team and product are different, so what works for one might not even be a viable option for another. In a case where all attempts to prevent such issues do not bear fruit, and you unwillingly land in the dreadful scene of having to fix a broken system, there are still ways to do it. Now what I'm going to suggest might seem reasonable to some and controversial to others, but I can assure you that many companies already in the past, during many moments of downtime where the only priority was to have the working product back online, have ended up doing this, i.e. prioritize Black-box testing on production environment and keep the white-box tests for testing environments and only very critical flows in production. That way you get the ability to develop new functionality and components for the production environment without having the added responsibility of making the unit and integration tests work every time you add something. While far from ideal, this could be adopted in an emergency or difficult situation where feedback from a select group of users is more helpful than your QA strategy. In conclusion, we recommend all participants in the product development/management process be aware of, and actively work to avoid these pitfalls in order to gain the maximum benefit from a well designed microservices ecosystem.

Web Application Testing guide for beginners

Vivek Mannotra — Tue, 05 Jul 2022 03:40:09 +0000

The internet, World Wide Web (WWW) or simply the web, is the most revolutionary information technology and application development platform in the world. The present world economy majorly depends on the web.

The simplest way to conceptualize it is that it is a network of interconnected nodes called servers and clients, exchanging data over well-defined protocols like TCP/IP, HTTP, FTP etc. Read more about its inception and specifications.

When building for the web as per today’s industry standards, it is important to have an updated and relevant stack of tools, with an architecture that supports CI/CD and QA at all levels.

Structure of web applications

A web application or website is a package loaded on the client-side containing multiple layers of interdependent modules, built on the core web technologies :

HTML: Hypertext markup language or HTML, is the tag-based document marking system which defines the structure and individual components of, what is compiled into the Document Object Model or DOM.
CSS: Cascading style sheet or CSS is the native style description framework which is used to identify and style different parts of the DOM into visible area of the page. It provides features like element selection by id, class and relation to other DOM elements.
JavaScript: JS is a high-level interpreted scripting language upon which all the dynamic behavior of the application is scripted and executed.

These core technologies are augmented with layers of other technologies to increase the functional capability. From popular JavaScript frameworks like Angular, React and Vue, to CSS pre/post processors like Less, Sass and also other HTML templating engines, the domain of technologies related to web is truly vast.

Besides the front-end layer, in most applications, there is also a backend or server-side layer with APIs built on micro-services and Databases, which hold all the data, business logic and abstract all the information into relevant contracts that the front-end can access via HTTP methods, with proper request and credentials.

Web Application Architecture Diagram

Depending on the tools used, and the nature of the website, it will need an appropriate hosting strategy and infrastructure. Websites can be hosted on a variety of systems, which can be broadly classified into:

Static web hosts: Used for static websites, file storage based delivery platforms which provide domains, email, DNS and other features like SSL encryption and 3rd party integrations. E.g. Godaddy, Hostinger etc.
Dynamic web hosts: For dynamic web apps, cloud platforms like AWS, Google Cloud, Azure, Salesforce and IBM cloud are preferred, which provide a range of computing features like Virtual Machines, Databases, on-demand resource scaling, etc. as services over the internet. These platforms are the standard for deploying web applications for business purposes because of their advanced performance and security features. They also provide a range of cutting edge AI/ML based tools.

A web application, even with a pretty simple initial setup, will tend to increase in complexity as more pages, content and functionality are added to it. After a certain threshold of complexity, it becomes very difficult to manage the behavior of the application and it gets more difficult to audit resource consumption and allocation.

Depending on the application, developers might prefer certain approaches to the application structure like:

Single-page app: This refers to a web app which has a single document model within which all functionality is contained. It involves a large amount of functional logic having to be packaged and sent to the client’s computer, with appropriate security and performance optimizations. Some popular examples are Gmail, Facebook, GitHub etc.
Multiple page app: Most commonly used structure, where the application is divided into multiple pages, accessible through various route URLs. Server-side app frameworks and template engines are used to build such applications, and they have an inherent security advantage over SPAs.
Progressive web app: This is a modern way of using WebView providers on handheld devices to run web applications as native apps using service workers, manifests and an app shell.

No matter what approach you take, implementing test plans at the level of development and beyond in order to ensure an efficient DevOps workflow is equally important for all.

Testing web applications

The components of a good website testing plan include a strategy, test objectives, test approach, test schedule, and test environment. The test strategy should be designed to ensure that the website meets the business requirements and is fit for purpose.

The test approach should include:

Unit Testing: Testing parts of the codebase through unit tests. Read more about unit testing in Javascript, Python. With the advent of no-code/low-code and AI-based tools you can now automate parts of the Unit test writing process also.
Integration Testing: Involves testing various parts of the website code as interdependent functions or modules through test code or other tools triggered as the code is merged with the parent repository. All major cloud hosts like Github, Gitlab, Bitbucket have native support for CI/CD integration.
System Testing: Testing the workings of the website at the level of user interface and features like login, signup, and other supported flows, which validates parts of the website working together. Selenium is the most widely adopted automation framework for web browsers.
Acceptance Testing: This is usually the final stage of testing in which the fully assembled application with data is tested in a live or pre-production environment. This involves testing with actual or mock users. Automated visual testing is the most efficient way to manage change approval process on a rapidly changing product UI.
Performance Testing: With an increase in users, it becomes vital to ensure that the servers can handle the request loads at peak usage times. Also, maintaining end-to-end security at each point of contact between the website and users through properly using HTTP headers and metadata analysis.

Role of web browsers

All websites need another application, known as the ‘browser’, running on top of the operating system of a device. Web browsers are built by many companies and are usually free.

Leading web browsers as of 2021

Web browsers play an important role, not only in making websites accessible to users, but also in helping the developers with a wide set of tools to test and debug various aspects of a web application under development. Most browsers provide development tools as an auxiliary interface to developers who want to peek under the hood and access the inner workings of a rendered web app. These tools can generally be accessed by right-clicking on a page and selecting the option ‘inspect’.

Let’s have a look at some of the primary tools provided in the dev tools section of Chrome:

Elements: The element explorer provides access to the compiled DOM with a host of features to add/remove components, set states like hover, focus etc.
Console: This is a log of the console output from the JavaScript execution, very useful for debugging. This panel can also be used to run JS code snippets on the active webpage and see the output.
Sources: Under the sources panel you can see a list of all the source code files downloaded by the web page, listed under the domain provider name. On the right side of this tab, you have a script debugger which can be used to put breakpoints and debug the website execution in real-time.
Network: This panel logs all the network calls to and from the webpage, with a long list of details like type, status, request/response, timing etc. It also has an option to simulate network availability scenarios by using the throttling feature.
Performance: Under this panel, you get the ability to record runtime performance by recording page load events and analyzing the rich breakdown provided after the recording is analyzed.
Memory: You can take memory heap snapshots and analyze things like memory leaks, object size, visualize memory usage over time etc.
Application: This can be used to evaluate, edit and debug service workers, worker caches etc.
Security: Gives you a summary of the SSL certificate validity.

Chrome Dev Tools

There is also a built-in device toolbar which lets you simulate user interface scenarios across a range of devices with a list of preset resolution profiles, network throttle, zoom level, screen rotation and the ability to enter custom resolution for testing responsiveness.

The Chrome dev tools section is constantly getting newer and more advanced tools like Lighthouse, Recorder etc. which provide deeper ways to analyze the overall health of your application.

Furthermore, beyond the dev tools even, browsers let you access a library worth of APIs for things like I/O, camera, location, audio/speech, network, cookies etc. The DOM and many other components of an active session are available for access at the object level.

Testing local deployments

Before you move to set up a live testing flow in a browser, it is also recommended to implement a unit testing workflow in your application to ensure a strong and clean codebase.

When developers make a website, they check all changes and updates in a browser on their work computer. After preliminary functional testing, the next important part is the device and browser compatibility testing, which actually gives us a sense of how the app looks and performs on different device-browser combinations.

Because of the fact that a developer has limited access to the range of all possible devices, it becomes a bit trickier to capture some bugs. Despite the fact that web has well-defined standards and specifications, discrepancies in the interpretation of different browsers arise due to different implementations in the underlying browser engine and JS interpreter. Overall a modern web browser by itself provides a loaded toolkit to open up and debug any aspect of the application.

Automated Visual testing

From the perspective of a user, the application is a visual experience, and the journey a user takes on a website is defined by the design specifications set by the business analysis and design processes. To ensure that the developed website stays true to the design goals, one has to ensure a proper regiment of visual testing and approval of changes to the code base.

This is where the QA team plays a leading role. The process of visual testing used to be mostly manual about a decade ago but now, it is increasingly being automated. One efficient way to implement visual testing is by using tools like Percy in your stack which offer a user-friendly and tightly integrated way to perform visual tests on the CI/CD line.

You can add your team members and configure it for the production environment, where each build can be evaluated for visual differences and the team can collaborate on approvals.

We have laid out a clear path of testing on your local machine manually, with automation and also discussed how to implement visual testing to test and control UI changes. After you successfully implemented all these layers, your website is good to go to production or live environment.

But that does not mean that the role of testing is over. It just means that if your test cases are properly configured and your CI/CD flow runs smooth, you will have to spend less time on validations, error checks and more time on actually developing new features. To ensure a healthy testing system, just keep in mind these points:

Test runs at all stages and levels need to be monitored because automation will help you execute and detect errors, but to understand the severity of the defect and then find a way to resolve it, we need proper communication and allocation within the team.
Dashboards provide a great way to monitor the status of builds and test runs, but having too many separate dashboards will lead to difficulty in tracking and also waste time, so it is suggested that integration of layers wherever possible should be implemented.
Any change in business or design specifications has to flow from the leadership to the development process and finally into the test scripts. In a healthy dev process, the flow of information from top to bottom is seamless and quick.

There are many different approaches to testing websites and many different combinations of tools that can be applied, but with the new cloud-based technologies web testing has become more feature-rich and reliable. The ability to collaborate with the team on the same platform for building test plans supercharges the output as all members can access shared resources on the platform.

Things Every Product Manager Must Know About Testing

Vivek Mannotra — Tue, 05 Jul 2022 03:25:46 +0000

Being responsible for the development and maintenance of a rapidly evolving tech product is arguably one of the most technical and fast-paced jobs out there. The fact that the whole landscape of relevant technologies is also evolving outside your company and product adds another layer of complexity to the mix.

New devices, new features on your favorite cloud platform, latest APP frameworks, new DB paradigms, best IDE for your team, etc. The number of right decisions that a product team needs to make to ensure a smooth functioning application is quite large.

Despite variabilities like business and programming paradigms, application testing as a complement to application development has been widely adopted as an effective and indispensable tool in the software development life cycle. Let’s try to briefly cover all the basics:

Why is it important to test software?

Establishing a healthy system for testing, which is running in sync with the design and development part of the application development effort, is very important. It is a scientific way of capturing, documenting, sharing, and resolving problems in a software application in a collaborative manner. It helps prevent issues from causing business loss or poor user experiences.

For example, in 2019, British Airways had to cancel 100 and delay 200 flights due to issues with their online portal, which caused a crowd of people and many long queues outside Heathrow airport, Or how about this report which explains how a financial service provider in the US lost $440 Million because of a software error. Needless to say, no business wants to go through any such tragedy, and software testing is the most effective way to ensure such events don’t occur.

Visualizing the testing workflow

We can say that software testing, when done correctly, ensures quality and prevents business failure and customer unhappiness.

Right strategy and tools

How and what to test during the development of a software application depends mainly on the particulars of the business use case. Developing the most efficient and cost-effective testing strategy is not a trivial task. It is easy to get lost in the sea of online advice and waste time, but you can prevent that if you understand the fundamentals we are laying out here :

Types of testing:

Testing is a broad field of study, and there are many different ways to classify types, but these are the basic layers at which tests are usually implemented:

Unit testing: Testing parts of the codebase through test code, usually part of the same bundle as the working application code.
Integration testing involves testing sections of the codebase as interdependent functions or modules through test code or other tools.
System testing: Testing the workings of an application at the level of features like login, signup, and other supported flows. This helps validate parts of the application working together.
Acceptance testing: This is usually the final stage of testing in which the fully assembled application with data is tested in a live or pre-production environment. This involves testing with actual or mock users.
Performance and security testing: With an increase in number of users, it becomes important to ensure that the servers can handle the request loads at peak usage times. Also, maintaining end-to-end security at each point of transaction between the app and user.

Modes of testing:

Based on who is testing:

Manual Testing: Testing of any component or aspect of a software application, carried out by a human.

Automated Testing: Testing of software components carried out by other applications and tools, with minimum human intervention.

Based on the relation between tester and application:

Black-box testing: When the tester is evaluating an application from outside, while not being aware of the internal structure and functions.

White-box testing: When testing is carried out with know-how and consideration for the internal structure of the application.

Testing DevOps

Development Operations or DevOps is a widely used term which refers to a range of activities including the coding, testing, building and deployment of software applications. For a product team, the main task is to implement a unit and integration testing plan as part of the technical architecture. Depending on the level of automation, teams use:

Continuous Integration (CI): This term is used to describe workflows where the process of validating code commits of individual contributors and merging with the parent repository is carried out in a seamless automated manner.
Continuous Delivery/Deployment (CD): This is when a code commit is validated, merged with the parent and in the same flow, also deployed to a testing or production server.

Testing in a CI/CD workflow usually involves :

Writing unit tests: This is a default part of all major programming language packages. Read more about unit testing in Javascript, Python, Java/Kotlin, Swift and Go. With the advent of no-code/low-code and AI-based tools like Devmate, Ponicode etc., you can now automate parts of the Unit test writing process also.
Testing source code and build: This involves running unit tests in batches, with other tests on the code to test functionality and code quality. After coding the next task is to bundle the application into a deployable format as specified by the technical architecture. It involves post-processing the source code and installing packages and dependencies as needed.

All major cloud hosts like Github, Gitlab, Bitbucket have native CI/CD support.

Testing backend and database: For APIs and backend applications like micro-services, integration testing is a very crucial step. Because of complicated architecture and many dependencies, it is important to ensure proper documentation, performance and security of REST APIs.

Also, the data being passed along needs to be tested to ensure proper values, constraints etc. Testing DB schema, tables, triggers etc. separately through specially designed queries and tools is sometimes referred to as Database Testing.

Using tools like Swagger, Apigee will allow you to generate, document and monitor your API workflows.

Testing UI: Testing the application user interface in depth is the job of QA teams, but most developers will have a test deployment setup on their dev machine for a brief round of validations that can be performed before code commits. The main goal for this is to ensure proper integration between database, APIs, dependencies and the user interface.
Testing business with users:

Once an application is deployed, the QA team starts work on the system and acceptance level testing. The starting point for this activity is the test plan document which captures in detail the expected behavior of the application during various business scenarios. It highlights all the mission-critical workflows and helps the testing team in understanding the business deeply. With a test plan at hand, the next step is to :

Document test cases: For a manual scenario, test cases are maintained in a collaborative documentation platform, and the tester is responsible for execution and reporting on the test runs. Tools like JIRA, Trello etc. are popular choices for this task.

But in an automation scenario, the test cases are encoded into a script or 3rd party automation tool like Selenium.

Prepare mock data: Most test scenarios require users to input information, this behavior is replicated during system testing by preparing and using mock data as a substitute for user data.

Tools like Visual Studio, Devart, DTM etc. can help you generate mock data.

Test execution: With the right script and data, the test run takes place. A report is generated at the end, which captures various metrics, helping gauge the outcome of the run. Another priority at this step is to execute on an applicable range of devices and browsers to ensure a wide range of compatibility and accessibility.
Documenting issues and exceptions: During and after test execution, all unexpected behavior is captured and documented into the tracking tool, where it is marked up with meta-data like date/time, location, screenshots/videos, steps to recreate, severity, ownership etc.
Iterate until fixed: The dev team fixes the issues logged in the bug tracker and the cycle repeats until the QA leads give a green light to the app.

Role and perspective of different stakeholders

At the level of business leadership, the expectation from the testing process is to ensure that the application(s) is deployed and available to the users while ensuring the best performance, accessibility, and experience. All the documented business requirements and design standards need to be implemented without any discrepancies. Ultimately, the product should be able to fulfil the business goals e.g. core workflows like sign up, log in, booking, product purchase, etc. should work in users’ hands without any hiccups.

From a technical leadership perspective, including PMs, the main goal is to achieve continuous development and integration flow (CI/CD). The development and QA/QC teams work with each other in a synchronized manner. Agile has been widely accepted as the leading Product Management methodology in today’s industry. The technical leaders will maintain a healthy QAOPS workflow, ensuring that the scripts are running as scheduled and generating relevant reports periodically. Constantly monitoring these reports and keeping a keen eye to ensure that the whole application ecosystem delivers on business goals.

At the level of a programmer, the goal of testing is to check and validate any new additions or changes to the code base, before and after they are deployed. In other words, testing for ‘bugs’ in the application code in one’s development, testing/staging, and finally, the production environment. Guidelines for testing within the scope of a particular module are shared with the team, and each contributing member has to write unit tests to cover any new functions or changes they have added to the codebase.

After the initial sanity check on the development side, the QA/QC team takes on testing it further. The testing process can be manual, automated, or mixed. Testers will document application behavior and anomalies in the form of defects logged in a reporting portal, where they can be assigned for fixing and tracking.

After QA approves a version of the app, it is released to a controlled set of users for the first stages of acceptance testing.

Reporting and communications

Once you have a sound strategy, right tools and team in place. You need dashboards, emails and messages flowing between the stakeholders, acting as a stream of relevant information regarding tests, bug fix status, performance etc. It usually involves:

The business team evaluates the current state of application performance, superimposes near future business goals, and communicates the vision to the technical leadership and design team in form of business requirement documents.
Technical leadership with the QA team updates the test plan to include the overall approach collaboratively, involving all stakeholders.
Updating the test cases in the bug tracker, like a traceability matrix. Assigning ownership and passing the tasks to relevant developers.
Updating manual and automation scripts to include new cases and doing mock runs.
Reports that gather and display the status and outputs of all the test runs in a time frame, going out as emails, messages, and notifications.

It is important to have the relevant information reach all parties to ensure that the right action can be taken in the event of a discrepancy. When an application is being worked on regularly, it helps the stakeholders gauge the overall health of the application.

Conclusion:

It is imperative to set up a system where all participants can effectively collaborate on the testing process. To get the best results, you need the whole team to work together. Armed with the right knowledge and tools, you can effectively improve the quality of deliverables and achieve great things with your product. Although some would argue about the resource-heavy and time-consuming nature of the testing workflow, in a proper development environment, testing provides a unique opportunity to improve the quality of any software application without having to risk any real user or business opportunities.