DEV Community: Wahome Stephen

AI-Based systems need regulation. Part of the regulation is to ensure high quality characteristics.
ISO/IEC 25059 provides defines out characteristics that AI-based systems should have

AI Functional correctness
Functional adaptability
User controllability
Transparency
AI robustness
Intervenability
Societal and ethical risk mitigation.

Every AI-based system should therefore aim at achieving the above quality characteristics. (Hold on we will discuss each of these in a few)

Before then, it is important to note that achieving each of these quality requirements is not easy. AI-based systems have a fair share of challenges which include;

Vague specifications
Non-determinism
Self-learning
Limited explainability
Evolving standards

AI-specific Quality Characteristics

AI Functional Correctness

AI-based systems especially those that use probabilistic ML, cannot guarantee perfect accuracy. A certain error rate is expected in AI outputs. ISO/IEC 25059 evaluates functional correctness by considering both correct and incorrect outputs and defining acceptable thresholds for correct results, reflecting the inherent variability in AI-based system outputs.
AI Functional correctness relates to product quality

Functional adaptability

This is a new sub-characteristic of functional sustainability that also relates to product quality. Functional adaptability simply means the ability of an AI-system to autonomously adapt to changes in its operational environment after it has been deployed.

User Controllability

User controllability relates to product quality and is a new sub-characteristic of interaction capability. In this context, interaction capability is a new name for usability.
This quality characteristic refers to: Human/ external agent can intervene in its functioning in a timely manner.

Transparency

This relates to the degree in which appropriate information about the AI-based system is communicated to stakeholders.
Transparency is a new sub-characteristic of interaction capability and satisfaction and relates to both product quality and quality in use.

AI robustness

It is a product quality sub-characteristic of reliability.
Simply put, AI robustness is the ability of an AI-based system to maintain its level of AI functional correctness regardless of circumstances.
Such circumstances include;
- Presence of biased, adversarial or invalid data inputs
- External interference
- Adverse environmental conditions
- Operator misuse

Intervenability

Refers to ability of an operator to intervene in an AI based system functioning in a timely manner to prevent harm or hazard.
Intervenability is a product quality sub-characteristic of security.

Societal and ethical mitigation

Related to 'Quality in use' and is a new characteristic of 'Freedom from risk'
This quality characteristic considers many areas to mitigate both societal and ethical risks such as human centered designs, security, safety and fairness.

AI & Safety

Safety related systems have potential to cause injury or harm to people, property or environment. Developing and testing non-AI safety-related systems can take a lot of effort, but is feasible; however, for AI-based systems, there are several additional challenges:

Specifications

Unlike traditional safety-related systems that have requirements defined for a complete system upfront, requirements for AI-based systems often begin with vague goals and are then implicitly provided via the training data that encodes patterns rules and objectives. All details are not formalized upfront.
As a result, we often have inadequate requirements and implementation traceability.

Non determinism

Non-determinism makes it challenging to guarantee the precise behavior of AI systems. Even rigorously tested models exhibit unexpected behavior due to factors such as random number generation and slight variations in input values.

Self learning

AI based system self-learn and improve with new data. Therefore, systems behavior of AI-based systems often move from originally tested behavior.

One way to deal with self-learning is managing how a model learns as well as the data it uses can help avoid emergence of new problematic behavior.
Alternatively, use safety guards to prevent model from learning or making decisions that could compromise safety.

Explainability & Transparency

It is important to understand how and why systems make certain decisions. AI based systems can be complex - they use billions of parameters making it difficult for humans to comprehend. As a result, their decision making process is often not transparent.
AI techniques such as LIME (Local interpretable
model-agnostic explanations) can be used to provide insights into AI-based systems' reasoning. However, they are not widely available and may compromise system performance.

Evolving regulations

The regulatory landscape for safety-related AI-based systems is constantly evolving. In a recent development; The EU AI Act [EU AI Act] classifies AI systems used as safety components (such as in aviation, medical devices, or automotive) as high-risk and imposes strict requirements on their development and testing.

Acceptance Criteria for AI-Based Systems

When evaluating the quality of an AI-based system, it is essential to consider both functional and non functional quality characteristics. This helps confirm that the AI-based system functions as intended and satisfies broader quality requirements.

Understanding Test Coverage: A Beginners Guide for Modern QA Teams

Wahome Stephen — Tue, 11 Aug 2026 10:37:14 +0000

How would do you ensure test coverage during testing? This is a question I have faced more times than I can count across different interviews that I have done. While it may look simple on the surface, the truth is quite the opposite. In this piece we explore how identify what metrics matter most, how to set realistic targets and implement coverage strategies that improve quality.

Why Should QA Teams care about Test Coverage?

Think of that time you confidently deployed code only to find critical bugs in production. Worse even, a customer reported this. I can already bet that you did have a hard time convincing your stakeholders that your tests were enough. Now, here is the catch; most QA teams think test coverage as just hitting arbitrary numbers. This is a wrong perspective. Test coverage instead is about meaningful coverage that actually prevents defects. To achieve this, we always want to focus on business-critical paths and critical user journeys.

Understanding Different Types of Coverage

When buying a car, you do not want to just look at the color of the car. Many other factors such as mileage, practicality and reliability must also come into the picture.
Likewise, software testing should also not focus on just one type of coverage.

Line Coverage: The Foundation

This is the most basic form of coverage. It checks what lines of code were executed during testing. Using the car analogy, this would be checking that all doors, wheels are present before further examining them.

Branch Coverage: The Decision Points

A big tree branch is basically a giant hierarchy made of smaller branches, which then split into twigs. For a car, you want to try each door. Think of branch coverage as checking conditional statements such as if-else and switch cases.

Functional Coverage

This tells you what methods or functions were called during testing. It is a good indicator of unused code but it doesn't tell you how roughly each function was tested.

Statement Coverage

Statement coverage checks if each executable statement was run, giving you a more granular view than line coverage.

Conditional Coverage: The Deep Inspection

It verifies that each Boolean subexpression was evaluated to both true and false.

Coverage Type	What It Really Tells You	When to Use	Target Range
Line	Basic code execution	Initial assessment	70-80%
Branch	Decision path execution	Logic validation	60-70%
Function	Method utilization	Dead code detection	80-90%
Statement	Detailed execution	Thorough testing	70-80%
Condition	Boolean evaluation	Complex logic testing	60-70%

Common Coverage Pitfalls to Avoid

Many teams aim at 100% coverage. Here's is where this is often counterproductive.

Cost vs. Benefit - Moving from 80% to 100% coverage, most times, outweighs the benefits.
False Confidence - High coverage doesn't not always translate into high quality tests.
Maintenance burden - With more tests, comes more maintenance.

Past SDET insights suggest that having a combination of branch coverage for critical business logic (targeting 70%) and line coverage for general code (targeting 80%) is a sweet spot between effort and risk mitigation.

The Wrong Tests Problem

While high coverage is important, having high coverage with wrong tests beats the purpose.

Happy Path Only: One of the most commin mistakes is writing tests that only validate expected user behavior. Effective test cases should have each parts
- Happy path: Verify expected behavior under normal conditions. e.g. Valid login with correct credentials.
- Unhappy path: Verify system handles errors correctly. e.g. Login with invalid password.
- Edge cases: Test boundary conditions and unusual scenarios e.g. Password at maximum allowed length, empty fields, special characters
Shallow Assertions - Some tests execute code correctly but fail to validate meaningful outcomes.
Poor Example
- User clicks Submit
- Test passes because no exception occurred
Better Example
- User clicks Submit
- Verify:
  - Correct API response is returned
  - Database record is created
  - Success message is displayed
  - Business rules are enforced
The AAA Principle

Every automated test should follow the Arrange-Act-Assert (AAA) pattern:
Arrange: Set up data, mocks, and prerequisites
Act: Execute the behavior being tested
Assert: Verify the expected outcome

If either the Act or Assert phase is missing, the test provides little value.

A test that executes code without validating results is closer to a script than a test.
Brittle Tests - Tests that break with minimal code changes. This creates false alarms, increases maintenance effort, and reduces trust in automation. Automation tests are most affected.

Common Causes
- Dependence on UI styling
- Dynamic page structures
- Hard-coded waits
- Unstable locators
- Environment-specific data

How to make sure everything is tested

Ensure every tester on the QA team is aware of the requirements and the testing methods.
- Ask clarifying questions to relevant stakeholders
- Clear any blockers encountered.
Prioritize your Requirements and focus your energy where it is most needed. It is important to focus on risk based testing to ensure that most important requirements are covered first.
Not every release is like the other. A tester should be aware of how a certain release is different from the previous one. This way they can identify critical requirements more accurately and focus on maximum positive coverage
Adapt Test Automation. Test automation acts as the primary engine for scaling and deepening test coverage, enabling software teams to execute vast, complex test suites that are functionally impossible to manage manually
Use Test Management tools to always stay in the know. Tools like TestRail, PractiTest, Qase and Testiny centralize QA work offering unified testing and real-time reporting.
Smart Work Assignment – Channel your best resources towards critical tasks and let new testers explore more for a fresh perspective
Maintaining a checklist for all tasks and miscellaneous activities
Interact more with other stakeholders such as developers, scrum master and the BA team. This allows tester to get insights into the application behavior.
Keep track of all your build cycles and fixes.
Identify the most impacting problems in the initial build itself (when possible) so the later ones can work for better stability and reach those areas blocked by prior problems

Regulations and Standards for AI

Wahome Stephen — Mon, 10 Aug 2026 13:08:14 +0000

AI regulations and standards are crucial for the responsible development, deployment, and use of AI.
Regulations have been put in place to

Foster trust in AI
Help promote realization of AI's benefits while mitigating potential harms.

Ideally, compliance with such regulations and standards should guarantee that AI-based systems are safe, fair, transparent, sustainable, accountable, ethical, and used responsibly.

Internationally, the OECD AI Principles OECD AI and the UN report on Governing AI for Humanity [UN Gov AI] serve as influential soft law instruments that foster a shared understanding of responsible AI
stewardship.
They act as a compass for national governments and organizations as they formulate their own AI strategies. These principles emphasize human-centric AI, ethical considerations, and the
importance of international cooperation.

The EU AI Act represents a landmark regulatory step, demonstrating a risk-based approach to regulating AI.
By categorizing AI-based systems by risk, from minimal to unacceptable, regulations are tailored accordingly. High-risk systems, particularly those that impact fundamental rights or safety, face stringent requirements that encompass rigorous testing, data governance, and human oversight. The substantial
financial penalties for non-compliance, based on a percentage of global turnover, underscore the EU's commitment to enforcement. In contrast, many nations outside the EU are adopting a more permissive approach, favoring lighter-touch regulations to encourage innovation.

Technical standards, developed by organizations such as ISO and IEEE, are crucial for translating high level aspirations into practical implementations. They provide concrete technical specifications and best practices, bridging the gap between policy and practice.
For example, ISO/IEC TR 29119-11 provides
detailed guidance on testing AI-based systems, a critical element in demonstrating regulatory compliance.
Meanwhile, the ISO/IEC 42119 series is being developed to cover various aspects of AI-based system testing.
Furthermore, sector-specific regulations are emerging in areas such as healthcare and finance, recognizing the unique risks posed by AI in these domains.

To effectively navigate this evolving landscape of AI governance, continuous dialogue and collaboration are paramount. Governments, industry, academia, and civil society must actively engage to achieve a harmonized and effective approach to AI governance worldwide. Moreover, given the dynamic nature of AI, regulations and standards must be regularly reviewed and updated to remain relevant and effective in guiding responsible AI development, deployment, and use.

Machine Learning Development Frameworks

Wahome Stephen — Mon, 10 Aug 2026 12:57:26 +0000

ML development frameworks provide a toolkit for building and training ML models. Typical functionality provided by these frameworks includes:

Data Handling: They assist with loading, preprocessing, and managing the data used to train and test the model. This might involve cleaning, formatting, and transforming the data into a suitable format for the chosen model.
Model Building: These frameworks offer libraries of ML algorithms and tools to design the architecture of the constructed ML model. This includes specifying the type of model (e.g., neural network, decision tree), the number of layers and connections, and the mathematical operations performed within the model.
Training and Optimization: Frameworks provide algorithms that iteratively adjust the model's internal parameters based on the training data and on the desired result. The goal is to optimize the model's performance in accomplishing the desired task (e.g., classification, ML regression). Some frameworks may support distributed training and enhance or fine-tune pretrained models.
Evaluation: They offer tools to evaluate how well the trained model performs on unseen data. This might involve measuring accuracy, precision, and recall for classification tasks, or error rates for ML regression tasks (see 3.1.1).
Deployment: Some frameworks provide capabilities for deploying the trained model for real-world use. This could involve converting the model into a format suitable for integration with web applications, mobile devices, edge devices or embedded systems. These frameworks can operate at different levels of abstraction. Some offer a lower-level application programming interface (API), providing developers with more control over model building but requiring more coding expertise. Others offer a higher-level API, simplifying model creation but offering fewer customization options.

Different frameworks can focus on different application domains. Some are general purpose and support a wide range of application areas. In contrast, others are more specialized, focusing on specific areas such as image recognition, speech recognition, and language translation.
Selecting the most appropriate framework can depend on several factors, such as:

the application area;
the need for a user-friendly interface for rapid prototyping;
configurability for complex models;
the expertise of the users;
deployment considerations, as some frameworks are better suited for resource-constrained environments;
level of (community) support;
ecosystem maturity.

Development and Hosting of AI Models

Wahome Stephen — Mon, 10 Aug 2026 12:48:42 +0000

There are two ways to acquire AI-based systems

Through 3rd party vendors
Developed privately in an organization

AI-based systems typically rely on pretrained models and can be deployed on-premises or in the cloud.
These models are hosted either on-premises or in the cloud, where cloud-based options are often accessed as a service (AIaaS).

Third-party AI-based systems typically come as pretrained models or AI as a Service (AIaaS), enabling faster deployment and quicker time-to-market.
Private AI-based systems (on-premises or customized cloud setups) can be better tailored to specific requirements, but their development will likely require specialized skills, either through in-house experts or outsourced teams. Local development enables direct control and privacy.

Development

Small models, such as decision trees or compact neural networks, can be developed on personal computers, while mid-sized models may require dedicated GPUs.
For large-scale models, high-performance on-premises server clusters become necessary with their associated energy, cooling, and hardware costs.

Cloud development offers significant flexibility.

Public clouds, in particular, provide pre-configured environments with pay-as-you-go pricing, limiting initial hardware investment and scales easily.
Private clouds can provide enhanced security and privacy for applications that require it, but this control necessitates a greater upfront infrastructure investment.

Many organizations adopt hybrid approaches, including developing prototypes locally before scaling to cloud infrastructure, maintaining sensitive components on-premises, such as the preparation of private data, and leveraging cloud resources for compute-intensive tasks.

Hosting

AI models can be hosted in various environments, ranging from local setups to cloud-based platforms.
Local hosting involves running smaller models on personal computers or smartphones, offering privacy
and eliminating cloud licensing costs, although this provides limited hardware capabilities.
For larger AI models, organizations may establish dedicated servers, which require a significant upfront investment but provide enhanced control.

Cloud hosting of AI models can be on public or private clouds.

Public cloud services provide scalable access to robust, powerful infrastructure, eliminating maintenance concerns and making them ideal for fluctuating workloads.
Private clouds offer similar benefits, with enhanced security and customization options, and are either managed in-house or through dedicated providers, obviously at a higher cost.
Hybrid approaches combine these methods, allowing organizations to run some operations locally while leveraging cloud elasticity for intensive tasks.

The optimal development and hosting solutions, which are typically decided upon separately, depend on factors such as model size, complexity, performance requirements, budget constraints, security and data privacy considerations, deployment needs and regulatory requirements. Some organizations adopt multitiered strategies to balance efficiency and control.

Hardware for Machine Learning Systems

Wahome Stephen — Mon, 10 Aug 2026 12:31:02 +0000

Various hardware is used for Machine Learning systems during training and inference.
ML benefits from hardware that offers

Ability to work with large data structures
Massively parallel processing e.g. to support matrix multiplication.
Low-precision arithmetic (Quantization) Uses fewer bits for computation resulting in faster processing, lower power consumption, smaller and more cost-effective chips and reduced bandwidth requirements.

General-purpose CPUS provide support for complex operations with high precision that is not required in ML application. But provide only a few cores.
Graphics Processing Units (GPUs) have thousands of cores and are designed to perform massively parallel, yet relatively simple, graphics processing. As a result GPUs outperform CPUs in ML applications even though CPUs run at higher clock speeds. For small scale ML work, GPUs generally offer the best option.

Certain hardware is specifically designed for AI, such as purpose-built Application-Specific Integration Circuits (ASIC) and System-On-a-Chip devices. These have multiple cores, specialized data management and capability to perform in-memory processing.
Best suited for edge computing, while training of the ML models is performed in the cloud using specialized hardware.
AI-specific hardware architectures continue to be developed. This includes neuromorphic processors,
which do not use the traditional von Neumann architecture but rather brain-inspired designs mimicking
neuronal structures.

Generative AI

Wahome Stephen — Mon, 10 Aug 2026 12:07:41 +0000

Generative AI (GenAI) refers to AI based systems specialized in creating new content, such as text, images, videos, music or complex data. Many also support classification and prediction.
They learn from vast amounts of data to produce outputs that resemble their training data. Hence;

More creative
Practical applications

Technologies behind GenAI

Generative Adversarial Networks (GAN) Use two neural networks in competition to create highly realistic synthetic data.
Diffusion models Generate content by gradually adding and then removing noise from data resulting in high quality outputs.
Transformer models Support LLMs Utilize self attention mechanisms to generate coherent and contextually relevant text. - Are being adapted for multimodal tasks.

*Disadvantages of GenAI *

While GenAI has its unique technical foundations, these systems raise significant societal and ethical concerns.

Misuse - Can be exploited to create deepfakes thus undermining trust in digital media
Ease to create synthetic data amplifies risks related to privacy, security and manipulation.
Impact on employment - May lead to workforce disruption and necessitate widespread reskilling.
Sustainability - Training and running GenAI models involves substantial computational resources, resulting in high energy consumption and significant carbon footprint.

Most practical GenAI tools today are based on foundation models, which are then fine-tuned for specific
applications. The field is also advancing toward multimodal models capable of processing and generating
content across text, images, and audio, enabling richer, more flexible AI-based systems. Regulatory
frameworks, such as the EU AI Act [EU AI Act], are emerging to guide the responsible development and
use of these technologies.

What are the components of a bug ticket? What should a bug report contain? These are questions I’ve encountered in multiple QA interviews. Before you wonder what to answer in your next interview, let’s discuss. Got questions or a different approach?

Wahome Stephen — Mon, 10 Aug 2026 09:57:45 +0000

Wahome Stephen

Aug 6

Understanding Bug Tickets

#testing #agile #software #programming

3 min read

Different Types of AI technologies

Wahome Stephen — Mon, 10 Aug 2026 09:39:32 +0000

Artificial Intelligence (AI) encompasses a wide range of technologies each suited to specific tasks and challenges. One of the branches of AI is ML which allows systems to learn from data and build models without explicit programming. Some ML systems can adapt to improve continuously with new data throughout their lifetime. Others require explicit retraining to update their capabilities.

Supervised learning - Utilizes labeled data and algorithms such as linear regression and decision trees to perform tasks such as prediction and classification.
Unsupervised learning - Uncovers patterns in unlabeled data using techniques such as clustering.
Reinforcement learning - Enable intelligent agents to learn optimal behavior by interacting with the environment through trial and error.

Deep learning, a subset of ML methods use deep neural networks to solve complex problems.

Convolutional Neural Networks - Used for image recognition and object detection
Recurrent neural networks - Specialize in processing sequential data e.g. text and time series.
Transformers handle long range dependencies in sequences, powering models for natural language processing and vision transformers for images.

Other specialized AI technologies include;

NLP for language analysis
Computer vision - Analyzes visual data, supports apps like facial recognition and robotics.
Fuzzy logic - reasoning under uncertainty
Search algorithms = solving optimization problems e.g. navigation and strategic decision making.
Rule based reasoning systems/ expert systems - for structured decision support.

Full integration of these AI technologies remains limited. However, new developments such as LLM
demonstrate the potential to combine different AI technologies into unified, intelligent systems.
Agentic AI extends these technologies through autonomous agents that plan, reason, and act independently to
achieve goals in dynamic environments.

Narrow AI Vs General AI Vs Super AI

Wahome Stephen — Mon, 10 Aug 2026 09:12:47 +0000

AI has different capabilities. These capabilities can be categorized into Narrow AI, General AI and Super AI.

Narrow AI
Also known as Weak AI, this category represents all the deployed versions of AI today. It operates with limited domain knowledge and is designed to perform specific tasks e.g. image recognition, speech processing and language translation.

Cannot generalize beyond the functions learned.
Frontier AI is the most advanced version of these systems.

General AI
Also known as Strong AI. Possess ability to perform most intellectual tasks that a human can. Can learn and apply knowledge across a broad range of tasks without the need to be retrained on each new task. We do not have an AI-based system with this capabilities just yet.

Super AI
Also known as artificial super intelligence. This AI based system continuously self-improve without the need for human control.

For supper AI to be possible access to internet is not strictly required. However, its access could significantly expand its capabilities and influence. It would surpass human intelligence and general AI possibly posing an existential risk to humanity.
The transition from general AI to Super AI , if it will ever come to happen is called technological singularity.