confident_prep

Posted on Oct 11 Originally published at confidentprep.com

When is machine learning the wrong answer to a business problem?

#aws #machinelearning #ai

A manager asks for "an AI solution" to calculate shipping fees. The fees already come from a published rate table: weight band, destination zone, a fixed surcharge. Training a model would mean spending effort to produce an approximate version of a rule the business has already written down. The AWS Certified AI Practitioner exam (AIF-C01, Exam Guide version 1.1) checks whether you can spot this situation. One of the abilities it validates is to "identify the appropriate use of AI/ML and GenAI technologies to solve business problems".

The claim of this article: machine learning is the wrong answer in five situations:

the rule is already known;
the output must be exactly predictable;
the data is missing, poor or biased;
the business value cannot be measured;
a simpler, ready-made or narrower technique would do the job.

Each follows from how machine learning works. It infers a relationship it does not know in advance from examples, and its output is probabilistic rather than certain. The answer is written the way a strong candidate would give it in an interview, followed by five follow-up questions. Each follow-up has a short model answer and the mistake weaker candidates tend to make.

The strong answer: machine learning earns its place only when the rule is unknown and the data can teach it

Asked "When is AI or machine learning the wrong answer?", a strong candidate starts from the mechanism, not a list of slogans.

Machine learning starts from the idea that some mathematical relationship links inputs to outputs. The model does not know this relationship in advance. It can estimate it if it sees enough examples. The standard illustration is simple. Give an algorithm the pairs (2,10), (5,19) and (9,31), and it works out that the output is three times the input plus four. Asked about an input of 7, it predicts 25.

That example contains the whole argument. If you already knew "times three, plus four", you would write that line of code and skip the training. Machine learning helps when nobody can write the rule down but examples of it exist.

The strong answer therefore names the conditions that remove the case for machine learning:

a known rule;
a need for guaranteed, repeatable output;
too little data, or data of poor quality;
no measurable business goal;
a problem a simpler or ready-made technique already solves.

Machine learning is justified only after the cheaper questions are answered

Follow-up 1: a known rule beats a learned guess when output must be predictable

Interviewer: "Why not use machine learning anyway? It would learn the rule eventually."

Model answer: Because the two kinds of system make different promises. Most software responds predictably, so you can say "if the user does this, he gets that". Machine learning learns from observation, so it is probabilistic. The statement becomes "if the user does this, there is an X% chance of that happening".

Tax calculations, fee schedules, eligibility checks against fixed thresholds and access permissions all need the first kind of promise. Replacing an exact rule with a probability adds error and buys nothing.

The mistake weak candidates make: treating "learns automatically" as an advantage in every case. In practice, the giveaway is an answer that never mentions that a learned model can be wrong on inputs where the written rule would always be right.

Think of it as the difference between a printed timetable and a commuter's guess about when the bus usually arrives. The guess is useful only when no timetable exists.

Rule-based software promises an outcome; machine learning promises a likelihood

Follow-up 2: without enough clean, representative data there is no model worth building

Interviewer: "The rule is unknown, but we only have a few hundred messy records. Is machine learning still the answer?"

Model answer: Probably not yet. A model's performance depends on the quality of the data it was trained on. Missing values, inconsistent entries and noise can significantly degrade accuracy. A dataset that is not large enough can stop the model from learning effectively.

Data that looks plentiful can still be skewed. When some classes appear far more often than others, a model can do well on the majority class and fail on the minority class. For example, a hiring model trained on history that favoured one demographic may keep favouring it. Training an unbiased system needs large volumes of data and good data-quality processes behind them.

Supervised learning is the style trained on labelled examples, where each input is tagged with its correct output. Labelling at the scale it needs is itself a challenge.

The mistake weak candidates make: assuming the model will "clean up" the data, or that more training fixes bad inputs. A sensible default is to treat data readiness as a gate. If the data cannot be fixed, the project is not ready for machine learning.

Follow-up 3: a project with no measurable business goal cannot be justified

Interviewer: "Leadership wants AI on the roadmap. Where do you start?"

Model answer: With the problem, not the technology. Guidance on adopting machine learning puts "business goal" first in the lifecycle:

Identify the problem to solve and the value machine learning would add.
Check whether that value can be measured against specific success criteria.

Next comes problem framing. You decide what is observed and what should be predicted, and which performance and error metrics matter. Only after that do data processing, model development, deployment and monitoring follow.

If you cannot say what the model predicts, or how anyone would know it is working, the honest answer is that machine learning is premature.

The business goal and problem framing come before any data or model work

The mistake weak candidates make: answering with a service or a model type before naming a measurable outcome. On a scenario question, any option that skips "define the goal and success criteria" deserves suspicion.

Follow-up 4: when a decision must be explained, an opaque model carries a real cost

Interviewer: "A lender wants a model to approve loans. Any concerns?"

Model answer: Two concerns: explainability and bias.

Explainability. As models become more complex, and deep learning models especially, their decisions become harder to interpret. This affects usability, trust and the ethics of deploying them. Deep learning is the family of methods that passes data through many layers of artificial neurons.

Bias. Imbalanced historical data can lead a model to keep reproducing the patterns in that data.

Governance. Deploying AI means managing data quality, privacy and security, within regulatory restrictions and privacy laws. The organisation stays accountable for customer data. Responsible AI is the practice of attending to fairness, transparency and accountability in how AI is built and used.

None of this rules machine learning out automatically. It does mean that a decision that must be justified to the person affected is a weaker fit for a model whose reasoning is hard to read. The choice is between a more interpretable approach and accepting the cost of bias checks and oversight.

The mistake weak candidates make: treating responsible AI as a separate topic from "is this the right tool". In practice, the two often turn out to be the same question.

Four warning signs that machine learning may be the wrong answer

Follow-up 5: choosing the wrong type of AI is also a wrong answer

Interviewer: "Fine, AI fits here. Should we just use the biggest generative model available?"

Model answer: Not by default. Foundation models are large deep learning models trained on broad, unlabelled data that can perform many different tasks from a prompt. They are not automatically the best choice. Many organisations still use conventional machine learning models for many tasks, and those models can outperform foundation models for many use cases.

The same care applies within machine learning, because each learning style has a known limit:

Unsupervised learning, which finds structure in unlabelled data, cannot give precise predictions.
Reinforcement learning, which learns by collecting rewards, struggles because real-world environments change often and with little warning.

Pretrained AI services offer ready-made intelligence for common use cases. When one fits, it can remove the need to train anything.

Cost is the last factor. Training large models demands significant computing resources and can be time-consuming and costly. Processing power can be expensive and limit how far a system scales.

The mistake weak candidates make: equating "most capable" with "most appropriate". A sensible rule is to choose the narrowest technique that meets the success criteria. Choose a conventional classifier when the task is a fixed set of categories with labelled history. Choose a generative model when the output itself is new text or images.

How AIF-C01 turns this judgment into scored questions

The exam guide lists four things the exam validates:

describing AI, machine learning and generative AI concepts;
identifying the appropriate use of these technologies for business problems;
determining the correct types of technology for specific use cases;
using them responsibly.

The second and third map directly onto the follow-ups above.

The target candidate uses AI/ML solutions on AWS but does not necessarily build them. Coding models, feature engineering, hyperparameter tuning and statistical analysis of models are explicitly out of scope. The exam therefore asks you to judge fit, not to build the fix.

The question formats suit judgment. Multiple choice has one correct answer and three distractors. Multiple response needs every correct option selected. Matching pairs responses with three to seven prompts. Ordering asks you to place three to five responses in sequence.

In practice, a lifecycle ordering question or a use-case matching question is a natural home for "business goal first" and "right type for the task". That is an interpretation of the format, not a published statement of which questions appear.

Scored content is spread across five domains; judging fit draws on several of them

Weighting:

Domain	Share of scored content
Fundamentals of AI and ML	20%
Fundamentals of GenAI	24%
Applications of Foundation Models	28%
Guidelines for Responsible AI	14%
Security, Compliance, and Governance for AI Solutions	14%

Scoring:

50 questions count toward the score; 15 more are unscored and not identified.
Results are reported on a scale of 100 to 1,000, and 700 is the passing score.
Scoring is compensatory, so you need to pass only the overall exam, not each section.
Unanswered questions count as incorrect, and there is no penalty for guessing.

Key takeaways

Machine learning infers an unknown relationship from examples. If the rule is already known, ordinary code is the better answer.
Machine learning output is probabilistic. Where the outcome must be guaranteed and repeatable, a learned model adds error.
Poor, scarce or imbalanced data, or a goal with no measurable success criteria, means the project is not ready for machine learning.
Explainability, bias and governance are part of judging fit, not an afterthought.
"Most capable" is not "most appropriate". Conventional models and ready-made services often fit better than a large generative model.
AIF-C01 tests this as judgment, not building: identifying appropriate use and choosing the right type of technology for a use case.

The honest limit: this article draws on the exam guide's overview and two general explainers. It does not reproduce the detailed task statements inside each domain, so it cannot say exactly how many questions address inappropriate uses of AI or how they are worded.