confident_prep

Posted on Oct 20 Originally published at confidentprep.com

How training and inference work: build and test a model by hand

#aws #ai #career #beginners

You want to train your first model, but "training" and "inference" still sound like the same fuzzy thing.

This article argues that the fastest way to make those words concrete is to train and test a tiny model yourself, on paper, before you open any tool. You will run a learning procedure over six examples, get a model out of it, score it on examples it has never seen, and then break it on purpose so you can watch overfitting happen. Every no-code tool hides these steps behind buttons. If you have done them once by hand, you will know what each button is doing and what its scores mean.

You need a pen, a sheet of paper and nothing else

There is nothing to install, no account to open and no code to write. The exercise costs nothing, and cleanup means putting the sheet in a drawer. The aim is not a useful model. The aim is to watch four separate things happen in order.

Four words name those things, and the exercise depends on keeping them apart:

Algorithm: the learning procedure, the recipe that turns example data into a model.
Training: the stage where the algorithm runs over the data.
Model: the trained result, the thing that takes a new input and produces a prediction.
Inference: the stage where the trained model is used on new input.

Think of it as a bakery. The recipe is the algorithm, baking is training, the loaf is the model, and serving a slice is inference. The recipe is not the loaf. The loaf is not being baked while someone eats it.

An algorithm trained on data produces a model, and the model alone answers new input

Step 1: Write down labeled training data

Draw a small table. Each row is a delivery, with its distance and whether it arrived late. The figures are invented for this exercise and describe no real company.

Six labeled deliveries: the late column records the correct answer for each row

Notice two things about this table. First, it is labeled data: the correct answer, late or not, is recorded for every example. Because the answers are recorded, you can use supervised learning, which means learning from examples paired with their correct outputs. If nobody had written down which deliveries were late, this route would be closed, however many rows you had.

Second, it is structured data: it follows a fixed schema of rows and columns. A pile of scanned delivery notes would hold the same facts as unstructured data, with no set data model, and it would need far more preparation before any learning could start.

Below the table, draw a line and write four more rows you will not look at during training: 3 km, not late; 6 km, not late; 11 km, late; 14 km, late. This is your test data. Covering it up is the most important move in the exercise. It is the only way to find out whether your model learned the pattern or just learned the six rows.

Step 2: Run the algorithm and get a model

Here is your algorithm, written as plain instructions:

Try a cut-off distance between each pair of neighbouring distances.
For each cut-off, predict "late" for every delivery farther than it.
Count how many of the six training rows that prediction gets wrong.
Keep the cut-off with the fewest mistakes.

Now run it. A cut-off at 8.5 km predicts late for E and F only. It gets C wrong, because C was late at 5 km, and gets everything else right: one mistake. A cut-off at 4.5 km also makes exactly one mistake, on D. Neither does better, so choose 8.5 km as your tie-break and move on.

What you are holding now is a model: the single sentence "late if distance is over 8.5 km". It is not the four-step procedure. It is what the procedure produced. If you collect more deliveries next month and want a better cut-off, you run the algorithm again over the new data. That is retraining, and it is the only way this model changes.

Write the training score next to the model: 5 out of 6 correct.

Step 3: Uncover the test rows and score the model

Uncover the four test rows and apply your sentence to each one. At 3 km it predicts not late, which is correct. At 6 km it predicts not late, also correct. At 11 km and 14 km it predicts late, and both are correct. That gives 4 out of 4.

You now have two numbers: one from the data the model learned on, one from data it never saw. Reading those two numbers together is the core skill in judging a model. A good score on training data alone tells you very little. The model has already seen those answers.

Training happens once per build; inference repeats every time the model is used

Step 4: Use the model on a new delivery

A new delivery comes in at 9 km. Look at your sentence, and nothing else: over 8.5 km, so predict late. That is inference. You did not look at the training table. You did not rerun the four steps. The algorithm and the training data play no part at this stage. Only the model does.

This is the distinction that tends to stay abstract until you do it by hand. Training is effortful and happens when the model is built or rebuilt. Inference is quick and happens every time the model is used. A common mistake runs straight across that line: proposing to "retrain" when the real problem is how answers are being delivered to the people who need them.

Step 5: Break it on purpose by memorizing

Now build a second model with a lazier algorithm: "copy the training table exactly; for any distance not in the table, answer not late."

Score it on training data and it gets 6 out of 6, a perfect result. Every answer it gives was copied straight from that table. Score it on the test rows and the picture changes. None of 3, 6, 11 or 14 km appears in its table, so it answers "not late" four times. That is right twice and wrong twice: 2 out of 4.

This is overfitting: a model that scores well on its training data and poorly on new data, because it learned the examples, quirks and all, rather than the pattern behind them. Think of a student who memorized last year's answer sheet and is lost on a new paper. Notice that the memorizer even reproduces delivery C, the odd late one at 5 km. Your cut-off model sensibly got C wrong, and that one training mistake is part of why it held up on new data.

For completeness, try the opposite failure. Use a model that always says "not late". It gets 3 of 6 on training and 2 of 4 on test. That is low on both, which is underfitting: a model too simple to find the pattern at all.

The gap between training and test scores tells overfitting from underfitting

The rule to keep is a number test. High on training and low on new data means overfitting. Low on both means underfitting. Good on both means a good fit. The fix for overfitting is to make the model generalize better, for example by making it less flexible. Regularization is one name for techniques that do this. Your one-sentence cut-off is about as inflexible as a model can be, which is exactly why it beat the memorizer.

Step 6: Decide how the model would answer people

Your model works. The next practical question is how it would be put to use. There are four common ways to serve a trained model, and the deciding question is about the people asking, not about the model.

Real-time: someone is waiting for the answer right now, such as a dispatcher on the phone.
Serverless: requests come in bursts with idle gaps between them, and a slower first answer after a quiet spell is acceptable, because capacity scales down to nothing when idle.
Asynchronous: each request is large or slow, so it is queued, and the result is collected later from a storage location.
Batch: a whole dataset is scored in one job, such as every delivery on tomorrow's schedule, and nothing needs to stay running between jobs.

Think of it as a post office: a counter for people who are waiting, a sack of mail sorted overnight, and a large parcel left to be picked up later.

Who is waiting decides first, then dataset shape, then request size, then traffic pattern

The pairing that most often causes confusion is asynchronous against batch. A huge file that belongs to a dataset scored in one run is still batch. A huge file that arrives on its own and joins a queue is asynchronous. A second trap: data that arrives continuously is not the same as a person waiting for an answer. Continuous arrival on its own does not call for real-time serving.

Why the paper version transfers

In our view, any no-code tool you use next will ask you these same questions, in different words. Which column holds the answer you want predicted? How should the data be split for testing? What do the two scores look like? How will predictions be requested? If you know that the column you pick is the label, that "build" means training, and that "predict" means inference, then the tool's screens become easier to read.

The exercise also builds a habit worth keeping. Ask whether the outcome was recorded before you ask anything else. Asking "is it labeled?" closes off or opens up supervised learning. Asking "how much data is there?" settles nothing on its own. Then compare two scores before trusting either. Then decide serving from who is waiting.

Key takeaways

The algorithm is the procedure, the model is its output. Training builds the model, and inference only uses it, many times over.
Supervised learning needs labeled data. Whether the correct answer was recorded matters more than how many rows you have.
Always hold back test data. A model is judged by comparing its training score with its score on examples it never saw.
High-then-low means overfitting, low-on-both means underfitting. A model that memorizes its examples can look perfect and still fail on new cases.
Choose a serving mode from who is waiting, whether a whole dataset is scored at once, how large each request is, and how bursty the traffic is.

This article settles the concepts behind training and testing a first model, worked through by hand. It does not walk through the screens, settings or charges of any particular no-code tool. Those vary by product, and checking them against the tool's own current documentation is the honest next step.

DEV Community