Training a small classifier for your own inbox (refund requests, shipping questions, spam) starts with a few hundred labelled messages, and most people have three. The usual workaround is to have a language model write the rest. That works, but only if the model is told exactly which labels exist, is shown what your real messages look like, and is asked for a small batch at a time. The steps below build a 100-row dataset of online-shop customer messages in the Datasets screen of the osFoundry desktop app. The rows in the screenshots were written by Qwen2.5-3B-Instruct, a 3-billion-parameter model running on the same Mac, in 4-bit.
Describe the rows before asking for them
Open Models, then Datasets, then the Generate tab. The left column lists templates; Text Classification gives a schema with two fields, Text and Label.
Change the Label field's type to Enum and add the labels you want: refund, shipping and spam here. Under them, set Samples to 20, write a Domain such as "Online shop customer messages", and set the Language.
These fields are not decoration. The app turns them into the prompt it sends to the model, and an Enum field becomes a line that reads [allowed values: refund, shipping, spam] next to the field's name. Without it, the model invents labels of its own.
Paste three real messages
Under Few-Shot Examples, click Paste JSON and paste a handful of real messages with their labels, then Import. Three were enough here:
[
{"text": "wheres my order?? its been 2 weeks", "label": "shipping"},
{"text": "mug arrived in pieces. want my money back", "label": "refund"},
{"text": "CONGRATS you won a $500 gift card, claim now", "label": "spam"}
]
The difference is easy to see. Without examples, the same model wrote five tidy, polite paragraphs ("Hey there, I've had this product for a year already…") and got two of the five labels wrong. With them, it wrote short, lower-case, slightly annoyed messages like the ones a shop actually gets. The examples go into the prompt as JSON, followed by an instruction to generate more rows "following the same style and patterns", so typos and missing punctuation in your examples are worth keeping.
Preview five rows and read every label
Pick a model under Model and click Preview (5 rows). The five rows appear in a table on the right.
Read the labels, not just the text. Row 3, "urgent response needed! my service is late", came back as spam; it is a shipping complaint. Click the label cell, type the right value and confirm. Each row also has buttons to regenerate or delete it. Five rows is the cheapest point to notice that the labels drift, and if most of them are wrong, the fix is better examples, not more rows.
Generate in batches of 20
Click Generate 15 more to fill up the first batch of 20, then Generate More (append) until the table reaches about 100 rows. Each click sends the same prompt again for another 20 rows and adds them under the ones you have.
It is tempting to set Samples to 100 and ask once. With this 3B model that went badly: asked for 95 rows in one go, it wrote 302 complete rows before the reply reached its 8,192-token limit, only 24 of them different, cycling through the same three sentences, and it never closed the JSON array. Small models drift into repeating their own output over a long reply. Twenty rows is short enough that each batch finishes cleanly.
Batches still repeat themselves a little. The screenshot shows "tracking number doesnt work. i want my money back" seven times in one batch. Delete the copies before saving, or remove them afterwards in whatever tool you train with.
Count the labels, then save
Type a name in the box above the table and click Save as Data Channel. The dataset appears under My Datasets with its row count, ready for training.
Before training on it, count the labels. This run ended with 75 refund, 23 shipping and 3 spam. A classifier trained on that will learn to answer "refund" and will rarely see spam at all. The fix is cheap at this stage: add a few more spam examples to the few-shot list and generate another batch or two, or delete some of the refund rows. In a Japanese run of the same setup, 15 of 99 rows came back with a label that was never in the list, so it is worth scanning the label column for anything unexpected as well.
The model writes the rows; checking them is still your job. Once the dataset looks balanced, the same Models screen can train on it, as described on the training custom models page. Saved datasets also show up next to uploaded CSV and JSONL files, covered on the your datasets page. The app is free from the osFoundry download page.
Top comments (0)