Gendered Nouns in French AI-Generated Text

#ai #debugging #llm #machinelearning

You asked for French and got La nouvelle système est très puissante, or Le photo que j’ai pris, or a paragraph that calls the same person il in one sentence and elle in the next. All three are the same failure seen at three different points in the sentence.

The output you are looking at

French gender is not a property of a word you look up once. It is a constraint that propagates. Take a single noun and watch how many other words it controls:

la  nouvelle  version  que  j'ai  installée  est  rapide ;  elle  marche
ART ADJ       NOUN(f)  REL  —     PART(f)   —    ADJ(inv)  PRON(f)

le  nouveau   serveur  que  j'ai  installé   est  rapide ;  il    marche
ART ADJ       NOUN(m)  REL  —     PART(m)    —    ADJ(inv)  PRON(m)

One decision — the gender of version or serveur — sets the article, the pre-nominal adjective, the past participle agreeing with a preceding direct object, and the personal pronoun that picks the noun up in the next clause. Five surface forms, one underlying fact. When you see a wrong ending on the participle, the participle is not the bug. It is the last link in a chain that broke earlier.

This is why the error looks erratic. A model generating left to right commits to la before it has produced the noun, and once la is in the context the cheapest continuation is a feminine noun — so it will often quietly swap the noun rather than contradict the article. Conversely, when the noun is correct and the adjective is twenty tokens downstream, the constraint has to survive everything in between, and it sometimes does not.

Gender is a chain, not a label

Four agreement targets are worth knowing individually, because they fail for different reasons.

Determiners and pre-nominal adjectives — le / la, un / une, ce / cette, nouveau / nouvelle. These are generated before the noun, so they are the model’s prediction of the noun’s gender, not a reaction to it.
Post-nominal adjectives — intéressant(e), ouvert(e). These follow the noun and are usually right when adjacent, and are where distance starts to matter.
Past participles. With être the participle agrees with the subject: elle est partie. With avoir it agrees only with a direct object that comes before it: les photos que j’ai prises but j’ai pris les photos. That rule depends on word order rather than on the words themselves, and it is the single most commonly wrong ending in generated French.
Anaphoric pronouns. il / elle for things as well as people, because French has no it. English it gives the model no gender to copy, so a translated paragraph has to reconstruct the gender of every inanimate referent from the noun it replaced — often several sentences back.

Where the model is forced to guess

Separate the cases where the model made an error from the cases where the input did not contain the answer. English The teacher was tired and left early has exactly one French translation per gender of the teacher, and the English sentence does not say:

Le professeur était fatigué et il est parti tôt.
La professeure était fatiguée et elle est partie tôt.

Four forms change. No amount of prompt engineering recovers information that is not in the source; the only fixes are to supply the gender, to ask for both, or to rewrite around it (L’enseignante ou l’enseignant is clumsy, Le personnel enseignant is not). Google’s Translate team shipped gender-specific alternatives for exactly this class of sentence in December 2018 — see Google’s own announcement — which is a good indication that the ambiguity is treated as unresolvable rather than as a bug to be trained away.

When the source is silent and you do not intervene, output skews masculine. That is not a moral property of the model; the masculine is the morphologically unmarked form in French and is correspondingly more frequent in any corpus, so it is the higher-probability continuation. The same effect is documented across languages and systems in Stanovsky, Smith and Zettlemoyer’s ACL 2019 evaluation of gender bias in machine translation.

Nouns whose gender the model gets wrong outright

Distinct problem: the noun has one fixed gender and the model picked the other. This clusters in predictable places.

Greek-derived nouns in -ème and -me are masculine against the surface pattern that -e means feminine: le problème, le système, le thème, le programme, le poème. La système is the single most recognisable tell of generated French.
Nouns in -age and -isme are masculine (le message, le stockage) with a short list of famous exceptions (la page, la plage, l’image).
Recent loanwords have contested gender. une app and un app both occur; le wifi is usual in France. A model reproduces the mixture, so it will be inconsistent within one document rather than consistently wrong.
Homographs with two genders. le livre (book) versus la livre (pound), le poste (job, set) versus la poste (post office), le tour versus la tour. Here the gender carries the meaning, and getting it wrong changes what the sentence says.
Epicene nouns — journaliste, architecte, élève — have one form and take their gender from the determiner alone. There is no ending to check, so an inconsistency here is invisible until the pronoun arrives.

What actually fixes it

In order of how much they are worth. First, supply the gender in the input rather than hoping. If you are translating records, the gender of a person is usually a field somewhere; put it in the prompt as data (“subject: female”) instead of expecting the model to infer it from a name. Second, keep the referent close: translating sentence-by-sentence destroys the antecedent that elle depends on, so translate at paragraph granularity even when your pipeline prefers sentences.

Third, check rather than re-prompt. Agreement is one of the few generation errors a deterministic tool catches reliably: run the output through a French grammar checker such as LanguageTool, which flags determiner–noun and noun–adjective disagreement, and re-generate only the sentences it flags. Asking the model “are you sure about the gender?” is much weaker, because the model will happily produce a confident justification for either answer.

Fourth, constrain the construction. Much of the participle-agreement risk disappears if you ask for the active voice and for direct objects after the verb. And if you are generating UI strings rather than prose, avoid interpolating a noun into a sentence frame at all — a frame like Votre {noun} a été supprimé cannot be right for both fichier and image, and the fix is separate strings, not a cleverer template. The same structural point applies to German, where the article carries case as well as gender, and to job titles specifically, where the masculine default is most visible.