<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Yanng</title>
    <description>The latest articles on DEV Community by Yanng (@yanng981).</description>
    <link>https://dev.to/yanng981</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4142559%2F54a20e5d-1ced-410b-97c7-95dc67f32f2a.png</url>
      <title>DEV Community: Yanng</title>
      <link>https://dev.to/yanng981</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/yanng981"/>
    <language>en</language>
    <item>
      <title>How I keep faces the same when editing photos with Nano Banana and GPT Image</title>
      <dc:creator>Yanng</dc:creator>
      <pubDate>Tue, 03 Nov 2026 13:30:00 +0000</pubDate>
      <link>https://dev.to/yanng981/how-i-keep-faces-the-same-when-editing-photos-with-nano-banana-and-gpt-image-3of5</link>
      <guid>https://dev.to/yanng981/how-i-keep-faces-the-same-when-editing-photos-with-nano-banana-and-gpt-image-3of5</guid>
      <description>&lt;p&gt;The most common complaint about AI photo editing is not quality. It's "that's not me anymore". You ask for a different jacket and get a different person wearing it.&lt;/p&gt;

&lt;p&gt;I build &lt;a href="https://aiphotoedit.ai" rel="noopener noreferrer"&gt;AI Photo Edit&lt;/a&gt;, an AI photo editor where you upload a photo and type one sentence, like "dress him in a navy suit". It runs on two image models, Nano Banana 2.1 (Google) and GPT Image 2.5 (OpenAI). Both follow instructions well. Neither knows that for my users the face is the one thing that must not change.&lt;/p&gt;

&lt;p&gt;What fixed it was not a model or a face-swap step. It was one paragraph appended to every prompt, plus a check for edits that need more than one photo. Here is how that paragraph is built, the version that broke, and the edits that still don't work.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt; Put a short paragraph of rules after the user's instruction. Protect identity, not appearance, and give every rule a way out for when the user asks for exactly what it protects. Then test it on the edits it might block. A copyable template and an eight-edit test list are below.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Users write the change, the server writes the rules
&lt;/h2&gt;

&lt;p&gt;Users only type what they want changed. They never see the protective wording. The server adds it before the request goes to the model:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// The user's sentence first, our rules after it.&lt;/span&gt;
&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;editPrompt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;userPrompt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;userPrompt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;()}&lt;/span&gt;&lt;span class="s2"&gt;\n\n&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;RULES&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;RULES&lt;/code&gt; is a short paragraph. I'm not going to paste our production wording, because it took a lot of test runs to get right, but each of its sentences has one job:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Apply the change above, and only that change.&lt;/li&gt;
&lt;li&gt;Keep every person recognizable as themselves.&lt;/li&gt;
&lt;li&gt;Leave everything the instruction doesn't mention where it was.&lt;/li&gt;
&lt;li&gt;Say what happens when the user asks for a new art style.&lt;/li&gt;
&lt;li&gt;Keep the original light and color unless the user asks otherwise.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Here is a version you can copy. It is not our production text, and I haven't run it through my test set, so treat it as a starting point:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Edit request: {what the user typed}

Rules for this edit:
- Change only what the edit request asks for. People, objects, framing
  and background stay as they are.
- Every person stays recognizable: same face shape, eyes, nose, mouth,
  skin tone and age. Their expression changes only if the request asks.
- If the request asks for an art style (painting, cartoon, sketch),
  apply it to the whole image and keep each person recognizable in it.
- Keep the original lighting, colors and camera look unless the request
  changes them.
- Do not add text, watermarks or extra people.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you try it, test "make him laugh" and "turn it into a watercolor painting" first. Those are the two edits that broke my first version, as the next section shows. The full test list is near the end. I'd like to know in the comments what you get.&lt;/p&gt;

&lt;p&gt;The same function feeds every provider. Nano Banana takes the result as &lt;code&gt;prompt&lt;/code&gt; with the photos in &lt;code&gt;image_input&lt;/code&gt;, and GPT Image gets the same text with &lt;code&gt;input_urls&lt;/code&gt;. Only the body shape changes. The rules are always in English, whatever language the interface is in, so both models get the same instruction.&lt;/p&gt;

&lt;p&gt;The user's sentence goes first. The rules come after it and point back to it, so the request stays the main instruction and the rules only limit everything around it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Version one was too strict
&lt;/h2&gt;

&lt;p&gt;My first paragraph tried to protect everything: the face, the expression, the hair, the pose and the "photographic look". On normal edits it worked. Then I ran my test list, and two edits came back almost unchanged:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;"Make him laugh": the expression rule won, so he kept the same polite smile.&lt;/li&gt;
&lt;li&gt;"Turn the photo into a watercolor painting": the "keep the photographic look" rule won, so it stayed a photo.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The models weren't failing. They were obeying the rules I gave them over the user's request.&lt;/p&gt;

&lt;p&gt;The fix was to protect identity, not appearance. Asking for the same recognizable person still allows a laugh, a beard or a painted version of the same man. Asking for the same expression and the same look does not. Then each rule that could block an edit got an explicit way out:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;art style: if the user asks for a new style, the whole image changes style, not just part of it.&lt;/li&gt;
&lt;li&gt;lighting and color: kept, unless the user asks to change them.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Every protective rule needs to know what happens when the user asks for exactly the thing it protects. If it doesn't say, the model guesses, and it usually picks the rule.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does on one photo
&lt;/h2&gt;

&lt;p&gt;Every edit below starts from the same original. They are not chained, and they are not retouched. The portrait is a test image I generated, not a customer photo.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fi0ebrlrbdq35r0crniu6.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fi0ebrlrbdq35r0crniu6.webp" alt="Original: a man in a grey sweater at an outdoor cafe" width="720" height="894"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Instruction&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Dress him in a navy suit with a white shirt and a dark tie&lt;/td&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4d3pcvb4p94uzepxxij0.webp" alt="Same man in a navy suit" width="720" height="894"&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Put him in a snowy mountain village in winter, wearing a warm puffer jacket&lt;/td&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fi626u600ibbr2cv6j7pr.webp" alt="Same man in a snowy village" width="720" height="894"&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Give him a full beard and longer, swept-back hair&lt;/td&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0s437uh5dei4kbdew7wg.webp" alt="Same man with a beard" width="720" height="894"&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Turn the photo into a watercolor painting&lt;/td&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fd5yxi4cevpdh8cxmbjzu.webp" alt="Same scene as a watercolor" width="720" height="894"&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The watercolor shows the style rule working. The whole frame changes style, and the café, the cup, the people in the background and the man himself all stay where they were.&lt;/p&gt;

&lt;h2&gt;
  
  
  Tested on both models
&lt;/h2&gt;

&lt;p&gt;Before launch I ran the same instructions through both models with this exact prompt setup, at 2K, keeping the original aspect ratio:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Instruction&lt;/th&gt;
&lt;th&gt;Nano Banana 2.1&lt;/th&gt;
&lt;th&gt;GPT Image 2.5&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;New outfit, new background, add an accessory, remove tourists, overcast to sunset, change text on a sign&lt;/td&gt;
&lt;td&gt;Works; nothing else moves&lt;/td&gt;
&lt;td&gt;Works&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Watercolor painting&lt;/td&gt;
&lt;td&gt;Works (after the fix above)&lt;/td&gt;
&lt;td&gt;Barely changes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Make him laugh&lt;/td&gt;
&lt;td&gt;Works (after the fix above)&lt;/td&gt;
&lt;td&gt;Works&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Time per edit, upload included&lt;/td&gt;
&lt;td&gt;about 30–50 s&lt;/td&gt;
&lt;td&gt;about 50–70 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A few things I learned from that run and the ones after it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;GPT Image is weak at full style changes.&lt;/strong&gt; It edits content well but resists turning a photo into a painting. For a painting, use Nano Banana.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Some instructions barely change anything on either model.&lt;/strong&gt; "1970s film look" and "make it daytime" on an indoor restaurant photo came back nearly identical to the original. I took them out of the suggested prompts rather than promising something the models don't do.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Each model wins on different photos.&lt;/strong&gt; When I asked both to make AI-looking images more realistic, Nano Banana gave better skin on portraits (pores, fine lines, natural shine), and GPT Image was clearly better on products and interiors (glass, color, walls). So users can switch models per edit.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Provider timeouts happen.&lt;/strong&gt; I hit four "generate task timeout" errors in about fifteen Nano Banana runs on one provider. A failed edit is now retried once on a second provider, and if that fails too, the credits go back automatically.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Edits that need a second photo
&lt;/h2&gt;

&lt;p&gt;Some edits reference another image: "put the jacket from the second photo on him". If the user hasn't uploaded that second photo yet, the model doesn't ask. It invents a jacket.&lt;/p&gt;

&lt;p&gt;So the editor reads the instruction before sending it and counts how many photos it refers to:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;ORDER&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Record&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;second&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;2nd&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;other&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;another&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;third&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;3rd&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;fourth&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;4th&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;PHOTO&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;(?:photo|picture|image|pic|shot)s?&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;mention&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;RegExp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="se"&gt;\\&lt;/span&gt;&lt;span class="s2"&gt;b(&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nb"&gt;Object&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;keys&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ORDER&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;|&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)}&lt;/span&gt;&lt;span class="s2"&gt;)&lt;/span&gt;&lt;span class="se"&gt;\\&lt;/span&gt;&lt;span class="s2"&gt;s+&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;PHOTO&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="se"&gt;\\&lt;/span&gt;&lt;span class="s2"&gt;b`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;gi&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;photosInPrompt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;count&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;match&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;matchAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;mention&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;count&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;ORDER&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;match&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;toLowerCase&lt;/span&gt;&lt;span class="p"&gt;()]&lt;/span&gt; &lt;span class="o"&gt;??&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;count&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The real version also catches "both photos" and "these three pictures". If the instruction needs two photos and only one is uploaded, the button waits and asks for the other one, and nothing is charged.&lt;/p&gt;

&lt;p&gt;With both photos in place, the same paragraph keeps the person intact while the jacket comes from the reference:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Person&lt;/th&gt;
&lt;th&gt;Jacket&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzewwtmk919m9qgk1j43i.webp" alt="Man in a white t-shirt" width="600" height="745"&gt;&lt;/td&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkcd3f1jm3q2qtrf7exjc.webp" alt="Suede jacket" width="320" height="320"&gt;&lt;/td&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fu2cqemp9hxpmnmkes116.webp" alt="Same man wearing the suede jacket" width="600" height="745"&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The same flow, with up to four photos, is what the &lt;a href="https://aiphotoedit.ai/ai-image-combiner" rel="noopener noreferrer"&gt;AI image combiner&lt;/a&gt; page does.&lt;/p&gt;

&lt;h2&gt;
  
  
  A test list for your own prompt
&lt;/h2&gt;

&lt;p&gt;These are the eight kinds of edits in my test list. Use one portrait for every edit that involves a person, so a changed face is easy to spot.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Kind&lt;/th&gt;
&lt;th&gt;Example instruction&lt;/th&gt;
&lt;th&gt;What a failure looks like&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Outfit&lt;/td&gt;
&lt;td&gt;Dress him in a navy suit with a white shirt and a dark tie&lt;/td&gt;
&lt;td&gt;The face or body changes along with the clothes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Background&lt;/td&gt;
&lt;td&gt;Put him in a snowy mountain village in winter&lt;/td&gt;
&lt;td&gt;The person is redrawn instead of placed in the new scene&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Add something&lt;/td&gt;
&lt;td&gt;Give the dog a red scarf and a ball&lt;/td&gt;
&lt;td&gt;The dog's breed, color or pose changes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Remove something&lt;/td&gt;
&lt;td&gt;Remove all the people in the background and keep the man in the green jacket&lt;/td&gt;
&lt;td&gt;The man who stays is changed too, or smudges are left behind&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Style&lt;/td&gt;
&lt;td&gt;Turn the photo into a watercolor painting&lt;/td&gt;
&lt;td&gt;It stays a photo, or only part of the frame changes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Light&lt;/td&gt;
&lt;td&gt;Replace the grey sky with a dramatic orange sunset&lt;/td&gt;
&lt;td&gt;The sky changes but the light on the scene doesn't&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Text&lt;/td&gt;
&lt;td&gt;Remove all the writing from the chalkboard sign&lt;/td&gt;
&lt;td&gt;Other parts of the photo get cleaned up too&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Expression&lt;/td&gt;
&lt;td&gt;Make him laugh&lt;/td&gt;
&lt;td&gt;The same smile as before (rules too strict) or a different person (rules too loose)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The style and expression rows matter most: they show whether your rules are too strict or too loose. The other six check that the rules keep the rest of the photo in place.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Put the user's instruction first and your rules after it, pointing back to it.&lt;/li&gt;
&lt;li&gt;Protect identity (the same recognizable person), not appearance. Expression, hair and style are things people want to change.&lt;/li&gt;
&lt;li&gt;Give every protective rule an exit for when the user asks for exactly what it protects.&lt;/li&gt;
&lt;li&gt;Test the edits your rules might block (a laugh, a painting), not only the ones they're meant to protect.&lt;/li&gt;
&lt;li&gt;Check whether an instruction refers to photos that aren't there before you call the model.&lt;/li&gt;
&lt;li&gt;Drop the prompts a model can't do from your suggestions, instead of letting users find out.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you're building something similar, I'd like to hear how you handle identity: prompt only, like this, or a face-similarity check after generation? And if you ran the template above, which edits did it break? I haven't added a similarity check yet, because in about twenty test edits neither model quietly returned the photo unchanged. That may change as real traffic grows.&lt;/p&gt;

</description>
      <category>showdev</category>
      <category>ai</category>
      <category>promptengineering</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Open-source Jev alternatives: System One models you can self-host</title>
      <dc:creator>Yanng</dc:creator>
      <pubDate>Fri, 25 Sep 2026 12:55:18 +0000</pubDate>
      <link>https://dev.to/yanng981/open-source-jev-alternatives-system-one-models-you-can-self-host-i5e</link>
      <guid>https://dev.to/yanng981/open-source-jev-alternatives-system-one-models-you-can-self-host-i5e</guid>
      <description>&lt;p&gt;TypeSafe Jev is the hosted model that introduced &lt;strong&gt;System One models&lt;/strong&gt; on September 15, 2026: you send a state and typed questions, and it returns answers with probabilities instead of generated text. It is generally available and billed per token: $0.042 per million input tokens, with output tokens free.&lt;/p&gt;

&lt;p&gt;Within days, open-source Jev alternatives appeared that do the same job on your own hardware. This post lists the ones worth testing, which of them accept Jev API requests unchanged, and what it takes to self-host one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why look for a Jev alternative?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Data stays in your infrastructure.&lt;/strong&gt; Tickets, emails and user messages never leave your servers.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost at volume.&lt;/strong&gt; A self-hosted model costs the same whether you make ten decisions or ten million.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No vendor dependency.&lt;/strong&gt; Open weights keep working if a hosted API changes its pricing, limits or terms.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fine-tuning.&lt;/strong&gt; You can train an open model on your own labelled decisions. Jev's weights are not public.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Offline and edge use.&lt;/strong&gt; Some models run on a laptop CPU or an Apple Silicon Mac.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Best Jev alternatives to try first
&lt;/h2&gt;

&lt;p&gt;If you already call Jev, start with a model that serves the same &lt;code&gt;POST /v1/systemone&lt;/code&gt; endpoint, so existing client code only needs a new base URL:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Size&lt;/th&gt;
&lt;th&gt;License&lt;/th&gt;
&lt;th&gt;Serves the Jev API&lt;/th&gt;
&lt;th&gt;Good first choice when&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://laya-ai.com/models" rel="noopener noreferrer"&gt;Laya&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;322M / 421M&lt;/td&gt;
&lt;td&gt;Apache 2.0&lt;/td&gt;
&lt;td&gt;Yes, through laya-serve&lt;/td&gt;
&lt;td&gt;You want a small model that runs on CPU, or 100+ languages&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://laya-ai.com/system-one-models/kev" rel="noopener noreferrer"&gt;Kev&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;0.8B to 27B&lt;/td&gt;
&lt;td&gt;Apache 2.0&lt;/td&gt;
&lt;td&gt;Yes, the TypeSafe SDK works unchanged&lt;/td&gt;
&lt;td&gt;You have a GPU and want the closest drop-in&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://github.com/Mapika/decider" rel="noopener noreferrer"&gt;Decider&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;2B / 4B / 35B MoE&lt;/td&gt;
&lt;td&gt;Apache 2.0&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;You serve with vLLM and want many-option choices&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://laya-ai.com/system-one-models/von" rel="noopener noreferrer"&gt;Von&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;395M&lt;/td&gt;
&lt;td&gt;Apache 2.0&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;You need low latency on modest hardware&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://github.com/iapp-technology/openthai-systemone" rel="noopener noreferrer"&gt;OpenThai-SystemOne&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;0.8B&lt;/td&gt;
&lt;td&gt;Apache 2.0&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Your input is Thai&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Other open options use their own APIs: Bespoke Nimble (9B), GLiNER2.5-Decide (340M), Together AI's &lt;a href="https://laya-ai.com/system-one-models/tev1" rel="noopener noreferrer"&gt;Tev1-4B&lt;/a&gt;, and the training-free AnyJev and SemIf. See the &lt;a href="https://laya-ai.com/system-one-models" rel="noopener noreferrer"&gt;full System One model comparison&lt;/a&gt; for all twelve.&lt;/p&gt;

&lt;h2&gt;
  
  
  Jev-compatible API servers
&lt;/h2&gt;

&lt;p&gt;These projects put a Jev-style HTTP API in front of an open model, so you can switch models later without touching client code:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Server&lt;/th&gt;
&lt;th&gt;Language&lt;/th&gt;
&lt;th&gt;Notes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;laya-serve&lt;/td&gt;
&lt;td&gt;Python&lt;/td&gt;
&lt;td&gt;Ships with the upstream Laya package. See &lt;a href="https://laya-ai.com/guides/self-host-laya" rel="noopener noreferrer"&gt;Self-host Laya&lt;/a&gt;.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://github.com/0xBakeer/arbiter" rel="noopener noreferrer"&gt;Arbiter&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Python&lt;/td&gt;
&lt;td&gt;Routing, batching, metrics and a built-in playground&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://github.com/alvarobartt/sys1" rel="noopener noreferrer"&gt;sys1&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Rust&lt;/td&gt;
&lt;td&gt;candle-based, token batching, CPU, CUDA and Metal&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://github.com/ollaya-dev/ollaya" rel="noopener noreferrer"&gt;ollaya&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Rust&lt;/td&gt;
&lt;td&gt;Ollama-style CLI and daemon; pulls models by name&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://github.com/1Panel-dev/laya-server" rel="noopener noreferrer"&gt;laya-server&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;TypeScript&lt;/td&gt;
&lt;td&gt;Docker image with a web console and API keys&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For the two Rust servers side by side, see &lt;a href="https://laya-ai.com/runtimes/rust" rel="noopener noreferrer"&gt;Run Laya in Rust&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to switch from Jev to a self-hosted model
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Pick a model&lt;/strong&gt; from the table above and start its server locally.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Point your client at it.&lt;/strong&gt; With the TypeSafe SDK, change the base URL; ollaya, for example, documents &lt;code&gt;TYPESAFE_BASE_URL=http://localhost:11435&lt;/code&gt;. With plain HTTP, the request body stays the same:
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-X&lt;/span&gt; POST http://localhost:8080/v1/systemone &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"state":{"message":"I was charged twice"},"questions":{"refund":{"type":"noul","instructions":"Does the customer ask for a refund?"}}}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Compare on your own data.&lt;/strong&gt; Send the same labelled requests to Jev and to the new model, and compare accuracy and latency before moving production traffic.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Recalibrate thresholds.&lt;/strong&gt; Probabilities from different models are not interchangeable. If you automate a decision above a confidence cut-off, set that cut-off again on held-out data.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Jev vs open alternatives: what you give up
&lt;/h2&gt;

&lt;p&gt;Hosted Jev still leads out of the box on several published comparisons, especially on tasks with many labels. On Banking77, with 77 intents, upstream Laya reports 0.425 at default settings against Jev's 0.870. Open models close much of the gap after fine-tuning on your own data, and they win on cost, privacy and latency when you run them close to your application. See &lt;a href="https://laya-ai.com/laya-vs-jev" rel="noopener noreferrer"&gt;Laya vs Jev&lt;/a&gt; for the details and caveats.&lt;/p&gt;

&lt;p&gt;Benchmarks in this space are young and mostly self-reported on different datasets, so treat any single number as a starting point, not a verdict.&lt;/p&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Is there an open-source version of Jev?
&lt;/h3&gt;

&lt;p&gt;No. Jev's weights are not public. The Jev alternatives above are independent open models, mostly Apache 2.0, that answer the same kind of typed questions. Several of them also accept the same &lt;code&gt;POST /v1/systemone&lt;/code&gt; requests.&lt;/p&gt;

&lt;h3&gt;
  
  
  Which Jev alternative runs on a CPU?
&lt;/h3&gt;

&lt;p&gt;The encoder models are the lightest: Laya (322M and 421M parameters), Von (395M) and GLiNER2.5-Decide (340M). See &lt;a href="https://laya-ai.com/guides/laya-cpu-performance" rel="noopener noreferrer"&gt;Laya on CPU&lt;/a&gt; for measured latency.&lt;/p&gt;

&lt;h3&gt;
  
  
  Can I keep using the TypeSafe SDK?
&lt;/h3&gt;

&lt;p&gt;Yes, with a server that speaks the Jev API. Kev and ollaya say the SDK works unchanged, and laya-serve says an existing Jev client only needs a new base URL.&lt;/p&gt;

&lt;h2&gt;
  
  
  Related
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://laya-ai.com/system-one-models" rel="noopener noreferrer"&gt;System One models compared&lt;/a&gt;: all twelve models side by side&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://laya-ai.com/laya-vs-jev" rel="noopener noreferrer"&gt;Laya vs Jev&lt;/a&gt;: open weights, speed, accuracy and tradeoffs&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://laya-ai.com/guides/self-host-laya" rel="noopener noreferrer"&gt;Self-host Laya&lt;/a&gt;: run a Jev-compatible API yourself&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://laya-ai.com/guides/laya-cpu-performance" rel="noopener noreferrer"&gt;Laya on CPU&lt;/a&gt;: latency and sizing without a GPU&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Last verified: September 25, 2026.&lt;/em&gt;&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I maintain &lt;a href="https://laya-ai.com" rel="noopener noreferrer"&gt;laya-ai.com&lt;/a&gt;, an independent Laya resource site; this article first appeared there as &lt;a href="https://laya-ai.com/jev-alternatives" rel="noopener noreferrer"&gt;Jev alternatives&lt;/a&gt;. The site also keeps an &lt;a href="https://github.com/yanng981/awesome-system-one" rel="noopener noreferrer"&gt;auto-updated list of System One models on GitHub&lt;/a&gt;. It was drafted with AI assistance, and every fact was checked against the project pages linked above.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>ai</category>
      <category>opensource</category>
      <category>python</category>
    </item>
    <item>
      <title>How fast is Laya on a CPU? Reading the 4-core server benchmarks</title>
      <dc:creator>Yanng</dc:creator>
      <pubDate>Fri, 25 Sep 2026 08:56:57 +0000</pubDate>
      <link>https://dev.to/yanng981/how-fast-is-laya-on-a-cpu-reading-the-4-core-server-benchmarks-2mgb</link>
      <guid>https://dev.to/yanng981/how-fast-is-laya-on-a-cpu-reading-the-4-core-server-benchmarks-2mgb</guid>
      <description>&lt;p&gt;Can you run Laya without a GPU? Yes, and the upstream project now publishes enough CPU measurements to say how fast, what it costs per question, and which settings matter. This guide reads those numbers so you can size a CPU deployment before you buy hardware.&lt;/p&gt;

&lt;p&gt;All figures below come from the upstream &lt;a href="https://github.com/NandhaKishorM/laya/blob/main/BENCHMARKS.md" rel="noopener noreferrer"&gt;Laya BENCHMARKS.md&lt;/a&gt;. Each is tied to the hardware and setup it was measured on.&lt;/p&gt;

&lt;h2&gt;
  
  
  The short answer
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;On a 4-core server CPU, &lt;strong&gt;&lt;code&gt;laya-multilingual&lt;/code&gt; answers one question in about 193 ms&lt;/strong&gt;; the English and typed-decisions checkpoints take about &lt;strong&gt;580 ms&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;On CPU, cost grows almost linearly with the number of questions. &lt;strong&gt;Batching questions into one call saves little&lt;/strong&gt;, unlike on a GPU.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Thread settings can matter more than the model.&lt;/strong&gt; On one laptop, fixing torch's thread defaults made calls about 12x faster.&lt;/li&gt;
&lt;li&gt;Cold loads take seconds, and several loaded checkpoints can use around 9 GiB of RAM, so &lt;strong&gt;preload what you serve&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The server CPU numbers
&lt;/h2&gt;

&lt;p&gt;Measured in-process on an AWS &lt;code&gt;m7a.xlarge&lt;/code&gt; (AMD EPYC 9R14, 4 physical cores, no SMT, 16 GiB RAM), Laya v0.3.20, fp32, &lt;code&gt;OMP_NUM_THREADS=4&lt;/code&gt;. Each call alternates a 3-option &lt;code&gt;choice&lt;/code&gt; and a &lt;code&gt;noul&lt;/code&gt; question. Values are p50; p95 was within 2% of p50 on every row.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;checkpoint&lt;/th&gt;
&lt;th&gt;1 question&lt;/th&gt;
&lt;th&gt;5&lt;/th&gt;
&lt;th&gt;10&lt;/th&gt;
&lt;th&gt;50&lt;/th&gt;
&lt;th&gt;cold load&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;laya&lt;/code&gt; (English)&lt;/td&gt;
&lt;td&gt;580 ms&lt;/td&gt;
&lt;td&gt;3,072 ms&lt;/td&gt;
&lt;td&gt;6,244 ms&lt;/td&gt;
&lt;td&gt;35,969 ms&lt;/td&gt;
&lt;td&gt;4.4 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;laya-multilingual&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;193 ms&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;912 ms&lt;/td&gt;
&lt;td&gt;1,842 ms&lt;/td&gt;
&lt;td&gt;11,157 ms&lt;/td&gt;
&lt;td&gt;2.5 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;laya-typed-decisions&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;584 ms&lt;/td&gt;
&lt;td&gt;2,819 ms&lt;/td&gt;
&lt;td&gt;6,031 ms&lt;/td&gt;
&lt;td&gt;35,653 ms&lt;/td&gt;
&lt;td&gt;0.5 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Two things stand out.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The multilingual checkpoint is about 3x faster on CPU.&lt;/strong&gt; It is built on mmBERT-base (322M parameters) rather than ModernBERT-large (421M). The upstream write-up does not break down where the rest of the gap comes from, so treat 3x as a measured result on this machine, not a rule.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The p95 is almost the same as the p50.&lt;/strong&gt; On an otherwise idle server, CPU latency was very predictable. In our reading, that makes capacity planning simpler than on a shared GPU.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why batching does not help on CPU
&lt;/h2&gt;

&lt;p&gt;Up to 10 questions per call, each extra question costs about &lt;strong&gt;600 ms&lt;/strong&gt; on the English and typed-decisions checkpoints and about &lt;strong&gt;185 ms&lt;/strong&gt; on multilingual. At 50 questions the cost per question rises by another 15 to 20%. In other words, ten questions in one call take roughly as long as ten separate calls.&lt;/p&gt;

&lt;p&gt;GPUs behave differently:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;hardware&lt;/th&gt;
&lt;th&gt;1 question&lt;/th&gt;
&lt;th&gt;50 questions&lt;/th&gt;
&lt;th&gt;cost per extra question&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Tesla T4, &lt;code&gt;laya&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;39.5 ms&lt;/td&gt;
&lt;td&gt;771.3 ms&lt;/td&gt;
&lt;td&gt;about 15 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;NVIDIA GB10, &lt;code&gt;laya-typed-decisions&lt;/code&gt; (over HTTP)&lt;/td&gt;
&lt;td&gt;100.2 ms&lt;/td&gt;
&lt;td&gt;443.1 ms&lt;/td&gt;
&lt;td&gt;about 7 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;EPYC 4-core CPU, &lt;code&gt;laya&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;580 ms&lt;/td&gt;
&lt;td&gt;35,969 ms&lt;/td&gt;
&lt;td&gt;about 600 ms up to 10 questions, more beyond&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;On the GB10, roughly 93 ms of every call is fixed overhead, so packing questions into one call is where the speedup is. On CPU there is almost no fixed overhead to amortize, so the design choice is simpler: ask only the questions you need.&lt;/p&gt;

&lt;h2&gt;
  
  
  Thread settings: the easiest 12x
&lt;/h2&gt;

&lt;p&gt;The upstream project also reports a laptop measurement (Ryzen 9 6900HX, WSL2) that is worth copying before anything else. With torch's default thread settings on a busy host (10 intra-op and 5 inter-op threads on 10 vCPUs), a three-question call over HTTP took &lt;strong&gt;9,396 ms&lt;/strong&gt; at p50. Setting two lines brought it to &lt;strong&gt;783 ms&lt;/strong&gt;, about 12x faster with no code change:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;torch&lt;/span&gt;

&lt;span class="n"&gt;torch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set_num_threads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;          &lt;span class="c1"&gt;# about the number of physical cores
&lt;/span&gt;&lt;span class="n"&gt;torch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set_num_interop_threads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  &lt;span class="c1"&gt;# one forward pass per call: nothing to overlap
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On the same laptop, one question in-process took 910 ms with 1 thread, 374 ms with 4, &lt;strong&gt;329 ms with 8&lt;/strong&gt;, and got worse again at 388 ms when every vCPU was used. The upstream advice is to use the physical core count plus a little, not one thread per vCPU, because SMT siblings contend. If you run the self-hosted server, &lt;code&gt;LAYA_THREADS&lt;/code&gt; caps the same setting.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cold starts and memory
&lt;/h2&gt;

&lt;p&gt;A cold load took 4.4 s for the English checkpoint and 2.5 s for multilingual on the EPYC machine. These figures depend on the OS file cache, so treat them as approximate. Two practical consequences:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Preload&lt;/strong&gt; the checkpoints you serve (&lt;code&gt;Router(preload=True)&lt;/code&gt;) instead of loading on the first request.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Budget memory.&lt;/strong&gt; The benchmark script peaked at 9.3 GiB with up to five checkpoints loaded at once. Serving fewer checkpoints should need less, but the upstream run does not report a per-checkpoint figure, so measure on your own machine.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  When is a CPU enough?
&lt;/h2&gt;

&lt;p&gt;This table is our interpretation of the numbers above, not a benchmark result. It assumes one question per call:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;your situation&lt;/th&gt;
&lt;th&gt;CPU is fine?&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Background jobs, queues, nightly batch classification&lt;/td&gt;
&lt;td&gt;Yes. Latency rarely matters; cost does.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Interactive routing or triage in non-English text&lt;/td&gt;
&lt;td&gt;Usually. &lt;code&gt;laya-multilingual&lt;/code&gt; at about 0.2 s is fast enough for many UIs.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Interactive English decisions with several questions per request&lt;/td&gt;
&lt;td&gt;Borderline. Five questions on the English checkpoint take about 3 s.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;High-throughput, low-latency serving&lt;/td&gt;
&lt;td&gt;Use a GPU. A T4 answers one question in 32.8 to 39.5 ms.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These are single-process, in-process numbers. Real deployments add HTTP, queuing, and concurrency effects, so benchmark your own hardware with your own questions before committing. The upstream script is &lt;code&gt;research/scripts/bench_latency.py&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://github.com/NandhaKishorM/laya/blob/main/BENCHMARKS.md" rel="noopener noreferrer"&gt;Laya BENCHMARKS.md&lt;/a&gt;: server CPU, laptop CPU, T4, and GB10 sections&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/NandhaKishorM/laya/blob/main/research/results/latency_cpu_m7a_xlarge_20260924.json" rel="noopener noreferrer"&gt;Raw EPYC results&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://laya-ai.com/benchmarks" rel="noopener noreferrer"&gt;Laya benchmarks tracker&lt;/a&gt;: upstream and independent results with their test conditions&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Last verified: September 25, 2026.&lt;/em&gt;&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I maintain &lt;a href="https://laya-ai.com" rel="noopener noreferrer"&gt;laya-ai.com&lt;/a&gt;, an independent Laya resource site; this article first appeared there. It was drafted with AI assistance, and every number was checked against the upstream benchmark files linked above.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>python</category>
      <category>performance</category>
      <category>ai</category>
    </item>
  </channel>
</rss>
