<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Alex Babintsev</title>
    <description>The latest articles on DEV Community by Alex Babintsev (@alexbabintsev).</description>
    <link>https://dev.to/alexbabintsev</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4212848%2F8d93d606-98ab-4f52-9cbc-f0b50e2bd797.jpeg</url>
      <title>DEV Community: Alex Babintsev</title>
      <link>https://dev.to/alexbabintsev</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/alexbabintsev"/>
    <language>en</language>
    <item>
      <title>Why is my XML valid but still rejected?</title>
      <dc:creator>Alex Babintsev</dc:creator>
      <pubDate>Fri, 30 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/alexbabintsev/why-is-my-xml-valid-but-still-rejected-55mi</link>
      <guid>https://dev.to/alexbabintsev/why-is-my-xml-valid-but-still-rejected-55mi</guid>
      <description>&lt;p&gt;"But it's valid XML" is the most common reply to a rejected file, and it is usually true. It just answers a different question from the one the receiver asked. "Valid" covers at least four separate checks, each stricter than the one before, and a file can pass one and fail the next.&lt;/p&gt;

&lt;p&gt;To show where each one draws its line, I took a small UBL invoice that passes the XML Schema and all of the EN 16931 business rules, then broke it in different ways. Every message below is real output from libxml2 and the official rule sets, not a paraphrase.&lt;/p&gt;

&lt;h2&gt;
  
  
  Level 1: is it XML at all
&lt;/h2&gt;

&lt;p&gt;The first check is whether the file is well-formed: tags closed and properly nested, one root element, attribute values in quotes, special characters escaped. No schema is involved. A browser that shows the file as a tree has only done this much.&lt;/p&gt;

&lt;p&gt;Change the item name to &lt;code&gt;Toner &amp;amp; drum&lt;/code&gt; and the parser stops:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;line 53 xmlParseEntityRef: no name

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;An ampersand starts an entity reference in XML, so a bare one in text has to be written &lt;code&gt;&amp;amp;amp;&lt;/code&gt;. This is almost always a sign that the XML was built by joining strings rather than with an XML library, which would have escaped it. Other errors of this kind are a &lt;a href="https://xmlvalid.com/errors/opening-and-ending-tag-mismatch" rel="noopener noreferrer"&gt;tag closed in the wrong order&lt;/a&gt;, &lt;a href="https://xmlvalid.com/errors/extra-content-at-the-end-of-the-document" rel="noopener noreferrer"&gt;text after the root element&lt;/a&gt; and &lt;a href="https://xmlvalid.com/errors/premature-end-of-data" rel="noopener noreferrer"&gt;a file cut off halfway&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;A well-formedness error stops everything. The parser cannot tell where the damage ends, so there is no schema check and no business rule check until it is fixed. If a validator reports exactly one error and it is one of these, that does not mean the rest of the file is fine; nothing else has been looked at yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  Level 2: does it follow its schema
&lt;/h2&gt;

&lt;p&gt;A schema describes what the document may contain: which elements, in what order, how many times, and of what type. For an invoice that is an XSD, but DTD and RelaxNG do the same job for other documents. Here the parser checks the file against UBL 2.1's &lt;code&gt;UBL-Invoice-2.1.xsd&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Write the issue date the way people write dates in Germany:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;line 7 Element 'cbc:IssueDate': '30.10.2026' is not a valid value of the atomic type 'xs:date'.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;xs:date&lt;/code&gt; is &lt;code&gt;2026-10-30&lt;/code&gt; and nothing else, whatever the locale. Or keep every value right and swap two elements, putting the currency code before the invoice type code:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;line 9 Element 'cbc:InvoiceTypeCode': This element is not expected. Expected is one of ( cbc:TaxCurrencyCode, cbc:PricingCurrencyCode, cbc:PaymentCurrencyCode, … ).

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;XSD sequences are ordered, so the same elements in another order are a different document. The message names the element it found and lists what it would have accepted at that point; the element you meant is somewhere further down the schema. Both messages have their own pages: &lt;a href="https://xmlvalid.com/errors/not-a-valid-value-of-the-atomic-type" rel="noopener noreferrer"&gt;not a valid value of the atomic type&lt;/a&gt; and &lt;a href="https://xmlvalid.com/errors/this-element-is-not-expected" rel="noopener noreferrer"&gt;this element is not expected&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;This is where most people stop, and for a lot of XML it is the right place to stop: configuration files, data feeds, a DTD-based publishing format. The &lt;a href="https://xmlvalid.com/xsd-validator" rel="noopener noreferrer"&gt;XSD&lt;/a&gt;, &lt;a href="https://xmlvalid.com/dtd-validator" rel="noopener noreferrer"&gt;DTD&lt;/a&gt; and &lt;a href="https://xmlvalid.com/relaxng-validator" rel="noopener noreferrer"&gt;RelaxNG&lt;/a&gt; guides cover this level for each schema language.&lt;/p&gt;

&lt;p&gt;What a schema cannot do is relate one value to another. It can say that &lt;code&gt;TaxInclusiveAmount&lt;/code&gt; is a decimal amount. It cannot say that it must equal the amount without tax plus the tax.&lt;/p&gt;

&lt;h2&gt;
  
  
  Level 3: does it follow the business rules
&lt;/h2&gt;

&lt;p&gt;That job belongs to business rules, usually written in &lt;a href="https://xmlvalid.com/schematron-validator" rel="noopener noreferrer"&gt;Schematron&lt;/a&gt;: XPath assertions over the whole document, each with an id and a message. For invoices in Europe the base set is EN 16931, published by CEN: over 200 &lt;code&gt;BR-&lt;/code&gt; rules on the invoice's content, plus syntax rules for UBL and CII.&lt;/p&gt;

&lt;p&gt;Add 0.19 to the total with VAT (and to the amount due, so the two still agree) and the file is still schema-valid. The rules are not satisfied:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;BR-CO-15 Invoice total amount with VAT (BT-112) = Invoice total amount without VAT (BT-109) + Invoice total VAT amount (BT-110).

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;100.00 + 19.00 is not 119.19. In real invoices this is usually a rounding difference: VAT rounded per line, then summed, against VAT computed once on the total.&lt;/p&gt;

&lt;p&gt;Or remove the buyer's legal name. UBL makes &lt;code&gt;cac:PartyLegalEntity&lt;/code&gt; optional, because UBL is a general format used for much more than invoices. EN 16931 decides what an invoice needs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;BR-07 An Invoice shall contain the Buyer name (BT-44).

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The BT numbers are EN 16931's business terms, the same field whatever the syntax: BT-44 is &lt;code&gt;cac:PartyLegalEntity/cbc:RegistrationName&lt;/code&gt; in UBL and &lt;code&gt;ram:BuyerTradeParty/ram:Name&lt;/code&gt; in CII. Each rule has a page explaining what it checks and how to fix it, for example &lt;a href="https://xmlvalid.com/rules/BR-CO-15" rel="noopener noreferrer"&gt;BR-CO-15&lt;/a&gt; and &lt;a href="https://xmlvalid.com/rules/BR-07" rel="noopener noreferrer"&gt;BR-07&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;One practical consequence: business rules only make sense on a document that matches its schema, so xmlvalid runs them after the schema check passes. Fix the last schema error and a new batch of findings can appear. That is not the validator moving the goalposts; those rules simply had not run yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  Level 4: does it follow the receiver's rules
&lt;/h2&gt;

&lt;p&gt;EN 16931 is the common core. Most of the time the invoice is sent under a national or network profile that adds its own rules on top: XRechnung in Germany, Peppol BIS across Europe and beyond, CIUS-RO in Romania, France's CTC rules.&lt;/p&gt;

&lt;p&gt;I took the invoice from the start, which passes both levels above with no findings, changed only its &lt;code&gt;CustomizationID&lt;/code&gt; to say it is an XRechnung 3.0, and ran KoSIT's XRechnung rules on it. Ten errors and one warning:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Rule&lt;/th&gt;
&lt;th&gt;What it wants&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;BR-DE-1&lt;/td&gt;
&lt;td&gt;Payment instructions (BG-16)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BR-DE-2&lt;/td&gt;
&lt;td&gt;A seller contact (BG-6)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BR-DE-3, BR-DE-4&lt;/td&gt;
&lt;td&gt;The seller's city and post code&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BR-DE-8, BR-DE-9&lt;/td&gt;
&lt;td&gt;The buyer's city and post code&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BR-DE-15&lt;/td&gt;
&lt;td&gt;The buyer reference (BT-10), where German public buyers expect their Leitweg-ID&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;PEPPOL-EN16931-R001&lt;/td&gt;
&lt;td&gt;A business process (BT-23)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;PEPPOL-EN16931-R010, R020&lt;/td&gt;
&lt;td&gt;Electronic addresses for buyer and seller&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BR-DE-TMP-32 (warning)&lt;/td&gt;
&lt;td&gt;A delivery date or invoicing period&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;None of these are mistakes as far as EN 16931 is concerned; they are fields that Germany's public buyers require, which the European standard leaves optional. KoSIT's messages are in German ("Das Element "Buyer reference" (BT-10) muss übermittelt werden."), which is another small surprise for anyone sending an XRechnung from outside Germany.&lt;/p&gt;

&lt;p&gt;The rule set is chosen by the &lt;code&gt;CustomizationID&lt;/code&gt; (BT-24). A file that claims the wrong profile is checked against the wrong rules, and a validator set to plain EN 16931 will happily pass an invoice the receiver then rejects. When you open an e-invoice in the &lt;a href="https://xmlvalid.com/validate" rel="noopener noreferrer"&gt;validator&lt;/a&gt;, it reads BT-24 and picks the matching profile from the &lt;a href="https://xmlvalid.com/schemas" rel="noopener noreferrer"&gt;schema catalog&lt;/a&gt; for that reason.&lt;/p&gt;

&lt;h2&gt;
  
  
  Level 5: what no file check can see
&lt;/h2&gt;

&lt;p&gt;Some rejections have nothing to do with the file's structure, and no validator, ours included, can predict them:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Registries.&lt;/strong&gt; Whether the Leitweg-ID, the Peppol participant id or the VAT number exists and belongs to the right party. A correctly formed id can still be unknown to the receiver.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;State.&lt;/strong&gt; Duplicate invoice numbers, a purchase order that has already been invoiced, a credit note for an invoice the receiver never saw.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The receiver's own policy.&lt;/strong&gt; Purchase order numbers required by one company and not by another, attachments in a format their system cannot open, size limits of the platform in between.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Clearance systems.&lt;/strong&gt; Italy's SdI, Spain's VERI*FACTU and Mexico's CFDI check some things against government records at submission time. A validator can only cover the checks that need nothing but the file.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If a file passes levels 1 to 4 and is still rejected, the rejection message itself, not a validator, is the place to look.&lt;/p&gt;

&lt;h2&gt;
  
  
  Reading a rejection
&lt;/h2&gt;

&lt;p&gt;The wording of the message usually tells you which level failed:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;The message mentions&lt;/th&gt;
&lt;th&gt;Level&lt;/th&gt;
&lt;th&gt;Where to look&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;parser error&lt;/code&gt;, &lt;code&gt;xmlParseEntityRef&lt;/code&gt;, &lt;code&gt;Opening and ending tag mismatch&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;1, well-formedness&lt;/td&gt;
&lt;td&gt;The line it names, or the code that wrote the XML&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;Element '…': This element is not expected&lt;/code&gt;, &lt;code&gt;not a valid value of the atomic type&lt;/code&gt;, &lt;code&gt;cvc-complex-type.2.4.a&lt;/code&gt; (Java)&lt;/td&gt;
&lt;td&gt;2, schema&lt;/td&gt;
&lt;td&gt;The schema and the element order&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A rule id such as &lt;code&gt;BR-CO-15&lt;/code&gt;, &lt;code&gt;BR-07&lt;/code&gt;, &lt;code&gt;BR-S-08&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;3, EN 16931&lt;/td&gt;
&lt;td&gt;The rule's page; usually amounts or a missing field&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;BR-DE-…&lt;/code&gt;, &lt;code&gt;PEPPOL-EN16931-…&lt;/code&gt;, &lt;code&gt;BR-FR-…&lt;/code&gt;, &lt;code&gt;BR-RO-…&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;4, the profile&lt;/td&gt;
&lt;td&gt;The profile's documentation and the &lt;code&gt;CustomizationID&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A duplicate, an unknown id, something about the account or the order&lt;/td&gt;
&lt;td&gt;5, the receiver&lt;/td&gt;
&lt;td&gt;The receiver&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Java and .NET word the same schema problems differently from libxml2. The &lt;a href="https://xmlvalid.com/errors" rel="noopener noreferrer"&gt;error guide&lt;/a&gt; lists their versions alongside, so &lt;code&gt;cvc-complex-type.2.4.a&lt;/code&gt; and &lt;code&gt;This element is not expected&lt;/code&gt; lead to the same explanation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;The broken invoices from this post are easy to recreate: open the &lt;a href="https://xmlvalid.com/schemas/en16931-ubl-invoice" rel="noopener noreferrer"&gt;EN 16931 example&lt;/a&gt; or the &lt;a href="https://xmlvalid.com/schemas/xrechnung-3-ubl" rel="noopener noreferrer"&gt;XRechnung one&lt;/a&gt; in the &lt;a href="https://xmlvalid.com/validate" rel="noopener noreferrer"&gt;validator&lt;/a&gt;, change a total or a date, and run it. It checks the schema and the profile's business rules together, in the browser, so the invoice is not uploaded anywhere.&lt;/p&gt;

</description>
      <category>softwaredevelopment</category>
      <category>development</category>
      <category>programming</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>Validating a 5 GB XML file in the browser</title>
      <dc:creator>Alex Babintsev</dc:creator>
      <pubDate>Fri, 23 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/alexbabintsev/validating-a-5-gb-xml-file-in-the-browser-265</link>
      <guid>https://dev.to/alexbabintsev/validating-a-5-gb-xml-file-in-the-browser-265</guid>
      <description>&lt;p&gt;Most online XML validators stop at a few megabytes. They upload the file, parse it into a tree on the server and walk the tree against the schema. That is fine for a config file and hopeless for a 2 GB data export, a year of bank statements or a product feed.&lt;/p&gt;

&lt;p&gt;xmlvalid validates in the browser instead: the document never leaves the machine. To see how far that goes, I built a 5.2 GB file (5,242,879,782 bytes) out of a book catalog, broke one &lt;code&gt;&amp;lt;year&amp;gt;&lt;/code&gt; on line 38,269,200 so it read &lt;code&gt;20x5&lt;/code&gt;, and opened it in the validator with its XSD. Sixty-six seconds later the page said:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Document is invalid: 1 issue
line 38,269,200 Element 'year': '20x5' is not a valid value of the atomic type 'xs:gYear'.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The tab's memory stayed flat the whole time. Here is how that works, and the few places where size still bites.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why a tree does not work
&lt;/h2&gt;

&lt;p&gt;The obvious way to validate XML in JavaScript is to read the file into a string and hand it to a parser. Two walls stand in the way long before 5 GB:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Strings.&lt;/strong&gt; V8 caps a string at 2&lt;sup&gt;29&lt;/sup&gt; characters, a little over 500 million, so &lt;code&gt;file.text()&lt;/code&gt; fails on anything near a gigabyte.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Memory.&lt;/strong&gt; A WebAssembly module compiled for 32 bits addresses at most 4 GB, and Emscripten's default ceiling is 2 GB. A DOM of the document is several times the document's size anyway.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So the document has to stream: read a piece, validate it, forget it, read the next one. Memory then depends on how deep the document nests, not on how long it is.&lt;/p&gt;

&lt;h2&gt;
  
  
  A streaming validator
&lt;/h2&gt;

&lt;p&gt;libxml2 has done this for twenty years. Its &lt;code&gt;xmlTextReader&lt;/code&gt; is a pull parser that can validate against XSD, RelaxNG or a DTD while it reads, and it frees each node once the reader has moved past it. It gets its bytes from whatever you give it: a file, a socket, or a callback.&lt;/p&gt;

&lt;p&gt;The engine behind xmlvalid is a small Rust program around that reader. The same code runs natively on the server (for the API and the command-line client) and, compiled with Emscripten, in the browser. In the browser build the byte source is a function the page provides:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="n"&gt;JsReader&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;impl&lt;/span&gt; &lt;span class="n"&gt;Read&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;JsReader&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;read&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="k"&gt;mut&lt;/span&gt; &lt;span class="k"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;buf&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="k"&gt;mut&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;u8&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt; &lt;span class="k"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nn"&gt;io&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nb"&gt;Result&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nb"&gt;usize&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;match&lt;/span&gt; &lt;span class="k"&gt;unsafe&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nf"&gt;xv_js_read&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;buf&lt;/span&gt;&lt;span class="nf"&gt;.as_mut_ptr&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="n"&gt;buf&lt;/span&gt;&lt;span class="nf"&gt;.len&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="k"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;Ok&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nb"&gt;usize&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="k"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;Err&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;io&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;Error&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;other&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"host read failed"&lt;/span&gt;&lt;span class="p"&gt;)),&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;xv_js_read&lt;/code&gt; is a two-line Emscripten library function that calls back into JavaScript. libxml2 asks for a few kilobytes at a time and never sees more than that.&lt;/p&gt;

&lt;h2&gt;
  
  
  Feeding it from a File
&lt;/h2&gt;

&lt;p&gt;The engine runs in a Web Worker, which matters for two reasons: the page stays responsive, and workers have &lt;code&gt;FileReaderSync&lt;/code&gt;. libxml2's read callback is synchronous, and in a worker a &lt;code&gt;Blob&lt;/code&gt; can be read synchronously, so there is no need for Asyncify or SharedArrayBuffer tricks:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;reader&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;FileReaderSync&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;CHUNK&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;byteReader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;blob&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;buffer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Uint8Array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="k"&gt;return &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;len&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="nx"&gt;blob&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;size&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
            &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;end&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;len&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;CHUNK&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="nx"&gt;blob&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;size&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
            &lt;span class="nx"&gt;buffer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Uint8Array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;reader&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;readAsArrayBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;blob&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;end&lt;/span&gt;&lt;span class="p"&gt;)));&lt;/span&gt;
            &lt;span class="nx"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;end&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;subarray&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;len&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="nx"&gt;buffer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;subarray&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;out&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;blob.slice()&lt;/code&gt; does not read anything; it only describes a range of the file. Each &lt;code&gt;readAsArrayBuffer&lt;/code&gt; call has a fixed cost, so the reader takes a megabyte at a time and serves libxml2's small requests from it. The callback copies the bytes into the module's heap with &lt;code&gt;HEAPU8.set&lt;/code&gt; and returns the count; zero means end of file. The engine's heap stays around 20 MB whether the file is 5 MB or 5 GB.&lt;/p&gt;

&lt;p&gt;Progress and errors go the other way as JSON lines, posted to the page as they happen, so the first error shows up while the rest of the file is still being read.&lt;/p&gt;

&lt;h2&gt;
  
  
  Things that break past 4 GB
&lt;/h2&gt;

&lt;p&gt;Streaming removes the obvious limit and leaves a few quieter ones:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Counters.&lt;/strong&gt; In wasm32 &lt;code&gt;usize&lt;/code&gt; is 32 bits. Anything that counts bytes in it wraps at 4 GB, so the engine's byte and error counters are &lt;code&gt;u64&lt;/code&gt;, and the file size crosses from JavaScript as a double, exact up to 2&lt;sup&gt;53&lt;/sup&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Line numbers.&lt;/strong&gt; libxml2 keeps a node's line number in 16 bits. The engine passes &lt;code&gt;XML_PARSE_BIG_LINES&lt;/code&gt;, which tells it to keep the real number past line 65,535, so an error on line 38 million says so.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Huge text.&lt;/strong&gt; By default libxml2 refuses a single text node over 10 MB, a guard against hostile input. &lt;code&gt;XML_PARSE_HUGE&lt;/code&gt; raises that to 1 GB, which is the one hard limit left: one text node, not the document.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Encodings.&lt;/strong&gt; The browser build of libxml2 has no iconv, so a document in windows-1251 or Shift_JIS is transcoded to UTF-8 with a streaming &lt;code&gt;TextDecoder&lt;/code&gt; on the way in, and its XML declaration rewritten to match. Line numbers stay the same.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The numbers
&lt;/h2&gt;

&lt;p&gt;Same 5.2 GB file and XSD, same Mac:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Engine&lt;/th&gt;
&lt;th&gt;Time&lt;/th&gt;
&lt;th&gt;Throughput&lt;/th&gt;
&lt;th&gt;Memory&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Native (Rust + libxml2)&lt;/td&gt;
&lt;td&gt;42 s&lt;/td&gt;
&lt;td&gt;118 MB/s&lt;/td&gt;
&lt;td&gt;4 MB resident&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Browser (WebAssembly, headless Chromium)&lt;/td&gt;
&lt;td&gt;66 s&lt;/td&gt;
&lt;td&gt;79 MB/s&lt;/td&gt;
&lt;td&gt;flat, about 20 MB of heap&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The browser time is from the click to the verdict on the page, so it includes the worker, the copies into the heap and the page updating its progress bar. WebAssembly at two thirds of native speed is about what you would expect for a parser.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where size still hurts
&lt;/h2&gt;

&lt;p&gt;Two things do not stream as nicely.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Business rules.&lt;/strong&gt; E-invoices are checked against Schematron rules (EN 16931, Peppol, XRechnung) after the schema. Those are XSLT, run with SaxonJS, and XSLT wants the whole document as a tree. That costs about 2.7 seconds per megabyte of invoice, and a 50 MB invoice takes a gigabyte of memory, so the rules stop at 50 MB. Real invoices are a few hundred kilobytes; the large ones are almost always a PDF attachment in base64, so past 50 MB those runs are cut to a stub before the rules see the document.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Big text nodes.&lt;/strong&gt; Those base64 attachments were also where libxml2 itself slowed down: validating a single large text node against XSD took time quadratic in its length, 12 seconds for a 30 MB attachment. We patched it (now 0.09 seconds) and sent the fix upstream; that is a story for another post.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;The validator is at &lt;a href="https://xmlvalid.com/validate" rel="noopener noreferrer"&gt;xmlvalid.com/validate&lt;/a&gt;. It takes files up to 50 MB without an account and up to 1 GB with a free one. The 1 GB is where we drew the line for the page, not where the engine stops: if you need to check something larger, &lt;a href="mailto:support@xmlvalid.com"&gt;write to us&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>webassembly</category>
      <category>rust</category>
      <category>xml</category>
      <category>javascript</category>
    </item>
  </channel>
</rss>
