DEV Community: Chan Lebron

I Built a JSONL to CSV Converter. Could Developers Test It?

Chan Lebron — Mon, 12 Oct 2026 08:13:43 +0000

I've been building DataToolForge, a collection of browser-based data utilities.

One tool I'd particularly like feedback on is a JSONL / NDJSON to CSV converter.

It supports:

Flattening nested objects into CSV columns

Preserving arrays as JSON strings

Reporting malformed JSONL records

Local browser processing, without uploading files

Try it: https://datatoolforge.com/tools/jsonl-to-csv

I'd appreciate feedback on three things:

Did the conversion produce the output you expected?

Was anything confusing about the interface?

What would prevent you from using it for a real task?

You can use the built-in example or a small, non-sensitive test dataset.

I'm the developer, and I'm looking for practical feedback rather than promotional reviews. Critical feedback is welcome.

Your JSON Parses. So Why Did One Value Disappear?

Chan Lebron — Sun, 11 Oct 2026 01:07:01 +0000

Imagine reviewing a configuration file for a nightly import job.

You expect the job to stop after processing 1,000 rows. But the application keeps going until 10,000.

The file looks reasonable at first glance:

{
  "task": "nightly-import",
  "limits": {
    "max_rows": 1000,
    "max_rows": 10000
  }
}

There are no missing commas or broken brackets. Your JSON parser doesn't throw an error.

Look again at max_rows.

The key appears twice, with two different values.

That small detail changes what the application actually receives.

What JavaScript does with this input

Save this as check.js and run it with Node.js:

const source = `{
  "task": "nightly-import",
  "limits": {
    "max_rows": 1000,
    "max_rows": 10000
  }
}`;

const config = JSON.parse(source);

console.log(config.limits.max_rows);
// 10000

console.log(Object.keys(config.limits));
// [ 'max_rows' ]

console.log(JSON.stringify(config, null, 2));

The final output contains only one max_rows property:

{
  "task": "nightly-import",
  "limits": {
    "max_rows": 10000
  }
}

The earlier value didn't survive parsing.

This is the part that makes the bug inconvenient: if you log only the parsed object, the evidence is gone.

The log shows a normal object with one property. Nothing tells you that the original payload contained two competing values.

If the raw document came from a generated configuration, a webhook, or a file assembled from multiple sources, the mistake may have happened several steps earlier.

Is duplicate-key JSON actually invalid?

RFC 8259, Section 4, says that member names within a JSON object SHOULD be unique.

That's an important distinction from MUST be unique.

The specification explains that implementations can handle duplicate names differently. Some keep the last occurrence; others reject the document or preserve multiple pairs.

So the safer engineering assumption is not "every parser will reject this."

It's "a document with duplicate object names may not be interpreted consistently."

For configuration files and API contracts, I prefer a stronger rule: reject duplicate names before the application uses the parsed values.

Why checking the object afterward doesn't work

You might try validating config after calling JSON.parse().

But by then, JavaScript has already reduced the duplicate entries to a single property.

A JSON.parse() reviver can't recover the overwritten member either. It receives values from the resulting parse, not an ordered record of every original occurrence.

The same limitation applies when a JSON Schema validator is given an already-parsed object. It can check the surviving value's type, range, and required fields, but it cannot reconstruct a duplicate member that was discarded before validation.

This is why duplicate-key detection belongs at the parsing boundary.

A small Python parser that rejects duplicates

Python's standard json module provides object_pairs_hook.

Instead of immediately reducing every JSON object to a dictionary, this hook receives its ordered member pairs.

That gives us a chance to reject duplicates.

import json


def reject_duplicates(pairs):
    result = {}

    for key, value in pairs:
        if key in result:
            raise ValueError(
                f"Duplicate JSON key: {key!r}"
            )

        result[key] = value

    return result


def load_unique_json(source):
    return json.loads(
        source,
        object_pairs_hook=reject_duplicates,
    )

Now test a duplicate in a nested object:

source = '''
{
  "task": "nightly-import",
  "limits": {
    "max_rows": 1000,
    "max_rows": 10000
  }
}
'''

load_unique_json(source)

Instead of choosing one value, it raises:

ValueError: Duplicate JSON key: 'max_rows'

The hook also runs for nested objects, including objects inside arrays.

We don't need to scan the raw file with regular expressions or write an entire JSON parser.

Three regression cases I would keep

A duplicate detector needs to reject repeated names within the same object, not across unrelated objects.

This should pass:

valid = '''
{
  "primary": {"id": 1},
  "backup": {"id": 2}
}
'''

assert load_unique_json(valid) == {
    "primary": {"id": 1},
    "backup": {"id": 2},
}

These should fail:

invalid_cases = [
    # Duplicate in a top-level object
    '{"mode": "safe", "mode": "fast"}',

    # Duplicate inside an array element
    '{"items": [{"id": 1, "id": 2}]}',

    # Same decoded key, different source spelling
    r'{"name": "first", "\u006eame": "second"}',
]

for source in invalid_cases:
    try:
        load_unique_json(source)
    except ValueError as error:
        print("Rejected:", error)
    else:
        raise AssertionError(
            "Duplicate key was not rejected"
        )

The third case is particularly easy to overlook.

In JSON, \u006e represents the letter n.

So these two names:

"name"
"\u006eame"

decode to the same string.

A simple search for repeated quoted text can miss the duplicate. The parser-based approach checks the decoded names instead.

This test is about exact decoded-key equality. It does not attempt to detect visually similar Unicode characters or perform Unicode normalization.

One limitation worth knowing

Our Python function detects duplicate keys, but its error only reports the member name.

It does not tell you the exact path, line, or column.

For a large configuration containing many objects with fields named id or timeout, that may not be enough to locate the source.

A production-grade diagnostic tool should ideally report something like:

Duplicate member:
  path: $.limits.max_rows
  first occurrence: line 4
  second occurrence: line 5

That requires retaining more source-location information than this short example does.

For a small configuration or a CI test, though, rejecting the document is already much better than silently choosing a value.

Put the check where data enters your application

For a configuration-loading pipeline, my preferred order is:

Read the original JSON text.
Reject duplicate object-member names.
Parse the accepted document.
Validate its schema and application-specific constraints.
Only then use it to configure the application.

In the Python example, steps 2 and 3 happen together.

The important point is to avoid discarding evidence before you have checked the rule you care about.

A browser-based syntax checker is still useful for malformed JSON. For example, I maintain a JSON Validator on DataToolForge for ordinary syntax and structure checks. That is a different task from the duplicate-key rejection implemented in this article.

A successful parse is useful evidence that the syntax was accepted.

It is not evidence that every member of the original document survived.

References

The configuration example is synthetic. The JavaScript and Python examples were tested with Node.js 22.16.0 and Python 3.13.5.

Disclosure: DataToolForge is my project. AI assistance was used to draft, research, and test the examples in this article.

JSON to XML: Why an Array Needs a Root Element

Chan Lebron — Tue, 06 Oct 2026 23:48:24 +0000

A JSON object with one key can still need an extra wrapper when that key holds an array. This is the small input I use to check that case:

{
  "item": [1, 2]
}

The array contains two values. An XML document still needs one document element. The XML specification defines that requirement.

Two items, one document

For comparison, this deliberately broken document has two top-level elements:

<item>1</item>
<item>2</item>

Those elements could be placed inside a larger document, but this text alone cannot be parsed as a well-formed XML document.

The tested converter adds a wrapper:

<root>
  <item>1</item>
  <item>2</item>
</root>

The name root is a mapping choice. If a receiving API expects <items> or a namespace-qualified element, this output can parse successfully and still have the wrong shape for that API.

Here is a small check using Python's standard ElementTree parser:

import xml.etree.ElementTree as ET

array_xml = """<root>
  <item>1</item>
  <item>2</item>
</root>"""

root = ET.fromstring(array_xml)
assert root.tag == "root"
assert [child.tag for child in root] == ["item", "item"]
assert [child.text for child in root] == ["1", "2"]
print("array_root: passed")

The parser checks the document structure. The assertions check the wrapper, element names, values and order expected by this particular mapping.

Add a nested object and text that needs escaping

My second fixture is a synthetic order:

{
  "order": {
    "@id": "00123",
    "customer": "Lee & Tom",
    "note": "<review>",
    "item": [
      { "sku": "A1" },
      { "sku": "B2" }
    ]
  }
}

Here, the single non-array key order supplies the document element. This converter treats the @ prefix as an attribute convention. Its output is:

<order id="00123">
  <customer>Lee &amp; Tom</customer>
  <note>&lt;review&gt;</note>
  <item>
    <sku>A1</sku>
  </item>
  <item>
    <sku>B2</sku>
  </item>
</order>

00123 stays a string in the source. The ampersand and angle brackets are escaped in the XML text, then decoded when a parser reads it:

order_xml = """<order id="00123">
  <customer>Lee &amp; Tom</customer>
  <note>&lt;review&gt;</note>
  <item>
    <sku>A1</sku>
  </item>
  <item>
    <sku>B2</sku>
  </item>
</order>"""

order = ET.fromstring(order_xml)
assert order.tag == "order"
assert order.attrib == {"id": "00123"}
assert order.findtext("customer") == "Lee & Tom"
assert order.findtext("note") == "<review>"
assert [item.findtext("sku") for item in order.findall("item")] == ["A1", "B2"]
print("nested_order: passed")

Run the two Python blocks in order. The second uses the import from the first. Keep the XML escapes in the document; the parsed values show whether the text survived.

Agree on the mapping before sending the result

For an integration, I would check the receiving contract against three choices:

The required document element, including any namespace.
How arrays are represented: repeated elements or a collection wrapper.
Which fields become attributes, child elements or text.

@id is a converter convention, not a rule defined by JSON. The receiving system's schema or API documentation should determine the required structure. The assertions above verify these fixtures; they do not validate an XSD or prove that another API will accept them.

Both examples were run against the live DataToolForge JSON to XML converter on 2026-10-07, UTC+08:00, in a remote Chrome browser. The two Python blocks were also executed with Python 3.12.14. The screenshots show the actual tool results, and all inputs are synthetic.

You can paste either fixture into the converter and compare the result with the checks above. If your XML API needs a different array shape, which element names and wrapper does it require?

Disclosure: DataToolForge is my project. AI assistance was used to draft, test and edit this article.

Excel changed my CSV IDs. Catch the problem before import.

Chan Lebron — Mon, 05 Oct 2026 16:49:00 +0000

A CSV can open without an error and still lose the information you needed.

The awkward case is an identifier that looks like a number. An account ID of 00123 and a long ID of 9007199254740993 are strings of digits. If an importer turns them into numeric cells, their original spelling may not survive.

For identifiers, keep the text until you have a reason to convert it. Adding a numeric format later cannot recover digits that were already lost.

A small file that exposes the problem

Save this as import-check.csv, or use it as pasted text:

account_id,name,comment
00123,Alice,Ready
9007199254740993,Bob,=1+1

There are two records. Three values deserve review before a spreadsheet import:

00123 needs its leading zeros preserved.
9007199254740993 needs all of its digits preserved.
=1+1 may be interpreted as a spreadsheet formula rather than displayed as the original text.

The formula is deliberately harmless. These rows use synthetic data, so the example does not need real customer records.

Check the source before blaming the export

Python's standard CSV reader keeps these fields as text unless your own code converts them:

import csv
import io
import json

source = """account_id,name,comment
00123,Alice,Ready
9007199254740993,Bob,=1+1
"""

records = list(csv.DictReader(io.StringIO(source)))

assert records[0]["account_id"] == "00123"
assert records[1]["account_id"] == "9007199254740993"
assert records[1]["comment"] == "=1+1"

print(json.dumps(records, indent=2))

The result includes quoted JSON strings for both IDs:

[
  {
    "account_id": "00123",
    "name": "Alice",
    "comment": "Ready"
  },
  {
    "account_id": "9007199254740993",
    "name": "Bob",
    "comment": "=1+1"
  }
]

That gives you a reference to compare with the imported data. CSV quoting handles CSV structure; it is not a reliable instruction to a spreadsheet to use Text cells.

Excel and JavaScript have different numeric limits

Microsoft documents a 15-significant-digit limit for Excel numeric values. Long account numbers should be imported as text when every digit matters. Leading zeros can also disappear when a field is converted to a number.

JavaScript has a different boundary. Number.MAX_SAFE_INTEGER is 9007199254740991. Try this in a JavaScript console:

JSON.parse('{"account_id":9007199254740993}').account_id;
// 9007199254740992

JSON.parse('{"account_id":"9007199254740993"}').account_id;
// "9007199254740993"

Both JSON documents have valid syntax. Only the second represents this identifier as a string. Keeping an ID quoted avoids this particular Number conversion; calculations on genuinely numeric values need their own precision policy.

Choose the import type before loading the column

Use Excel's text/CSV import flow instead of relying on a double-click to choose types for you. In Power Query, set identifier columns to Text before numeric conversion loses information. Check any automatically added type-conversion step: changing a rounded value back to text does not restore the original ID. Menu labels vary between Excel versions.

After loading, compare the two IDs with the source. Check the actual cell value rather than judging only its display. If you export the workbook to CSV, close it and inspect the exported file too. A workflow that preserves an ID on the first open can still change it in a later step.

Review formula-like values separately

Preserving an ID is not the same task as preventing formula interpretation. The comment field demonstrates why: =1+1 can parse as a CSV string and later act as a formula in a spreadsheet.

OWASP's CSV injection guidance covers formula prefixes and explains why escaping must be tested against the destination. There is no universal rewrite that suits every spreadsheet and downstream consumer. A warning should show the original cell and let you choose a destination-specific treatment.

To try the example without uploading a file, open DataToolForge's Excel Import Preflight, paste the sample and run the check. The report is intended to surface import risks, not prove that a file satisfies every business rule. A broader CSV Import Preflight also checks file structure.

The useful habit is simple: compare the source text, the imported values and the next exported file. Check exact IDs and formula-like cells before using the data in another system.

Disclosure: DataToolForge is my project. AI assistance was used to draft and edit this article.

CSV Formula Injection: Why =, +, -, and @ Can Be Dangerous

Chan Lebron — Tue, 29 Sep 2026 12:26:23 +0000

CSV looks simple, but a file can be structurally valid and still contain values that behave unexpectedly when opened in spreadsheet software.

CSV syntax is not the whole story

CSV itself does not define formulas. A parser may see =1+1 as an ordinary string, while spreadsheet software may interpret the same value as a formula after import.

id,value
1,=1+1
2,+10+20
3,@SUM(A1:A2)

Nothing in that example is necessarily malformed CSV. The risk appears later, when another application decides how to interpret the cell value.

Why this matters

Formula-like values matter most when CSV data contains user-controlled or externally supplied content. Typical sources include form submissions, CRM exports, support tickets, marketplace data, and administrative reports.

username,comment
alice,Looks good
bob,"=HYPERLINK(""https://example.com"",""Click"")"

A parser can accept this file while a spreadsheet application may treat the second row differently from plain text.

Characters worth checking

Common spreadsheet formula-triggering prefixes include:

=
+
-
@

A match is not automatically malicious. -12 may be a negative number, and +44 may be part of a phone number.

Spreadsheet behavior also varies by application and locale. Security guidance such as OWASP's CSV Injection material additionally calls out tab, carriage return, line feed, and some full-width variants as cases worth considering.

That is why reporting the risk and testing against the target spreadsheet application is usually better than assuming one universal rule.

Why automatic sanitization can be risky

One defensive technique is to prefix a value with an apostrophe. That may force some spreadsheet applications to treat the value as text, but it also changes the original dataset.

For exports, migrations, and audit data, silent rewriting may be undesirable. Some spreadsheet applications may also transform quoting or escaping when a CSV file is saved and reopened.

A safer workflow is:

parse the CSV;
identify formula-like cells;
show the row, column, and original value;
let the user decide whether the value is intentional;
apply a mitigation appropriate to the target spreadsheet application.

Formula risk is different from malformed CSV

An unclosed quote is a structural CSV error. A quoted value such as "=1+1" can still be structurally valid while remaining interesting from a spreadsheet-risk perspective.

Successful parsing does not guarantee safe downstream interpretation.

Takeaway

A CSV file can be valid and still contain values that downstream software interprets in surprising ways.

Reference

OWASP: CSV Injection

Originally published on DataToolForge.

AI-assisted disclosure: I used AI as a drafting/editing assistant and reviewed the technical content before publication.