DEV Community

Emir Mrkaljević
Emir Mrkaljević

Posted on

tail: cannot open '7': when localisation reformats your file names

#rust #i18n #testing #cli

Ask uutils tail (the Rust rewrite of GNU coreutils) for a file that doesn't exist and it tells you so. Unless the file is called 007:

$ tail 007          # GNU coreutils 9.12
tail: cannot open '007' for reading: No such file or directory
$ tail 007          # uutils main
tail: cannot open '7' for reading: No such file or directory
Enter fullscreen mode Exit fullscreen mode

1e3 comes back as 1000, 1.50 as 1.5, +5 as 5. Try tail 1e3 on a directory and you get "error reading '1000'".

Where the 7 comes from

uutils translates its messages with Fluent. Every argument goes through a translate! macro, and that macro checks whether the value parses as a number. If it does, it hands Fluent a number instead of a string, so the locale can format it (digit grouping, decimal separator). That's what you want for "3 files failed". It's not what you want for a file name that just happens to look like a number: Fluent prints the number, not the text you typed.

The project already has translate_text! for exactly this, and date and the sort --check "disorder" line were fixed the same way earlier. The tail fix (#15534, merged a few hours after I opened it) switches the call sites that take a file name, tac (#15541) was merged the same day, and nl (#15544) and the shared checksum code behind md5sum/cksum (#15548) were merged the next morning. The same pattern still shows up in dd, sort, shred and in clap's "unexpected argument" error, all listed in #15533.

The general lesson, if you localise a CLI with Fluent or ICU MessageFormat: anything the user typed (paths, operands, input lines) should go in as a string. Let only real counts be numbers.

Catching it automatically

I found this with behave, which probes a command as a black box, records what the original does and replays it against a rewrite. Its generated cases always included "missing file", but with the name does-not-exist.txt, which no number parser will touch. Since v0.1.6 behave probe also adds a missing-file case with the names 007 and 1e3:

✗ missing file named like a number  0008-missing-file-named-like-a-number.yaml
  $ tail 007 1e3
  stderr:
    - $CMD: cannot open '007' for reading: No such file or directory
    - $CMD: cannot open '1e3' for reading: No such file or directory
    + $CMD: cannot open '7' for reading: No such file or directory
    + $CMD: cannot open '1000' for reading: No such file or directory
Enter fullscreen mode Exit fullscreen mode

It costs one case per tool and checks something every port that translates its messages can get wrong.

go install github.com/emirmrkaljevic/behave/cmd/behave`latest
# or: brew install emirmrkaljevic/tap/behave
behave probe tail --out spec-tail
behave diff spec-tail --impl ./target/release/tail
Enter fullscreen mode Exit fullscreen mode

behave is open source (MIT): https://github.com/emirmrkaljevic/behave · more from me on GitHub

Top comments (1)

Collapse
 
c-wind profile image
c-wind •

The point about the translate! macro quietly turning anything that parses as a number into a Fluent number is a subtle one. I wouldn't have guessed a name like 1e3 would come back as 1000. Did you hit any case where locale digit grouping changed the output for a real file name, or is it mostly leading zeros and exponent forms?