DEV Community

Super Funicular
Super Funicular

Posted on

Why Did ls Sort clip_10 Before clip_1? Three Orders for One Folder of Numbered Clips

Twelve clips named clip_1.mp4 through clip_12.mp4 should be the easiest folder in the world to put in order. I listed the same twelve files with ls, a shell glob and Python on one Linux machine and got three different orders back. In one of them clip_10 came before clip_1, which surprised me, and none of them gave the order the clips were recorded in.

Disclosure: I make an Android camera app. Nothing here uses its code or describes how it names its files. The names below are ones I made up so the effect is easy to see, and everything runs on a computer, with GNU coreutils 8.32, bash and Python 3.10.

What the tests showed:

  • ls in the C locale put clip_10, clip_11 and clip_12 between clip_1 and clip_2. That is the order most people expect to see and dislike.
  • ls in en_US.UTF-8 put all three before clip_1. Same files, same command, one environment variable different.
  • A shell loop over clip_*.mp4 follows the locale too, so a play-in-order list built that way started with clip_10.
  • Python's os.listdir and glob.glob returned the files in neither order. sorted() matched the C locale, not the recording order.
  • ls -v, sort -V and zero-padded names were what got me 1 to 12, and padding was the one that made every tool agree.

The test folder

#!/bin/bash
# twelve clips, numbered the way a counter numbers them
rm -rf clips && mkdir clips && cd clips
for i in $(seq 1 12); do touch "clip_$i.mp4"; done
Enter fullscreen mode Exit fullscreen mode

Two locales, two orders

#!/bin/bash
cd clips
echo "== LC_ALL=C ls";            LC_ALL=C ls | tr '\n' ' '; echo
echo "== LC_ALL=en_US.UTF-8 ls";  LC_ALL=en_US.UTF-8 ls | tr '\n' ' '; echo
echo "== bash glob, C";           LC_ALL=C bash -c 'echo clip_*.mp4'
echo "== bash glob, en_US.UTF-8"; LC_ALL=en_US.UTF-8 bash -c 'echo clip_*.mp4'
Enter fullscreen mode Exit fullscreen mode
== LC_ALL=C ls
clip_1.mp4 clip_10.mp4 clip_11.mp4 clip_12.mp4 clip_2.mp4 clip_3.mp4 clip_4.mp4 clip_5.mp4 clip_6.mp4 clip_7.mp4 clip_8.mp4 clip_9.mp4 
== LC_ALL=en_US.UTF-8 ls
clip_10.mp4 clip_11.mp4 clip_12.mp4 clip_1.mp4 clip_2.mp4 clip_3.mp4 clip_4.mp4 clip_5.mp4 clip_6.mp4 clip_7.mp4 clip_8.mp4 clip_9.mp4 
== bash glob, C
clip_1.mp4 clip_10.mp4 clip_11.mp4 clip_12.mp4 clip_2.mp4 clip_3.mp4 clip_4.mp4 clip_5.mp4 clip_6.mp4 clip_7.mp4 clip_8.mp4 clip_9.mp4
== bash glob, en_US.UTF-8
clip_10.mp4 clip_11.mp4 clip_12.mp4 clip_1.mp4 clip_2.mp4 clip_3.mp4 clip_4.mp4 clip_5.mp4 clip_6.mp4 clip_7.mp4 clip_8.mp4 clip_9.mp4
Enter fullscreen mode Exit fullscreen mode

The C result is plain byte order. Character by character, clip_1.mp4 and clip_10.mp4 agree for the first six characters; then one has . and the other has 0, and . comes earlier in ASCII, so clip_1 wins. clip_10 then beats clip_2 because 1 comes before 2. Nothing about that is numeric. Every name is read as text.

The en_US.UTF-8 result is the one I did not expect. The GNU coreutils FAQ explains it in its entry titled "Sort does not sort in normal order!": with that locale set, "case is folded and punctuation is ignored because ‘en_US.UTF-8’ specifies this behavior." Take the underscore and dot out and the comparison becomes clip1mp4 against clip10mp4. The sixth characters are now m and 0, digits come before letters, and clip_10 goes first.

The FAQ is also clear that this is not a bug in the tool: "The locale collating table selected by the locale setting is not part of the GNU program but part of your vendor’s system release." Which means the same script can list a folder one way on a server and another way on a laptop, without any change to the script.

The shell loop follows the locale too

Lots of small scripts do not call ls at all. They loop over a glob, and bash sorts the glob for them. The second half of the output above shows the glob order matching ls in both locales. Here is what that does to the usual list-building one-liner:

#!/bin/bash
# the one-liner that builds a "play these in order" list
cd clips
LC_ALL=en_US.UTF-8 bash -c 'for f in clip_*.mp4; do echo "file $f"; done' | head -4
Enter fullscreen mode Exit fullscreen mode
file clip_10.mp4
file clip_11.mp4
file clip_12.mp4
file clip_1.mp4
Enter fullscreen mode Exit fullscreen mode

If a list like that feeds anything that plays or joins files in order, the result starts with the tenth clip and reaches the first one fourth. Nothing fails, nothing warns, and a quick look at the first line of the list does not reveal it, because clip_10 looks like a perfectly reasonable first line.

Python did not sort at all

import glob, os, re
os.chdir("clips")
print("listdir:", os.listdir("."))
print("glob:   ", glob.glob("clip_*.mp4"))
print("sorted: ", sorted(os.listdir(".")))
num = lambda s: [int(t) if t.isdigit() else t for t in re.split(r"(\d+)", s)]
print("natural:", sorted(os.listdir("."), key=num))
Enter fullscreen mode Exit fullscreen mode
listdir: ['clip_3.mp4', 'clip_5.mp4', 'clip_11.mp4', 'clip_9.mp4', 'clip_12.mp4', 'clip_6.mp4', 'clip_1.mp4', 'clip_4.mp4', 'clip_2.mp4', 'clip_8.mp4', 'clip_7.mp4', 'clip_10.mp4']
glob:    ['clip_3.mp4', 'clip_5.mp4', 'clip_11.mp4', 'clip_9.mp4', 'clip_12.mp4', 'clip_6.mp4', 'clip_1.mp4', 'clip_4.mp4', 'clip_2.mp4', 'clip_8.mp4', 'clip_7.mp4', 'clip_10.mp4']
sorted:  ['clip_1.mp4', 'clip_10.mp4', 'clip_11.mp4', 'clip_12.mp4', 'clip_2.mp4', 'clip_3.mp4', 'clip_4.mp4', 'clip_5.mp4', 'clip_6.mp4', 'clip_7.mp4', 'clip_8.mp4', 'clip_9.mp4']
natural: ['clip_1.mp4', 'clip_2.mp4', 'clip_3.mp4', 'clip_4.mp4', 'clip_5.mp4', 'clip_6.mp4', 'clip_7.mp4', 'clip_8.mp4', 'clip_9.mp4', 'clip_10.mp4', 'clip_11.mp4', 'clip_12.mp4']
Enter fullscreen mode Exit fullscreen mode

The listdir and glob lines are not a sort of any kind. They are whatever order the directory hands back. The Python documentation says so directly for glob: "The pathnames are returned in no particular order. If you need a specific order, sort the results." It also says that whether the results are sorted "depends on the file system", so a script that happens to come out right on one disk can come out wrong on another.

sorted() fixes the randomness but not the numbering. It compares code points, the way the C locale does, so it lands on exactly the C order: clip_1, clip_10, clip_11, clip_12, clip_2. The natural line uses a small key function that splits each name into text and number pieces and compares the numbers as numbers. That is the one Python line here that produced 1 to 12.

Three fixes, and the one I would use

#!/bin/bash
cd clips
echo "== ls -v, en_US.UTF-8"; LC_ALL=en_US.UTF-8 ls -v | tr '\n' ' '; echo
echo "== sort -V";            ls | sort -V | tr '\n' ' '; echo
# rename to two digits, then every tool agrees
for f in clip_?.mp4; do n=${f#clip_}; mv "$f" "clip_0$n"; done
echo "== padded, C";          LC_ALL=C ls | tr '\n' ' '; echo
echo "== padded, en_US.UTF-8"; LC_ALL=en_US.UTF-8 ls | tr '\n' ' '; echo
python3 -c 'import os; print("== padded, python sorted:", " ".join(sorted(os.listdir("."))))'
Enter fullscreen mode Exit fullscreen mode
== ls -v, en_US.UTF-8
clip_1.mp4 clip_2.mp4 clip_3.mp4 clip_4.mp4 clip_5.mp4 clip_6.mp4 clip_7.mp4 clip_8.mp4 clip_9.mp4 clip_10.mp4 clip_11.mp4 clip_12.mp4 
== sort -V
clip_1.mp4 clip_2.mp4 clip_3.mp4 clip_4.mp4 clip_5.mp4 clip_6.mp4 clip_7.mp4 clip_8.mp4 clip_9.mp4 clip_10.mp4 clip_11.mp4 clip_12.mp4 
== padded, C
clip_01.mp4 clip_02.mp4 clip_03.mp4 clip_04.mp4 clip_05.mp4 clip_06.mp4 clip_07.mp4 clip_08.mp4 clip_09.mp4 clip_10.mp4 clip_11.mp4 clip_12.mp4 
== padded, en_US.UTF-8
clip_01.mp4 clip_02.mp4 clip_03.mp4 clip_04.mp4 clip_05.mp4 clip_06.mp4 clip_07.mp4 clip_08.mp4 clip_09.mp4 clip_10.mp4 clip_11.mp4 clip_12.mp4 
== padded, python sorted: clip_01.mp4 clip_02.mp4 clip_03.mp4 clip_04.mp4 clip_05.mp4 clip_06.mp4 clip_07.mp4 clip_08.mp4 clip_09.mp4 clip_10.mp4 clip_11.mp4 clip_12.mp4
Enter fullscreen mode Exit fullscreen mode

ls -v and sort -V both gave 1 to 12, and both say the same thing about themselves in their help text: "natural sort of (version) numbers within text". They work, but they only work when the tool reading the folder is told to use them. A file manager, a photo viewer, or someone else's script will not know.

Padding the number to a fixed width moves the fix into the names. Once every name has the same number of digits, plain text order and numeric order are the same thing, so C, en_US.UTF-8 and Python's sorted() all returned 01 to 12. The rename loop above only handles single digits, because that is all twelve clips need. If a folder can reach a hundred clips, the padding has to be three digits from the first clip on, not added once the count rolls over.

Dates in names have the same problem, sooner

Many recorders name clips by date and time instead of a counter. The test below creates four clips recorded across one New Year's Eve, from 11:50 PM to 9:15 AM, named once in month-day-year with a 12-hour clock and once in year-month-day with a 24-hour clock:

#!/bin/bash
# the same four clips, named three ways
rm -rf dated && mkdir dated && cd dated
touch "12-31-2025 11-50 PM.mp4" "01-01-2026 12-10 AM.mp4" "01-01-2026 01-30 AM.mp4" "01-01-2026 09-15 AM.mp4"
echo "== month-day-year, 12-hour"; LC_ALL=C ls | sed 's/^/   /'
rm ./*
touch 2025-12-31_2350.mp4 2026-01-01_0010.mp4 2026-01-01_0130.mp4 2026-01-01_0915.mp4
echo "== year-month-day, 24-hour, C";     LC_ALL=C ls | tr '\n' ' '; echo
echo "== year-month-day, 24-hour, en_US"; LC_ALL=en_US.UTF-8 ls | tr '\n' ' '; echo
Enter fullscreen mode Exit fullscreen mode
== month-day-year, 12-hour
   01-01-2026 01-30 AM.mp4
   01-01-2026 09-15 AM.mp4
   01-01-2026 12-10 AM.mp4
   12-31-2025 11-50 PM.mp4
== year-month-day, 24-hour, C
2025-12-31_2350.mp4 2026-01-01_0010.mp4 2026-01-01_0130.mp4 2026-01-01_0915.mp4 
== year-month-day, 24-hour, en_US
2025-12-31_2350.mp4 2026-01-01_0010.mp4 2026-01-01_0130.mp4 2026-01-01_0915.mp4 
Enter fullscreen mode Exit fullscreen mode

The month-first names came back scrambled. The first clip recorded, the last one of the old year, sorted last, because 12 beats 01. Within the new year, 12-10 AM sorted after 09-15 AM, even though ten past midnight was the first clip of the new year. Year-month-day with a 24-hour clock sorted correctly in both locales, because every field runs from the largest unit to the smallest and every field has a fixed width.

That rule has one known hole. A name that records local time can repeat when clocks fall back, which I wrote up separately in the hour that happens twice. Sorting by name stays reliable only if the clock writing the names does not run backwards.

What I would check before trusting a folder's order

  1. Look at the tenth file, not the first. Single-digit names hide the problem until the counter reaches two digits.
  2. Run the listing twice, once with LC_ALL=C and once without. If the two disagree, the order depends on the machine.
  3. Sort every Python listdir or glob result before anything reads it in order, and use a natural key if the names carry unpadded numbers.
  4. Prefer names that sort as plain text: fixed-width numbers, year first, 24-hour time.

Things I did not test

  • Phone and desktop file managers. Many of them do their own natural sort, and I have not checked which ones, so I do not know how they would show the unpadded folder.
  • Other platforms. Everything here ran on one Linux machine with glibc locale tables. The FAQ's point is that those tables belong to the system, so other systems may well differ.
  • Locales other than C and en_US.UTF-8. Those were the two installed in my test environment.

If you want a phone that records in the background so there is a folder of clips to sort, the app I make is Background Camera RemoteStream on Google Play. The scripts above do not depend on it.

A folder that is only in order on your own machine is not in order yet.

Sources. Quotes are from the GNU Core Utilities FAQ entry "Sort does not sort in normal order!", the Python glob documentation, and the --help output of ls and sort from GNU coreutils 8.32. Test results come from the scripts above, run on one Linux machine with bash, GNU coreutils 8.32 and Python 3.10.

Top comments (0)