MacOS vs Linux — the cp command will trip you up!

While building www.firecode.io, I was recently confronted with more than a hundred failing unit tests when running the test suite locally on my MacOS machine, rather than in a Linux Docker container the platform is configured to run tests in.

I use a MacBook Pro for software development, primarily because the BSD based MacOS provides a development experience that’s very similar to other Linux distros. All of my most frequently used navigation commands — cd, cp, mv, mkdir, touch, etc...work with 1:1 parity across both flavors of operating systems, or at least that’s what I thought. As I discovered after spending a good hour debugging these failures, there’s a subtle difference between the BSD and GNU implementations of the cp -R command, which I got hit with when running my tests on different environments.

Here’s the difference, illustrated with an example run in both MacOS and Linux:

# Creates "source_directory" in the current working directory
$ mkdir source_directory

# Creates 2 new empty files within "source_directory"
$ touch source_directory/{file1,file2}

# Creates "destination_directory" in the current working directory
$ mkdir destination_directory

# Intended to copy the contents of "source_directory" to "destination_directory", hence the trailing slashes
$ cp -R source_directory/ destination_directory/

# Lists the contents of "destination_directory"
$ ls destination_directory

So what would you expect to be printed with ls? Turns out the Linux GNU implementation of cp copies the source_directory directory to destination_directory, whereas on BSD MacOS the contents are unpacked and copied, as I’d expected it to behave on both environments:

GNU (Linux):

$ ls destination_directory
source_directory

BSD (MacOS):

$ ls destination_directory
file1 file2

The trailing slash is significant in BSD, whereas the GNU implementation treats both source_directory/ and source_directory the same. The workaround, thankfully, is really simple — append a period when you intend for the contents to be copied and you’ll see the same behavior on both BSD and GNU : cp -R source_directory/. destination_directory/.

I hope this tidbit helps you write better cross environment code on MacOS and helps you save some time debugging unexpected results.

Comments

[Jason C. McDonald]

And, see, the Linux behavior is what I'd expect. If I ask for a directory itself to be copied, I name only the directory. If I want it unpacked, I glob it as such:

cp -R source_directory/* destination_directory/

(Yeah, okay, I use globs instead of . Same diff in this case.)

Copy really needs to always behave as "I want this thing in that place". The BSD way is actually more surprising when you least expect it...and the surprise (unpacking the files) is more dangerous than if Linux's behavior surprises you! Just consider this...

cp -R my_vacation_pics_with_lousy_names/ pictures_to_sort/

Well, crud. On Mac, now you have just crammed a bunch of garble-named photos into an already crowded folder, and lost the one thing that you had going for you: the containment of the source directory. That's non-trivial to undo. Meanwhile, on Linux, you can find pictures_to_sort/my_vacation_pics_with_lousy_names/ and go from there. Much better.

Yet the reverse surprise, not unpacking when you expect it to, is easily remedied with a single command.

Surprises are bad, hard-to-undo surprises even more so.

[Yifei Kong]

One thing that makes GNU cp a surprise is the inconsistent behavior of cp -R . ../foo, the source is clearly a directory, however, this time the contents are copied.

[Ben Sinclair]

Something I mentioned in a different comment, rsync behaves the way cp does on a Mac. So for people used to rsync, it's not a surprise, and GNU cp is the surprise!

I think it's telling though that there are so many articles about rsync warning you you can screw up if you don't get the slashes right - the majority of people over the last couple of decades have been used to doing it the GNU way.

[Jason C. McDonald]

There's also a reason rsync behaves that way, and it too is something I'd expect. You sync source-directory to destination-directory. As with any other form of syncing, it would be expected that you'd have two folders which are made to match one another.

"Copy X to Y", find X in Y.
"Sync X and Y", find X and Y are identical.

Simple.

[Paweł Kowalski]

One thing that wasted me a lot of time: macOS filenames are not case sensitive. So on linux you can have "file and "File" next to each other, on macOS you are going to have a bad time.

$ touch TEST
$ ls
TEST
$ touch test
$ ls
TEST

I would never in million years figure it out if i didnt encounter it myself. Couldn't believe this :x

[Ben Sinclair]

That's why I don't really like these shells that do tab-completion in a "smart" case way.

It's actually worse than this. Macs keep metadata in files with... the same names as the original. Or at least they did up until a couple of years ago. That used to mean that when someone sent an email with an attachment to someone on a non-Mac, their naive client had a 50/50 chance of detaching the correct file or a zero-byte piece of nonsense.

Don't get me started on __MACOSX directories and the fact that they end up in zip files and repositories and and and...

[Fernando B 🚀]

If you use mac and Linux I'd recommend to save you some pain and just install gnu coreutils on mac.

[Ben Sinclair]

It never occurred to me that BSD cp would work that way. It's interesting that rsync - which is GNU licensed even if it's not part of GNU utils - behaves the way BSD cp works.