How macOS, Linux, and Windows detect file changes (and why it's hard to catch them)

#go #programming #testing #development

File watching seems simple on the surface: "tell me when a file changes" But the reality is three completely different APIs with three fundamentally different philosophies about how computers should monitor file systems.

The three main approaches

I recently spent weeks building FSWatcher, a cross-platform file watcher in Go, and the journey taught me that understanding file watching means understanding how each operating system approaches the problems differently.

🍎 macOS: FSEvents (Directory-first)

Apple's philosophy is to monitor directory trees, rather than individual files.

// You say: "watch /Users/me/fswatcher"
// macOS says: "OK, I'll tell you when ANYTHING in that tree changes"

Pros

Low CPU usage
Efficient for large directories
Event-driven (no polling)

Cons

Gives you directory-level info, not file-level
Can flood you with redundant events
You must filter what you actually care about

Example event

//Event: /Users/me/project/src changed
// You have to figure out WHAT changed in /src

🐧 Linux: inotify (File-First)

Linux's philosophy: granular control over specific files and directories.

// You say: "watch /home/me/fswatcher/main.go"
// Linux says: "OK, I'll tell you exactly what happens to that file"

Pros

Precise, file-level events
You know exactly what changed
Low-level control

Cons

Each watch = one file descriptor (limited resource)
Easy to hit system limits on large projects
More prone to event flooding

Example event

// Event: /home/me/fswatcher/main.go MODIFIED
// Event: /home/me/fswatcher/main.go ATTRIBUTES_CHANGED
// Event: /home/me/fswatcher/main.go CLOSE_WRITE
// Same file save = 3 events

🪟 Windows: ReadDirectoryChangesW (Async-first)

Windows philosophy: asynchronous I/O with overlapping operations.

// You say: "watch C:\project and give me async notifications"
// Windows says: "I'll buffer changes and notify you asynchronously"

Pros

Fast asynchronous I/O
Efficient buffering
Scales well

Cons

Requires careful buffer management
Can lose events if the buffer overflows
Complex synchronization needed

Example event

// Event: C:\project\main.go MODIFIED (buffered)
// Event: C:\project\test.go CREATED (buffered)
// Events may be batched by Windows

The real challenges

Challenge 1: Event Inconsistency
Same action (save a file) → different events per OS:

// macOS
Event: /project changed (directory-level)

// Linux  
Event: file.go MODIFIED
Event: file.go ATTRIB
Event: file.go CLOSE_WRITE

// Windows
Event: file.go MODIFIED (buffered)

Challenge 2: Editor Spam
Modern editors (VSCode, GoLand) don't just save once:

1. Create temp file (.file.go.tmp)
2. Write content to temp
3. Delete original
4. Rename temp to original
5. Update attributes
6. Flush buffers

That's 6+ events for ONE save operation!

Challenge 3: Bulk Operations
When you run git checkout, thousands of files change instantly:
bashgit checkout main

# 10,000 files changed
# = 10,000+ file system events in ~1 second

Your watcher must handle this flood without crashing.

A unified pipeline to solve inconsistency

In FSWatcher, I built a pipeline that normalizes all these differences:

┌─────────────┐
│ OS Events   │ (platform-specific)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ Normalize   │ (consistent Event struct)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ Debounce    │ (merge rapid duplicates)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ Batch       │ (group related changes)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ Filter      │ (regex include/exclude)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ Clean Event │ (to consumer)
└─────────────┘

Debouncing in action

// Without debouncing:
Event: main.go changed at 10:00:00.100
Event: main.go changed at 10:00:00.150
Event: main.go changed at 10:00:00.200
Event: main.go changed at 10:00:00.250
Event: main.go changed at 10:00:00.300

// With debouncing (300ms window):
Event: main.go changed at 10:00:00.300 (final)

Batching in action

// Without batching:
10 separate event notifications

// With batching:
Batch: []Event{10 events} (one notification)

Real example

Here's a hot-reload system using FSWatcher:

go get github.com/sgtdi/fswatcher

package main

import (
    "context"
    "fmt"

    "github.com/sgtdi/fswatcher"
)

func main() {
    // Only watch .go files and ignore .go files under test dir
    fsw, _ := fswatcher.New(
        fswatcher.WithIncRegex([]string{`\.go$`}),
        fswatcher.WithExcRegex([]string{`test/.*\.go$`}),
    )

    ctx := context.Background()
    go fsw.Watch(ctx)

    fmt.Println("Starting..")
    for e := range fsw.Events() {
        fmt.Println(e.String())
        fmt.Println("Changed..")
    }
}