Fix Git Hash Object No Filters CRLF Divergence

Engineers often face checksum mismatches between git hash-object and raw disk bytes when working across platforms.

Quick solution:

git hash-object --no-filters -- file.txt

When developers and continuous integration pipelines verify file integrity or synchronize local assets with remote APIs, they frequently encounter an unexpected problem. The calculated object ID does not match the raw bytes sitting on the local disk or the checksum computed by standard utilities like shasum. This divergence is caused by a fundamental Git architectural feature known as the working tree clean filter.

By default, when you invoke git hash-object <file> on a working-tree file path, Git does not read raw bytes blindly. Instead, Git inspects active repository configurations and rules defined in git attributes to transform content before computing its object identifier. Understanding this behavior is essential when debugging cross-platform build discrepancies between Windows and Linux runners. Before deploying complex pipelines, teams should validate content briefs before AI article generation to ensure consistent technical specifications and align their documentation.

The Clean Filter Pipeline and Repository Identity

Invoking git hash-object <file> initiates the repository clean filter pipeline. Git evaluates core configuration settings such as core.autocrlf and attribute definitions like text=auto. If a text file contains Windows carriage returns (\r\n), Git normalizes them into standard Unix line endings (\n) in memory before calculating the SHA-1 or SHA-256 hash. Consequently, the output represents what the file would look like if committed to the database, rather than the exact state of the working tree.

To understand why this happens, we must examine how Git indexes and stores objects. When a file enters the staging area, Git runs the clean filter to strip platform-specific irregularities, ensuring that a file committed on Windows yields the exact same blob hash as the same logical file committed on Linux or macOS. This guarantees repository identity consistency across distributed teams. However, this normalization creates friction when developers attempt to compare Git's internal object database against local filesystem artifacts that retain their original line endings or file attributes.

For example, when working with shared teams across operating systems, developers often reference documentation like the official Git Hash Object Documentation to understand plumbing behavior. However, standard documentation often glosses over the precise mechanics of how working tree transformations alter the resulting hash value, leaving engineers confused when their build scripts report unexpected checksum failures.

Common Failure Modes: CRLF Endings and Git LFS Pointers

Two distinct scenarios cause severe verification headaches in automated pipelines across mixed environments:

1. CRLF vs. LF Normalization: On Windows environments, files checked out with automatic line ending conversion contain literal carriage returns. Default hashing normalizes these bytes, creating a mismatch against S3 ETags, container registries, or CDN caches that inspect raw disk bytes without applying Git filters.
2. Git LFS Pointer Swapping: Large assets tracked by Git LFS substitute local binary files with small text pointers on disk. Running a standard git hash-object on a tracked asset returns the hash of a 130-byte text pointer instead of the actual media file, leading to severe asset corruption if the pipeline uploads the pointer instead of the payload.

To manage these attribute rules effectively across complex codebases, teams must consult specifications such as the Git Attributes Documentation to verify how filters interact with specific file patterns. Without proper attribute scoping, unexpected filters can silently modify binary payloads or text streams during hashing operations.

Deep Dive Into Working Tree Versus Repository State

When troubleshooting cryptographic mismatches, distinguishing between the repository state and the working tree state is vital. The standard git hash-object command reads from the working tree by default, but it applies the clean mechanism unless explicitly told otherwise. This means the command is not simply reading bytes from disk; it is simulating what Git would store if the file were added to the index right now.

If your local repository has configuration settings such as core.safecrlf enabled, or if custom filter drivers are registered in your local environment, these components inject themselves into the hashing pipeline. As noted in discussions such as CoderLegion reader feedback by lisarconnect (Comment #a31678), engineers frequently lose hours trying to reconcile S3 storage checksums with Git object IDs because they assume that reading a file from the working directory implies reading its raw disk representation. By passing the --no-filters flag, you instruct Git to bypass the clean filter entirely, reading the exact stream of bytes currently residing on the storage medium.

Architectural Decision Matrix

Choosing the correct command depends entirely on whether your verification target is the repository object database or the raw filesystem artifact.

Command	Target Domain	Behavior & Filters Applied
git hash-object <file>	Repository Object Parity	Applies clean filters, normalizes CRLF to LF, and converts LFS binaries to text pointers.
git hash-object --no-filters -- <file>	Working Tree Disk Parity	Bypasses all attributes, reading exact bit-for-bit bytes from the local filesystem.
`{ printf 'blob %d\0' ...; cat; }	shasum`	Zero-Dependency Runtime Parity

Verification Checklist

Execute these steps to verify filter divergence and confirm disk byte parity:

1. Initialize a Test Repository: Create a temporary directory, initialize a Git repository, and define an end-of-line attribute rule in a .gitattributes file.
2. Create a CRLF File: Write a test file containing Windows carriage returns (\r\n) using a local script or text editor.
3. Compare Filtered and Raw Hashes: Run both git hash-object test.txt and git hash-object --no-filters -- test.txt to confirm that the outputs diverge.
4. Validate Stream Parity: Pipe the file contents through a POSIX header format command and verify that the resulting checksum matches the unfiltered Git hash exactly.