As it happens, you would like to initialize a git repository with git-lfs (large file storage) functionality to store some really large files. For someone a file of a 10MB size may seem large, for somebody else it would have to be as 1GB in size, but regardless of that, this tutorial explains how to efficiently initialize a repository with git-lfs.
Prerequisites
First, make sure that you have git lfs installed (how-to install for: macOS, Linux). In short, for macOS: brew install git-lfs.
Initializing git-lfs
Assuming that you have not initialized your repository with the "normal" git command yet, run the following commands:
git init
git lfs install
For the sake of the example (to show you some difficulties you may encounter), let's assume that the files that you wish to add to git-lfs are stored in two separate directories (e.g., data and supplementary_materials). The tree of your repository looks somehow like that:
.
├── ...
├── README.md
├── data
│ ├── data1.png
│ ├── data2.zip
│ └── label.txt
├── images
│ └── some_pict.png
└── supplementary_materials
├── first_subdir
│ └── somefile.png
├── second_subdir
│ └── somefile.png
└── third_subdir
└── somefile.zip
First, some basic stuff about synchronizing with your remote repository:
git remote add origin https://github.com/mikbuch/your-repository.git
Then:
git branch -M main
git pull origin main
Now, there are basically two ways of adding files to git-LFS.
The first (recommended) method
Create .gitattributes file:
vim .gitattributes
and add the following lines:
data/**/* filter=lfs diff=lfs merge=lfs -text
supplementary_materials/**/* filter=lfs diff=lfs merge=lfs -text
Note! In our example we have two separate directories that you wish to add to git-lfs in which you have some subdirectories. (1) The syntax has to be exactly like the one in the code above, or else you won't add all files in all subdirectories. (2) If you have several directories, from the point of view of your current working directory, you have to list them all, one by one.
The second (not recommended) method
Use the following commands to add directory and all files in this directory and its subdirectories:
git lfs track data/**/*
git lfs track supplementary_materials/**/*
Note! The command git lfs track supplementary_materials/** will not be sufficient to add all files in all subdirectories.
Why this method is not recommended?: It will list/add all tracked files to your .gitattributes (one by one!), e.g.:
➜ git:(main) ✗ cat .gitattributes
data/data1.png filter=lfs diff=lfs merge=lfs -text
data/data2.zip filter=lfs diff=lfs merge=lfs -text
data/label.txt filter=lfs diff=lfs merge=lfs -text
...
supplementary_materials/first_subdir/somefile.png filter=lfs diff=lfs merge=lfs -text
supplementary_materials/second_subdir/somefile.png filter=lfs diff=lfs merge=lfs -text
supplementary_materials/third_subdir/somefile.zip filter=lfs diff=lfs merge=lfs -text
...
What you would like to have in your .gitattributes is just:
data/**/* filter=lfs diff=lfs merge=lfs -text
supplementary_materials/**/* filter=lfs diff=lfs merge=lfs -text
It is much more clearer approach. See the first (recommended) method described above.
Preview the .gitattributes file
cat .gitattributes
Add all files to your repository:
git add --all
Show which files are tracked with (git lfs ls-files)
git lfs ls-files
Note! git lfs ls-files works only after you have added the files to the repository with git add.
Show which files were added to the repository (regardless of whether LFS or not):
git status
Make sure that the files are tracked with git-LFS:
du -sh .git
du -sh .git/lfs
Note on GitHub
If you are using GitHub, please make sure that you are using the proper SSH file:
cat ~/.ssh/config
In GitHub, also make sure that your key is added to you profile (not just one repository!). See the output of a command:
ssh -T -ai ~/.ssh/my-key.pem git@github.com
Source: https://superuser.com/a/232406/950943
If you used this key for one repository only you will see somethink like this:
Hi mikbuch/your-repository! You've successfully authenticated, but GitHub does not provide shell access.
You should see output like this:
Hi mikbuch! You've successfully authenticated, but GitHub does not provide shell access.
Committing and pushing to the server
Commit the files (both LFS and "normal"):
git commit -a -m "Initializing commit with git LFS"
Finally, you can push your reporitory to the remote server:
git push -u origin main
In order to reset (if needed/optional) during the process:
rm -rf .git
rm .gitattributes
And that's it, I hope you will find this tutorial useful, have a nice day!
Additional resources
045 Introduction to Git LFS (Large File Storage) by Dan Gitschooldude: https://youtu.be/xPFLAAhuGy0
Example use: https://github.com/git-lfs/git-lfs/wiki/Installation#debian-and-ubuntu
Source for the cover photo image: Photo by Wesley Tingey on Unsplash
Top comments (0)