Copying something is easy. Copying it in a way that still knows where it came from is the part worth learning. Day 23 was two copies of exactly that kind, one on a Git server and one between two S3 buckets, and both have a smarter option sitting next to the obvious one.
One Git task, one AWS task. Fork a repository and connect it back to the original, then migrate the contents of one S3 bucket into another. The tasks come from the KodeKloud Engineer platform.
Forking: the Git operation that isn't a Git command
Here is what surprised me. There is no git fork. Type it, and Git tells you it is not a command, because forking does not happen on your machine at all. It is a feature of the Git server, Gitea or GitHub or GitLab, which makes a full copy of a repository under a different owner and quietly records where that copy came from.
So the fork happens in a browser. What happens afterwards is where the Git work starts:
# Clone YOUR fork, not the original
git clone http://git.example.com/<your-user>/<repo>.git
cd <repo>
# origin points at your copy
git remote -v
At this point you have a copy that has already forgotten its parent, as far as your local repo is concerned. One remote, pointing at you. The fix is to add the original back by hand:
# Add the original repository as a second remote
git remote add upstream http://git.example.com/<original-owner>/<repo>.git
# Pull new work from the original into your fork
git fetch upstream
git merge upstream/master
git push origin master
origin and upstream are just names, not Git keywords, but the convention is worth following because everyone uses it: origin is your copy, the one you can push to, and upstream is the original, which you can usually only read from.
Three words get muddled constantly here, so it is worth being blunt about them. A fork is a copy on the server under a new owner. A clone is a copy on your local disk. A branch is a pointer inside one repository. Only the fork changes who owns the code, and that is exactly why it exists: you cannot push to a repository you do not own, so you fork it, push to your own copy, and ask the owner to pull your work in. That is the entire pull request model on every public project you have ever contributed to.
The trap is that a fork is a snapshot of one moment. It does not track the original. Leave it alone for a month, and it drifts behind, and you will find out when your pull request no longer applies cleanly.
S3 migration: sync knows what it already did
The AWS task was moving the contents of one bucket into another. The obvious command works:
aws s3 cp s3://source-bucket s3://destination-bucket --recursive
The better one is barely different to type and behaves nothing alike:
# Rehearse first — transfers nothing, prints everything it would do
aws s3 sync s3://source-bucket s3://destination-bucket --dryrun
aws s3 sync s3://source-bucket s3://destination-bucket
cp --recursive copies everything, every time, whether it changed or not. sync compares each object's size and last-modified time and transfers only the differences. On a small bucket, you will not notice. On a large one, that is the difference between re-running a failed transfer in seconds and starting the whole thing again from zero.
Two details make this better than it first looks. The transfer is server-side: S3 copies bucket to bucket inside AWS, so the objects never travel down to your terminal and back up again. Your laptop only issues API calls, which means a slow connection does not slow the migration. And because sync is incremental, it is safe to run repeatedly, which is what makes it usable as a scheduled job rather than a one-off.
Two flags deserve respect. --delete turns a copy into a mirror, removing anything in the destination that has no counterpart in the source. And aws s3 mv deletes from the source once each object lands, which without versioning enabled is not recoverable. Both are the right tool sometimes. Both should meet --dryrun first.
One thing to note is that objects arrive in the destination bucket's default storage class unless you pass --storage-class. If your source was sitting in something cheaper, a naive sync promotes the whole lot back to Standard, and you find out on the bill.
Copies that remember
Both tasks had a naive version that works and a better version that keeps context. Cloning a fork without adding upstream gives you code with no way back to its source. Copying a bucket with cp --recursive gives you data with no memory of what was already transferred. In both cases, the smarter option costs one extra line and saves you the moment where you realise you have lost the thread.
So here is the Day 23 question. Would you rather make a copy that stands alone and hope you never need to reconcile it, or spend one extra command keeping the link to where it came from?
Day 23 down. Seventy-seven to go
Top comments (0)