DEV Community: Jing Xue

How to manage your local changes with git

Jing Xue — Sun, 30 Jan 2022 05:32:23 +0000

When we are working a project, often we need to make changes specific only to the local development environment we are working in. For example, point to some development or even personal database, or plug in our own credentials. Those are truly local only changes that we don't want to push to the remote repo or merge to any team level branches. On the other hand, we don't want to have to manually make these changes every time when we start working on a new feature branch, or even when we have just pulled from the remote repo.

That is not a problem if the changes can be isolated to a local-only small file, or externalized into environment variables, because we can simply ignore the local configuration file in git.

However, when circumstances don't allow local changes to be isolated or ignored, e.g. when it's a couple lines of changes in a large config file that's always tracked, or the project does not have the infrastructure to support environment specific configuration files, here is the workflow I often use:

First I create the feature branch that will contain the commits implementing the feature, and will be pushed to the remote repo:

$ git checkout -b my-feature

Then I create the local branch I will actually be working on. It will always be the feature branch plus the commit containing my local changes.

$ git checkout -b my-feature-local

Assuming A is the current tip of master, so this is the commit graph we are starting with:
A(master,my-feature,my-feature-local,HEAD)

I make whatever local changes necessary.

$ git commit -am "LOCAL ONLY, DO NOT MERGE"

Now the graph looks like (where L is the "LOCAL ONLY" commit):
A(master,my-feature) - L(my-feature-local,HEAD)

I start hacking away at the actual task, to a good saving point. I may have created a couple more commits, B and C, containing actual coding for the new feature by now.
A(master,my-feature) - L - B - C(my-feature-local,HEAD)

I'm ready to push my work to the remote repo. But I don't want to push my local changes, so I do:

$ git rebase -i my-feature

During the interactive rebase, I move the "LOCAL ONLY" commit to the top, so now the graph looks like:
A(master,my-feature) - B - C - L(my-feature-local,HEAD)

Now I have moved the commits for implementing the feature to before the local commit, and onto the my-feature branch from which I'll actually do the push:

$ git rebase my-feature-local~ my-feature

The graph at this point:
A(master) - B - C(my-feature,HEAD) - L(my-feature-local)

I'm ready to push:

$ git push -u my-feature
$ git checkout my-feature-local

Now I'm back on my local branch. After I make some more commits, I just repeat the rebase-and-push process above.

When I am working with other people on a feature, and we all push code to the my-feature branch, there is just one extra step after every time I have pulled my-feature from the remote repo:

$ git rebase my-feature my-feature-local

At the cost of a couple of extra steps, this workflow has some advantages, compared to manual changes and/or git-ignore the files locally changed:

Changes, even though local, are still tracked and controlled properly by git. If the same files are changed as part of the implementation, by you or the rest of the team, git will merge them with your local changes during the rebase.
No need to make the same manual changes again and again. Simply create a "local" branch on top of whichever branch you need to work on, cherry pick the "local only" commit over, and you are good to go.
Local changes don't have to be limited to one commit. They can be organized into a series of "local" commits and allow more advanced use cases by manipulating the series in git.

Why You Don't Want to Use the S3 Referer Only Policy

Jing Xue — Tue, 15 Dec 2020 19:02:50 +0000

When you have an S3 bucket configured as a static web site, S3 allows you to restrict access to the objects in the bucket only when the requests came from a particular referrer, using a condition in the bucket policy. For instance, in cases where you want visitors to get to some images only via links on your own web site.

Well, the problem is, the "referrer" comes in the HTTP request header, so a client can fake any referrer and easily bypass this condition. For example, with curl, it's as easy as --referer http://url.to.the.referer.page.

I wish AWS would at least put up a warning next to the sample policy (linked to above), because it really just adds some false sense of security.

My Tags Missing Upon Going Back to dev.to Front Page

Jing Xue — Wed, 29 Jul 2020 23:12:57 +0000

when I load up the dev.to front page, my tags show up correctly:

https://imgur.com/a/jRBETVe

I click into an article, read it, and come back out, my tags are gone:

https://imgur.com/a/jia09RY

If I reload the page, they are shown correctly again.

Anybody else having this issue?

(Also inline image in the post editor apparently doesn't work for me for some reason. When I put the image links above into the inline image tag like ![my tags](https://imgur.com/a/jRBETVe\), I get broken links to res.cloudinary.com.)

List to Tuple: a Little Compilation Optimization

Jing Xue — Sat, 04 Jul 2020 23:17:16 +0000

I was reading Julien Maury's Python pro tips, and ran into this code snippet:

if myvar in [9,1,7]

I was about to leave a comment to suggest using a tuple instead, because instinctively you would think tuples should be faster than lists, right?

if myvar in (9,1,7)

And then I decided not to trust my instinct and take a look at the bytecode first. Well, what do you know... both code compiles to exactly the same:

0 LOAD_FAST                0 (myvar)
2 LOAD_CONST               1 ((9, 1, 7))
4 COMPARE_OP               6 (in)

So the compiler is smart enough to realize that even though I am creating a mutable list, I won't actually make any changes to it, and will use it only once, so it converts the list to a tuple.

Taking one step further, I changed the code to:

l = [9, 1, 7]
if myvar in l

Now the compiler creates a list:

0 LOAD_CONST               1 (9)
2 LOAD_CONST               2 (1)
4 LOAD_CONST               3 (7)
6 BUILD_LIST               3
8 STORE_FAST               1 (l)

10 LOAD_FAST                0 (myvar)
12 LOAD_FAST                1 (l)
14 COMPARE_OP               6 (in)

Although I suppose one could argue that in this case the compiler should still have detected that l is never changed, and converted it to a tuple.

Missing 'distutils.util' after upgrading from Python 3.7 to 3.8

Jing Xue — Mon, 25 May 2020 21:08:11 +0000

I recently upgraded to Pop OS(essentially Ubuntu) 20.04, which had Python upgraded to 3.8. My ongoing AWS Lambda project build is broken right away:

Running PythonPipBuilder:ResolveDependencies

Build Failed
Error: PythonPipBuilder:ResolveDependencies - Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/pip/_internal/main.py", line 14, in main
    from pip._internal.utils.entrypoints import _wrapper
  File "/usr/lib/python3/dist-packages/pip/_internal/utils/entrypoints.py", line 3, in <module>
    from pip._internal.cli.main import main
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/main_parser.py", line 7, in <module>
    from pip._internal.cli import cmdoptions
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/cmdoptions.py", line 19, in <module>
    from distutils.util import strtobool
ModuleNotFoundError: No module named 'distutils.util'

All the solutions from googling the error message seems to be "make sure to run 'apt sudo install python3-distutils'. However, I do have python3-distutils installed under /usr/lib/python3.8.

I noticed that the traceback above was produced by pip from python3/dist-packages, however I do have a later version of pip installed under my .local/lib/python3.8/site-packages. So why did that not get picked up? The two symptoms together made me think that when SAM runs pip, it's not using the Python 3.8 search path which would contain all the 3.8 packages.

After a couple of hours of pouring through SAM source code and debugging, it turns out SAM was using python 3.7 to build my project, including running pip in, because when I upgraded I only remembered to update Pipfile to use '3.8', but (completely) forgot to update the '3.7' references in the CloudFormation template file.

While I should obviously take the bulk of the blame for this silly trip, it would have helped if SAM could somehow print something more obvious then the error message I got.