DEV Community

Siddhartha Varma
Siddhartha Varma

Posted on

Create a folder and push multiple files under a single commit through GitHub API

#javascript #github #git #api

Intro

I could not understand how to push multiple files under a single commit, using GitHub API. Some googling landed me on Git Database API Docs. Wasn't very clear how it worked so I tried getting my hands dirty! Documenting it here for anyone who's looking for it.

Disclosure:

  1. This API needs atleast 1 commit to work. Empty repositories don't work.

Git Database API

The Git Database API enables you to read and write raw Git objects to your Git database on GitHub and to list and update Git references (branch heads and tags).

The whole process is as follows: create blobs corresponding to files, create tree, create a commit for the changes and finally updating refs to reflect the commit.

Image description

Creating blobs

A Git blob (binary large object) is the object type used to store the contents of each file in a repository. The file's SHA-1 hash is computed and stored in the blob object.

To create a blob we need to hit the following endpoint:



POST https://api.github.com/repos/{user}/{repo}/git/blobs
Enter fullscreen mode Exit fullscreen mode

Let's create a blob for a file called main.py with 1 line of python code:



{
    "content":"print(\"hello world !\")",
    "encoding": "utf-8"
}
Enter fullscreen mode Exit fullscreen mode

We can create blobs for each file we want to upload. In the response, we get SHA of the blob which we need to send while creating the tree. Sample response:



{
    "sha": "638eff25696b982124deeb1f3dfcceabfdc81a93",
    "url": "https://api.github.com/repos/BRO3886/git-db-example/git/blobs/638eff25696b982124deeb1f3dfcceabfdc81a93"
}
Enter fullscreen mode Exit fullscreen mode

Creating a tree

Now, we need to create a tree with the files we added. For that, first we need to get the SHA of the base tree. Then we need to create a tree for the files we need to add:

Get SHA for base_tree

The base tree can be thought of as the folder where you'd like to create your commit. If you want to create it at the root folder, you can make request to:



GET https://api.github.com/repos/{user}/{repo}/git/trees/{branch}
Enter fullscreen mode Exit fullscreen mode

It returns a response like:



{
    "sha": "0d43a3b20104b4baa402c09a6c9c6c3298390e4a",
    "url": "{url of tree}",
    "tree": [
        {
            "path": "App",
            "mode": "040000",
            "type": "tree",
            "sha": "{sha}",
            "url": "{url of folder/tree}"
        },
        {
            "path": "README.md",
            "mode": "100644",
            "type": "blob",
            "sha": "{some SHA}",
            "size": 885,
            "url": "{some URL}"
        },
        ...
    ],
    "truncated": false
}
Enter fullscreen mode Exit fullscreen mode

Here, we can extract the SHA from response.sha or SHA of the folder under response.tree[i].sha.

Create tree

Then, we need to hit the create tree api:



POST https://api.github.com/repos/{user}/{repo}/git/trees
Enter fullscreen mode Exit fullscreen mode

With the body as:



{
    "tree":[
        {
            "path":"helloworld/main.py",
            "mode":"100644",
            "type":"blob",
            "sha":"638eff25696b982124deeb1f3dfcceabfdc81a93"
        },
        {
            "path":"helloworld/main2.py",
            "mode":"100644",
            "type":"blob",
            "sha":"638eff25696b982124deeb1f3dfcceabfdc81a93"
        }
        ...
    ],
    "base_tree":"3c408bafa55eda6b1c51de5df0fc36304f37414c"
}
Enter fullscreen mode Exit fullscreen mode

Here request.tree would have an array of blobs that you want to push. For each blob we need the SHA that we got as a response from create blob API.

For the mode:
The file mode; one of 100644 for file (blob), 100755 for executable (blob), 040000 for subdirectory (tree), 160000 for submodule (commit), or 120000 for a blob that specifies the path of a symlink.

GitHub doesn't store folders, so the only way to make a folder is to set the path for the blob as {folderName}/{fileName}

Here, we get another SHA in the response, which we need when we create a commit corresponding to the changes:



{
    "sha": "a69117177bb067933189072b2b8799c63f388f32",
    "url": "https://api.github.com/repos/BRO3886/git-db-example/git/trees/a69117177bb067933189072b2b8799c63f388f32",
    "tree": [
        {
            "path": "README.md",
            "mode": "100644",
            "type": "blob",
            "sha": "bc7b1321063b4075c97bf16e6f8130b6f9fa6537",
            "size": 54,
            "url": "https://api.github.com/repos/BRO3886/git-db-example/git/blobs/bc7b1321063b4075c97bf16e6f8130b6f9fa6537"
        },
        {
            "path": "helloworld",
            "mode": "040000",
            "type": "tree",
            "sha": "82a82f6788b44fe93774597ff2e76ac66ae1e657",
            "url": "https://api.github.com/repos/BRO3886/git-db-example/git/trees/82a82f6788b44fe93774597ff2e76ac66ae1e657"
        }
    ],
    "truncated": false
}
Enter fullscreen mode Exit fullscreen mode

Here we need response.sha for the next step.

Add a commit

This step is pretty simple. We just need the response.sha from the previous step. We make a request to the following endpoint:



POST https://api.github.com/repos/{user}/{repo}/git/commits
Enter fullscreen mode Exit fullscreen mode

Along with the body:



{
    "tree":"a69117177bb067933189072b2b8799c63f388f32",
    "message":"some commit msg",
    "parents": ["3c408bafa55eda6b1c51de5df0fc36304f37414c"]
}
Enter fullscreen mode Exit fullscreen mode

parents: The SHAs of the commits that were the parents of this commit. If omitted or empty, the commit will be written as a root commit. For a single parent, an array of one SHA should be provided; for a merge commit, an array of more than one should be provided.

In our case, for the parent, since we want to add another commit on a particular branch, we need to get the SHA of it.

For that, we need to make a request to the following endpoint:



GET https://api.github.com/repos/BRO3886/git-db-example/git/refs/heads/{branch}
Enter fullscreen mode Exit fullscreen mode

It returns a JSON with details about the branch:



{
    "ref": "refs/heads/main",
    "node_id": "REF_kwDOG87gc69yZWZzL2hlYWRzL21haW4",
    "url": "https://api.github.com/repos/BRO3886/git-db-example/git/refs/heads/main",
    "object": {
        "sha": "3c408bafa55eda6b1c51de5df0fc36304f37414c",
        "type": "commit",
        "url": "https://api.github.com/repos/BRO3886/git-db-example/git/commits/3c408bafa55eda6b1c51de5df0fc36304f37414c"
    }
}
Enter fullscreen mode Exit fullscreen mode

For the parent SHA, we need response.object.sha.

In the response of create commit API, we'll get another SHA:



{
    "sha": "544aa83c4d4a784c4c8490d6548c248b0e57d0ac",
    "node_id": "C_kwDOG87gc9oAKDU0NGFhODNjNGQ0YTc4NGM0Yzg0OTBkNjU0OGMyNDhiMGU1N2QwYWM",
    "url": "https://api.github.com/repos/BRO3886/git-db-example/git/commits/544aa83c4d4a784c4c8490d6548c248b0e57d0ac",
    "html_url": "https://github.com/BRO3886/git-db-example/commit/544aa83c4d4a784c4c8490d6548c248b0e57d0ac",
...
}
Enter fullscreen mode Exit fullscreen mode

We'll need this SHA value for the last step.

Updating ref

This step is for updating the ref from where we pulled the SHA under parents (create commit step). For my case, since the ref was main, I'll update it. Make a request to the following endpoint:



PATCH https://api.github.com/repos/{user}/{repo}/git/refs/heads/{branch}
Enter fullscreen mode Exit fullscreen mode

Along with the body:



{
    "sha":"544aa83c4d4a784c4c8490d6548c248b0e57d0ac"
}
Enter fullscreen mode Exit fullscreen mode

If in the response the object.sha is same as the one sent as a part of the request, along with status code 200, that means your changes will be reflected on GitHub.

I hope you got some idea. Here is the GitHub link of the sample repo you can refer: github.com/BRO3886/git-db-example

Top comments (6)

Collapse
 
spitf1re profile image
sPITf1re-dev •

Hi and thanks a lot for such a cool guide. It saved me tons of time. I have a question.
This part:
"Get SHA for base_tree"
you received: "sha": "0d43a3b20104b4baa402c09a6c9c6c3298390e4a",

Then you create a tree and use this:
"base_tree":"3c408bafa55eda6b1c51de5df0fc36304f37414c"

which is not what you got in the previous step. But this is the SHA you're obtaining in the next step as the latest commit SHA of a needed branch.

Which one I should actually use as a base tree SHA?
I guess that it's a typo (because you don't use base_tree SHA after obtaining) but not sure.

Please advice, thanks in advance.

Collapse
 
spitf1re profile image
sPITf1re-dev •

UPD: Actually, I did it assuming that there was a typo in your guide and it worked well to me. So I think it is a typo

Collapse
 
bro3886 profile image
Siddhartha Varma •

Hey, yes seems like a typo, thanks for pointing out. Or, I might have wanted to show you can use the sha for any of the folders present in the tree. Basically:

tree
|
|-some_folder
|-{newfile}
Enter fullscreen mode Exit fullscreen mode

For above case use sha of base tree

tree
|
|-some_folder
|   |-{newfile}
|-file.py
Enter fullscreen mode Exit fullscreen mode

For above case, use the sha of some_folder. And this is recursive so you can traverse the folder to get the sha of subfolder.

Hope this answers your query. Sorry I'm a tad bit late in answering...

Collapse
 
rbkwin profile image
Rohit •

Hi Siddhartha,
Thanks for such a good explanation, I need some input in one issue I am facing.
I created one tree under my branch with one file inside tree(folder/file).
structure is like -- branch/folder/file.
Now I want add another tree(folder1/file1) in above tree, but when I commit these thing I am not getting new tree added in above tree, I used all configurations correctly. result should be like (branch/folder{file}/folder1/file1)
resulting structure is like branch{file}/folder1/file1 (also it has file just below branch) one folder got removed i.e. : "folder"
Can you advice.

Collapse
 
njfamirm profile image
S. Amir Mohammad Najafi •

thanks! i want to use them to push multi file into large repository in github action

Collapse
 
thilllon profile image
thilllon •

the last part, updating ref, saved my life. thx!

Read next

lithephp profile image

Creating a TodoList with PHP and the Lithe Framework: A Complete Guide

Lithe -

amitkumar13 profile image

Memory Leak Detection in React Native with LeakCanary

Amit Kumar -

whiskeygerman profile image

Web canvas with generative neural networks with elements of visual programming.

German Whiskey -

siisee11 profile image

Langgraph - Human In the Loop

Jaeyoun Nam -