DEV Community: David MacEachern

How do you work with someone else's fork?

David MacEachern — Wed, 10 Feb 2021 13:34:51 +0000

This might be useful if you want to work with a fork that someone else has started. I found a solution on Stackoverflow which described how to get their work into a branch that can be worked with. [1]

They had forked the project we were talking about on Github, and in their local they checked out a branch of their own to work on, done some awesome work, and pushed their branch back to their fork on Github.

Step 1: Fork our own repository

We can click on the fork button to make a repository of our own that we can work with.

Step 2: Clone our own repository to our local machine


 bash
wsl@DESKTOP-DCRMGFD:~$ git clone git@github.com:davidmaceachern/rocket-lamb.git
Cloning into 'rocket-lamb'...
remote: Enumerating objects: 206, done.
remote: Total 206 (delta 0), reused 0 (delta 0), pack-reused 206
Receiving objects: 100% (206/206), 45.74 KiB | 564.00 KiB/s, done.
Resolving deltas: 100% (120/120), done.

Step 3: Get the work that the other person has started into our fork

I ran the following:


 bash
$ cd rocket-lamb
$ git remote add other git@github.com:dbanty/rocket-lamb.git
$ git fetch other
$ git checkout -b add-other-changes
$ git pull other rocket-0.5

The output looked like:


 bash
wsl@DESKTOP-DCRMGFD:~$ cd rocket-lamb
wsl@DESKTOP-DCRMGFD:~/rocket-lamb$ git remote add other git@github.com:dbanty/rocket-lamb.git
wsl@DESKTOP-DCRMGFD:~/rocket-lamb$ git fetch other
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 15 (delta 9), reused 15 (delta 9), pack-reused 0
Unpacking objects: 100% (15/15), 3.58 KiB | 215.00 KiB/s, done.
From github.com:dbanty/rocket-lamb
 * [new branch]      master     -> other/master
 * [new branch]      rocket-0.5 -> other/rocket-0.5
wsl@DESKTOP-DCRMGFD:~/rocket-lamb$ git checkout -b add-other-changes
 pull other rocket-0.5Switched to a new branch 'add-other-changes'
wsl@DESKTOP-DCRMGFD:~/rocket-lamb$ git pull other rocket-0.5
From github.com:dbanty/rocket-lamb
 * branch            rocket-0.5 -> FETCH_HEAD
Updating 6b28c56..6a995b1
Fast-forward
 Cargo.toml            |   9 +++--
 src/builder.rs        |  26 ++++++++-----
 src/handler.rs        | 220 ++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------------
 src/request_ext.rs    |  56 +++++++++++++-------------
 tests/path_tests.rs   |   2 +-
 tests/rocket_tests.rs |  11 +++---
 6 files changed, 155 insertions(+), 169 deletions(-)

Step 4: Make some changes then push to Github

Once changes have been committed locally they can be pushed to our remote for others to benefit from.


 bash
$ git push origin HEAD

They can then add our fork as an other remote on their local so they make use of the work we did, if what we did was correct of course!

Wrapping up

Thoughts on this? Have you done something similar or maybe there is a better way to go about working with a fork?

If this helped don't forget to go to Stackoverflow and upvote the solution provided there!

References

[1] Git pull from someone else's fork https://stackoverflow.com/questions/42156971/git-pull-from-someone-elses-fork#42157071

How understanding software versions can help you get unstuck

David MacEachern — Fri, 15 Jan 2021 13:59:04 +0000

Today I thought I would document how I got past a problem related to software versions that I encountered whilst using a tool on Windows Subsystem for Linux.

tl;dr

In a more general sense, I remember when I started working with software tools 4 years ago, and it never occurred to me that the problems we face working with software on a daily basis can often be solved by:

Downgrading or upgrading the version we are using
using workarounds provided by other individuals
raising an issue ourselves with the maintainer, Stackoverflow, or asking elsewhere on the internet

For example for the Rust ecosystem's package manager "Cargo" we can see the different versions available here.

The Problem

OK, so I recently ran into an issue publishing a crate from Windows System for Linux (WSL) for the first time.

It seems the cargo publish command does not work when the repository is based in the mounted Windows directory.

So the directories involved in this process look like.

|-- ~/
|-- /mnt/c/Users/david/Documents/Github/nixy

You can see that my project is in the Windows user's documents folder.

Has someone else solved the problem?

To avoid reinventing the wheel, also I didn't immediately think of where to go, I Googled the error message I received when I tried to run cargo publish.

There is an issue where this is being tracked here and more technical detail about why this actually happens here.

Trying one of the workarounds

Based on a comment in #8439 I took the following steps to fix my problem for the day.

I checked whether the directory in the workaround existed.

wsl@DESKTOP-DCRMGFD:/mnt/c/Users/david/Documents/GitHub/nixy$ cd ~

Then I returned to the previous directory I was in.

wsl@DESKTOP-DCRMGFD:~$ cd -

Then I went to the parent directory of the project.

wsl@DESKTOP-DCRMGFD:/mnt/c/Users/david/Documents/Github/nixy$ cd ..

Then I recursively copied the project to the WSL home folder.

$ wsl@DESKTOP-DCRMGFD:/mnt/c/Users/david/Documents/Github/$ cp -R ./nixy ~

Then I navigated to the second project location in the WSL home folder and ran the publish command again.

wsl@DESKTOP-DCRMGFD:/mnt/c/Users/david/Documents/Github$ cd ~
wsl@DESKTOP-DCRMGFD:~$ ls
nixy
wsl@DESKTOP-DCRMGFD:~$ cd nixy
wsl@DESKTOP-DCRMGFD:~/nixy$ ls
Cargo.toml  LICENSE  README.md  src  target  tests
wsl@DESKTOP-DCRMGFD:~/nixy$ cargo publish
    Updating crates.io index
   Packaging nixy v0.1.0 (/home/wsl/nixy)
   Verifying nixy v0.1.0 (/home/wsl/nixy)
   Compiling nixy v0.1.0 (/home/wsl/nixy/target/package/nixy-0.1.0)
    Finished dev [unoptimized + debuginfo] target(s) in 2.53s
   Uploading nixy v0.1.0 (/home/wsl/nixy)
wsl@DESKTOP-DCRMGFD:~/nixy$

Do I have to use the workaround?

No! You can now upgrade to the latest version of Cargo and this should work across the mounted /mnt/c/ directory.

I did this by checking my installed Cargo version, which told me I'm on a version from 2020-10-14.

wsl@DESKTOP-DCRMGFD:~/nixy$ cargo --version
cargo 1.48.0 (65cbdd2dc 2020-10-14)

I can see on #8950 that the fix was merged to master on 7th December.

So I update the rust toolchain, which in turn updates the Cargo publish command.

wsl@DESKTOP-DCRMGFD:/mnt/c/Users/david/Documents/Github$ rustup update

This gives me the following Rust version which is actually just before the fix was merged, might need to switch to the nightly channel to access 1.50 and any bug fixes associated with it.

wsl@DESKTOP-DCRMGFD:/mnt/c/Users/david/Documents/Github$ cargo --version
cargo 1.49.0 (d00d64df9 2020-12-05)

Wrapping up

Thanks to those awesome people who contribute to the Rust ecosystem, mostly in their free time, to add fixes like the one I benefitted from in this post!

I'll try to remember to update this post once I publish a new version of my Crate to test the problem.

Feel free to join in the discussion below, until next time 👋.

It's time to unleash your social potential

David MacEachern — Mon, 11 Jan 2021 18:12:47 +0000

The issue I have had with social media in the past is only being able to see the negative aspects. I'm also someone that by default prefers to keep things fairly quiet, an air of mystery if ever that was a thing.

social adjective - Of, relating to, or occupied with matters affecting human welfare. Of or relating to human society and its modes of organization. Interacting with other people and living in communities. [1]

The truth is all technology work is now online. I chose this career, and I have learned how to use technology to bring business value to organisations, why not to myself?

potential adjective - Capable of being but not yet in existence; latent or undeveloped. noun The inherent ability or capacity for growth, development, or future success. [2]

It's time I opened my eyes to what has been staring me in my face the whole time. Having an online presence is important, it can demonstrate so many qualities in oneself but the most important one to me right now during Covid is connection.

Working alone at home in quarantine, looking for a job is exhausting, and it's my first time tackling technical interviews. I have never done one, so have no idea how that is going to go. For now, my goal is to get to the interviews and see what companies expect.

But how can you connect with people?

Social media as a tool

Like any tool, social media has potential, and success is made when we can use this potential to provide value. However when we forget to reason, "does this tool actually address a problem we are facing right now", this is when we don't get what we want.

Too often I have reached to eliminate sources of distraction entirely but maybe there are other options we can consider. It actually might be more effective to understand how we want things to serve us, as Free solo famous Alex Honnold discusses in an interview about his work [3].

Is blogging social media?

So let's talk about how this might go. Wikipedia describes a blog as consisting of "often informal diary-style text entries" [4]. And you can see on this very blogging platform some cool examples of how you might approach this opportunity.

Ok great, so you have made your first blog post, so what?

validation noun The act of giving validity; a strengthening, enforcement, or confirming; an establishing or ratifying [5]

Validation gets a bad wrap, it can be an important element in the journey of discovering whether an idea is going to deliver some value. Will our customers like our product, will our developers enjoy their job, is my understanding of this subject correct?

If we don't communicate with other human beings, then we miss out on a valuable chance to improve our situation, there's always a way, right?

We could think of blogging as a social media, a place we can connect with others.

But what about micro-blogging?

Wikipedia, appears to explain microblogging[6] as a petite form of blogging, something that takes less time but allows you to share your thoughts all the same.

Twitter is the atypical example of the microblog (I had a Myspace and a Bebo once upon a time but they might not be relevant anymore...).

Yes you can use a microblog to kill time if that's what you really want, or you could build an inspiration machine, it's all a matter of perspective.

Archillect

@archillect

21:54 PM - 10 Jan 2021

Think of the ways that you could use a microblog to your advantage, maybe you can connect with people in ways you'd never imagined were possible.

If you need more ideas for Twitter specifically, I can't find the tweet but Josh W. Comeau shared a great guide, the link is in the references [7].

For me, I'm starting here:

David MacEachern

@maceacherndjh

A great listen if you're into Adventure, in essence, it's about finding what's challenging where you are as opposed to chasing what is expected by everyone else.

open.spotify.com/episode/1ZHPxY…

12:28 PM - 11 Jan 2021

Let's see how this goes.

Wrapping up

Does this resonate with you? Are you looking for a job too? Or just feel like talking to someone? My messages are open, or comment down below so everyone else can join in!

Have a great week!

References

[1] Social https://www.wordnik.com/words/social
[2] Potential https://www.wordnik.com/words/potential
[3] Alex Honnold talks about deleting social media amongst other things
[4] Blogging https://en.wikipedia.org/wiki/Blog
[5] Validation https://www.wordnik.com/words/validation
[6] Microblogging https://en.wikipedia.org/wiki/Microblogging
[7] Josh W. Comeau's helpful guide to Twitter https://docs.google.com/document/d/1b7P71-OjEt5dHlgKEf9EbUBTkXRVrpDQaz7giXUd3n4

How to generate Type Definitions for Distribution on NPM

David MacEachern — Fri, 11 Dec 2020 13:35:31 +0000

tl;dr

If in doubt create a toy application quickly using a project template, for Typescript this could be done using tsdx
A checklist of the things required to test from the point of view of the end user might be helpful, how will they consume the library in the end?
It always helps to scroll through the docs to understand what the tools we are working with can do, to solve the problems I encountered whilst writing this I visited, tsc, babel, and npm.

Problem

I was recently testing out a web application, and in doing so I created a toy application and imported a small open-source library I had created.

# main.ts
import { Nosozu } from 'nosozu';

VS Code started giving me an error related to missing type definitions, despite having installed the package using npm before using npm i nosozu.

Cannot find module '@nosozu' or its corresponding type declarations.

What I did to generate type definitions

In the tsconfig.json I added the field which tells the Typescript compiler to generate the type definition.

{
+  "declaration": true,

In the package.json I checked what was declared as the main entry point to the library, and then added a line for where the types will be located.

  "main": "dist/index.js",
+ "types": "dist/index.d.ts",

Tidying up the NPM Package

Checking the first version I published of Nosozu, the user would install our package using npm i nosozu@2.0.1-alpha, in doing so and checking what files are in ./node_modules/nosozu/ they would find 49 files totaling 63.6 kB.

Taking a look at other node_module directories within the project, helped to understand what files are generally included when publishing a library like this.

Then also reading about what the .npmignore file is for over here helped in figuring out how to exclude sending things like test and example folders to npm.

I then settled with adding the following allowed list to the package.json

+  "files": [
+    "/dist"
+  ]

This led to a node_modules directory structure like:

.
|-- node_modules
|   |-- nosozu
|   |   |-- LICENSE
|   |   |-- README.MD
|   |   |-- package.json
|   |   |-- CHANGELOG.md
|   |   |-- dist 
|   |   |   |-- command-builder.d.ts
|   |   |   |-- command-builder.js
|   |   |   |-- error.d.ts
|   |   |   |-- error.js
|   |   |   |-- index.d.ts
|   |   |   |-- index.js
|   |   |   |-- json-client.d.ts
|   |   |   |-- json-client.js
|   |   |   |-- protocol.d.ts
|   |   |   |-- protocol.js

Testing the changes worked before publishing the fix

Before publishing the new version I checked whether these changes fixed the original import issue. I did so by copying the contents of the dist folder and the package.json into the node_modules/nosozu of the project I was testing in the beginning.

In Bash this looked something like:

cd playground-timber-nosozu/playground-timber-nosozu/
rm -rf ./node_modules/nosozu/                         # remove the installed library
mkdir ./node_modules/nosozu                           # make the folder again
cp -r ../../nosozu/dist/  ./node_modules/nosozu       # copy the modified library with types
cp ../../nosozu/package.json  ./node_modules/nosozu   # copy package.json so our application can see the "main" and "types"

This could also be done using npm link.

The changes to that project can be seen here, I did change the package name before publishing it to NPM, as well as messing around with unpublishing versions on NPM. If you do make mistakes it's possible to undo them but consider if there are any users that this might affect!

Wrapping up

Checking the package again, the size has increased to 123 kB however the number of files has reduced to 15. I reinstalled this in the toy project and found everything appeared to be working, except for an error being thrown inside the library itself, work for another day!

Maybe you want to learn more about Typescript, or maybe you know something I don't!

Contributions of any sort are more than welcome, you can see open issues over here

How to query JSON data in the terminal

David MacEachern — Tue, 24 Nov 2020 17:12:43 +0000

Today we will be using jq to work with JSON data, you should be able to get it here.

tl;dr

Return top-level keys from multiple json files in present working directory

jq -rs 'reduce .[] as $item ({}; . * $item) | keys[]' ./*.json

Get the data

Go ahead and clone the assets if you want to work with this too.

git clone https://github.com/davidmaceachern/playground-query-json.git

Navigate to where the JSON documents are.

cd playground-query-json/assets/requests

Have a look at the JSON document names if you wish.

ls

cat add_backend.json

Prints the following as the contents.

{
  "id": "ID_TEST",
  "version": 0,
  "type": "PROXY",
  "data": {
    "type": "ADD_BACKEND",
    "data": {
      "app_id": "xxx",
      "backend_id": "xxx-0",
      "ip_address": "127.0.0.1",
      "port": 8080
    }
  }
}

The problem

Ok so FYI this is test data for a project I am working on.

I need to decide on which properties I need the application to generate. The goal is to reduce them down as much as possible so the user only provides the items which they need to control and the application handles the rest.

At this stage, each of these files can be considered 1 single dataset.

ls -1q *json | wc -l

If we run the above command we should find there are 16 files that match a '.json' ending.

Combine all the documents into a single dataset

First we can print as we did using cat before but using jq instead:

jq . add_backend.json

jq is the application we are invoking. The . represents the first argument jq needs which is a filter, or a thing we want to do to change our input, which right now is nothing. The third item is the filename which is our input.

Exactly the same but prettier:

{
  "id": "ID_TEST",
  "version": 0,
  "type": "PROXY",
  "data": {
    "type": "ADD_BACKEND",
    "data": {
      "app_id": "xxx",
      "backend_id": "xxx-0",
      "ip_address": "127.0.0.1",
      "port": 8080
    }
  }
}

Let's combine two files and see what that looks like.

jq . add_backend.json remove_backend.json

Which returns:

{
  "id": "ID_TEST",
  "version": 0,
  "type": "PROXY",
  "data": {
    "type": "ADD_BACKEND",
    "data": {
      "app_id": "xxx",
      "backend_id": "xxx-0",
      "ip_address": "127.0.0.1",
      "port": 8080
    }
  }
}
{
  "id": "ID_TEST",
  "version": 0,
  "type": "PROXY",
  "data": {
    "type": "REMOVE_BACKEND",
    "data": {
      "app_id": "xxx",
      "backend_id": "xxx-0",
      "ip_address": "127.0.0.1",
      "port": 8080
    }
  }
}

Ok great but we want to merge all the files together, along with handling nested
structures:

jq -rs 'reduce .[] as $item ({}; . * $item)' ./*.json

Which returns:

{
  "id": "ID_TEST",
  "version": 0,
  "type": "UPGRADE_MASTER",
  "data": {
    "type": "STATUS",
    "data": {
      "app_id": "xxx",
      "hostname": "yyy",
      "path_begin": "xxx",
      "fingerprint": "ab2618b674e15243fd02a5618c66509e4840ba60e7d64cebec84cdbfeceee0c5"
    },
    "path": "./config_dump.json"
  },
  "proxy_id": 0
}

We can output that result to a file to use later if we wish.

jq -rs 'reduce .[] as $item ({}; . * $item)' ./*.json > ../aggregate.json

For now though I only care about keys, so inside the jq filter we can add a pipe
which only returns the keys.

jq -rs 'reduce keys[] as $item ({}; . * $item) | keys[]' ./*.json

Which prints the unique top-level keys:

data
id
proxy_id
type
version

Great, now I might encounter more issues with the JSON request format further into development, but for now a quick look at these keys is exactly what I
needed.

So what?

Ok so, if we were actually writing a bash script this might be helpful, or using a language such as Python we can actually use jq queries inside our code using a package such as found here.

jq is a great way to work with JSON data in the terminal, picking up knowledge of a couple of filters can come in handy. I've enjoyed using it in the past to explore JSON data and it can be handy for helping to make decisions.

You can achieve similar things manually in VSCode, with JSON linting and copy/paste and find/replace but if it's a task you do over and over again it can be worthwhile to use tools like jq.

I'm writing a Javascript application, so I could have done this type of thing inside my unit tests, as Javascript is great when it comes to JSON, but I thought to share this approach today.

References

How to fix Github Docker Containers Built with Actions

David MacEachern — Thu, 12 Nov 2020 13:11:39 +0000

Following on from having set up the Github Container Registry in my previous post I look at how I can combine Github Actions with the Registry to automate the build step of my application.

tl;dr

You can use Github Actions to build containers based on Dockerfiles in your repository.
Have a Google for the answer but don't hesitate to run your problematic containers interactively.
You can build and interact with your Docker images locally docker build . then docker run -it $IMAGE_ID /bin/bash

Problem

I have a Dockerfile I am building to package up my application to be deployed somewhere, I do so using a Github Action upon pushing changes
to Github.

Github Actions have many example templates to base workflows on if you already have a project you can navigate to: "https://github.com/" + $YOUR_USERNAME + "/" + $REPOSITORY_NAME + "/actions/new

When I create a Dockerfile, I develop locally and commit changes to the repository. Upon a push to Github my Action Workflow is triggered, this is continuous integration (CI) at it's best, but when we need to look inside a container things can get a little tricky.

What follows are some notes about how you might fix your CI container build.

Lessons learnt

When I attempt to build this Dockerfile, which looks like this:

FROM rustlang/rust:nightly-slim AS build
WORKDIR /src/rust-web-template
COPY . .
RUN cd ./collector && cargo build --release && cd ..
FROM ubuntu:18.04
COPY --from=build /src/rust-web-template/target/release/collector /usr/local/bin/collector
CMD ["usr/local/bin/client"]

Here are some issues I ran into:

pkg-config is required for locating dependencies such as SSL within the container's linux.

It looks like you're compiling on Linux and also targeting Linux. Currently this
requires the `pkg-config` utility to find OpenSSL but unfortunately `pkg-config`
could not be found. If you have OpenSSL installed you can likely fix this by
installing `pkg-config`.

Docker container builds steps that require approval must have a yes flag passed to them apt-get install -y pkg-config.

Do you want to continue? [Y/n] Abort.
The command '/bin/sh -c apt-get update &&     apt-get install pkg-config' returned a non-zero code: 1

We need to run apt-get update to ensure our container knows where to get packages we want to install.

E: Package 'pkg-config' has no installation candidate

Dependencies such as Openssl need to be installed as well to be available when compiling an application that depends on it.

run pkg_config fail: "`\"pkg-config\" \"--libs\" \"--cflags\" \"openssl\"` did not exit successfully: exit code: 1\n--- stderr\nPackage openssl was not found in the pkg-config search path.\nPerhaps you should add the directory containing `openssl.pc\'\nto the PKG_CONFIG_PATH environment variable\nNo package \'openssl\' found\n"

How can we debug a Docker container without Googling?

So our Dockerfile now looks like this:

FROM rustlang/rust:nightly-slim AS build
WORKDIR /src/rust-web-template
COPY . ./collector
RUN apt-get update -y && \
    apt-get install -y pkg-config \
    libssl-dev
RUN cd ./collector && cargo build --release && cd ..
FROM ubuntu:18.04
COPY --from=build /src/rust-web-template/target/release/collector /usr/local/bin/collector
CMD ["usr/local/bin/client"]

I received the following error, which tells me that a directory path was not correct.

 COPY failed: stat /var/lib/docker/overlay2/640cf4644f3f681a1b78f9507d80e70f3cae1a189a65dfd6a1f26f7313748161/merged/src/rust-web-template/target/release/collector: no such file or directory

To fix this I comment out the code, as we need to build the container and then interact with it.

FROM rustlang/rust:nightly-slim AS build
WORKDIR /src/rust-web-template
COPY . ./collector
RUN apt-get update -y && \
    apt-get install -y pkg-config \
    libssl-dev
RUN cd ./collector && cargo build --release && cd ..
+ #FROM ubuntu:18.04
+ #COPY --from=build /src/rust-web-template/target/release/collector /usr/local/bin/collector
+ #CMD ["usr/local/bin/client"]

Next we build it by running $ docker build -t debugimage ., and then we can run the container interactively to explore it docker run -it --name debugimage:latest /bin/bash

Once inside we can run our bash commands to investigate.

t@d0e323387e63:/src/rust-web-template# ls
collector
root@d0e323387e63:/src/rust-web-template# cd collector/
root@d0e323387e63:/src/rust-web-template/collector# ls
Cargo.lock  Cargo.toml  Dockerfile  src  target
root@d0e323387e63:/src/rust-web-template/collector# cd tar
bash: cd: tar: No such file or directory
root@d0e323387e63:/src/rust-web-template/collector# cd target
root@d0e323387e63:/src/rust-web-template/collector/target# ls
CACHEDIR.TAG  release
root@d0e323387e63:/src/rust-web-template/collector/target# cd release/
root@d0e323387e63:/src/rust-web-template/collector/target/release# pwd
/src/rust-web-template/collector/target/release
root@d0e323387e63:/src/rust-web-template/collector/target/release# ls
build  collector  collector.d  deps  examples  incremental
root@d0e323387e63:/src/rust-web-template/collector/target/release# pwd
/src/rust-web-template/collector/target/release
root@d0e323387e63:/src/rust-web-template/collector/target/release#

Then update the path in the Docker container, and uncomment out lines that were not working.

- COPY --from=build /src/rust-web-template/target/release/collector /usr/local/bin/collector
+ COPY --from=build /src/rust-web-template/collector/target/release/collector /usr/local/bin/collector

And then we can $ exit our interactive container session.

Upon building the image again the path problem shouldn't cause the build to fail!

The working code for this can be found here.

Wrapping up

So that's a couple of common problems building linux containers, hopefully it won't happen again!

Do you have better ways to resolve build problems?

Feel free to share any thoughts or findings of your own down below 🙏🏻.

Using Github Container Registry

David MacEachern — Sat, 07 Nov 2020 13:11:35 +0000

Using Docker is common place, it's useful in various points of the SDLC, and solves some headaches when it comes to reproducing a code build on different architectures.

I've been looking at setting up a container registry to enable some experimentation with Kubernetes, which requires having a registry for custom images, whether I will actually do anything with Kubernetes
we will see.

Today I discovered Github has a container registry service (or 'GHCR'), for personal use this is appealing with limits that are not so low that we cannot experiment with them.

Plan	Storage	Datatransfer out within Actions	Datatransfer out without Actions
Free	500MB	Unlimited	1GB per month
Pro	2GB	Unlimited	10GB per month

Luckily I have Pro plan so let's have a look at what we can do.

Storing a container in the GHCR

First we need to register our github container registry with our Docker client:

Create a token here
Navigate to "https://github.com/" + $YOUR_GITHUB_USER + "?package_type=Docker&tab=packages"
Read the instruction titled # Step 1: Authenticate and understand what it needs, then run this in your shell locally.

Next thing we need is a custom Dockerfile that we can upload to the registry, you can refer to this repository if you're not sure how to proceed.

Create a new Dockerfile in a local directory.
Add the build you want, or stick in hello world build steps to keep it simple, remember we're just here to test out GHCR.
Build the image, if you used the repository I created then you might run docker build -t playground-ghcr:latest .
Then we will want to get the container image id, docker image list, my image was 118MB in size, which is within the limits mentioned previously.
Continue with # Step 2: Tag don't forget to understand what these commands will do that you're being asked to run!
Piping docker image list | grep ghcr should return the image with two tags now (or even more if you made mistakes like me!).
Continue with # Step 3: Publish, as I mentioned before I made a mistake where I didn't swap out the /repository-name/, silly me!

Reviewing usage

So I uploaded an image that was 118MB in size and it took 6m 28s on my slow network speed.

Pulling the image back down should begin using the quota we have on our Github plan.

We can remove the image we have locally docker image rm -f $IMAGE_ID
Then we can request the image we uploaded before from GHCR docker pull docker.pkg.github.com/davidmaceachern/playground-ghcr/playground-ghcr:latest
After we can verify if we wish that this is indeed our Docker image docker container run $IMAGE_ID.
Finally we can navigate here to view billing details.

So a little disappointing, it appears that billing data might not be available until the quota resets Data transfer quota resets in 4 days,
or not at all given that the service is still in public beta:

During the beta, storage and bandwidth is free.
— Github

Checking the recently released github cli docs there is no sign of a billing function. Maybe this data could be grabbed using the graphql feature?

What about Dockerhub?

When we compare the experience with Dockerhub for a second, the limits for private repositories appear to be much lower than those provided with the free tier of Dockerhub even after they announced new rate limits for their service.

Vincent kindly pointed out in the comments, that the billing details I referred to are for hidden repositories, Github is offering a very appealing pricing model.

"The limits that you cite are only for private. If your containers are public then there are no limits, even for free accounts."

Docker has however provided a better way to obtain how much usage you have incurred which they explained here

So I will probably continue using the service until the official pricing is announced.

Wrapping up

So there ends a little investigation into using Github Container Registry, it seems like it's still a work in progress for Github but it should do the job for projects going forward.

Feel free to share any thoughts or findings of your own down below if you want 🙏🏻.

Learning Rust: Collecting Data from an API

David MacEachern — Fri, 11 Sep 2020 08:21:52 +0000

Wondering where to begin with programming? Why not give Rust a try, and collect some data from an API as we go!

In this article we will cover:

Find some data to use
Query an API and handle some HTTP
Collect some data saving it locally as JSON