DEV Community: Andy Yang

Release 0.43

Andy Yang — Sat, 12 Dec 2020 05:19:11 +0000

The Final

When I still work on the second issue, I found another issue to work with https://github.com/eupn/macrotest/issues/54. This is the pull request I made. https://github.com/eupn/macrotest/pull/57

Migrate from Travis CI to Github Actions

This is a rust project wants move to Github Actions. This is the draft I made.

name: Rust

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ${{matrix.os}}

    strategy: 
      matrix:
        rust: [stable]
        os: [ubuntu-latest, macos-latest, windows-latest]

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Use stable rust toolchain
        uses: actions-rs/toolchain@v1
        with:
          profile: minimal
          toolchain: stable
          override: true

      - name: install
        if: matrix.os == 'windows-latest'
        run: |
          rustup target add x86_64-pc-windows-msvc
      - name: before script
        run: |
          rustup toolchain install nightly
          rustup component add rustfmt
          cargo install $TARGET cargo-expand
      - name: script
        run: |
          cargo build $TARGET --verbose --all
          cargo test
          cd test-project
          cargo test -- --nocapture
          cd ../test-procmacro-project
          cargo test -- --nocapture

There are some problems. The author wants a minimal version of rust being tested. When they use export MSRV=1.34.0, but it not working on windows in github action. I am working on something similar to handle this issue. The potential solution could be use set MSRV=1.34.0 in windows.

Release 0.42 - Three ideas about automating release.

Andy Yang — Fri, 11 Dec 2020 17:10:27 +0000

The current release process

The telescope project https://github.com/Seneca-CDOT/telescope.git use release-it https://www.npmjs.com/package/release-it as the release tool. To automate the release process, I have three ideas.

Idea one: release-it

The release-it tool have a ci mode release-it --ci, it will automate the process without any prompt. We can also continue our release process like before, just moving to the cloud.

But I have two problems:

Problem one: read the GITHUB-TOKEN.

I have tried

- name: export env
        run: export GITHUB_TOKEN="${{ secrets.GITHUB_TOKEN }}"

and

- name: Create Release
  run: npm run release
  env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}"

none of them work.

Problem two: triger the release

We triger the release when push to github with the tag v*, but release-it also add a new tag when create the new release.

Idea two:

We maintain a file CHANGLOG.md.

on:
  push:
    tags:
      - 'v*'

name: Create Release

jobs:
  build:
    name: Create Release
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Create Release
        id: create_release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.ref }}
          release_name: Release ${{ github.ref }}
          body_path: ./CHANGELOG.md
          draft: false
          prerelease: false

Idea Three: Release Drafter

We can draft a release as we commiting to the repo.
https://github.com/marketplace/actions/release-drafter

name: Release Drafter

on:
  push:
    # branches to consider in the event; optional, defaults to all
    branches:
      - $default-branch

jobs:
  update_release_draft:
    runs-on: ubuntu-latest
    steps:
      # Drafts your next Release notes as Pull Requests are merged into "master"
      - uses: release-drafter/release-drafter@v5
        with:
          # (Optional) specify config name to use, relative to .github/. Default: release-drafter.yml
          # config-name: my-config.yml
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Release Go package

Andy Yang — Sun, 06 Dec 2020 04:44:59 +0000

Release Method

For the Go program, I choose https://pkg.go.dev/ to release my goURL. And the release address https://pkg.go.dev/github.com/yzwdroid/goURL

Read the document.

The first thing I do is read google's document for release. It is hard to find a clue, but as a programmer, I did some experiments.

Visiting that page on pkg.go.dev, and clicking the “Request” button. For example:
https://pkg.go.dev/example.com/my/module

I guess this is meaning my released address should be

https://pkg.go.dev/github.com/yzwdroid/goURL

This is hard to guess. But google treated all the programmer knows everything.

I created my first release three months ago. But when I add a new release today, it didn't refresh. Maybe it takes some time to refresh.

Release 0.41

Andy Yang — Mon, 30 Nov 2020 16:30:18 +0000

For release 0.4, I will work on the CI/CD area on the telescope project. I already fixed one issue https://github.com/Seneca-CDOT/telescope/issues/1336, the pull request https://github.com/Seneca-CDOT/telescope/pull/1345

It takes a little time to this issue. I learned a lot about Github action from this issue.

Adding if condition

- if: matrix.os == 'ubuntu-latest'
        run: npm run test-ci

Run multiple jobs in parallel.

  Prettier-Check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
      - run: npm install && npm run prettier-check
  Lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
      - run: npm install && npm run eslint

add cache.

 - name: Cache node modules
        uses: actions/cache@v2
        env:
          cache-name: cache-node-modules
        with:
          # npm cache files are stored in `~/.npm` on Linux/macOS
          path: ~/.npm
          key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}

We will redo the cache if the file changed package-lock.json.

Next step.

I am currently working on the second issue https://github.com/Seneca-CDOT/telescope/issues/1301 to automate the release process about telescope.

lab8-unit test and CI

Andy Yang — Sat, 28 Nov 2020 04:50:58 +0000

Overview

This week we practiced unit tests and continuous integration using Github action. This is the first time I write a unit test and with the help of go document and go test. I feel more confident to write the test first and then write the logic code.

Set up testing framework

Go program language has a standard testing package built-in. It's easy to test and benchmark with the standard library. After I learn some basic concepts, I started my work with git checkout -b testing.

My first test

func Test_removeDuplicate(t *testing.T) {
    type args struct {
        urls []string
    }
    tests := []struct {
        name string
        args args
        want []string
    }{
        // TODO: Add test cases.
        {"name", args{[]string{"http://www.google.ca", "http://www.google.ca"}}, []string{"http://www.google.ca"}},
        {"name", args{[]string{"http://www.google.ca", "http://zyang.ca", "http://www.google.ca"}}, []string{"http://www.google.ca", "http://zyang.ca"}},
    }
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            if got := removeDuplicate(tt.args.urls); !reflect.DeepEqual(got, tt.want) {
                t.Errorf("removeDuplicate() = %v, want %v", got, tt.want)
            }
        })
    }
}

It looks complicated, but to be honest, most of the code is generated by vscode.

Mock the HTTP server

I spend a longer time searching for the right syntax to mocking in go. I found mocking a HTTP server is also built in the Go language. "net/http/httptest"

    ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(404)
        w.Write([]byte("Not Found"))
    }))
    defer ts.Close()
    url := ts.UR

run the test

we can run all the test

go test -v .

or we can just run one test file

go test filename

test coverage

go test cover

git rebase

I forgot to do the rebase in the testing branch. After commit to the master with all the commit history. Instead of git rebase master -i, I found the commit number from git log and did a rebase on the master branch. git rebase 312ef0446a -i

Github action

Actually, I added the GitHub action a month ago. I already make some contributions to the CI/CD area.

name: Go

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:

  build:
    name: Build
    runs-on: ubuntu-latest
    steps:

    - name: Set up Go 1.x
      uses: actions/setup-go@v2
      with:
        go-version: ^1.13

    - name: Check out code into the Go module directory
      uses: actions/checkout@v2

    - name: Build
      run: go build -v .

    - name: Test
      run: go test -v .

  lint:
    name: Lint
    runs-on: ubuntu-latest
    steps:

    - name: Set up Go 1.x
      uses: actions/setup-go@v2
      with:
        go-version: ^1.13

    - name: Check out code into the Go module directory
      uses: actions/checkout@v2
    - run: go get -u golang.org/x/lint/golint
    - name: lint
      run: golint .
  fmt:
    name: Fmt
    runs-on: ubuntu-latest
    steps:

    - name: Set up Go 1.x
      uses: actions/setup-go@v2
      with:
        go-version: ^1.13

    - name: Check out code into the Go module directory
      uses: actions/checkout@v2
    - name: fmt
      run: gofmt .

Test Pull Request

To test the pull request and github action. I make this PR https://github.com/yzwdroid/goURL/pull/12. All the tests passed, and I merged the PR to the master branch.

Update contibuting.md

## Tests

You are welcome to add tests to this project.

go test filename
// or 
go test .
Test coverage
go test -cover

Add tests to another project

integrate linter

lint:
    name: Lint
    runs-on: ubuntu-latest
    steps:

    - name: Set up Go 1.x
      uses: actions/setup-go@v2
      with:
        go-version: ^1.13

    - name: Check out code into the Go module directory
      uses: actions/checkout@v2
    - run: go get -u golang.org/x/lint/golint
    - name: lint
      run: golint .

Integrated test into VS Code

Install the Go plugin, generate the simple test as mentioned above.

Release 3.2 - Github action

Andy Yang — Fri, 20 Nov 2020 03:13:11 +0000

For this release, I choose to work with github action as the external one. The issue I found is https://github.com/infinyon/future-aio/issues/36. The pull request is https://github.com/infinyon/future-aio/pull/44

What is Github action

Github action is a CI/CD tool. We can rent a virtual machine(Linux, Windows,macOS) from GitHub(Azure). For me, it's like a git hook on the cloud. Everytime we change our code (push, PR, tag), we can specify some commands to run, and the results will show on our repo.

How to use it.

All the command for github action is located at .github/workflows/*.yaml. yaml is a superset of JSON. The syntax is like this:

name: node-js-ci

on:
  pull_request:
    branches:
      - master
  push:
    branches:
      - master

jobs:
  build:
    runs-on: ${{matrix.os}}

    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        node-version: [12.x]

We can have different actions for different branches of our repo.

Work with the `rust` project.

I am new to rust, so first install rust on my MacBook.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

After reading the rust document, I know the package tool cargo like npm. For this issue https://github.com/infinyon/future-aio/issues/36, I need to create a release when we have a push with a tag v*. And this is done by the code below.

on:
  push:
    tags:
      - 'v*'

name: Create Release

jobs:
  build:
    name: Create Release
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Create Release
        id: create_release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.ref }}
          release_name: Release ${{ github.ref }}
          body_path: ./changelist.md
          draft: false
          prerelease: false

I also created a ./changelist.md file to track the next release notes. So the release body with replaced by the content of this file.

Release 3.1 - telescope

Andy Yang — Fri, 20 Nov 2020 02:43:33 +0000

For this release, I contribute to this issue https://github.com/Seneca-CDOT/telescope/issues/1351. The pull request https://github.com/Seneca-CDOT/telescope/pull/1352

This is a relatively easy issue, We need to change the environment variable MAX_POSTS_PER_PAGE. The first thing I do is search this variable in our code.

Three results from three files.
Here are the steps I did.

create a new branch with the issue number.

git checkout -b issue#1351

Then, I set the value of this variable to 5. The coding job is done.

remove the old .env file, and copy the new env.example to .env

cp env.example .env

Then test the code in my local machine. It works, we have 5 posts per page.

Problem to push the code.

When I push the code to the remote git push origin issue#1351, git gives me a warning no permission to push. I checked the remote address git remote -v. I make a mistake and clone the repo from telescope. Now the original address is https://github.com/Seneca-CDOT/telescope.git.

How I solve the problem.

If I am not in this class, I will delete everything and clone a new repo from my forked repo git@github.com:yzwdroid/telescope.git.

But I am a git master now (Thanks for @humphd), the remote repo is just a name origin. I can just add another remote with the command git remote add yzwdroid git@github.com:yzwdroid/telescope.git. Then just push the current branch $issue1351 to the yzwdroid remote repo.

git push yzwdroid issue#1351

Then I do the pull request from github.

Go development with VSCode and vim

Andy Yang — Sun, 15 Nov 2020 02:46:35 +0000

Gofmt

Go takes a strong stance on code formatting. The gofmt tool rewrites code into the standard format.

gofmt filenames

Golint

Linter is a tool that analyzes source code to flag programming errors, bugs, stylistic errors, and suspicious constructs.
Using Linter provides many benefits.

Find bugs
Speed up code review
Manage code quality

go get -u golang.org/x/lint/golint
golint filenames

VSCode setup

Install the Go extension maintained by Go Team at Google(previously maintained by Microsoft).

You should immediately see a prompt in the bottom-right corner of your screen titled Analysis Tools Missing. This extension relies on a suite of command-line tools, which must be installed separately. Accept the prompt, or use the Go: Install/Update Tools command.

Also, see the tutorial https://code.visualstudio.com/docs/languages/go

vim setup

Install the vim-go plugin
Plug 'fatih/vim-go', { 'do': ':GoUpdateBinaries' }

The official toturial https://github.com/fatih/vim-go/wiki

Here are some settings from my vimrc file

augroup go
  autocmd!

  autocmd FileType go nmap <silent> <Leader>v <Plug>(go-def-vertical)
  autocmd FileType go nmap <silent> <Leader>s <Plug>(go-def-split)
  autocmd FileType go nmap <silent> <Leader>d <Plug>(go-def-tab)

  autocmd FileType go nmap <silent> <Leader>x <Plug>(go-doc-vertical)

  autocmd FileType go nmap <silent> <Leader>i <Plug>(go-info)
  autocmd FileType go nmap <silent> <Leader>l <Plug>(go-metalinter)

  autocmd FileType go nmap <silent> <leader>b :<C-u>call <SID>build_go_files()<CR>
  autocmd FileType go nmap <silent> <leader>t  <Plug>(go-test)
  autocmd FileType go nmap <silent> <leader>r  <Plug>(go-run)
  autocmd FileType go nmap <silent> <leader>e  <Plug>(go-install)

  autocmd FileType go nmap <silent> <Leader>c <Plug>(go-coverage-toggle)


  autocmd Filetype go command! -bang A call go#alternate#Switch(<bang>0, 'edit')
  autocmd Filetype go command! -bang AV call go#alternate#Switch(<bang>0, 'vsplit')
  autocmd Filetype go command! -bang AS call go#alternate#Switch(<bang>0, 'split')
  autocmd Filetype go command! -bang AT call go#alternate#Switch(<bang>0, 'tabe')
augroup END

Hello Telescope

Andy Yang — Sat, 07 Nov 2020 04:13:23 +0000

What is Telescope

Telescope is an example of planet: a tool for
aggregating blog feeds. It reminds me of google reader which I used for a long time to read the news.

Setup Environment

I am working on a fedora33 with KDE Plasma Desktop. Docker still not support CgroupsV2. It takes a little time to install docker and I have written a blog post for the installation https://dev.to/yzwdroid/fedora-33-docker-445k.

For some reason, redis has a permission error when docker-compose

Starting telescope_redis_1        ... done 
Starting telescope_elasticsearch_1 ... done 
Attaching to telescope_redis_1, telescope_elasticsearch_1 
redis_1         | find: '.': Permission denied 
redis_1         | chown: changing ownership of '.': Permission denied 
telescope_redis_1 exited with code 1

So I installed Redis by sudo dnf install redis, and start it with
redis-server

All other part went well, I just follow the environment-setup.md

Modify goURL

To work with the REST API of telescope, I added a new flag -u --url, fetch
the json data, convert the data to a slice of post, get each post's contents by
the url. I just added one new function to make this work. Below is the diff I
made.

git diff HEAD main.go utils.go > lab6.diff

The diff

diff --git c/main.go w/main.go
index 4679a1a..fd3d249 100644
--- c/main.go
+++ w/main.go
@@ -25,6 +25,7 @@ var js = flag.BoolP("json", "j", false, "output json format to stdout")
 var fp = flag.StringP("file", "f", "", "file name to check")
 var ignore = flag.BoolP("ignore", "i", false, "ignore url patterns")
 var failOnly = flag.Bool("fails", false, "show only urls that failed")
+var urlFlag = flag.BoolP("url", "u", false, "read telescope restful API")

 func main() {
    flag.Parse()
@@ -43,14 +44,20 @@ go run main.go -v or --version check version.
        os.Exit(-1)
    }

-   dat, err := ioutil.ReadFile(*fp)
-   check(err)
-
+   var urls []string
+   var dat []byte
+   if *urlFlag {
+       dat = dataTelscope()
+   } else {
+       var err error
+       dat, err = ioutil.ReadFile(*fp)
+       check(err)
+   }
    // use xurls tool to exact links from file. Strict mod only match http://
    // and https:// schema
    rxStrict := xurls.Strict()
    // urls is a slice of strings
-   urls := rxStrict.FindAllString(string(dat), -1)
+   urls = rxStrict.FindAllString(string(dat), -1)
    urls = removeDuplicate(urls)

    if *ignore {
diff --git c/utils.go w/utils.go
index 5c27fc7..aee4859 100644
--- c/utils.go
+++ w/utils.go
@@ -2,7 +2,9 @@ package main

 import (
    "bufio"
+   "encoding/json"
    "fmt"
+   "io/ioutil"
    "log"
    "net/http"
    "os"
@@ -18,6 +20,43 @@ type urlStatus struct {
    Status int
 }

+type post struct {
+   ID  string
+   URL string
+}
+
+func dataTelscope() []byte {
+   var data []byte
+   var posts []post
+   resp, err := http.Get("http://localhost:3000/posts")
+   check(err)
+   defer resp.Body.Close()
+
+   if resp.StatusCode == http.StatusOK {
+       body, err := ioutil.ReadAll(resp.Body)
+       check(err)
+       if err := json.Unmarshal(body, &posts); err != nil {
+           panic(err)
+       }
+       for _, p := range posts {
+           resp, err := http.Get("http://localhost:3000" + p.URL)
+           if err != nil {
+               fmt.Println(err)
+           }
+           defer resp.Body.Close()
+           if resp.StatusCode == http.StatusOK {
+               bodyData, err := ioutil.ReadAll(resp.Body)
+               if err != nil {
+                   fmt.Println(err)
+               }
+               data = append(data, bodyData...)
+           }
+       }
+   }
+
+   return data
+}
+
 func removeDuplicate(urls []string) []string {
    result := make([]string, 0, len(urls))
    temp := map[string]struct{}{}

The gist version is here https://gist.github.com/yzwdroid/26b6cb71f75f3799dfccd5fb0ed9ec79

Release 2.0 - pr4

Andy Yang — Mon, 02 Nov 2020 04:41:18 +0000

Release 2.0 - pr4

For the fourth pull request, I worked on this issue [https://github.com/JIITODC/jiitodc.github.io/issues/86#event-3923513143(https://github.com/JIITODC/jiitodc.github.io/issues/86#event-3923513143). The pull request is
https://github.com/JIITODC/jiitodc.github.io/pull/88.

I helped this repo to add a horizontal scrollbar on the table if the content is
overflow. This is done by adding a style to the CSS file.

display: block;
overflow-x: auto;
white-space: nowrap;

How I failed to contribute to a large program.

I planned to fix this issue https://github.com/goreleaser/goreleaser/issues/1878 , but after three days of working, I found that is not an easy problem, I have to learn some third party libraries. And I asked for help on this issue, for now, no response yet.

What I learned in this year's hacktoberfest

Before this festival, I don't know how the open-source world work, how to file issue, how to help, how to ask help. I learned that communication is a key ability to work with others. Before we contribute to the program we should read the rules usually on a file named contribute.md.

This festival gives me a chance to really start. It's a small step but the most important step for me to contribute to the real open source world.

Thank Digital Ocean and my professor David.

Fedora 33 - Docker

Andy Yang — Thu, 29 Oct 2020 17:03:27 +0000

Docker and Fedora 33

With the release of Fedora 33, Docker is not supported by Fedora 33 officially.
This article will help you install Docker on Fedora 33.

Remove old version

sudo dnf remove docker-*
sudo dnf config-manager --disable docker-*

Enable old CGroups

Docker still not support CgroupsV2.

sudo grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"

Set up firewall

sudo firewall-cmd --permanent --zone=trusted --add-interface=docker0
sudo firewall-cmd --permanent --zone=FedoraWorkstation --add-masquerade

Install Moby

Moby is the open-source version of Docker.

sudo dnf install moby-engine docker-compose
sudo systemctl enable docker

restart sudo reboot

Test

sudo docker run hello-world

Runing without `sudo`

sudo groupadd docker
sudo usermod -aG docker $USER
//reboot to take effect

release 2.0 - translation

Andy Yang — Tue, 27 Oct 2020 09:36:05 +0000

Release 2.3

For the third PR, I work on this issue https://github.com/photoprism/photoprism/issues/514. This is the PR I made https://github.com/photoprism/photoprism/pull/569. It's hard to find the project than actually work on the project.

How I worked with it

The instruction is so long to work with. I followed the instruction on this issue, but it did not work when I do this command npm run gettext-compile.

I realize it's a node module. So I tried npm install, then everything works.

The translation is easy to deal with with the help of Poedit https://poedit.net/. I translated both the frontend and backend of the software. And then test it on my local laptop. Then I did the pull request.

I am so excited to contribute to software that has so many users. I can do more translations with this set-up experience.

DEV Community: Andy Yang

Release 0.43

The Final

Migrate from Travis CI to Github Actions

Release 0.42 - Three ideas about automating release.

The current release process

Idea one: release-it

Idea two:

Idea Three: Release Drafter

Release Go package

Release Method

Read the document.

Release 0.41

Next step.

lab8-unit test and CI

Overview

Set up testing framework

My first test

Mock the HTTP server

run the test

test coverage

git rebase

Github action

Test Pull Request

Update contibuting.md

Add tests to another project

integrate linter

Integrated test into VS Code

Release 3.2 - Github action

What is Github action

How to use it.

Work with the rust project.

Release 3.1 - telescope

Problem to push the code.

How I solve the problem.

Go development with VSCode and vim

Gofmt

Golint

VSCode setup

vim setup

Hello Telescope

What is Telescope

Setup Environment

Modify goURL

Release 2.0 - pr4

Release 2.0 - pr4

How I failed to contribute to a large program.

What I learned in this year's hacktoberfest

Fedora 33 - Docker

Docker and Fedora 33

Remove old version

Enable old CGroups

Set up firewall

Install Moby

Test

Runing without sudo

release 2.0 - translation

Release 2.3

How I worked with it

Work with the `rust` project.

Runing without `sudo`