DEV Community: PanupongDeve

Create Pipeline build and push docker to Docker Registry with Github actions

PanupongDeve — Fri, 29 Nov 2024 10:17:43 +0000

Hi! everyone, Now I learning github actions and I would like to know how use docker for build source code by github action.

and I very love about feature actions. someone can create actions and deploy to action marketplace and then anyone can download and use them pipelines that is make productivity when you have create pipeline tasks

I write file workflow yml

name: Build and Push Docker to Dockerhub

on:
    # Allows you to run this workflow manually from Actions tab
    workflow_dispatch:
    # workflow_run:
    #   workflows: ["Build Cache and Push Docker to Dockerhub"]
    #   types:
    #     - completed
    push:
      branches:
        - main

permissions:
  contents: 'read'
  packages: 'write'

jobs:
  build:
    runs-on: ubuntu-latest
    environment:
            name: main
    steps:
        - uses: actions/checkout@v4

        - name: Set up Docker Buildx
          uses: docker/setup-buildx-action@v3

        - name: Log in to GitHub container registry
          uses: docker/login-action@v3
          with:
            # registry: ghcr.io
            username: ${{ secrets.DOCKER_USERNAME }}
            password: ${{ secrets.DOCKER_PAT }}

        - name: Set SHA SHORT
          id: set-sha-short
          run: echo "sha_short=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT

        - name: Build and push container image to registry
          uses: docker/build-push-action@v6
          with:
            push: true
            cache-from: type=registry,ref=${{ secrets.DOCKER_REPO_NAME }}:cache
            cache-to: type=registry,ref=${{ secrets.DOCKER_REPO_NAME }}:cache, mode=max
            tags: ${{ secrets.DOCKER_REPO_NAME }}:${{ steps.set-sha-short.outputs.sha_short }}
            file: ./Dockerfile

and Thank you about feature buildx of docker. They help me about cache you don't need to create Cache Dockerfile for create cache. They have feature cache-to, it's help create image cache after pipeline docker builded

[Plobem] nginx: worker_connections are not enough

PanupongDeve — Thu, 28 Nov 2024 07:57:01 +0000

When I work in company I see some error in nginx and show error in this picure.

and the my frontend can not load image in browser.

How to solve it

I increase worker_connection from 1024 to 4096 and try restart nginx

events { worker_connections 4096; }

Fix gitlab runner error with docker

PanupongDeve — Wed, 20 Nov 2024 22:31:35 +0000

in /etc/gitlab-runner/config.toml

privileged = true volumes = ["/cache","/var/run/docker.sock:/var/run/docker.sock"]

gitlab-runner restart

ref: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4794

How to clean disk from container running

PanupongDeve — Tue, 19 Nov 2024 20:42:07 +0000

remove container logs
sudo sh -c "truncate -s 0 /var/lib/docker/containers/*/-json.log"
remove containers component unused
docker system prune -a

How to learn anything for your improve skills?

PanupongDeve — Tue, 05 Jul 2022 08:11:47 +0000

Hi! today I think about learning that how to learn for anything, for better software engineer, for my career. I think about my learning skill is a better performance for use it.

Now, I have 2 ideas for improve my learning skills and improve productivity about thing in my learning.

First Idea from I watched video Let’s build a personal assistant and level up your skills! talk about his peronal project and how to improve technical skill for this project.

Second idea about working style of software engineer role from my friend. The Company of him work, They make MVP (Minimal Value Product) projects that tech me about when you learn something you don't need to completed your learning, you learn for something minimal enough for you can solve problem and improve your knowledge after new problem or feedback is coming.

the two ideas help me to understanding and improving about my learning skills.

In my thinking, Knowledge is something you known for to solve problem, when you learn something asking self that current learning can solve what interesting your problem.
We have two phase about learning. First, Begining phase is you don't known anything about you are interesting. Two, Knowlege base phase is you have enough knowleage for solving problem you find.
In begining phase for you are interesting, you should use 70% of learning time focus about input information such as you are going something new. You should learn tutorial, fundamental, or basic knownlege for solving your problem of you are interesting and 30% of learning time about output information, if you learning tools, you should creating projects and using information you known to solving problem in this project. If you are learning for exam such as IT certification, you should doing mock exam or quiz about your learning. other, You should have practical something you learn.
In Knowlege base phase for you are interesting, you should use 70% of learning time focus output information. You would like to improve performance for using knowlege to solve problem. You should keep doing on your projects or keep doing exam and something using your knowlege. 30% of learning time about input information, learn advance topic for you learn or something new knownlege and have relation between you have known and unknow.
You can't learn everything all the world. You should focus what you love, what you wwould like to be and what skills people would like to someone solving and they can paying when problem is solving.

For now, I'm interesting Fullstack Javascript Developer and English Skill, I learn some course and tutorial. I'am going to move to Knowleage base phase, That Every Day I'm planing for coding my created projects and have talking and writing English Time and focus result and recive some feedback for my improving skills.

I hope this content to make benefits to you. You can comment and feedback for my improving skills or share how to your learn anyting in this comment, I would like to read for develop my ideas to learning skills, Thank you for your reading.

Programming Rule Vs Machine Learning

PanupongDeve — Mon, 04 Jul 2022 09:16:28 +0000

Hi everyone, Today I'am seeing the people to activity such as post, comment, and upload something in social media, shopping with digital platform, control smart electrical appliance with application and other. This is reason to make a lot of data in every day.

A lot of data is incomming to make new problems such as Marketer cann't analyzing a lot of data is comming every day.
I'm web developer. I like to solve to problems. I would like to learn that how to solve problems in the news problem about a lot of data.

Now, Machine learning is one of solutions to solve problem about how to analyzing a big data. I'm starting to learn machine learning and start to learn with tensorflowjs becase I'm freindly to use javascript to make some services and applicaiton and I'm starting to read book Deep learning with Javascript. The book has overview the concept of programing rule and machine learning. This is good for understand and start learning machine learning

Programing rule is something you make application or some function. you write to rule when data input and return to something output when you write by rule.

Machine Learning is someting you have dataset and answer you putting to algorithm of machine to learn data and output to rule of data that cover the awnser when new data put the rule. The output of rule that we call Model.

The picture explain when you would like to use machine learning to solve problem in your application. we have 2 phase

Training Phase, First, You create machine learning model and need to have dataset and answer to trainning your model and when you trained model you have countinue to inference phase for using trained model

Inference Phase, Final, You apply trained model from model in Training Phase to your application or function in your code. when new data is comming to application or function the trainend model is processing your new data and output to your answer follow by tranined model that you make.

Monolithic vs Microservice, What do you choose?

PanupongDeve — Sat, 23 Jan 2021 05:59:03 +0000

I have already work software devlopment and my repository is Devops engineer. My company have devlopment process is agile, devops cuture and microservice for develop application in company.

I have a question that what do you choose monolithic or micro service for devlop software application.

Thank you for The book Microservice patterns
(https://www.amazon.com/Microservices-Patterns-examples-Chris-Richardson/dp/1617294543)
In the chapter one in the book make good decision for my result and The have many recommendation about implement, testing and deployment for you start develop microservice application.

Monolithic Architecture

picture from book Microservice pattern(page 3)

A monolithic architecture is the traditional unified model for the design of a software program and compose business logic application for the one source code and one service. (https://whatis.techtarget.com/definition/monolithic-architecture)

Benefit of Monolithic Architecture

- Simple to develop IDEs and other developer tools are focused on building a single application. - Easy to make radical changes to the application You can change the code and the database schema, build, and deploy. - Straightforward to test The developers wrote unit test, end-to-end tests that launched the application in one source code. - Straightforward to deploy have one source code for build and deploy result in easy to deploy (for small application I love it.) - Easy to scale can ran multiple instances of the application behind a load balancer(application sohuld be stateless. about stateless/stateful -> https://www.redhat.com/en/topics/cloud-native-apps/stateful-vs-stateless)

Drawback of Monolithic Architecture

- hard process develop for the big team imagin you have 100 developers in project. you have 50 or more commit, merging or pull request every day. they make overhaed process for you. - slow deployment when big source code very log for build, complie, deploy and testing. - hard for mange resource some bussiness logic code use a lot of resource such as generate report feature. you can't separate resource becase all bussiness logic store in same place source code - effect of compiled code broken some devloper make some broker code in source code result in all compiled code broken. - technology stack block example you have java application and have new feature. You try implement application with golang for new feature and golang application have performance and manage resource more than java application but you can't adapt to main application becase your main application is compiled with JAVA.

Productivity in Sofware Architecture

![Productivity in Sofware Architecture](https://martinfowler.com/bliki/images/microservice-verdict/productivity.png "Productivity in Sofware Architecture") (https://martinfowler.com/bliki/MicroservicePremium.html) In the picture you can see when you implement software with monolithic such as add feature or refactor code when source code is big you have a long time to compile unit test and build source code result in make one ploblem for slow producitvity but you implement software with microserivce same way like. The time conpile and build focus in new feature code or refactor code and not make effect of other service result and about git management you can separate source code and developer teams help you for reduce manage git such as many commit and pull request. All about of deployment time and proess development result in microservice faster productivity than monolic when your application is growing fallow the picture.

Microservice Architecture

![Microservice Architecture](https://dev-to-uploads.s3.amazonaws.com/i/gxh6unfnj4ctg5ug5xgq.png) picture from book Microservice pattern(page 13)

Microservices is an architectural style that structures an application as a collection of services that are

Highly maintainable and testable
Loosely coupled
Independently deployable
Organized around business capabilities
Owned by a small team

(https://microservices.io/)

Benefit of Micro Architecture

it enable the continuous delivery and deployment enable for CI/CD tools for deployment
Service are small easily maintained
Service are independency deployable about deployment you shouldn't recompile all service.
Service are independency scalable when one service scale all service needn't scale.
enable for separate team you can separate big team to many small team for implement and manage each service.
allow easy experimenting and adoption of new technologies you can separate service and team for implementing new framwork or new programing language.
increase reliability when some service is down but other service can still working.

Drawback of Micro Architecture

finding the right set of services is challenging
increase complex distributed systems are complex, which makes development, testing, and deployment difficult
increase networking traffic complex deploying features that span multiple services requires careful coordination.
adopt the microservice architecture is difficult micro service help your solution for you have big source code and big team developer but about many process suach as testing process, reliable of build ACID transaction process, deployment process and orther are change from monolithic result in difficult for adopt.

Conclusion

When you have a small project, demo project and small team monolithic architecture is a good way to use in devlop your software architecture. when your project is growing, a lot of feature is coming or you'll have a big team. Microservice architect is one of good way solution to adopt your software architecture development but when your adaopt you have a new many process develop and maintain your application and The book Microservice patterns (https://www.amazon.com/Microservices-Patterns-examples-Chris-Richardson/dp/1617294543) is one of good resource for help you develop application with microservice.

[AWS EKS] Setup Kubenetes Dashboard

PanupongDeve — Wed, 13 May 2020 22:57:38 +0000

step 0 - install package in linuux

sudo apt install jq curl

step 1 - deploy the K8s Metrics Server

DOWNLOAD_URL=$(curl --silent "https://api.github.com/repos/kubernetes-sigs/metrics-server/releases/latest" | jq -r .tarball_url)

DOWNLOAD_VERSION=$(grep -o '[^/v]*$' <<< $DOWNLOAD_URL)

curl -Ls $DOWNLOAD_URL -o metrics-server-$DOWNLOAD_VERSION.tar.gz

mkdir metrics-server-$DOWNLOAD_VERSION

tar -xzf metrics-server-$DOWNLOAD_VERSION.tar.gz --directory metrics-server-$DOWNLOAD_VERSION --strip-components 1

kubectl apply -f metrics-server-$DOWNLOAD_VERSION/deploy/1.8+/

step 2 - deploy the K8s dashboard

latest release of v2.xxx here: https://github.com/kubernetes/dashboard/releases

export DASHBOARD_RELEASE={{ latest releaseversion }}

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/$DASHBOARD_RELEASE/aio/deploy/recommended.yaml

kubectl apply -f admin-service-account.yaml

step 3 - access the dashboard

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret| grep {{ service_account_name }} | awk '{print $1}')

copy token

kubectl proxy --address='0.0.0.0' --accept-hosts='^*$'

open browser

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#!/login

[AWS EKS] Setup CloudWatch Metrics

PanupongDeve — Wed, 13 May 2020 22:15:54 +0000

Liquid syntax error: Unknown tag 'endraw'

[AWS EKS] Setup CloudWatch-logging

PanupongDeve — Wed, 13 May 2020 21:31:31 +0000

step1 - setup file cluster.yaml

step2 - enable cloudwatch log with eksctl

eksctl utils update-cluster-logging --config-file ((cluster-config.file.yaml)) --approve

loggin in aws console

CloudWatch -> log groups -> /aws/eks

disable cloudwatch logging

eksctl utils update-cluster-logging --name=((EKS-course-cluster-name)) --disable-types all --approve

[AWS EKS] set Autoscaler

PanupongDeve — Wed, 13 May 2020 02:20:25 +0000

Deploy the autoscaler

step1

kubectl apply -f https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

step2

kubectl -n kube-system annotate deployment.apps/cluster-autoscaler cluster-autoscaler.kubernetes.io/safe-to-evict="false"

step3

set your EKS cluster name at the end of property - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/<<EKS cluster name>>

set the image version at property image: asia.gcr.io/k8s-artifacts-prod/autoscaling/cluster-autoscaler:v1.18.1

see logs

kubectl -n kube-system logs deployment.apps/cluster-autoscaler

IAM-Policy for worker-nodes Auto scaling

IAM -> policies -> Create policy

attach policy in role of worker node for auto scaling

[AWS EKS] Setup Worker Nodes with EC2

PanupongDeve — Tue, 12 May 2020 21:10:35 +0000

install eksctl

https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html

install kubectl

https://kubernetes.io/docs/tasks/tools/install-kubectl/

eksctl-cluster-ec2.yaml

time eksctl create cluster -f eksctl-cluster-ec2.yaml

basic eksctl

eksctl get cluster

eksctl get nodegroup --cluster my-cluster-ec2

eksctl scale nodegroup --cluster=my-cluster-ec2 --nodes=5 --name=ng-1

update nodegroups

eksctl create nodegroup --config-file=cluster-ec2.yaml --include='ng-mixed'

delete nodegroups

eksctl delete nodegroup --config-file=cluster-ec2.yaml --include='ng-mixed' --approve