DEV Community: Nir Adler

Kubernetes Administrator (CKA)

Nir Adler — Sat, 03 Dec 2022 12:02:25 +0000

Hi, I'm doing the CKA certificate and wanted to share some details.

Purchase the exam certificate at this link, the exam cost 395$ and you get 2 tries, and 2 demo tries at killer.sh

Preparation course on Udemy

Kubernetes Overview

Setting up the local environment using minikube

brew install minikube
minikube start
kubectl get po -A

you can set a default kube editor altho I recommend staying with vim.

KUBE_EDITOR=nano

recommended shell configuration

alias k=kubectl # will already be pre-configured
export do="--dry-run=client -o yaml" # k create deploy nginx --image=nginx $do
export now="--force --grace-period 0" # k delete pod x $now

Introduction to Kubernetes Tools

Check out the kubectl cheatsheet, here are some examples of common tasks.

kubectl config get-contexts # print config contexts 
kubectl config use-context my-ctx # switch context to my-ctx
kubectl run nginx --image=nginx # create a pod with name nginx and image nginx:latest
kubectl get pods -o wide # list pods in defauult namespace with extra details
kubectl run nginx --image=nginx --dry-run=client -o yaml # print yaml configuration to run pod nginx
kubectl create deployment --image=nginx nginx --replicas=4 --dry-run=client -o yaml > nginx-deployment.yaml # print yaml configuration to create deployment nginx and direct the output to a new file
kubectl create -f nginx-deployment.yaml # create resources from yaml template
kubectl apply -f nginx-deployment.yaml # apply resources from yaml template
kubectl describe node cluster1-master1 # get resource details
kubectl get pod -A --sort-by=.metadata.uid # print all the pods in the cluster and sort them by the field .metadata.uid
kubectl get pod -o jsonpath="{range .items[*]} {.metadata.name}{.spec.containers[*].resources}{'\n'}" # manipulate output with jsonpath
k expose pod nginx --name nginx-service --port 80
k create ns my-namespace
k -n secret create secret generic mysecret --from-literal=user=myuser --from-literal=pass=1111
kubectl scale --current-replicas=2 --replicas=3 deployment/mysql

kubectl is not the only tool you will need, make sure to read about etcdctl, and others.

Terminal tools you should know: (short list for some of the advanced question more tools and Linux familiarity is needed)

vim
wc
grep

Topics you should master:

Configurations location:

/etc/kubernetes
/etc/kubernetes/manifests

you are allowed to use the Kubernetes docs while solving the exam, you can use these bookmarks for faster navigation in the Kubernetes docs, make sure to use only the CKA exam folder, you are not allowed to search in any site that is not the official docs while solving the test.

the bookmarks folder did not create by me, I found it online

Statikly - No hassle full stack framework

Nir Adler — Fri, 02 Dec 2022 13:27:34 +0000

Statikly is a full-stack framework for developers who want to create SSR/static sites, great for blog/static/content sites, and amazing for prototype and building internal tools and SEO optimisation.

Provide an alternative for writing frontend with endless packages and single page application frameworks like React/Vue, more and more heap is growing on server side UI, like next.js and remix, and I wanted to simplify it even more.

Statikly is based on fastify framework and ecosystem plugins, it's trying to not be very opinionated but to support strong defaults.

let's create our first project

npm i -g statikly nodemon
mkdir statikly
cd statikly
statikly init # clone niradler/statikly-demo
npm run watch # visit localhost:3000

The code behind this page can be found in views/index.ejs if you wonder where the rest of the HTML coming from we use the layout feature, the page layout is at partials\layout.ejs configured as env var

STATIKLY_LAYOUT=./partials/layout.ejs

in .env make sure you don't commit .env with secrets to git.

<section>
  <h1>Todo example</h1>
  <a href="/todos"><button>My Todos</button></a>
</section>

<figure>
  <img src="public/image.jpg" alt="Minimal landscape" />
</figure>

Go ahead and play with the demo todo app to figure out how you can pass data to your view with loaders, checkout the todos loader for example views\todos\loader.js

Supported env vars:

NODE_ENV=production # optional: set in production
STATIKLY_ROOT= # optional: set to override current folder
STATIKLY_STATIC_FOLDER=public # optional: for other public folder
STATIKLY_TEMPLATE=ejs # optional: template engine to use for the complete list @fastify/view
STATIKLY_LAYOUT= # optional: layout path
STATIKLY_VIEWS=views # optional: for other views folder
STATIKLY_PASSWORD=1234 # optional: basic auth
STATIKLY_USERNAME=user # optional: basic auth

Another helpful command wil be running statikly --help

Serverless Authentication with Golang

Nir Adler — Mon, 04 Jul 2022 12:40:16 +0000

Almost every project needs authentication to serve secure user content.

For most of my side projects, I prefer to pay per use for cost optimization.

continuing my experiment with Golang, I created Golang service that's going to run inside AWS Lambda function and store the data in Dynmodb database, I'm using serverless AWS services to fulfil my requirement of cost optimize service.

GitHub repo

Features

Email/password login
Forgot password
Provider login using google/github and more
Multi tenant(orgs)
Store user data on signup
Basic roles
Invite users
Emails template
JWT

Supported routes:

Cloudflare automatic DNS update for homelab

Nir Adler — Sat, 26 Mar 2022 08:31:40 +0000

First of all, I love Cloudflare, I think they have a great product, I use Cloudflare to manage all my DNS, anyone that has a homelab setup knows you need to figure out a way to keep Cloudflare IP update if your home IP is changed, there are many great solutions out there but I love to create my own tools, so I created cloudflaresync.

My home lab setup is a set of docker-compose files, everything is dockerized so this was the obvious choice, I'm trying to learn Golang so written in Go, and the code is very simple, but taylormade to my need.

# docker-compose
version: '3.4'

services:
  cloudflaresync:
    image: niradler/cloudflaresync
    env_file:
      - .env

The code is very straightforward, we get the configuration from a set of env vars, and configure a cron task to run every x time to update DNS records with new IP, I'm also using it to create new records when a new service is created.

// main.go
package main

import (
    ...

    "github.com/cloudflare/cloudflare-go"
    "github.com/joho/godotenv"
    "github.com/robfig/cron/v3"
)

....

func main() {
    err := godotenv.Load()
    if err != nil {
        log.Println("Error loading .env file")
    }
    updateRecords()
    log.Println("Start cron", time.Now())
    cronExpression, exist := os.LookupEnv("CRON")
    if !exist {
        cronExpression = "0 * * * *"
    }
    log.Println("cron:", cronExpression)
    c := cron.New()
    id, err := c.AddFunc(cronExpression, updateRecords)
    if err != nil {
        log.Fatal("cron AddFunc error:", err)
    }
    log.Println("cron id:", id)
    c.Start()
    log.Println("Cron Info: ", c.Entries())

    go forever()

    quitChannel := make(chan os.Signal, 1)
    signal.Notify(quitChannel, syscall.SIGINT, syscall.SIGTERM)
    <-quitChannel

    fmt.Println("done")
}

Supported env vars:

CLOUDFLARE_API_TOKEN=<key>
CLOUDFLARE_DOMAIN=example.com
CLOUDFLARE_SUB_DOMAINS=test,home
CRON=*/2 * * * *
PROXIED=true

Running example for raspberry pi.

docker run --name cloudflaresync --env CLOUDFLARE_API_TOKEN=<key> --env CLOUDFLARE_SUB_DOMAINS=<app,home> --env CLOUDFLARE_DOMAIN=<example.com> --restart unless-stopped niradler/cloudflaresync:armv7

Policyer Github Action

Nir Adler — Sun, 05 Dec 2021 22:14:26 +0000

My submission to the GitHub Actions x DEV Hackathon 2021!

My Workflow

Policyer is an open-source project (more like a vision) I created after being inspired by policy engines that become very popular lately (OPA,Checkov)

Policyer going to focus on providing a platform to run and create meaningful reports, data engagement and a plugin system to let you provide any data, sometimes it can be k8s YAML and in other cases, it can be user data.

Policyer Action

The policyer-action lets you the option to run policyer as part of your CI process, in my example I'm going to validate GitHub SDK calls.

The provider is like a plugin for policyer engine, it provides the data so the engine can run it against the checks (polciies)

It's important for me to emphasise that Policyer provide a platform, and eventually I will want to see a marketplace full of people custom providers.

The action can use any provider either local or published to NPM (support for private registries is on the way). In my example, I created a simple provider to run GitHub SDK calls.

Example Repo

Example Check:

---
configuration:
  provider: github-provider
  type: rest
  validEvents:
    - pull_request
    - push
  domain: pulls
  action: listRequestedReviewers
  args:
    owner: context.payload.pull_request.base.user.login
    repo: context.payload.pull_request.base.repo.name
    pull_number: context.payload.pull_request.number
checks:
  - id: validate-reviewers
    name: check if reviewers exists.
    severity: High
    steps:
      - path: data.users
        condition: includes
        value: "nirtester"
        utility: map
        utilityProps:
          - "login"

(just a reminder this is a policy example and the GitHub action will evaluate it and output it as a report)

Check flow:

first of all we setup the configuration section where we can provide metadata for the check, in this example I'm asking the provider to do an SDK call:

SDK[pulls]listRequestedReviewers

octokit docs
next, we going to dive into the actual policy, in this policy we want to verify a certain user is a reviewer, so after the call I'm going to point to the "users" array, then use the condition includes ([...users].includes(value)), utilities function by default includes all Lodash functions, you can add custom utilities in the provider level.

I'm going to use the map utility function to prepare an array of reviewers' usernames.
the final step is the results:

Submission Category:

Wacky Wildcards

Action YAML

# Add github action file .github/workflows/policyer.yml
name: Policyer

on: [pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Policyer GitHub Action
        uses: policyerorg/policyer-action@v0.0.3-alpha
        with:
          verbose: false
          provider: policyer-github
          internal: false
          checks_path: ./checks

Additional Resources / Info

Visit Policyer for more information this is just the beginning

Packages used

chalk
figlet
jmespath
lodash
moment
yaml
yargs
@actions/core/github

Policyer Action

Nir Adler — Sun, 05 Dec 2021 21:54:55 +0000

My Workflow

Policyer is an open source project (more like a vision) I created after inspired by policy engines that become very popular lately (OPA,Checkov)
Policyer going to focus on providing platform to run and create meaningful reports, data engagement and plugin system to let you provide any data, some time it can be k8s yaml and in other it can be user data.

Policyer Action

The policyer-action let you the option to run Policyer as part of your CI process, in my example I'm going to validate GitHub SDK calls.

Provider is like a plugin for policyer engine, it provide the data so the engine can run it against the checks (polciies)

Its important for me to emphasize that Policyer provide a platform, and eventually I will want to see marketplace full of custom providers.
The action can use any provider either local or published to NPM (support for private registries is on the way). In my example I created a simple provider to run GitHub SDK calls.

Example Repo

Example Check:

---
configuration:
  provider: github-provider
  type: rest
  validEvents:
    - pull_request
    - push
  domain: pulls
  action: listRequestedReviewers
  args:
    owner: context.payload.pull_request.base.user.login
    repo: context.payload.pull_request.base.repo.name
    pull_number: context.payload.pull_request.number
checks:
  - id: validate-reviewers
    name: check if reviewers exists.
    severity: High
    steps:
      - path: data.users
        condition: includes
        value: "nirtester"
        utility: map
        utilityProps:
          - "login"

(just a reminder this is a policy example and the Github action will evaluate it and output as a report)

Check flow:

first of all we setup the configuration section where we can provider meta data for the check, in this example I'm asking from the provider to do an SDK call:

SDK[pulls][listRequestedReviewers]({
                  owner: ...pull_request.base.user.login
                  repo: ...pull_request.base.repo.name
                  pull_number: ...pull_request.number
})

octokit docs

next we going to dive in to the actual policy, in this policy we want to verify a certain user is a reviewer, so after the call im going to point to the "users" array, then use the condition includes ([...users].includes(value)), utilities function by default includes all Lodash functions, you can add custom utilities in the provider level. I'm going to use the map utility function to prepare an array of reviewers usernames.
final step is the results:

  ____       _ _                      
 |  _ \ ___ | (_) ___ _   _  ___ _ __ 
 | |_) / _ \| | |/ __| | | |/ _ \ '__|
 |  __/ (_) | | | (__| |_| |  __/ |   
 |_|   \___/|_|_|\___|\__, |\___|_|   
                      |___/           
Visit us at policyer.org
Event name: pull_request
Valid events: pull_request,push

┌────────────────────┬──────────┬──────────────────────────────┬──────────────┬─────────────────┬───────────┬──────────┐
│      (index)       │ hasError │            check             │ stepsResults │ inspectedValues │  status   │ severity │
├────────────────────┼──────────┼──────────────────────────────┼──────────────┼─────────────────┼───────────┼──────────┤
│ validate-reviewers │  false   │ 'check if reviewers exists.' │   [ true ]   │   [ [Array] ]   │ 'success' │  'High'  │
└────────────────────┴──────────┴──────────────────────────────┴──────────────┴─────────────────┴───────────┴──────────┘

Submission Category:

Wacky Wildcards

Action yaml

# Add github action file .github/workflows/policyer.yml
name: Policyer

on: [pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Policyer GitHub Action
        uses: policyerorg/policyer-action@v0.0.3-alpha
        with:
          verbose: false
          provider: policyer-github
          internal: false
          checks_path: ./checks

Additional Resources / Info

Visit Policyer for more information this is just the beginning

Packages used

chalk
figlet
jmespath
lodash
moment
yaml
yargs
@actions/core/github

Policyer - the first JS policy engine

Nir Adler — Sun, 21 Nov 2021 11:10:57 +0000

Policyer is an open-source project (more like a vision) I created after being inspired by policy engines that become very popular lately (OPA,Checkov)

Policyer going to focus on providing a platform to run and create meaningful reports, data engagement and a plugin system to let you provide any data, sometimes it can be k8s YAML and in other cases, it can be user data.

GitHub repo

Getting started

checkout the example repo

git clone https://github.com/niradler/policyer-todo
cd policyer-todo
npm i
node cli.js -o ./report.json -p ./checks -i false

Checks format:

---
configuration:
  provider: todo-provider
  type: resource
  resource: todo
  payload:
    id: 1
checks:
  - id: todo-id-check
    name: check if todo has an id.
    severity: High
    steps:
      - path: id
        condition: equal
        utility: isInteger
        value: true
      - path: id
        condition: equal
        value: 1
  - id: todo-title-check
    name: check if todo has a title.
    severity: Warning
    steps:
      - path: title
        condition: not
        utility: isEmpty
        value: true
  - id: todo-completed-check
    name: check if todo has a valid completed field.
    severity: Warning
    steps:
      - path: completed
        condition: includes
        value:
          - true
          - false

A check is a set of conditions or policies to enforce, many utilities can help you build complex policies, and you can also add your own when writing your provider.

Policyer is modular, and by nature prefer to be behind the scene and let you customise the user-facing side by the Provider as best fit to your need.

checkout the example repos, contribution is welcome.

Lyve Cloud hackathon Submission - 1st Place

Nir Adler — Mon, 26 Jul 2021 21:43:56 +0000

Introducing a new approach to cloud storage. Lyve Cloud from Seagate is your simple, trusted and efficient storage as a service. Long-term cost predictability means youll never be surprised by your cloud bill. Put your data to work with always-on availability, world-class security, and cloud flexibility from the global leader in mass data storage management.

What is s3-syncer?

s3 synchronization automation, deploy syncer to your AWS account and connect to the bucket you wish to sync with Lyve Cloud storage. using an s3 notification syncer will reflect any change in the source bucket and also in the target bucket and keep them in sync.

Demo: https://www.loom.com/share/11e098376d8548ddb35a1f6ec4266e2e

Presentation:

https://www.israelclouds.com/news/seagate-cloud-hackathon-winners

https://lyvehack.startisrael.co.il/

https://www.linkedin.com/posts/niradler_first-place-had-a-great-experience-activity-6831281878838849538-mUp5

AWS CloudFormation Outputs

Nir Adler — Sun, 14 Mar 2021 12:31:44 +0000

My favourite way to deploy resources on AWS is to use Cloudformation (cf), I'm using CDK/Serverless/SST to create my Cloudformation templates.

often you will want to share data between stacks, the recommended way will be stack outputs, to use this output in build (CI/CD) time I needed an easy tool to pull the output into a machine-readable file, for this task I created cfexport npm module.

let's look at the following example, we have cf template that deploys our API to AWS APIgateway service, next we have a pipeline to build our client code and upload it to s3, during the build we want to transfer our APIgateway URL to the client build script.

let's look at how we do that with cfexport

npm i cfexport

next, let's update our scripts filed in package.json

"pre-build":"cfexport compile -v --file \"./.env.template\" --region us-east-1 --output \"./.env\" --format \".env\""

and create our template file .env.template

REACT_APP_APPLICATION_URL=DEV-APIGATEWAY-URL // you can use --prefix DEV- and then just put APIGATEWAY-URL
REACT_APP_IGNORE_PARAM=!dont-change // value starting with ! will be ignored.

running npm run pre-build will create .env file, and replace DEV-APIGATEWAY-URL value with the export value (export name should be = DEV-APIGATEWAY-URL), values starting with ! will be ignored and ! will be removed.

REACT_APP_APPLICATION_URL=https://{restapi_id}.execute-api.{region}.amazonaws.com/{stage_name}/
REACT_APP_IGNORE_PARAM=dont-change

Cortx hackathon Submission - 2nd Place

Nir Adler — Fri, 11 Dec 2020 22:41:11 +0000

CORTX is a distributed object storage system designed for great efficiency, massive capacity, and high HDD-utilization. CORTX is 100% Open Source

I choose to leverage the advanced storage with great compatibility for creating Cloudinary clone, for those who don't know Cloudinary is a saas platform to manage images at scale, they have features like editing images on the fly to the size you need and many more, the project called CORTX-Images, you can find the code here

Demo:

Presentation:

Commit secrets to Git (encrypted)

Nir Adler — Sat, 04 Apr 2020 15:27:31 +0000

As part of the covid-19 extra free time, I'm learning google cloud and terraform.

My first experiment was to deploy a simple docker file to cloud run service and to set up a custom domain.

When I got the results needed, I wanted to commit the files to Github, but I've found out that the terraform files can expose secrets. To avoid using a third party solution, I decided to use a tool to encrypt the files before committing. I looked for a simple solution, self-contained and portable, so I decided to create my own solution.

git-secrets - simple npm package that can be used with husky (git hooks), to transparent encrypt and decrypt files in your repo.

Setup:

npm i -S git-secrets husky

  "scripts": {
    "start": "node src/server.js",
    "infra:init": "terraform init",
    "infra:plan": "terraform plan",
    "infra:deploy": "terraform apply",
    "infra:destroy": "terraform destroy",
    "secret:init": "./node_modules/.bin/git-secrets init",
    "secret:hide": "./node_modules/.bin/git-secrets hide",
    "secret:reveal": "./node_modules/.bin/git-secrets reveal"
  },
  "husky": {
    "hooks": {
      "pre-commit": "npm run secret:hide && git add .",
      "post-commit": "npm run secret:reveal"
    }
  },

npm run secret:init

Now, add files you would like to encrypt before committing them to the config file.

.git-secrets (can be any other file by setting env variable GIT_SECRETS_CONFIG)

terraform.tfstate
variables.tf
secrets.json

The next step is to choose your secret password and pass it to git-secrets. You can pass the key by cli param (--key=secret), env variable (GIT_SECRETS_KEY), and by creating the key file (make sure you add this file to .gitignore, filename=.git-secrets.key)

The final step is to test all our configurations, commit the changes we just added and check the files on GitHub to see the result.

disclaimer: this is a work in progress and not safe for production or enterprise projects, but can do the trick for self-projects when the risk is low.

Javascript object reamp

Nir Adler — Tue, 11 Feb 2020 09:59:30 +0000

in my daily job as a full stack js developer I often need to manipulate js objects, either remove fields or transform to a new structure etc.

I decided to create an npm module object-remap, to make this repeated task more reliable, reduce code, and create cleaner code.

let's look at the examples:

const objectRemap = require("object-remap");

const obj = {
  a: 1,
  b: 2,
  c: 3,
  d: {
    e: 4,
    f: [1, 2, 3]
  }
};

//simple usage
const fields = ["a", "b", "c"];

let newObj = objectRemap(obj, fields);
console.log({ obj, newObj });
/*
{
  obj: { a: 1, b: 2, c: 3, d: { e: 4, f: [Array] } },
  newObj: { a: 1, b: 2, c: 3}
}
*/

//advance usage
const fieldsMap = [
  {
    origin: "a",
    target: "a"
  },
  {
    origin: "d.e",
    target: "e"
  },
  {
    origin: "d.f",
    target: "f",
    formatter: data => `${data.length} items`
  }
];

newObj = objectRemap(obj, fieldsMap);
console.log({ obj, newObj });
/*
{ 
  obj: { a: 1, b: 2, c: 3, d: { e: 4, f: [Array] } },
  newObj: { a: 1, e: 4, f: '3 items' } 
}
*/

for getting started install the package with npm:

npm i -S object-remap