DEV Community: Mihindu Ranasinghe

Why AI Systems Become Expensive: Tokenization, Chunking, and Retrieval Design in the Cloud (AWS)

Mihindu Ranasinghe — Sat, 07 Mar 2026 15:40:08 +0000

When building modern AI knowledge systems, discussions often jump directly to prompts, retrieval pipelines, or model selection. However, long before a model generates an answer, something more fundamental happens that your data must be transformed into a format that models can understand and retrieve efficiently.

This transformation typically involves several foundational steps:

1. Tokenization – Converting raw text into model-readable units
2. Chunking – Splitting documents into manageable segments
3. Vectorization – Converting text into embeddings
4. Indexing – Storing vectors for efficient similarity search

These steps form the foundation of retrieval-based AI systems, and design decisions at this stage often have a greater impact on system performance than prompt engineering or model tuning.

These architectural considerations are also increasingly relevant for modern AI development tools such as Claude Code, OpenAI Codex–based systems, and other AI-powered coding assistants. Although these tools may appear to operate purely through conversational prompts, they often rely on similar retrieval pipelines under the hood, indexing codebases, documentation, and contextual information before generating responses. As a result, the same factors discussed in this article, such as tokenization, chunking strategies, embeddings, and retrieval design, can significantly influence both the performance and the token consumption of these systems.

In this article, we explore how these processes work together and how they can be implemented using Amazon Bedrock and Amazon OpenSearch Service.

The Retrieval Pipeline Begins Before Retrieval

Before a model retrieves or generates information, data must first pass through a preparation pipeline.

Each stage introduces trade-offs affecting:

Retrieval accuracy
Latency
Operational cost

Designing this pipeline carefully is essential for scalable GenAI systems.

Tokenization: How Models Read Text

Large language models do not process text directly. Instead, they operate on tokens, which are smaller units derived from text.

Tokens may represent:

Whole words
Parts of words
Punctuation
Whitespace

For example:
Cloud computing enables scalable AI systems.

might be tokenized as:
["Cloud", " computing", " enables", " scalable", " AI", " systems", "."]

Tokenization is necessary because models operate within fixed context windows, defined by the maximum number of tokens they can process at once.

However, token counts vary depending on the model's tokenizer.

Tokenization Algorithms

Tokenization is performed using trained algorithms rather than simple whitespace splitting.

Common approaches include:

Algorithm	Description
Byte Pair Encoding (BPE)	Merges frequently occurring character pairs
WordPiece	Used in many transformer-based models
Unigram Language Model	Probabilistic token selection
SentencePiece	Language-agnostic tokenizer framework

Many modern models rely on Byte Pair Encoding or similar subword tokenization techniques (Sennrich et al., 2016).

Example: Byte Pair Encoding

Consider the word: tokenization, A BPE tokenizer may split it as:
["token", "ization"] or ["token", "iz", "ation"] depending on how the tokenizer vocabulary was learned.

Subword tokenization allows models to represent rare or unseen words efficiently without requiring extremely large vocabularies.

Why Tokenization Matters

Tokenization affects several key aspects of AI system design.

Context Window Limits

Models can only process a limited number of tokens at once. Depending on the model, context windows may range from thousands to hundreds of thousands of tokens.

Large documents therefore need to be split into smaller chunks before processing.

Cost

Many generative AI platforms charge based on token usage. When generating embeddings or performing inference with Amazon Bedrock, token counts directly influence operational costs.

Retrieval Quality

Chunk sizes and overlaps are typically defined in tokens. Tokenization therefore influences:

Chunk boundaries
Embedding context
Retrieval precision

Token Cost Optimization in GenAI Pipelines

In production systems, whether powering AI assistants, coding copilots, or knowledge retrieval pipelines, token usage is not just a technical constraint, it is also a major cost driver. Most generative AI platforms charge based on the number of input and output tokens processed. When models are accessed through services such as Amazon Bedrock, inefficient prompt construction or overly large context windows can significantly increase operational cost.

Token usage grows quickly at scale. For example, a system sending 2,000 tokens per request and serving 10,000 requests per day processes 20 million tokens daily. Even small inefficiencies in chunking or retrieval strategies can therefore become expensive.

Design decisions such as chunk size, overlap, and retrieval precision directly influence token consumption. Smaller, well-structured chunks combined with accurate vector retrieval (for example using Amazon OpenSearch Service) help ensure that only the most relevant context is sent to the model.

Treating token efficiency as an architectural concern, not just a billing metric, can significantly improve the scalability and cost-efficiency of GenAI systems.

Chunking: Preparing Documents for Retrieval

Chunking refers to the process of splitting large documents into smaller segments prior to indexing.

Effective chunking improves:

Retrieval accuracy
Semantic coherence of embeddings
Query relevance

Poor chunking may result in fragmented information or irrelevant context.

Several strategies are commonly used.

1. Standard Chunking

The simplest method is fixed-size chunking, where text is split based on a predetermined token length.

Example configuration:

Chunk size: 300 tokens
Overlap: 20%

Overlap ensures that information spanning chunk boundaries remains available.

Typical parameters include:

Parameter	Purpose
Chunk size	Number of tokens per chunk
Overlap	Shared tokens between chunks

Many systems default to approximately 300 tokens per chunk, although optimal sizes vary depending on document structure and query patterns.

2. Hierarchical Chunking

Hierarchical chunking preserves document structure by splitting text into nested segments.

Example structure:

Document
 ├── Section
 │    ├── Paragraph
 │    │     ├── Sentence

This enables retrieval at multiple levels of granularity:

fine-grained paragraph retrieval
broader section-level context

Hierarchical chunking is particularly useful for large technical documentation or knowledge bases.

3. Semantic Chunking

Traditional chunking splits text mechanically. Semantic chunking instead splits text based on meaning and topic boundaries.

Instead of dividing text every fixed number of tokens, semantic chunking analyzes content to group sentences that represent a coherent concept.

Foundation models available through Amazon Bedrock can assist in identifying semantic boundaries.

This often produces more meaningful chunks, improving retrieval quality.

Vectorization: Converting Text into Embeddings

After chunking, each segment is converted into a vector embedding.

Embeddings represent text as numerical vectors capturing semantic meaning.

Example:
"cloud infrastructure"

might become:
[0.43, 0.24, 0.54, 0.12, 0.53]

Embeddings allow systems to compare semantic similarity between text segments.

Embedding generation can be performed using models provided through Amazon Bedrock.

Optimizing Embeddings

Embedding design has a direct impact on performance and cost.

Embeddings vary in Vector Dimensionality, and higher-dimensional vectors can capture more semantic nuance but require more storage and computational resources.

Indexing Embeddings for Retrieval

Once embeddings are generated, they must be stored in a structure supporting efficient similarity search.

Vector databases achieve this using Approximate Nearest Neighbor (ANN) algorithms, which allow searching across millions of vectors efficiently (Aumüller et al., 2020).

Amazon OpenSearch Service supports vector search capabilities using ANN indexing.

Typical workflow:

Example AWS Architecture

This architecture separates concerns across:

Ingestion
Processing
Embedding generation
Vector indexing
Retrieval

Example: Generating Embeddings with AWS Bedrock

import boto3
import json

bedrock = boto3.client("bedrock-runtime")

text = "Cloud architecture improves scalability and resilience."

response = bedrock.invoke_model(
    modelId="amazon.titan-embed-text-v1",
    body=json.dumps({"inputText": text})
)

embedding = json.loads(response['body'].read())

print(embedding)

The generated vector can then be indexed in Amazon OpenSearch Service for similarity search.

Final Thoughts

Tokenization, chunking, vectorization, and indexing form the core infrastructure of retrieval-based AI systems.
Although much attention is given to models and prompt engineering, the effectiveness of an AI system often depends more on how knowledge is structured and indexed before retrieval.
Platforms such as Amazon Bedrock and Amazon OpenSearch Service provide powerful building blocks for implementing these pipelines.
However, achieving high-quality retrieval ultimately depends on designing the data preparation process thoughtfully
Efficient tokenization and chunking strategies not only improve retrieval quality but also play a critical role in controlling token consumption and operational cost in large-scale GenAI systems.

References

Sennrich, R., Haddow, B., & Birch, A. (2016). Neural Machine Translation of Rare Words with Subword Units. ACL.
Vaswani, A. et al. (2017). Attention Is All You Need. NeurIPS.
Kudo, T., & Richardson, J. (2018). SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing.
Aumüller, M., Bernhardsson, E., & Faithfull, A. (2020). ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms.
Salton, G., & Buckley, C. (1988). Term-Weighting Approaches in Automatic Text Retrieval.
AWS Documentation – Amazon Bedrock
AWS Documentation – Amazon OpenSearch Vector Search

If you're building AI systems today, optimizing tokenization and chunking strategies early can save significant cost and complexity later.

Future of User Interface Design

Mihindu Ranasinghe — Wed, 06 Apr 2022 12:33:01 +0000

The next generation of User Interface (UI) design tools and how those UI trends impact on day to day life is really important. Before dive in to deep let’s see what user interface (UI) design is. UI can be defined as every visual aspect of a smart device, that enables the user to interact with the technology in a largely seamless manner. User interface (UI) design, also known as the process of creating user interfaces for machines and software, such as computers, household appliances, mobile phones, and other electronic devices, with the goal of increasing the ability to interact with them and improving the user experience. These days, design and user experience of applications are important than ever.

UI designing is now rapidly improving in the industry. Engineers are working on improves the look and feel and adding values to the products by developing better UIs. So it's time to look into upcoming designs shifts and trends and some incredible future UI designs.

User interface designs in future

Voice User Interface

Voice User Interface allows users to interact with a system through speech commands. Examples of VUI are virtual assistants, such as Siri, Google Assistant, and Alexa. With the better machine learning capability, the VUI comonad is now similar to human language. The advantage of VUI is work with voice interfaces is faster and saves time. For example, while we working on something we can provide voice commands without holding the current task.

Brain computer interface

In simpler words, a brain computer interface (BCI), also known as a direct neural interface or a brain machine interface, is a tool that allows a person (their brain) to communicate with a system (an external technology). Neurons in our brain send and receive signals from other nerve cells. These neurons produce brain waves, which operate the system in the brain-computer interface. The BCI captures brain waves and delivers them to a computer system to complete the work at hand. The brain generates electric impulses in response to our ideas, and each thought has its own brain wave pattern. The wave signal is used to control an object and express a concept.

Augmented Reality (AR)

Augmented reality is no longer a cutting-edge technology. Despite the fact that adoption is still in its early stages, companies have used the AR experience in a variety of apps, games, glasses, and systems. It has yet to reach its full potential, however.
AR augments the real-world environment by adding perceptual simulated or virtual material via computer-generated input, transforming objects in our environment into an interactive digital experience. It has expanded into a variety of industries, including healthcare, retail, gaming, entertainment, hospitality, tourism, education, design, and many more.

Dynamic Color Palets

Most mobile phones, adding feature of dynamic color palettes and helps the users to customize their phones and create a good-looking and harmonious result. Designers should keep dynamic color palettes in mind when it comes to UI. From 2022 onwards, such palettes have a high chance of being a staple on Android as more brands will have their own iteration of this trend.

Holographic user interfaces

A holographic display is a form of display that creates a virtual three-dimensional representation of an object via light diffraction. Holograms differ from other types of 3D imaging in that they do not require the use of special glasses or other external equipment to view the image. People have been able to express their views and imaginations through the medium of science fiction.

Biometric and cybernetic interaces

Cybernetics is a term used in computing to describe robotic systems, as well as their control and command. Biometrics, on the other hand, refers to biological markers that are present in every human being (as well as all other living things) and are generally unique to each individual. Fingerprint and retina scanners, for example, are frequently employed for security purposes.
Here are a few current biometric and cybernetic interface projects. For example, warfighter physiological, Status monitoring, fingerprint scanners, Digital glass...etc

Not only the explained above, users can gain many advantages in the future by using the interactive user interfaces. New technologies in UI facilitate users to do their tasks without wasting time in an enjoyable manner. When people looking at a screen/virtual object for a longer period human eyes may get tried. People may get isolated with their devices.

Designers should keep in mind the important elements that will keep the end users tight to the screen or the device.

WSO2 API Cloud - Adding mediation sequences, documents and API thumbnails using CLI Tool.

Mihindu Ranasinghe — Sun, 24 Jan 2021 14:07:41 +0000

WSO2 API Cloud delivers an enterprise-ready solution for creating, publishing, and managing all aspects of an API and its lifecycle. To register and get started with WSO2 API Cloud, go to http://wso2.com/cloud/api-cloud/

In my previous article, I was focusing on Designing, Initializing and Deploying APIs to the WSO2 API Cloud from the command line using the APIMCLI tool.

👉 Previous Article

WSO2 API Cloud projects initialization and deployment using the CLI Tool

In this article, we are focusing on adding mediation sequences, documents and API thumbnails from the command line using the APIMCLI tool, for APIs in the WSO2 API Cloud.

You can follow the same development environment configuration approach which I have explained in my previous article.

Once an API project is initialized and replaced the api.yaml file and swagger.json file with our actual files [EX api.yaml, swagger.json], I deployed the API using the CLI Tool to the WSO2 API Cloud platform as below :

The deployment approach was clearly explained in the previous article

Adding mediation sequences from CLI.

An API hosted in WSO2 API Cloud receives requests sent from your consumer applications to your backend, and then sends the responses from the backend to you. The default flow of events of an API hosted in WSO2 API Cloud is depicted in the diagram below.

If you need to modify or transform the requests and responses of your API, you need to engage a message mediation that intercepts the default flow. The API Gateway has a default mediation flow, which you can extend using mediation policies. You can create a custom mediation policy manually, or using a tool, and then engage it with the API. This will enable you to modify the default mediation flow for different use cases according to your requirement.

Learn more about WSO2 API Message Mediation Policies.
Here you can find out how to add mediation sequence policies from the WSO2 API Cloud Web UI

Let's see how we can do this with CLI

1. First of all you need to have prepared your in-flow, out-flow and
fault-flow Custom Mediation Policy files (.xml)

2. Here I have attached some sample sequence files for you to get hands-on experience and move forward with this article.

Download in-sequence , out-sequence, fault-sequence

3. You need to create a folder with the name of "Custom", inside every sequence directory in your API project folder structure and you have to keep your respective .XML files inside respective Custom folders.

As below :

SampleStore
│ ── api_params.yaml
│ ├── Docs
│ ├── Image
│ ├── Meta-information
│---------- ├── api.yaml
│---------- └── swagger.yaml
│ └── Sequences
│---------- ├── in-sequence
│-------------------- └── Custom
│------------------------------ └── in-seq.xml
│---------- ├── out-sequence
│-------------------- └── Custom
│------------------------------ └── out-seq.xml
│---------- └── fault-sequence
│-------------------- └── Custom
│------------------------------ └── fault-seq.xml

4. Then you need to enable custom mediation sequences in the API definition api.yaml file as well.

Just add these keys to the api.yaml :


# Mediation Sequence
inSequence: in_seq
outSequence: out_seq
faultSequence: fault_seq

Note that you can add and keep multiple sequence files in the custom folder and you can enable which one to be considered with these keys in api.yaml

5. Finally you need to log into the targeted tenant from the CLI as I explained in previous article, and import the API to the Targeted WSO2 API Cloud Tenant

#apimcli login <Environment> -u <Username> -p <Password> 
apimcli login wso2apicloud -u mihindu@wso2.com@development -p PASS54687 -k

Also you can login like this :


#apimcli login <Environment> 
apimcli login wso2apicloud
Username: mihindu@wso2.com@development
Password:

Logged into wso2apicloud environment

After logging in to the targeted tenant, you can import the API using the following command.


apimcli import-api -f ./path/SampleStore -e wso2apicloud --preserve-provider=false --update --verbose -k

Go back to WSO2 API Cloud Web Platform and you can confirm that your changes have been updated in web UI successfully like this.

Adding API Documents from the CLI.

WSO2 API Cloud enables you to upload documentations to your API projects from the Web UI.

Let's see how we can add documents to Cloud API Projects using the CLI tool.

1. You need to have prepared the documents that you want to upload to your API Project.

2. Keep your documents in the following directory.

SampleStore > Docs > FileContents > ...

SampleStore
│ ── api_params.yaml
│ ├── Docs
│---------- ├── FileContents
│-------------------- └── doc1.md
│---------- └── docs.json
│ ├── Image
│ ├── Meta-information
│---------- ├── api.yaml
│---------- └── swagger.yaml
│ └── Sequences
│---------- ├── in-sequence
│---------- ├── out-sequence
│---------- └── fault-sequence

3. Then you need to create and maintain a docs.json file to configure the documents with the API Cloud you added to the above directory.

SampleStore > Docs > docs.json

Here you can maintain a seperate object for each and every document you added and configure id,name,summary,sourceType,visibility,lastUpdate,filePath


[
  {
    "id": "61d20220-9b81-427f-a189-db933b06a070",
    "type": "HOWTO",
    "name": "ReadMe",
    "summary": "ReadMe",
    "sourceType": "FILE",
    "visibility": "API_LEVEL",
    "lastUpdated": "Jan 13, 2021 9:45:43 PM",
    "filePath": "README.md"
  }
]

5. Then you need to enable this key like below in api.yaml API definition as well.


# Enabling Documents Adding
documents: []

5. Finally again you need to log into the targeted tenant from the CLI as I explained above, and import the API to the Targeted WSO2 API Cloud Tenant


apimcli import-api -f ./path/SampleStore -e wso2apicloud --preserve-provider=false --update --verbose -k

Go back to WSO2 API Cloud Web Platform and you can confirm that your documents have been updated in web UI successfully like this.

Adding API Project Thumbnails from the CLI.

You can upload API Project Thumbnails from the WSO2 API Cloud Web UI as usual.

Here I will show you how you can add a thumbnail using the CLI tool.

1. There are few standard wso2 guidelines you need to follow when setting up a thumbnail image. (Download sample icon.jpeg)

Thumbnail image must be in the size of 100px X 100px
Thumbnail image must be renamed as "ïcon.jpeg"

2. Then you need to place the image in the following directory.

SampleStore
│ ├── Image
│---------- └── icon.jpeg

3. Finally again you need to log into the targeted tenant from the CLI as I explained above, and import the API to the Targeted WSO2 API Cloud Tenant


apimcli import-api -f ./path/SampleStore -e wso2apicloud --preserve-provider=false --update --verbose -k

Go back to WSO2 API Cloud Web Platform and you can confirm that your thumbnail changes have been updated in web UI successfully like this.

👉 What's Next?

Testing WSO2 API Cloud APIs using the CLI Tool

Thank You

Hope you all enjoyed and learned something from this. Let me know your comments and suggestions in the discussion section.

👉 Visit me - https://mihinduranasinghe.com/

WSO2 API Cloud projects initialization and deployment using the CLI Tool

Mihindu Ranasinghe — Sat, 23 Jan 2021 07:41:32 +0000

WSO2 API Cloud delivers an enterprise-ready solution for creating, publishing, and managing all aspects of an API and its lifecycle. To register and get started with WSO2 API Cloud, go to http://wso2.com/cloud/api-cloud/

You can take a look at the WSO2 API Cloud Documentation to quickly try out designing , creating and publishing APIs from the Web UI, subscribing to it from the Web UI of the WSO2 API Cloud

In this article, we are focusing on Designing, Initializing and deploying APIs from the command line using the APIMCLI tool.

👉 Pre-requisites

1.Download API Controller based on your preferred platform (i.e., Mac, Windows, Linux).

2.Extract the downloaded archive containing the CLI Tool to a preferred location and navigate to it from the terminal (using cd ).

3.Navigate to the working directory where the executable CLI Tool resides.

4.Execute the following command to start the CLI tool.

./apimcli

5.Add the location of the extracted folder to your system's $PATH variable to be able to access the executable from anywhere.

apimcli

1. Adding the cloud gateway environment


apimcli add-env -n wso2apicloud \
                      --registration https://gateway.api.cloud.wso2.com/client-registration/register \
                      --apim https://gateway.api.cloud.wso2.com/pulisher \
                      --token https://gateway.api.cloud.wso2.com/token \
                      --import-export https://gateway.api.cloud.wso2.com/api-import-export \
                      --admin https://gateway.api.cloud.wso2.com/api/am/admin/ \
                      --api_list https://gateway.api.cloud.wso2.com/api/am/publisher/apis \
                      --app_list https://gateway.api.cloud.wso2.com/api/am/store/applications

Make sure the environment is configured by this command :

apimcli list envs

2. Create a new API project

Usage:

You can follow one of these example methods to initialize a project.

apimcli init [project path] [flags]
Examples:

# Initialize with default swagger.yaml file and default API definition (You have to replace the generated API definition and Swagger with your actual values)
apimcli init SampleStore
# Initialize with your actual swagger file path
apimcli init SampleStore --oas petstore.yaml
# Initialize with your actual swagger file URL
apimcli init SampleStore --oas https://petstore.swagger.io/v2/swagger.json
# Initialize with your actual swagger file and API Definition file
apimcli init SampleStore --oas ./swagger.yaml -d definition.yaml

A project folder with the following structure will be created in the given directory.

├── api_params.yaml
├── Docs
│ └── FileContents
├── Image
├── Meta-information
│ ├── api.yaml
│ └── swagger.yaml
├── README.md
└── Sequences
│ ├── fault-sequence
│ ├── in-sequence
│ └── out-sequence

I have attached 2 sample files for api.yaml and swagger.json for your reference.

You can get the hands-on experience by replacing below files with generated Meta-information in your directory and move forward with the article.

If you want to learn more on api.yaml configuration, here you can find out gist

3. Deploying the API project to a targeted tenant in API Cloud

First of all you need to log into your targeted tenant in API Cloud by providing targeted tenant username & password .

#apimcli login <Environment> -u <Username> -p <Password> 
apimcli login wso2apicloud -u mihindu@wso2.com@development -p PASS54687 -k

Also you can login like this :

After logging in to the targeted tenant, you can import the API using the following command.


apimcli import-api -f ./path/SampleStore -e wso2apicloud --preserve-provider=false --update --verbose -k

Make sure the API project is deployed successfully by this command :

# apimcli list apis -e <environment> -k
apimcli list apis -e wso2apicloud -k

Here the deployed APIs are listed:

You can log into the WSO2 API cloud portal and see the project is deployed successfully.

How to export the API Project from the logged in tenant from the CLI ?


# # apimcli export-api -n <API-name> -v <version> -r <provider> -e <environment> -u <username> -p <password> -k
# # apimcli export-api --name <API-name> --version <version> --provider <provider> --environment <environment> --username <username> --password <password> --insecure
apimcli export-api -n SampleStore -v 1.0.0 -r mihindu@wso2.com@development -e wso2apicloud -k

Once you exported successfully you will receive a response with the exported directory in your local machine.

Response :

Successfully exported API!
Find the exported API at /Users/kim/.wso2apimcli/exported/apis/dev/SampleStore_1.0.0.zip

DO YOU KNOW

How to update an API Cloud Project Thumbnail from the CLI ?
How to add Mediation Sequences for the API Cloud Project from the CLI ?
How to add API Scopes from the CLI ?
How to add Documents for the API Cloud Project from the CLI ?
How to invoke an API in WSO2 API Cloud with a generated API Token, from the CLI ?

👉 What's Next?

These sections will be covered in-detail in the next article.

Thank You

Hope you all enjoyed and learned something from this. Let me know your comments and suggestions in the discussion section.

👉 Visit me - https://mihinduranasinghe.com/

Using Docker Containers In Jobs - GitHub Actions

Mihindu Ranasinghe — Fri, 04 Dec 2020 15:13:35 +0000

👉 Prerequisites

If you are new to GitHub Actions, I suggest you to read these articles

As we discussed in previous articles, we can run our workflows in virtual machines by using operating systems that GitHub Actions provides as GitHub Hosted Runners.

Using Docker Container

We can use a docker container which will be installed on the the virtual machine & then instead of running directly on virtual machine, our workflow will runs on a particular container.

This can be an image from Docker Hub.

Docker Hub

Docker Hub is the platform which allows users to create, manage, and deliver their container applications. Docker Hub is the largest library of container images.

Link: Docker Hub

Once you click on an image (For example Node), you can see all the dockerfile links with the respective tags.

How to use ?

You have to provide the,
publisher-dockerhub-username : docker-image-tag with the image tag under the "container:"

We can not use containers on windows and mac-os virtual machines.
We can use containers only on ubuntu machines.


name: workflow-name
on: push

jobs:
  job-name:
    runs-on: ubuntu-latest
    container:
      image: dockerhub-username:docker-image-tag

Example:


name: container
on: push

jobs:
  node-docker:
    runs-on: ubuntu-latest
    container:
      image: node:14.15.0-alpine3.12

Also you can provide some other parameters like environment variables, ports & options...etc


name: container
on: push

jobs:
  node-docker:
    runs-on: ubuntu-latest
    container:
      image: node:14.15.0-alpine3.12

     #env:
     #  ENV1: Available to this container only

     #ports: To expose our container

     #options: --cpus 1 --host

So in next figure you can confirm that steps are running inside the container we initialized with dockerhub image, instead of running directly on the virtual machine.


name: container
on: push

jobs:
  node-docker:
    runs-on: ubuntu-latest
    container:
      image: node:14.15.0-alpine3.12
    steps:
         - name: Log the node version
           run: |
             node -v
             cat /etc/os-release

Here we echo the node version of the image and some operating system information of the docker container.

Output:

We can also specify multiple containers to run as services.

If you have an application to run multiple services, you might need multiple images to run in our virtual machine. Normally we use Docker-Compose to do something like that, **but in GitHub Actions we have a similar way to run this type of applications.

Running multiple docker containers in a job.

Instead of having container here, we can provide "services: "
We can give the services as docker-images under the "services:" and they run as separate containers in our virtual machine.


name: workflow-name
on: push

jobs:
  job-name:
    runs-on: ubuntu-latest
    services:
      service-name-1:
        image: dockerhub-username:docker-image-1
        ports:
           - 3001:3000

      service-name-2:
        image: dockerhub-username:docker-image-2
        ports:
           - 4001:4000

    steps:
      - name: Get from service-1
        run: http://localhost:3001/api/users

      - name: Get from service-2
        run: http://localhost:4001/api/items


      - name: post a user example
        run: "curl -x POST http://localhost:3001/api/user -H 'Content-Type: application/json' -d '{\"username"\: \"hello"\, \"address"\: "sampleaddress"}' "

Here it creates two containers for service-1 & service-2 inside our virtual machine

In order to communicate between those two containers, we can use the service-name as the host name in the app.

Running docker containers in individual steps.

In GitHub Actions, we can use different containers in different steps as well.

We can use "with:" key to provide inputs to this container in the particular step
We can give an entrypoint in with: key and override the entrypoint of the dockerfile
To find some entrypoints of the echo, node ...etc use this commands

Ex 1:

type -a  echo

Output - Entrypoint of echo in ubuntu:

echo is  /bin/echo

Ex 2 :

type -a  node

Output - Entrypoint of node in ubuntu:

node is  usr/local/bin/echo

Use this "type -a " command in ubuntu to find some Entrypoints of the installed tools & softwares.

Example:


name: container
on: push

jobs:
  node-docker:
    runs-on: ubuntu-latest
    container:
      image: node:14.15.0-alpine3.12
      steps:
         - name: Log the parent container node version
           run: |
             node -v 
           # Echo the node version of the parent container


         - name: Log the step container node version
           uses: docker://node:12.14.1-alpine3.10
           with:
             entrypoint: usr/local/bin/echo
             args: -v
            # Echo the node version of this step container based on this 12.14.1-alpine3.10 image


         - name: Step with docker to echo a string
           uses: docker://node:12.14.1-alpine3.10
           with:
             entrypoint: /bin/echo
             args: Hello Docker !

Push the workflow file into the repository & check the Actions window of the repository

Outputs:

v14.15.0
v12.14.1
Hello Docker !

👉 What's Next?

Trigger workflows manually using the repository dispatch event

Thank You

Hope you all enjoyed and learned something from this. Let me know your comments and suggestions in the discussion section.

👉 Visit me - https://mihinduranasinghe.com/

Working with external actions - GitHub Actions (Part 3)

Mihindu Ranasinghe — Tue, 01 Dec 2020 20:06:34 +0000

👉 Prerequisites

If you are new to GitHub Actions, I suggest you to read these articles

Here is the cool thing about GitHub actions.

Instead of automating each and every step by your own from the scratch , you can use reusable codes in Github Actions.

Mainly there are two types of reusable actions.

We can create private reusable actions and use them inside our local workflows.
We can use public reusable actions from the marketplace

In this article, let's have a look at how we can use publicly available external actions from the marketplace and how we can use them inside our workflow.

GitHub Actions Marketplace

GitHub Actions Marketplace is where you can find out reusable codes/workflows also known as "actions" implemented by someone in the GitHub Actions community.

Most of the reputed cloud companies like AWS, Google, Azure have published their own public actions for their clients.

Not only companies, also the individuals in the Github Actions Community have build up and published some useful actions in the Marketplace

Link : Marketplace

You can search for actions in the search bar of the Marketplace.
Once you click on an action, you will be redirected to a readme(documentation) page and there you can find a brief about the action and how we can use it in our workflow and what is the reference(signature) of the particular workflow and what are the inputs we have to give when we use the action in our workflow.

How to reference a public action ?

There are several ways that we can reference a public action from the marketplace.

Referencing a branch
Referencing a version
Referencing a commit

Here is the basic structure of a step to use a public action :

steps:
 - name: Any-relevant-name-for-the-step
   id: step-1-id(Optional)
   uses: #reference
   with: #inputs

We have to provide a reference with the "uses:" tag and we can provide relevant inputs under "with:" tag.

Referencing a branch:


uses: publisher-username/repo-name@branch-name

Referencing a version:

Find the releases from the marketplace for the particular action


uses: publisher-username/repo-name@v1.1.0

Referencing a commit ID :

Explore the commit history of the action and find a specific commit ID


uses: publisher-username/repo-name@commit-ID

Note :

If you reference a branch, actually it is not really preferred. Some breaking changes might occur when the publisher changes the code in the particular branch you referred. Then your workflow might fail.
If you reference a specific Version, It will help to mitigate this problem.
The safest way is to have a stable workflow is to use specific commit ID to reference an action.

An easier way to find the reference of an action

Go to the action page in the marketplace.
Click on this green color box or drop down and find the specific version.
copy paste the reference.

For example lets use this Hello world javascript action

If you have read the documentation, there are few inputs you can provide.

I provide here the "who-to-greet" input with a sample name.


name: name-of-the-workflow

on: [push]

jobs:
    sample-job-1:
        runs-on: ubuntu-latest
        steps:
            - name: step-1
              uses: actions/hello-world-javascript-action@v1.1
              with:
                 who-to-greet: Mihindu

Output :

Hello Mihindu!

Another useful action for any workflow.

Before working with the repo files from the workflow steps, we need to clone our repo files into the virtual machine that we created in the job.

So this is something we do at very first of many workflows. So that you can use this checkout public action and clone repo files into VM very easily.


name: name-of-the-workflow

on: [push]

jobs:
    sample-job-1:
        runs-on: ubuntu-latest
        steps:
            - name: Clone-repo-files-into-vm
              uses: actions/checkout@v2.3.4

            - run: ls

There are many more useful actions in the marketplace. Take your time and refer them and use them in your workflows according to the purpose.

👉 What's Next?

Using Docker Containers In Jobs - GitHub Actions (Part 4)

Thank You

Hope you all enjoyed and learned something from this. Let me know your comments and suggestions in the discussion section.

👉 Visit me - https://mihinduranasinghe.com/

Working with environment variables - GitHub Actions (Part 2)

Mihindu Ranasinghe — Mon, 30 Nov 2020 16:07:15 +0000

👉 Prerequisites

If you are new to GitHub Actions, I suggest you to read my Introduction to Github Actions - GitHub Actions (Part 1) article.

There are mainly three types of environment variables in GitHub Actions.

Custom Environment Variables .
Default Environment Variables.
Encrypted Environment Variables.

1. Custom Environment Variables

Custom environment variables can be declared by ourselves inside any workflow file.

We can declare custom environment variables under different access scope in a workflow file.

Workflow level
Job level
Steps level

So then the declared variable can be accessed by the particular scope only.

Example:

Workflow level custom environment variables

Here I have declared a variable called PUBLICENV in workflow level and trying to echo it in different levels.

name: workflow-level-custom-environment-variables

on: [push]
# Here the environment variable is declared in workflow level
env:
    PUBLICENV: Available for all jobs in this workflow

jobs:
    sample-job-1:
        runs-on: ubuntu-latest
        steps:
            - name: step-1
              run: echo "${PUBLICENV}"
              # This will echo the value/string of PUBLICENV

            - name: step-2
              run: echo "${PUBLICENV}"
              # This will echo the value/string of PUBLICENV

    sample-job-2:
        runs-on: ubuntu-latest
        steps:
            - run: echo "${PUBLICENV}"
            # This will echo the value/string of PUBLICENV

Job level custom environment variables

Here I have declared a variable called JOBENV in "sample-job-1" level and trying to echo it in different levels.


name: job-level-custom-environment-variables

on: [push]

jobs:
    sample-job-1:
        env:
            JOBENV: Available for this specific job only
            # Here the environment variable is declared in job level
        runs-on: ubuntu-latest
        steps:
            - name: step-1
              run: echo "${JOBENV}"
              # This will echo the value/string of JOBENV

            - name: step-2
              run: echo "${JOBENV}"
              # This will echo the value/string of JOBENV

    sample-job-2:
        runs-on: ubuntu-latest
        steps:
            - run: echo "${JOBENV}"
            # This will NOT echo the value/string of JOBENV

Step level custom environment variables

Here I have declared a variable called STEPENV in "step-1" level inside the "sample-job-1" and trying to echo it in different levels.


name: step-level-custom-environment-variables

on: [push]

jobs:
    sample-job-1:
        runs-on: ubuntu-latest
        steps:
            - name: step-1
              env:
                  STEPENV: Available for this specific job only
                  # Here the environment variable is declared in step level

              run: echo "${STEPENV}"
              # This will echo the value/string of STEPENV

            - name: step-2
              run: echo "${STEPENV}"
              # This will NOT echo the value/string of STEPENV

    sample-job-2:
        runs-on: ubuntu-latest
        steps:
            - run: echo "${STEPENV}"
            # This will NOT echo the value/string of STEPENV

2. Default Environment Variables

GitHub provides some default environment variables for many useful parameters of your repository. We can use them anywhere inside the workflow.

Here is a list of default env variables in GitHub Actions

GITHUB_WORKFLOW The name of the workflow.
GITHUB_RUN_ID A unique number for each run within a repository. This number does not change if you re-run the workflow run.
GITHUB_RUN_NUMBER A unique number for each run of a particular workflow in a repository. This number begins at 1 for the workflow's first run, and increments with each new run. This number does not change if you re-run the workflow run.
GITHUB_ACTION The unique identifier (id) of the action.
GITHUB_ACTIONS Always set to true when GitHub Actions is running the workflow. You can use this variable to differentiate when tests are being run locally or by GitHub Actions.
GITHUB_ACTOR The name of the person or app that initiated the workflow.
GITHUB_REPOSITORY The owner and repository name.
GITHUB_EVENT_NAME The name of the webhook event that triggered the workflow.
GITHUB_EVENT_PATH The path of the file with the complete webhook event payload.
GITHUB_WORKSPACE The GitHub workspace directory path.
GITHUB_SHA The commit SHA that triggered the workflow.
GITHUB_REF The branch or tag ref that triggered the workflow.
GITHUB_HEAD_REF Only set for forked repositories. The branch of the head repository.
GITHUB_BASE_REF Only set for forked repositories. The branch of the base repository.
GITHUB_SERVER_URL Returns the URL of the GitHub server.
GITHUB_API_URL Returns the API URL.
GITHUB_GRAPHQL_URL Returns the GraphQL API URL.

Example:


name: default-environment-variables

on: [push]

jobs:
    sample-job:
        runs-on: ubuntu-latest
        steps:
            - name: echo-default-env-variables
              run: |
                  echo "Home: ${HOME}"
                  echo "GITHUB_WORKFLOW: ${GITHUB_WORKFLOW}"
                  echo "GITHUB_ACTIONS: ${GITHUB_ACTIONS}"
                  echo "GITHUB_ACTOR: ${GITHUB_ACTOR}"
                  echo "GITHUB_REPOSITORY: ${GITHUB_REPOSITORY}"
                  echo "GITHUB_EVENT_NAME: ${GITHUB_EVENT_NAME}"
                  echo "GITHUB_WORKSPACE: ${GITHUB_WORKSPACE}"
                  echo "GITHUB_SHA: ${GITHUB_SHA}"
                  echo "GITHUB_REF: ${GITHUB_REF}"

Outputs:

3. Encrypted Environment Variables

In GitHub Actions, we can create encrypted environment variables as well. We can use GitHub Secrets to store API keys and passwords kind of things.

Click on the settings in the repository
Click on the secrets
Click on the "New Repository Secret"
Give YOUR_SECRET_NAME and the VALUE and click on the "Add Secret"

Example:

Here is an example code how we can re use encrypted variables without exposing the value.


name: working-with-encrypted-environment-variables

on:
  push: 
    branches: [main, develop]

jobs:
    sample-job:
        runs-on: ubuntu-latest
        steps:

            - name: Deploy-to-staging
              if: github.event_name == 'push' && github.ref == 'refs/heads/develop'
              # Here this will be triggered only when the "push" event is performed only to the specific "develop" branch
              run: npx surge --project ./build --domain eight-circle.surge.sh
              env:
                  SURGE_LOGIN: ${{secrets.SURGE_LOGIN}}
                  SURGE_TOKEN: ${{secrets.SURGE_TOKEN}}
                  # SURGE_LOGIN & SURGE_TOKEN is stored as github secrets

👏 Bonus Tip :

In the Actions log screen, when any step is passed, it indicates a green check mark and when a step is failed, it indicates a red check mark but we can not see the exact error there unless enabling debugging.

To enable debugging in GitHub Actions, store these two encrypted environment variables in github secrets in the perticular repository. Then when a step is failed, you will see debug messages in the same log screen & it makes you easier to debug your errors.


Secret : ACTIONS_STEP_DEBUG 
Value  : TRUE

Secret : ACTIONS_RUNNER_DEBUG
Value  : TRUE

👉 What's Next?

Working with external actions - GitHub Actions (Part 3)

Thank You

Hope you all enjoyed and learned something from this. Let me know your comments and suggestions in the discussion section.

👉 Visit me - https://mihinduranasinghe.com/

Introduction to GitHub Actions ( Part 1 )

Mihindu Ranasinghe — Sun, 29 Nov 2020 13:36:26 +0000

What is GitHub Actions ?

GitHub Actions makes it really easy to automate all your software workflows with automating CICD pipelines, Building, Testing and Deploying your code directly from your GitHub repository even without any third party integrations like Jenkins.

We can automate code reviews, branch management and issues and we can trigger any workflow with the GitHub events.

GitHub Actions Overview

Here you can see and understand the big picture of GitHub Actions workflow.

GitHub Events

I hope you have a basic idea about what GitHub events are. If you are familiar with Git & GitHub you should have worked with GitHub events.

Pushing a code into a repository branch
Opening & closing a pull request
Code reviewing
Merging
Opening and closing Issues
Assigning issues
Re-opening issues like that, We can trigger any automated workflow when a GitHub event is performed.

So actually this is the beauty of GitHub Actions.

GitHub Server

GitHub actions are triggered inside a GitHub server. We can create multiple workflows & multiple jobs in GitHub actions. As you see on the above figure, each and every Job we create and script, is running on a separate virtual machine inside the GitHub server.

We can automate actions as steps inside each and every job such as step1 step2 step3...

Virtual machines in GitHub server which are also known as Runners.

VMs (Runners)

A runner is also known as any virtual machine with the GitHub Actions Runner Application is installed. Because GitHub Actions Runner is the core application part of GitHub actions which allows to run/trigger any workflow in this platform.

Mainly there are two types of runners as you see on this figure such as GitHub hosted runners and Self Hosted runners.

👏GitHub Hosted Runners

GitHub Hosted runners are some virtual machines that github team provides us to use directly.

For example, Linux. Windows, Mac OS virtual machines are there with pre-installed software like GIT, CURL, NPM, YARN, PIP, NODE.

These GitHub hosted machines are ,

Easy to use
We can not customize it as we want.
Fully maintained by the GitHub team.

👏Self Hosted Runners

Self hosted runners are the machines that we can manage and maintain as we want.

You have full control over the hardware, software, operating system of the vm and also the tools which need to be configured to automate some commands.

Most of the cloud companies like AWS Azzure GCP have developed their CICD automation action setups and published in GitHub Actions Marketplace for their clients.

Triggering Point

Mainly In GitHub actions we have to work with YAML files. If you are not familiar with YAML format and YAML files please feel free to read this article about YAML.

First of all we need to have a repository and that should be in public.

Specially note that private repositories cannot be used with GitHub actions in the free version of GitHub.

When it comes to the triggering point,

We have to create a folder called “.github‘ in the root of the repository and we have to create a folder called “workflows”, inside the “.github” folder. We have to create our own YAML file inside the “workflows” folder.

Here is the standard triggering point of GitHub actions.

Workflow Overview

Here you can see a sample screenshot of triggering a workflow.

Actions window is where we can monitor our workflows.
Here you can see the workflow name, and jobs on the left side of this window.
GitHub actions logs every step in the automation very clearly
When a step is passed, it indicates a green check mark or when a step is failed it indicates a red check mark as well.

Also we can debug those errors as well in this log screen when a step is failed.

Basic structure of a GitHub Actions Workflow

First of all create a github repository and initialize it and create a yml file in the triggering point of the repository. (.github > workflows > sample.yml)

👏👏 Here is the basic structure...


name: Any-Name #Name of the workflow

on: [push]
# On which events this workflow should be triggered. We can use this as an array as well

# on: [push, pull_request, issue]

# We can specify events like this as well
# on:
#   pull_request:
#     types: [opened, closed, reopened]

jobs:
    run-shell-command:         # Name of the job 1
        runs-on: ubuntu-latest # Selecting a GitHub Hosted Runner (Here selected an ubuntu machine)

        steps:                      # Automated Steps of the job
            - name: echo a string   # Name of the Step 1 (Optional)
              run: echo "Hello"     # command of the step 1

            - name: multiline commands  # Name of the Step 2 
              run: |                    # Multiline commands of the step 2
                  node -v
                  npm -v

            - name: cloning repo files into the vm   # Consuming pre published actions from the marketplace
              uses: actions/checkout@v2.3.4          # Signature of the action you want to use here from the market place

    run-windows-command:              # Name of the job 2
        runs-on: windows-latest       # Selecting a GitHub Hosted Runner (Here selected a windows machine)
        needs: ["run-shell-command"]  # If we want to run the first job before triggering the this job, we can set 'needs'
        steps:
            - name: Echo the Directory
              run: pwd

Please note that, keeping indents is a must when it comes to github actions workflows.
you can set many types of events to trigger the workflow in "on:" section as follows.

Events which can trigger a worfkflow Link

We can specify individual steps to be triggered for specific events as well with if conditions. (It will be explained in a separate article)
Once the scripting is done, you can push this into the remote repository and see how the workflow is triggering in "actions" window of the github repository.
Consuming public actions from the marketplace, an in-detail article will be available soon.

👉What's Next?

Working with environment variables - GitHub Actions(Part 2)

Thank You

Hope you all enjoyed and learned something from this. Let me know your comments and suggestions in the discussion section.

NOTE :

This article is for giving you a start to GitHub Actions and giving a brief introduction regarding what GitHub Actions is and what we can do with GitHub Actions.
I will be writing few articles with some hands-on demonstrations how we can use GitHub Actions in real world.

👉 Visit me - https://mihinduranasinghe.com/

Software Licensing

Mihindu Ranasinghe — Tue, 08 Sep 2020 03:44:59 +0000

A software license is a document that provides legally binding guidelines for the use and distribution of software.
Software licenses typically provide end users with the right to one or more copies of the software without violating copyrights. The license also defines the responsibilities of the parties entering into the license agreement and may impose restrictions on how the software can be used. Software licensing terms and conditions usually include fair use of the software, the limitations of liability, warranties and disclaimers and protections if the software or its use infringes on the intellectual property rights of others.

Why Software Licensing Is Important

Businesses must be savvy in the licenses they purchase to ensure firstly they are using software legally and secondly, they aren't paying for licenses that aren't being used. By acquiring too many software licenses you're wasting company resources, without enough you leave yourself liable to a potential lawsuit (which is quite costly). Finding the right license agreement can also make it easier to manage software in your company.

Ownership vs. licensing

Many proprietary or open source software houses sell the SW copy with a license to use it. There isn't any transferring of ownership of the good to the user, which hasn't the warranty of a for life availability of the software, nor isn't entitled to sell, rent, give it to someone, copy or redistribute it on the Web.

License terms and conditions may specify further legal clauses that users can't negotiate individually or by way of a consumer organization, and can uniquely accept or refuse, returning the product back to the vendor. This right can be effectively applied where the jurisdiction provides a mandatory time for the good decline right after the purchase (as in the European Union law), or a mandatory public advertisement of the license terms, so as to be made readable by users before their purchasing.

In the United States, Section 117 of the Copyright Act gives the owner of a particular copy of software the explicit right to use the software with a computer, even if use of the software with a computer requires the making of incidental copies or adaptations (acts which could otherwise potentially constitute copyright infringement). Therefore, the owner of a copy of computer software is legally entitled to use that copy of software. Hence, if the end-user of software is the owner of the respective copy, then the end-user may legally use the software without a license from the software publisher.

The ownership of digital goods, like software applications and video games, is challenged by "licensed, not sold" EULAs of digital distributors like Steam. In the European Union, the European Court of Justice held that a copyright holder cannot oppose the resale of a digitally sold software, in accordance with the rule of copyright exhaustion on first sale as ownership is transferred, and questions therefore the "licensed, not sold" EULA. The Swiss-based company UsedSoft innovated the resale of business software and fought for this right in court. In Europe, EU Directive 2009/24/EC expressly permits trading used computer programs

Methods for Licensing Software

An end-user license agreement (EULA) is one approach vendors can take to license their software. This is a contract between the licensor and purchaser, establishing the purchaser's right to use the software. The contract may include the different ways the software can be used as well as any rights the buyer has obtained by purchasing the software. This is one of the more basic and commonly used ways to license software.

If you're using SaaS and as a result your applications are cloud based, the software license is usually subscription based. That is, you will pay for each user on a monthly (or some other period) basis. This type of software license offers greater flexibility. It is also beneficial in that you only pay for what you need " allowing you to scale your business without repercussions.

There's also the question of whether you can re-sell your software license if you're no longer using it. There is no black and white answer to this question. The answer can generally be found in the EULA. From a legal standpoint there may be question marks over the enforceability of EULA's given to users after they purchased the software given they weren't aware of these conditions at the time the contract was formed. However I will not go into depth about questions of law.

Another method of licensing software is by white labelling it. This is where a product is created by one company and then re-branded by another company. As a result, the software/product belongs to the company that created it. There is a software license in place which permits the company benefiting from the white label to use the software.

Different types of software licenses require you to meet certain obligations if you want to reuse the code.

Here are 5 common types of software licenses.

If you write code, you also reuse code, including code snippets, libraries, functions, frameworks, and entire applications. All software code comes with certain rights and obligations if you want to add it to your codebase. Free and open source software (FOSS) is free of cost, but you aren’t free to use it as you wish. Even unlicensed code snippets copied from Stack Overflow have obligations for reuse. But formally developed code usually comes with a specific software license.

There are many different types of software licenses, and the penalties for license noncompliance can be harsh. If you reuse a component without following the obligations of its license, the licensor might sue, and you might be forced to publish your own source code. To protect your code and your organization, you need to understand these software licenses before using any code, including libraries and frameworks, you didn’t write yourself. See our list of the top open source licenses and their potential legal risks.

Public domain.

This is the most permissive type of software license. When software is in the public domain, anyone can modify and use the software without any restrictions. But you should always make sure it’s secure before adding it to your own codebase.

Warning:

Code that doesn’t have an explicit license is NOT automatically in the public domain. This includes code snippets you find on the internet.

Permissive.

Permissive licenses are also known as “Apache style” or “BSD style.” They contain minimal requirements about how the software can be modified or redistributed. This type of software license is perhaps the most popular license used with free and open source software. Aside from the Apache License and the BSD License, another common variant is the MIT License.

LGPL.

The GNU Lesser General Public License allows you to link to open source libraries in your software. If you simply compile or link an LGPL-licensed library with your own code, you can release your application under any license you want, even a proprietary license. But if you modify the library or copy parts of it into your code, you’ll have to release your application under similar terms as the LGPL.

Copyleft.

Copyleft licenses are also known as reciprocal licenses or restrictive licenses. The most well-known example of a copyleft or reciprocal license is the GPL. These licenses allow you to modify the licensed code and distribute new works based on it, as long as you distribute any new works or adaptations under the same software license.
For example, a component’s license might say the work is free to use and distribute for personal use only. So any derivative you create would also be limited to personal use only. (A derivative is any new software you develop that contains the component.)

The catch here is that the users of your software would also have the right to modify the code. Therefore, you’d have to make your own source code available. But of course, exposing your source code may not be in your best interests.

Proprietary.

This is the most restrictive. The idea behind it is that all rights are reserved. It’s generally used for proprietary software where the work may not be modified or redistributed.

How Does Licensing Work On The WorkingMouse Platform

The License Agreement between WorkingMouse and our customers gives them access to use the platform for developing and maintaining their software application(s). It allows customers to issue support tickets to WorkingMouse for assistance with server administration, bug fixing and minor enhancements. The license ensures WorkingMouse can support customers as they embark on their journey.

WorkingMouse also offers an innovation platform which is used by multiple businesses. Since all these businesses use the same underlying platform they will all feel the benefit of improvements. For this reason, a business that starts using the platform will feel many benefits immediately, e.g. a mobile-app can operate in an offline/online mode and re-sync with the server as required. Everyone gets the capability as it's built into the platform. If a company did not want to share a new capability it is possible to fork the code base and they could continue on their own branch.

However, this would mean they would then not receive any more benefits from being inside the business collective hence the innovation process slows down.

👉 Visit me - https://mihinduranasinghe.com/

Ethical Whistleblowing

Mihindu Ranasinghe — Sat, 25 Jul 2020 15:51:57 +0000

Today we are going to have a look at Ethical Whistleblowing.

What is Ethical Whistleblowing ?

Whistleblowing can be simply specified as passing on information concerning wrongdoing, such as corruption, harassment or actions believe to be misconduct, and not in line with the company’s Code of Conduct. It is the act of telling the authorities or the public that the organization you are working for is doing something immoral or illegal. This will affect your organization or government in good or bad. Whistleblowing is a valuable and powerful tool in organizational corporate governance strategy because it gives employees the way to deal with incidence of misconduct and protect the interest of individuals, organizations or public.

Who is a Whistleblower? ?

A whistleblower is a person who exposes secretive information or activity within a private or public organization that is deemed illegal, unethical, or not correct.
This can be classified into many categories violation of company policy/rules, law, regulation, or threat to public interest/national security, as well as fraud, and corruption.

Elements of ethical whistleblowing.

There are 4 elements of ethical whistleblowing.

1) the whistleblower
2) the whistleblowing act or complaint
3) the party to whom the complaint is made
4) the organization against which the complaint is lodged

There are two types of whistleblowers: internal and external.

-Internal whistleblowers are those who report the misconduct, fraud, or indiscipline to senior officers of the organization.

-External whistleblowers are those who report the wrongdoings to people outside the organization such as the media, public, or police.

Whenever we talk about act of whistleblowing there is a name that automatically comes up in history, Edward Snowden. He was a former CIA employee who leaked classified and restricted information to the public from the United States National Security Agency in 2013.

Link of a famous interview with Edward Snowden:
https://www.theguardian.com/world/2013/jun/09/edward-snowden-nsa-whistleblower-surveillance

Steps/procedure for whistleblowing

Hire a powerful attorney:- Recruit a gifted legal advisor with involvement with dealing with informant cases can exhortation lawfully to accumulate proof against the business and secure proficient notoriety.

Make sure you have evidence:- The legislature won’t pay you an award for bringing them flawed tattle of offenses, but instead explicit verification, for example, messages, inside examinations, charging records, test results, names, the contact data of included gatherings, and different types of narrative proof.

File a complaint:- Informants must document an objection in court and submit it to the national government with a Disclosure Statement that subtleties the speculated unfortunate behavior.

Play the waiting game:- On the off chance that the administration ought to choose to continue forward with your case, it is essential to show restraint.

Prepare to be “outed”:- You need to set yourself up to be “outed” and the outcomes that it might bring. It’s prescribed to look for a new position as quickly as time permits subsequent to recording an informant guarantee.

Be prepared to testify:- On the off chance that the administration chooses to make a move against your manager dependent on the consequences of its examination, you might be approached to affirm at a great jury continuing or during the preliminary. You should be set up to uncover your character, since your declaration may considerably affect the result of the case.

Thank you !

👉 Visit me - https://mihinduranasinghe.com/

Working with AJaX JSON

Mihindu Ranasinghe — Fri, 17 Jan 2020 09:41:26 +0000

IF YOU ARE NEW TO THIS ARTICLE, I RECOMMEND TO READ "Introduction To Asynchronous JavaScript" and "XmlHttpRequest (XHR) Object Request" ARTICLE BEFORE STARTING THIS.
Introduction To Asynchronous JavaScript
XmlHttpRequest (XHR) Object Request

Here we are going to GET some data from a .JSON file and display them in a simple web page

There are two types of data in a json file to GET

Get single object data from a json file
Get multiple data from an array of a json file (fetching an array)

Note:

We have created a customer.json file with a single customer's data

{
    "id": 1,
    "name": "Mihindu ranasinghe",
    "company": "CyberCatsIT",
    "phone": "11-222-555"
}

We have created a customers.json file with a multiple customers' data

[

{
    "id": 1,
    "name": "Mihindu ranasinghe",
    "company": "CyberCatsIT",
    "phone": "11-222-555"
},

{
    "id": 2,
    "name": "Kavindu ranasinghe",
    "company": "CyberCatsIT",
    "phone": "11-222-555"
},

{
    "id": 3,
    "name": "Supun ranasinghe",
    "company": "CyberCatsIT",
    "phone": "11-222-555"
} 

]

And a simple web page with two buttons as "Get Customer" and "Get Customers".

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/skeleton/2.0.4/skeleton.css" integrity="sha256-ECB9bbROLGm8wOoEbHcHRxlHgzGqYpDtNTgDTyDz0wg=" crossorigin="anonymous" />

    <title>Document</title>
</head>
<body>
   <div class="container">

       <button id="button1">Get Customer</button>
       <button id="button2">Get Customers</button>
       <br><br>
       <div class="" id="output"></div>
       <h2>Customer</h2>
       <div class="" id="customerOutput"></div>

       <h2>Customers</h2>
       <div class="" id="customersOutput"></div>
   </div> 
<script src="app.js"></script>
</body>
</html>

👏👏Let's code the JavaScript part - app.js...

Get single object data from the customer.json file

document.getElementById('button1').addEventListener('click',loadCustomer);

//Load single customer
function loadCustomer(e){
    const xhr = new XMLHttpRequest();
    xhr.open('GET','customer.json',true);

    xhr.onload = function(){
        if(this.status === 200){//Check Http status is Ok or not
            //Display on console
            console.log(this.responseText);

            //Displaying on web
              const customer = JSON.parse(this.responseText);

              const output = `
                <ul>
                    <li>ID: ${customer.ID} </li>
                    <li>Name: ${customer.name} </li>
                    <li>Company: ${customer.company} </li>
                    <li>Phone: ${customer.phone} </li>
                </ul>
            `;

            document.getElementById('customerOutput').innerHTML = output;


        }
    }
    xhr.send();

Get multiple data from an array of a customers.json file (fetching an array)

document.getElementById('button2').addEventListener('click',loadCustomers);

//load ustomers
function loadCustomers(e){
    const xhr = new XMLHttpRequest();
    xhr.open('GET','customers.json',true);

    xhr.onload = function(){
        if(this.status === 200){//Check Http status is Ok or not

            //Displaying on web
             const customers = JSON.parse(this.responseText);
            let output = '';
            customers.forEach(function(customer){
                output += `
                <ul>
                    <li>ID: ${customer.id} </li>
                    <li>Name: ${customer.name} </li>
                    <li>Company: ${customer.company} </li>
                    <li>Phone: ${customer.phone} </li>
                </ul>
            `;

            });

        document.getElementById('customersOutput').innerHTML = output;


        }
    }
    xhr.send();
}

You can do those two functions in a single app.js file.

Output

👏👏This works in the similar way if you are fetching something from an external URL from a public API. For example, getting the github users from their public API.

In this case we are using a local json file

👉What's Next?
IF YOU HAVE DONE READING "Working with AJaX JSON" ARTICLE, I SUGGEST YOU TO READ MY NEXT ARTICLE.

How to use external API?

Thank You

Hope you all enjoyed and learned something on this. Please let me know your comments suggestions and any questions you have about this blog.

👉 Visit me - https://mihinduranasinghe.com/

XmlHttpRequest (XHR) Object Request

Mihindu Ranasinghe — Fri, 17 Jan 2020 04:04:40 +0000

NOTE

IF YOU ARE NEW TO THIS ARTICLE, I RECOMMEND TO READ "Introduction To Asynchronous JavaScript" ARTICLE BEFORE STARTING THIS.
Introduction To Asynchronous JavaScript

Lets code how to Get some data from a text file & display them in a web page using XHR Object

Text file name "data.txt"
Html page "index.html" (Only contains a button with the #id of 'button').


<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/skeleton/2.0.4/skeleton.css" integrity="sha256-ECB9bbROLGm8wOoEbHcHRxlHgzGqYpDtNTgDTyDz0wg=" crossorigin="anonymous" />

    <title>Document</title>
</head>
<body>
   <div class="container">
       <button id="button">Get Data</button>
            <div class="" id="output"></div>
    </div> 
<script src="app.js"></script>
</body>
</html>

👏👏Let's code the JavaScript part - app.js...

document.getElementById('button').addEventListener('click',loadData);
function loadData(){

    //Create an XHR object
    const xhr = new XMLHttpRequest();

    //OPEN
    xhr.open('GET','data.txt',true); //Get Method used
    console.log('READYSTATE', xhr.readyState);//Just to check the ready state

    xhr.onload = function(){ //readystate is 4 
        console.log('READYSTATE', xhr.readyState);//just to check the ready state
        if(this.status === 200){//Check whether the http state is 200 or not

            document.getElementById('output').innerHTML=`<h1>${this.responseText}</h1>`;
        }
    }

   xhr.send();
}

Output When press 'Get Data'

So what is "READYSTATE" and "HTTP status" ??

READYSTATE means the status of your request readyness. There are some values of ReadyState with representing several meanings.

👏ReadyState Values:

    0: request not initialized
    1: server connection established
    2: request received
    3: processing request
    4: request finished and response is ready

👏HTTP Status:

    200:"OK"
    403:"Forbidden"
    404:"Not Found"

Also you can GET data using this below method instead of "xhr.onload",
You can try this method and get the same result.

NOTE

This method is now not in use... This is a very old method. I never recommend it.


//This one is old method

    xhr.onreadystatechange = function(){
        console.log('READYSTATE', xhr.readyState);
        if(this.status === 200 && this.readyState === 4){
            console.log(this.responseText);
        }
    }

👏👏onerror

You can display an error message if something bad happened to your request

xhr.onerror=function(){
        console.log('request error...');
    }

    xhr.send();

👏👏Bonus Tip

when the readystate is 3, that means it is in processing state. So you can implement a Spinner or a Loader while processing a request.

This code line will easily do it for you


xhr.onprogress = function(){ //readystate is 3, u can use it for loader, spinners
        //console.log('READYSTATE', xhr.readyState);
        //Spinner or Loader
    }

👉What's Next?
IF YOU HAVE DONE READING "XmlHttpRequest (XHR) Object Request" ARTICLE, I SUGGEST YOU TO READ MY NEXT ARTICLE.

How to make/code XHR Object Request using AJaX ?

Thank You

Hope you all enjoyed and learned something on this. Please let me know your comments suggestions and any questions you have about this blog.

👉 Visit me - https://mihinduranasinghe.com/