DEV Community: Kiit Framework

Using an Application Identity

kishore — Tue, 16 Feb 2021 02:33:12 +0000

This post discusses using an application identity ( a convention based unique name to identify any app ) to organize, identify an application, and properly link the identity to the logs, metrics, alerts, and other diagnostics of the application. This was designed for a Kotlin framework called Slate Kit; codebase at Git, and code for this component here at Identity.kt.

Example

We use the convention below to create a unique name representing the identity of any application. This name uniquely identifies an application, provides ownership, environment, version info and is used to appropriately link it to various diagnostics( logs, alerts, metrics) when running.

{area}.{name}.{type}.{env}.{version}.{instance}
signup.alerts.job.qat.v2.4a3b300b-d0ac-4776-8a9c-31aa75e412b3

Real-World

We already have conventions for identifying various resources, some obvious examples in the real world are below. So why not have one for an application ?

uri: https://github.com/slatekit/slatekit - account + repo
ip : 129.144.59.56 - network + host
phone : 800-123-4567 - u.s phone format
SSN : 123-45-6789 - u.s Social Security Number

Use-cases

The examples above clearly resolve the value to a computer or a person. This allows for supporting a lot of other things like lookup, ownership, tracing, and diagnostics. A production application usually involves the following

ownership : simply knowing who owns/manages an app
logs : log for info, debug, warnings, errors
metrics: metrics being sent to providers like newrelic/datadog
alerts : alerts being sent out ( e.g. pagerduty, slack )

An identity is especially useful for organizations that could have dozens to hundreds of applications. But even for small groups, startups, side projects, an identity can come in handy to distinguish between a background job, an api, or a bot that may all be related to a specific product area or service.

Packages

One way to typically identify a running app is to simply use a git repo and some type of namespace ( e.g. java package name ) to serve as some form of identity. For example slatekit.samples. But this is pretty vague doesn't contain useful information. It also says nothing about what type of app it is, its environment, version info, and/or some other attributes.

app: github.com/account/app
package : slatekit.samples

The package name may be used in logs, metrics. But a package name is a loose standard with not enough info, especially for a running app.

Solution

A simple solution is to come up with a strict naming convention for the identity of app ( and enforced in code ) that can capture some important attributes to build the name or identity. I've decided on the following convention.

{area}.{name}.{type}.{env}.{version}.{instance}
signup.alerts.job.qat.v2.4a3b300b-d0ac-4776-8a9c-31aa75e412b3

This examples represents a background job we use in the signup/registration portion of an mobile app being worked on, that generates alerts, emails, sms confirmations in a test/qat environment. It also contains the version number and instance id ( which is newly generated as a simple uuid on each deployment ) and also distinguishers between multiple instances of the application.

Attributes

I've decided to include the following attributes in the identity of an app. These parts can also be used to generate a short-form or long-form id for convenience.

Attribute	Example	Purpose
area	signup	product area / department
name	alerts	name of the application
type	job	`cli , api , job , bot`
env	qat	`dev , qat , uat , pro`
ver	v2-1	version of app
uuid	id	instance id

With these attributes, you can generate short or long form identities. For example :

Short id
# {area}.{name}.{type}.{env}
"signup.alerts.job.qat"

Version id
# {area}.{name}.{type}.{env}
"signup.alerts.job.qat.v2"

Full Id
# {area}.{name}.{type}.{env}.{version}.{instance}
"signup.alerts.job.qat.v2.4a3b300b-d0ac-4776-8a9c-31aa75e412b3"

Code

The application identity is strictly enforced in code using the Identity.kt class and enums for the types, environments.

/**
 * Simple Identity used to identity services / components
 * @param area    : area  | dept | org  - logical group 
 * @param service : user1 | job  | svc  - name of app/service
 * @param agent   : api   | app  | job  - type of app
 * @param env     : dev   | qat  | pro  - environment
 * @param instance: UUID                - for 2+ instances
 *
 *      = {area}.{service}.{agent}.{env}.{version}.{instance}
 * name = signup.alerts.job.qat
 * full = signup.alerts.job.qat.v2.4a3b300b-d0ac-4776-8a9c-31aa75e412b3
*/
data class SimpleIdentity(
        override val area:String,
        override val service:String,
        override val agent: Agent,
        override val env:String,
        override val instance:String = ULIDs.create().value,
        override val version: String = "LATEST",
        override val desc: String = "",
        override val tags:List<String> = listOf()) : Identity {
  // ... 
}

Diagnostics

We identify applications using this approach and use the full-id and use its attributes when linking it to slack alerts, logs and metrics. Here are some samples for Slack and Loggly.

Alerts

Here is a sample Slack alert with the identity of the application that sent the alert in the origin field.

Logs

Here is a simple example of a log entry with the identity of the application prefixed ( in both a local log file and sent to Loggly ).

signup.alerts.job.qat.v2.2.0: 2021-02-15 17:25 INFO [main] app Beta user activated   , uuid=0a32988c-4eed-42dc-8ba7-b932d82da3f0, phase=2
signup.alerts.job.qat.v2.2.0: 2021-02-15 17:26 INFO [main] app user welcome mail sent, uuid=0a32988c-4eed-42dc-8ba7-b932d82da3f0, phase=2

Because the identity is enforced in code and in configurations for various diagnostics, we now have a way of easily identifying apps and connecting/searching diagnostics more easily!

Registry

With application identities defined, you can also create a dashboard to serve as an application registry that lists all applications running in your environments.

Customize

All these attributes collectively provide a unique identity that can assist with ownership, and most importantly link with and provide insight into diagnostics and metrics. If you're using Kotlin for Server or Android, then checkout Identity.kt or build your own!

How to create a Homebrew installer

kishore — Mon, 18 Jan 2021 05:02:58 +0000

This post discusses how to create a homebrew installer to package any script, application or tool, in this case a Kotlin (Java) based application. This was done for a Kotlin framework called Slate Kit, and here on Git.

Building scripts, applications and tools is a part of software engineering and a pretty common task. This process though typically involves a runnable application, installing, uninstalling, upgrading and packaging some dependencies/resources. This can be trivial or somewhat complex. Fortunately, there are package managers like homebrew that handle most of the boilerplate. While homebrew is great, and easy to use as an end-user who installs software, I found the documentation and guides on creating an installer somewhat difficult to navigate.

Terms

Before we get into the details of coding an installer, we need to first understand some terms and concepts. Most of these are represented here at Homebrew Terms. However, I want to elaborate on a few that are relevant for this post. These are also represented in the diagram below

Package: A package is not listed in the homebrew terminology section but it is basically the script, application or tool that you want to install and use. This contains your executable and all the resources.
Formula: A formula is a Ruby installation script that defines and installs your package on Mac OS. Since homebrew is Ruby based, the formula is a Ruby script and it typically just moves your package executable and resources into the various install directories.
Tap : A tap is a git repository containing 1 or more Formulas. You direct homebrew to load your tap(git repo) as the source for your formulas. A tap must be prefixed with homebrew-.
Version: The version number of your package to be installed, e.g. v1.38.0

Diagram

Example

This is a quick example of the entire process of setting up your tap, installing your package and then using your package. In this case the tap is slatekit/slatekit ( homebrew automatically interprets this using the homebrew- prefix, so this actually is slatekit/homebrew-slatekit, the formula is also called slatekit ( slatekit.rb ) and finally the executable script is slatekit ( a bash script that runs the Kotlin application ).

# Set tap (shortcut to github.com/slatekit/homebrew-slatekit)
:> brew tap slatekit/slatekit

# Install formula slatekit.rb to install the package 
:> brew install slatekit 

# Finally use the package ( slatekit script )
:> slatekit new app -name="Sample1" -packageName="mycompany.apps"

Step 1: Package

You need to first create your git repo for the actual package that you plan to install and use. Its libraries, dependencies, and resources need to be stored in one git repo. In this case its Slate Kit CLI. There is a file called slatekit and this represents a shell script that is the actual tool that will be used. You also need to create a release of this package so there is a zip/tar containing all items in the repo, for example v1.34.5. Since this is a Kotlin/Java based application, the repo contains the libraries( jars ) and some resources, and an executable. This Kotlin/Java application was originally built using gradle 'application' plugin and running gradle distZip to package the slatekit shell script all the libraries.

Step 2: Tap

You need to create another git repo containing the homebrew formula that will perform the install. The tap can contain 1 or more formulas. In this example the tap is slatekit/homebrew-slatekit. This tap provides homebrew a place to look for formulas. Some formulas are submitted to Homebrew for proper approval and available without requiring a tap ( later step ), but this example uses a custom tap so that you don't need to go through the approval process.
NOTE: The name of the git repo must start with homebrew-

Step 3: Formula

Now you need to create a formula that defines your package and performs the installation steps. A formula contains a reference to the url/version of the package above, so brew can download the package and unzip it so the formula can install the contents. The formula in this example is slatekit.rb. The most relevant things in the formula are the url, sha, and the steps in the install method.

class Slatekit < Formula
  url "https://github.com/slatekit/slatekit-cli/archive/v1.34.5.tar.gz"
  sha256 "e95375f92a8c0e86082d9b22e11bd6a414ee0de9df77dda84d6bb6dc21061647"

  # ...

  def install
    bin.install 'slatekit'
    prefix.install Dir["lib"]
    prefix.install Dir["conf"]
    prefix.install Dir["templates"]
    puts "Completed install"
  end
end

There are 2 ways to get the sha for the package url, e.g. the e95375f92a8c0e86082d9b22e11bd6a414ee0de9df77dda84d6bb6dc21061647 in the forumla above.

Option 1

curl -L https://github.com/slatekit/slatekit- cli/archive/v1.34.5.tar.gz > 1.34.5.tar.gz
shasum -a 256 1.34.5.tar.gz

Option 2

Run the brew create command below and the script is created in /usr/local/Homebrew/Library/taps/homebrew/homebrew-core/Formula/slatekit-cli.rb which will have the sha.

brew create https://github.com/slatekit/slatekit- cli/archive/v1.34.5.tar.gz

Process

There a few important things regarding whats happening in the install step. This is where you specify what to install from the package version that was downloaded.

version since homebrew provides versioning of packages, this package directory will correspond to the version in the formula e.g. v1.34.5 and installed to /usr/local/Cellar/slatkit/1.34.5
bin.install: This call will move the slatekit shell script to the bin directory /usr/local/Cellar/slatkit/1.34.5/bin, make it executable, and available on the command line to call.
prefix.install This call will install the directory from the unzipped package into the folder for this version. e.g. the lib directory from package is moved to /usr/local/Cellar/slatkit/1.34.5/lib

Install

Finally, once you have all this set up, you can simply run the following commands to install the tap and package.

brew tap slatekit/slatekit
brew install slatekit

The first line brew tap slatekit/slatekit is really a call to download the tap and formula from https://github.com/slatekit/homebrew-slatekit. The second call brew install slatekit is call to run the formula slatekit.rb
NOTE:
You may experience a few issues during install

Execute

Now with the package installed, you can immediately start using it. In this case, slatekit generates new projects. So we can use it like this :

slatekit new app -name="MyApp1" -packageName="company1.apps"
slatekit new api -name="MyAPI1" -packageName="company1.apis"
slatekit new job -name="MyJob1" -packageName="company1.jobs"
slatekit new cli -name="MyCLI1" -packageName="company1.apps"

Uninstall

If you want to uninstall everything, you can do

brew uninstall slatekit
brew untap slatekit/slatekit

Upgrade

Lastly, if you want to upgrade your package to a new version, then you simply have to publish a new version of your package, change the url, sha in your formula and then run the following:

brew upgrade slatekit

Conclusion

Well, thats finally it. I created a homebrew installer to allow for a command line tool to generate slatekit projects making it very easy for new users to get started quickly. It sounds like quite a lot at first, but there are only 3 concepts, the package( your script, app, tool), the tap (git repo for formulas), and the formula (installer). Hope this helps!