DEV Community: Aldi Fianda Putra

Automate App Deployment with Cloud Build

Aldi Fianda Putra — Mon, 02 Jun 2025 03:37:29 +0000

After revisiting the basics of cloud infrastructure in my previous post, I figured it's a good time to talk about something equally essential: CI/CD.

In the past few years working with Google Cloud, I've had to set up multiple CI/CD pipelines—some simple, some complex. I’ve found that using Cloud Build makes it straightforward to automate deployments and integrate well with other GCP services.

In this post, I’ll walk through how to build a CI/CD pipeline using Cloud Build, and how it fits into a modern DevOps workflow on GCP.

What is Cloud Build?

Cloud Build is a serverless CI/CD platform from Google Cloud that lets you build, test, and deploy your code directly from repositories like GitHub or Cloud Source Repositories. It supports custom build steps using Docker images and integrates smoothly with other GCP services like Cloud Run, GKE, and Artifact Registry.

Setting Up Service Account and Required Roles for Cloud Build Deployment

Before using Cloud Build, you need to create a dedicated service account for it, which can be either the default service account or a custom one. Then, assign the necessary roles based on your build requirements.

In this post, since I will deploy the application to App Engine — a fully managed platform by Google Cloud that automatically handles infrastructure, scaling, and load balancing for web applications — I assign the following roles to the service account:

App Engine Deployer for deploying the app
Storage Admin because the staging process involves Cloud Storage, and
Logs Writer to allow logging build activities to Cloud Logging.

After creating the service account, the next step is to set up a connection to our code repository. In this example, I’m connecting to GitLab, so I create a host connection to my GitLab repository.

Once connected, you will see something like the image above, where the connection name and the linked repository appear in Cloud Build.

Creating a CI/CD Trigger in Cloud Build

To create a CI/CD trigger, go to the Cloud Build page and select the Triggers menu, then click "+ Create Trigger".

On this page, you can specify the event that will invoke the build trigger. These events can be repository-related, such as a push to a branch, a new tag push, or a pull request. Alternatively, the trigger can respond to non-repository events like manual invocation, Pub/Sub messages, or webhook events.

Next, select the repository to connect. Then, choose the build configuration method—whether based on a Cloud Build config file, a Dockerfile, or Buildpacks. In this example, I will use a Cloud Build configuration file (cloudbuild.yaml).

As an additional note, you can also add environment variables to the trigger using the Substitution variables option. However, the variable names must include an underscore and follow the format _VARIABLE; using names without the underscore (like VARIABLE) is not allowed.

Finally, select the service account that you created earlier to run the build.

Cloud Build Configuration for Deployment

The build configuration file (cloudbuild.yaml) is stored in the code repository alongside the application source code. This allows Cloud Build to automatically detect and use the configuration whenever a build is triggered.

Since in this example I am using App Engine to deploy a Node.js application, besides the Cloud Build configuration, there is also an app.yaml file.

The app.yaml file is the configuration file for App Engine that defines deployment settings such as the runtime environment, instance class, service name, and URL handlers.

Here is an example of the app.yaml content:

env: standard          # Specifies the App Engine environment (standard or flexible)
runtime: nodejs20      # Defines the runtime environment (Node.js version 20)
instance_class: F2     # Specifies the instance class which controls CPU and memory resources
service: testing       # Names the service/module for this deployment
handlers:              # Defines how URL paths are handled
  - url: /.*           # Matches all URL paths
    secure: always     # Enforces HTTPS for all requests
    script: auto       # Automatically selects the script to run (default behavior for Node.js)

Below is an example of a Cloud Build configuration (cloudbuild.yaml) that runs several build steps before deploying the app to App Engine:

steps:
  - name: 'node'
    entrypoint: 'yarn'
    args: ['cache', 'clean']
  - name: 'node'
    entrypoint: 'yarn'
    args: ['install']
  - name: 'node'
    entrypoint: 'yarn'
    args: ['build']
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: 'bash'
    args: ['-c', 'gcloud config set app/cloud_build_timeout 1600 && gcloud app deploy']
timeout: '1600s'
options:
  logging: CLOUD_LOGGING_ONLY

Explanation of each step:

yarn cache clean clears the local yarn cache to ensure a fresh install of dependencies.
Install Dependencies: yarn install installs the project dependencies as defined in package.json.
yarn build runs the build script, typically compiling or bundling the app for production.
Uses the Google Cloud SDK container to run a bash command that sets the build timeout and deploys the app with gcloud app deploy.
The timeout: '1600s' defines the maximum time the build can run before being cancelled (in this case, about 26 minutes).
The options.logging: CLOUD_LOGGING_ONLY setting ensures that build logs are sent only to Cloud Logging.

Monitoring Build Progress in Cloud Build

You can monitor the progress and status of your builds in the Build History page of Cloud Build. This page provides detailed logs for each build step, allowing you to troubleshoot issues or confirm successful deployments. Each entry shows timestamps, build results, and any error messages if the build fails.

With this setup, you now have a complete CI/CD pipeline that automates building and deploying your Node.js application to App Engine. By integrating Cloud Build with your repository and configuring triggers, deployment becomes consistent, repeatable, and easier to manage. In future posts, I plan to explore more advanced topics such as environment variable management, artifact storage, and secret handling to further optimize the pipeline.

References

Landing Zone: A Blueprint for Cloud Deployments

Aldi Fianda Putra — Tue, 22 Apr 2025 02:31:46 +0000

After working in the cloud industry for over two years—mostly with Google Cloud Platform—I began to think that I need to start documenting my experiences. Mainly as a personal reference, but hopefully others might find them useful too.

This is my first post in a long while (the last one was about two years ago). To kick things off, I want to start with something fundamental: Landing Zone in the cloud.

What is A Landing Zone?

A Landing Zone is an initial framework or foundation set up in the cloud (such as Google Cloud, AWS, or Azure) to enable organizations to deploy applications and services in a secure, structured, and scalable manner. Landing Zones are dynamic and evolve as your organization adopts more cloud-based workloads over time.

When creating a Landing Zone, there is no one-size-fits-all implementation. Each organization is free to design its Landing Zone based on specific business needs. The design is usually influenced by several factors, such as:

Industry type
Organizational structure and internal processes
Security and compliance requirements
Types of workloads to be migrated to the cloud
Existing infrastructure (either on-premises or in another cloud)
Business and customer locations

Sample of Landing Zone Architecture. Image credit: Google Cloud

The image above shows an example of Landing Zone architecture on Google Cloud. It highlights how different components come together to create a secure, scalable, and well-managed environment for deploying workloads in the cloud.

The core elements of Landing Zone that you should include when designing a Landing Zone:

Identity provisioning
Resource hierarchy
Networking
Security controls

On top of that, you can also add other components based on your business needs, such as:

Monitoring and logging
Backup and disaster recovery
Compliance
Cost efficiency and budget control
API management
Cluster management

Automating Landing Zone Creation — Is It Possible?

A Landing Zone, as previously explained, is a framework and foundation for our cloud infrastructure. Based on this foundation, we can set up our environment using whichever method we prefer—such as the GCP Console or Cloud Shell. However, if we need to configure many projects, doing so manually can be time-consuming. To simplify this, we can automate the setup of Landing Zone services using Terraform. To give you a better idea of how this works in practice, here's a simple example of how we can use Terraform to create a basic Landing Zone setup on Google Cloud.

main.tf

provider "google" {
  project = var.project_id
  region  = var.region
  zone    = var.zone
}

resource "google_project" "landing_zone_project" {
  name            = var.project_name
  project_id      = var.project_id
  org_id          = var.org_id
  billing_account = var.billing_account_id
}

resource "google_project_service" "enabled_apis" {
  for_each = toset([
    "compute.googleapis.com",
    "iam.googleapis.com",
    "cloudresourcemanager.googleapis.com"
  ])
  project = google_project.landing_zone_project.project_id
  service = each.key
}

resource "google_project_iam_member" "project_owner" {
  project = google_project.landing_zone_project.project_id
  role    = "roles/owner"
  member  = "user:${var.user_email}"
}

resource "google_compute_network" "vpc_network" {
  name                    = "landing-zone-vpc"
  auto_create_subnetworks = true
  project                 = google_project.landing_zone_project.project_id
}

resource "google_compute_instance" "vm_instance" {
  name         = "landing-zone-instance"
  machine_type = "e2-medium"
  zone         = var.zone
  project      = google_project.landing_zone_project.project_id

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-11"
    }
  }

  network_interface {
    network = google_compute_network.vpc_network.name
    access_config {
      # Ephemeral external IP
    }
  }
}

This main.tf template lays the groundwork for a basic Google Cloud Landing Zone. It starts by configuring the Google provider using project, region, and zone variables. Then, it creates a new GCP project under your organization, connects it to a billing account, and enables a few essential APIs like Compute Engine, IAM, and Cloud Resource Manager. To ensure proper access, it assigns the "Owner" role to a specified user email. After that, it sets up a default VPC network with automatic subnet creation. Finally, it launches a simple VM instance running Debian 11 and connects it to the VPC. This setup gives you a clean starting point to build more infrastructure on GCP. Since dev.to has a content limit, I’m not including the other Terraform files like variables.tf and terraform.tfvars here.

That’s a wrap! This setup might be simple, but it’s a solid starting point if you’re looking to build your own Landing Zone on GCP. There’s still a lot you can add and customize depending on your needs, but hopefully this gives you a head start. If you're trying it out or have suggestions, feel free to chat in the comments. Thanks for stopping by!

References

Landing Zones on Google Cloud.

ETL Process from NoSQL to SQL Database

Aldi Fianda Putra — Fri, 03 Mar 2023 16:28:01 +0000

Project Description

This is a my project for Data Engineering course during college to do some ETL processing on pokemon dataset which can be accessed on this link. On this project, I created some ETL processes to fix certain things in the dataset, like names, numbering, fixing missing value etc.

Dataset Description

This dataset is a dataset that discusses the data contained in the Pokemon video game. The data comes from Pokemon starting from Gen I (Pokemon Red, Green, Blue) to Gen 7 (Pokemon Sun & Moon, Lets Go). In this dataset there are 4 data or tables:

Pokedex

The Pokedex table is a sort of encyclopedia of all the pokemon in the pokemon universe. Pokedex contains information about the statistics of pokemon in general.

Moves

The moves table is a table that contains skills or attacks that can be trained on Pokemon. Usually known as TM and HM. Each move has its own statistics, and a maximum of 4 moves can exist in one Pokemon.

Types

The types table is a table that lists the types of pokemon or movesets in the Pokemon video game. There are 18 different types of Pokemon and existing movesets.

Items

The items table contains items that can be used directly by players or held by Pokemon during battle.

Table Relation

The relationship between each table in the dataset can be seen in the image above. From the picture, it can be seen that the pokemon has or is registered in the pokedex. Then, Pokemon has a maximum of up to two types. In terms of moves, each Pokemon has a maximum of four. And lastly, every Pokemon can have at least one item.

Extract

The first step is to carry out the extraction process. In this process, I also created a logging function to store information about what processes have been running. The logging function can be seen in the following program code.

!touch logfile.txt
def log(pesan):
    timestamp_format = '%Y-%h-%d-%H:%M:%S'
    now = datetime.now()
    timestamp = now.strftime(timestamp_format)
    with open("logfile.txt","a") as f:
        f.write(timestamp+','+pesan+'\n')

The next step is to download the dataset from the GitHub link that has been attached above. The dataset from that page is a dataset of type NoSQL with key and value types. For that, the data is first converted to Pandas dataframe format. While converting to Pandas, we can carry out the logging process by calling the log function that was created earlier by calling log(). The program code for downloading, the conversion function to a Pandas dataframe, and the conversion can be seen in the following program code.

#Downloading dataset
!curl -LJO https://raw.githubusercontent.com/fanzeyi/pokemon.json/master/items.json
!curl -LJO https://raw.githubusercontent.com/fanzeyi/pokemon.json/master/moves.json
!curl -LJO https://raw.githubusercontent.com/fanzeyi/pokemon.json/master/pokedex.json
!curl -LJO https://raw.githubusercontent.com/fanzeyi/pokemon.json/master/types.json

#extract function
def extract_from_json(file_to_process):
    dataframe = pd.read_json(file_to_process,orient=str)
    return dataframe

#calling extract function
log("Proses ETL dimulai")
log("Proses Extract : Mengekstrak 4 tabel JSON menjadi dataframe")
df_1=extract_from_json('pokedex.json')
df_2=extract_from_json('moves.json')
df_3=extract_from_json('types.json')
df_4=extract_from_json('items.json')
log("Proses esktraksi tabel JSON selesai")

The picture above is the result of calling the df_1 dataframe to check the output. It can be seen that there are four features, namely id, name, type, and basic stats. If you pay attention, the name, type, and basic stats have parentheses, and each value is separated by a colon. This means that the feature stores data in the form of keys and values. Therefore, it is necessary to carry out a transformation process to retrieve only some data from this NoSQL feature.

Transform

In the transformation stage, the transformation steps are carried out differently for each table, where the explanation is as follows:

On the pokedex table, the transformation that is done is:

Removing Japanese, Chinese, and French names
Flattening the type attribute
Flatten the basic stat attribute
Deleting nested attributes that exist in the initial dataframe
Merging previously fixed attributes into one

In the moves table, the transformations performed are

Removing Japanese, Chinese, and tm and max_pp names because they are not on the original reference page (references come from Bulbapedia)
Changing the value in the category column, which is still in Chinese.
Fill in the NaN value to 100.0, because NaN here in the game is of infinite value, which move will be successfully used without the slightest failure
Renaming the column ename to name
Tidy up the column according to the reference from bulbapedia.
Fixing flying moves that were recorded as fighting.

In the type table, the transformation performed is:

Removing Japanese and Chinese names from the type, then renamed the English column to name.
Creating an id column starting from 1

In the items table, the transformation that is done is

Flattening the name attribute and only taking English item names

The function for carrying out the transformation process on data can be seen in the following program code snippet.

def transform_pokedex(df):
    log("Memulai transformasi tabel pokedex")
    #Menghapus nama bahasa Jepang, China dan Perancis
    df_1_name = pd.DataFrame.from_dict(df)
    df_1_name = df_1_name['name'].apply(pd.Series)
    df_1_name = df_1_name.drop(['japanese','chinese','french'],axis=1)
    df_1_name = df_1_name.rename(columns={'english': 'Name'})

    #Melakukan flattening kepada atribut type
    df_1_type = pd.DataFrame.from_dict(df)
    df_1_type = df_1_type['type'].apply(pd.Series)
    df_1_type = df_1_type.rename(columns={0: 'Type 1', 1: 'Type 2'})
    df_1_type = df_1_type.fillna('')

    #Melakukan flattening kepada atribut base
    df_1_stats= pd.DataFrame.from_dict(df)
    df_1_stats = df_1_stats['base'].apply(pd.Series)

    #Menghapus nested atribut yang ada pada dataframe awal
    df = df.drop(['name', 'base', 'type'], axis = 1)

    #Menggabungkan atribut yang sudah diperbaiki sebelumnya menjadi 1
    df = df.join(df_1_name)
    df = df.join(df_1_type)
    df = df.join(df_1_stats)

    log("Transformasi tabel pokedex berhasil")
    return df

def transform_moves(df):
    log("Memulai transformasi tabel move")
    #Menghapus nama bahasa Jepang, China, tm dan max_pp karena tidak terdapat pada 
    #link referensi aslinya (bulbapedia)
    df_2_fix = pd.DataFrame.from_dict(df)
    df_2_fix = df_2_fix.drop(['id','jname','cname','tm','max_pp'],axis=1)

    #Mengganti value pada kolom 'category' yang masih berbahasa China
    df_2_fix = df_2_fix.replace('物理','Physical')
    df_2_fix = df_2_fix.replace('特殊','Special')
    df_2_fix = df_2_fix.replace('变化','Status')

    #Mengisi nilai NaN menjadi 100.0, karena NaN disini maksud di gamenya adalah infinity
    #dimana move akan berhasil tanpa kegagalan sedikitpun
    df_2_fix = df_2_fix.fillna(100.0)
    #Mengganti nama kolom ename menjadi name
    df_2_fix = df_2_fix.rename(columns={'ename': 'Move','type': 'Type','category':'Category'
                                       ,'pp': 'PP','power':'Power','accuracy':'Accuracy'})

    #Merapikan kolom sesuai dengan referensi dari bulbapedia
    df_2_fix = df_2_fix[['Move', 'Type', 'Category','PP','Power','Accuracy']]

    #Memperbaiki moves flying yang tercatat sebagai fighting

    df = df_2_fix
    df = df.reset_index()
    df = df.rename(columns={"index":"id move"})
    df['id move'] = df.index + 1
    #df.set_index('id', inplace=True)

    log("Transformasi tabel move berhasil")
    return df

def transform_moves_2(df):
    log("Memulai perbaikan terhadap semua moveset bertipe Flying yang sebelumnya bertipe Fighting")
    flying_move = 331,176,402,339,412,447,431,64,296,18,15,541,118,607,63,364,142,506,365,16

    #Memperbaiki moveset Flying yang awalnya bernilai Fighting
    for x in flying_move:
        df.loc[x] = df.loc[x].replace(to_replace="Fighting", value = "Flying")

    log("Transformasi untuk perbaikan moveset type Flying berhasil")
    return df

def transform_type(df):
    log("Memulai transformasi tabel type")
    #Menghapus nama Jepang dan nama China dari tipe, kemudian mengganti nama kolom 'english'
    #menjadi 'name'
    df_3_fix = pd.DataFrame.from_dict(df)
    df_3_fix = df_3_fix.drop(['chinese','japanese'],axis=1)
    df_3_fix = df_3_fix.rename(columns={'english': 'Name'})

    df = df_3_fix

    #Membuat kolom id yang dimulai dari 1
    df = df.reset_index()
    df = df.rename(columns={"index":"id"})
    df['id'] = df.index + 1
    #df.set_index('id', inplace=True)
    df = df.rename(columns={"id":"id type"})
    log("Transformasi tabel type berhasil")
    return df

def transform_items(df):
    log("Memulai transformasi tabel items")
    #Melakukan flattening ke atribut 'name' dan hanya mengambil nama item dalam 
    #bahasa inggris saja
    df_4_fix = pd.DataFrame.from_dict(df)
    df_4_name = pd.DataFrame.from_dict(df)
    df_4_name = df_4_name['name'].apply(pd.Series)
    df_4_name = df_4_name.drop(['japanese','chinese'],axis=1)

    df_4_fix = df.join(df_4_name)
    df = df_4_fix
    df = df.drop(['id','name'],axis=1)

    df = df.rename(columns={'english': 'Name'})
    df['id']= range(0, 0+len(df))

    df=df[['id','Name']]
    df = df.rename(columns={"id":"id item"})
    log("Transformasi tabel items berhasil")
    return df

Then, the following function is to call the transform function on the dataset.

log("Memulai transformasi tabel pokedex dengan melakukan flattening data")
pokedex_fix = transform_pokedex(df_1)
moves_fix = transform_moves(df_2)
moves_fix = transform_moves_2(moves_fix)
types_fix = transform_type(df_3)
items_fix = transform_items(df_4)
log("Transformasi semua tabel telah berhasil")

The picture above is an example of the results of the transformation in the pokedex table, where now there are features id, name, type 1, type 2, HP, attack, defense, Sp.Attack, Sp.Defense, and speed. For Pokemon with a single type, type 2 contains an empty string.

Because in this project I plan to make each Pokemon have four moves and hold items, Then a transformation process is carried out again to give a random move to the pokemon, with the condition that the move must be of the same type as the pokemon. For this reason, the function to provide random moves can be seen in the following program code snippet.

def addmove4(final_pkmn,awal,akhir):
    log("Memulai transaksi 4 move kepada semua pokemon dalam pokedex")
    count = 0
    for i in range(awal,akhir):
        moves_added = 0
        while(moves_added<4):
            count_same = 0
            rand=random.randint(0,611)
            random_name= moves_fix.iloc[rand]['Move']
            random_type = moves_fix.iloc[rand]['Type']
            t_random=random_type
            t1 = pokedex_fix["Type 1"][i]
            t2 = pokedex_fix["Type 2"][i]
            if (t1==t_random) or (t2 == t_random):
                if(moves_added<4):
                    id_pkmn = pokedex_fix.iloc[i]['id']
                    nama_pkmn = pokedex_fix.iloc[i]['Name']
                    moveset=random_name
                    tambah = {"id" : id_pkmn,"Move" : moveset}
                    count += 1
                    moves_added += 1
                    final_pkmn=final_pkmn.append(tambah,ignore_index=True) 
                    log("Menambahkan moveset "+moveset+"ke pokemon bernama "+nama_pkmn)
                else:
                    for x in range(moves_added):
                        if random_name==final_pkmn.iloc[count-x]['Move']:
                            count_same=0
                        else : 
                            id_pkmn = pokedex_fix.iloc[i]['id']
                            nama_pkmn = pokedex_fix.iloc[i]['Name']
                            moveset=random_name
                            tambah = {"id" : id_pkmn,"Move" : moveset}
                            count += 1
                            moves_added += 1
                            final_pkmn=final_pkmn.append(tambah,ignore_index=True)  
                            log("Menambahkan moveset "+moveset+"ke pokemon bernama "+nama_pkmn)
            else :
                 continue
    logggg ="Proses transaksi moveset dari pokemon dengan index "+str(awal)+" hingga "+str(akhir)+" berhasil"
    log(logggg)
    print(logggg)
    return final_pkmn

log("Mendeklarasikan tabel terakhir untuk menggabungkan atau JOIN beberapa tabel")
final_pkmn = pd.DataFrame()
final_pkmn["id"]=""
final_pkmn["Move"]=""

log("Menginisiasikan proses transaksi moveset kepada semua 809 pokemon")
final_pkmn = addmove4(final_pkmn,0,809)

The dataframe results obtained after the transformation are as shown above. It can be seen that a pokemon with ID 1 has four moves, and so on. Thus, the final step is to merge all tables.

def join_final_transform(final_pkmn):
    log("Memulai proses JOIN pada beberapa tabel")
    log("Memulai proses LEFT JOIN pada tabel pokedex dengan item -> pokemon memiliki hold item")
    pokemon_w_item = pokedex_fix
    item_join = items_fix
    item_join = item_join.rename(columns={"id":"Item id","Name":"Item Name"})
    pokemon_w_item = pokemon_w_item.join(item_join)
    log("Proses LEFT JOIN berhasil dilakukan")

    log("Memulai proses FULL OUTER JOIN antara tabel pokemon yang sudah memiliki hol item dengan transaksi move")
    final_pkmn = pd.merge(pokemon_w_item,final_pkmn,on='id')
    final_pkmn = pd.merge(final_pkmn,moves_fix,on="Move")
    final_pkmn = final_pkmn.sort_values(by=['id'])
    final_pkmn = final_pkmn.reset_index()
    final_pkmn = final_pkmn.drop("index", axis=1)
    log("Proses FULL OUTER JOIN antara tabel pokemon dengan transaksi move berhasil dilakukan")

    return final_pkmn

log("Melakukan transformasi final yakni melakukan proses join")
final_pkmn = join_final_transform(final_pkmn)
log("Semua proses transformasi berhasil dilakukan")

The final appearance after the final transformation of the dataset is as shown above. For example, the bulbasaur has four moves that have different stats and don't store items.

Load

During the load process, I load two different databases, namely SQLite and MySQL. The process of loading data into a SQLite database can be seen in the following program code snippet.

log("Membuat connection ke database sqlite dengan API sqlite3 beserta cursornya")
connection = s3.connect('preprocessed_pokemon.db')
cursor = connection.cursor()
engine = create_engine('sqlite:///home/stndb01/Documents/Data Engineering/UTS/preprocessed_pokemon.db')  

log("Meload semua tabel ke dalam datamart sqlite3")
final_pkmn.to_sql(name='Pokemon_Final', con=connection, if_exists='replace', index=False)
pokedex_fix.to_sql(name='Pokedex', con=connection, if_exists='replace', index=False)
moves_fix.to_sql(name='Moves', con=connection, if_exists='replace', index=False)
types_fix.to_sql(name='Type', con=connection, if_exists='replace', index=False)
items_fix.to_sql(name='Items', con=connection, if_exists='replace', index=False)
log("Proses load tabel ke datamart sqlite3 berhasil")

Meanwhile, the MySQL database has more or less the same function. The only significant difference lies in the data storage; previously, SQLite stored it in a local database, which is a file that had a .db format. While MySQL saves on localhost, which can be accessed through phpmyadmin, The process of loading data into the MySQL database can be seen in the following program code snippet.

log("Membuat connection ke database mysql dengan API mysql beserta cursornya")
engine_mysql = create_engine('mysql+pymysql://phpmyadmin:bruh@127.0.0.1/preprocessed_pokemon')
connection_mysql = engine_mysql.raw_connection()
my_cursor = connection_mysql.cursor()

log("Meload semua tabel ke dalam datamart mysql lokal")
final_pkmn.to_sql('Pokemon_Final', con = engine_mysql, if_exists='replace', index=False)  
pokedex_fix.to_sql('Pokedex', con = engine_mysql, if_exists='replace', index=False)  
moves_fix.to_sql('Moves', con = engine_mysql, if_exists='replace', index=False)  
types_fix.to_sql('Type', con = engine_mysql, if_exists='replace', index=False)  
items_fix.to_sql('Items', con = engine_mysql, if_exists='replace', index=False)  
log("Proses load tabel ke datamart mysql lokal berhasil")

SQLite and MySQL have similar query functions, so we can call the same query line for the same process. The difference is the function of Python to call queries from each database. SQLite queries can be executed with read_sql from pandas. But in MySQL, queries are run using engine_mysql and the execute function. The following are examples of queries and calls for each database.

log("Menampilkan isi dari tabel dari datamart sqlite3")
sql_read_1 = ''' SELECT * FROM Pokemon_Final'''
sql_read_2 = ''' SELECT * FROM Pokedex'''
sql_read_3 = ''' SELECT * FROM Moves'''
sql_read_4 = ''' SELECT * FROM Type'''
sql_read_5 = ''' SELECT * FROM Items'''

def read_table_sqlite3(query):
    return pd.read_sql(query,connection)

SQLite

MySQL

CRUD

The data that has been stored in the SQLite and MySQL databases can be changed using the Python environment. Following are examples of insert, update, and delete operations on SQLite and MySQL databases via Python.

SQLite

Insert

#Insert secara banyak ke tabel pokedex
log("Melakukan proses insert pada tiga tabel di sqlite3 dengan masing-masing 3 data")
new_pokemon = [(810,"Kadal Bersayap","Flying","Dragon",78,89,42,54,58,85),
              (811,"Naga Indosiar","Flying","Dragon",68,78,67,25,52,35),
              (810,"Kuda Bertanduk","Normal","Fairy",48,42,25,34,78,44)]
cursor.executemany('INSERT INTO Pokedex VALUES (?,?,?,?,?,?,?,?,?,?)', new_pokemon)

#Insert satu-satu
cursor.execute('INSERT INTO Moves VALUES (613,"Bruh","Normal","Special",5.0,100.0,90.0)')
cursor.execute('INSERT INTO Moves VALUES (614,"Fus Ro Dah","Dragon","Special",8.0,90.0,100.0)')
cursor.execute('INSERT INTO Moves VALUES (615,"Totsugeki","Flying","Physical",40.0, 100.0, 100.0)')

new_pokemon_w_move = [(810,"Kadal Bersayap","Flying","Dragon",78,89,42,54,58,85,1,"Master Ball","Totsugeki",615,"Flying","Physical",40.0, 100.0, 100.0),
                     (811,"Naga Indosiar","Flying","Dragon",68,78,67,25,52,35,1,"Master Ball","Fus Ro Dah",614,"Dragon","Special",8.0,90.0,100.0),
                     (810,"Kuda Bertanduk","Normal","Fairy",48,42,25,34,78,44,1,"Master Ball","Bruh",613,"Normal","Special",5.0,100.0,90.0)]
cursor.executemany('INSERT INTO Pokemon_Final VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', new_pokemon_w_move)
connection.commit()
log("Proses insert pada sqlite3 berhasil dilakukan")

Update

#Update data yang sudah ada
log("Melakukan update data pada yang sudah ada pada db sqlite3")
update_pokemon_1 = ("Mega-Bulbasaur","Bulbasaur") 
update_1 = '''UPDATE Pokemon_Final set Name = ? WHERE Name LIKE ?'''
cursor.execute(update_1,update_pokemon_1)

update_pokemon_2 = ("Poison","Melmetal")
update_2 =  '''UPDATE Pokemon_Final set "Type 2" = ? WHERE Name LIKE ?'''
cursor.execute(update_2,update_pokemon_2)

update_pokemon_3 = ("Mega-Charizard","Dragon","Charizard")
update_3 =  '''UPDATE Pokemon_Final set Name = ?, "Type 1" = ?  WHERE Name LIKE ?'''
cursor.execute(update_3,update_pokemon_3)
connection.commit()
log("Update data pada db sqlite3 berhasil dilakukan")

Delete

#Delete data yang sudah ada
log("Melakukan delete pada data yang sudah ada pada db sqlite3")
cursor.execute('DELETE FROM Pokemon_Final WHERE Name LIKE "Mega-Bulbasaur"')
cursor.execute('DELETE FROM Pokemon_Final WHERE Name LIKE "Ivysaur"')
cursor.execute('DELETE FROM Pokemon_Final WHERE Name LIKE "Venusaur"')
cursor.execute('DELETE FROM Pokemon_Final WHERE Name LIKE "Melmetal"')
cursor.execute('DELETE FROM Pokemon_Final WHERE Name = "Kadal Bersayap"')
connection.commit()
log("Delete data pada db sqlite3 berhasil dilakukan")

MySQL

Insert

#Insert data secara banyak ke mysql
log("Melakukan insert data pada db mysql")
insert_my_1 = "INSERT INTO Pokedex VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)"
nilai_ins_1 = [(810,"Kadal Bersayap","Flying","Dragon",78,89,42,54,58,85),
              (811,"Naga Indosiar","Flying","Dragon",68,78,67,25,52,35),
              (810,"Kuda Bertanduk","Normal","Fairy",48,42,25,34,78,44)]
my_cursor.executemany(insert_my_1,nilai_ins_1)

#Insert data satu persatu
engine_mysql.execute('INSERT INTO Moves VALUES (613,"Bruh","Normal","Special",5.0,100.0,90.0)')
engine_mysql.execute('INSERT INTO Moves VALUES (614,"Fus Ro Dah","Dragon","Special",8.0,90.0,100.0)')
engine_mysql.execute('INSERT INTO Moves VALUES (615,"Totsugeki","Flying","Physical",40.0, 100.0, 100.0)')


insert_my_3 = "INSERT INTO Pokemon_Final VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)"
nilai_ins_3 = [(810,"Kadal Bersayap","Flying","Dragon",78,89,42,54,58,85,1,"Master Ball","Totsugeki",615,"Flying","Physical",40.0, 100.0, 100.0),
                     (811,"Naga Indosiar","Flying","Dragon",68,78,67,25,52,35,1,"Master Ball","Fus Ro Dah",614,"Dragon","Special",8.0,90.0,100.0),
                     (810,"Kuda Bertanduk","Normal","Fairy",48,42,25,34,78,44,1,"Master Ball","Bruh",613,"Normal","Special",5.0,100.0,90.0)]
my_cursor.executemany(insert_my_3,nilai_ins_3)

connection_mysql.commit()
log("Insert data pada db mysql berhasil dilakukan")

Update

#Update data yang sudah ada
log("Melakukan update data pada yang sudah ada pada db mysql")
update_my_1 = "UPDATE Pokemon_Final set Name = %s WHERE Name LIKE %s"
nilai_upd_1 = ("Mega-Bulbasaur","Bulbasaur") 
my_cursor.execute(update_my_1,nilai_upd_1)

update_my_2 =  "UPDATE Pokemon_Final set `Type 2` = %s WHERE Name LIKE %s"
nilai_upd_2 = ("Poison","Melmetal")
my_cursor.execute(update_my_2,nilai_upd_2)

update_my_3 =  "UPDATE Pokemon_Final set Name = %s, `Type 1` = %s  WHERE Name LIKE %s"
nilai_upd_3 = ("Mega-Charizard","Dragon","Charizard")
my_cursor.execute(update_my_3,nilai_upd_3)
connection_mysql.commit()
log("Update data pada db mysql berhasil dilakukan")

Delete

#Delete data yang sudah ada
log("Melakukan delete pada data yang sudah ada pada db mysql")
engine_mysql.execute('DELETE FROM Pokemon_Final WHERE Name LIKE "Mega-Bulbasaur"')
engine_mysql.execute('DELETE FROM Pokemon_Final WHERE Name LIKE "Ivysaur"')
engine_mysql.execute('DELETE FROM Pokemon_Final WHERE Name LIKE "Venusaur"')
my_cursor.execute('DELETE FROM Pokemon_Final WHERE Name LIKE "Melmetal"')
my_cursor.execute('DELETE FROM Pokemon_Final WHERE Name = "Kadal Bersayap"')
connection_mysql.commit()
log("Delete data pada db mysql berhasil dilakukan")

Classification for Myocardial Infarction Dataset

Aldi Fianda Putra — Wed, 01 Mar 2023 15:12:27 +0000

What is Myocardial Infarction

Myocardial infarction, commonly known as the heart attack," is a very serious heart problem. This disorder occurs when the heart muscle does not get good blood flow. This condition will interfere with cardiac function and the flow of blood flow throughout the body, which can be fatal to humans.

This complication can be detected by a tool called an electrocardiogram, or EKG. This tool detects the heartbeat wave signal, of which each signal is classified into three kinds: Q, R, and S. There are 11 kinds of heartbeat waves, from this wave we can classify a person's heart disease.

Project Description

This is a project to classify a person heart disease from Myocardial Infarction Complication which can be accessed from this link. On this project, the models used are Naive Bayes, Decision Trees, and Support Vector Machines. But there are some problems with the dataset. The dataset contains missing values, and most of them are not normalized. Therefore, imputation needs to be done, and then the normalization process is also carried out using min-max normalization. This project itself has also been published in an article that can be accessed on this page.

Dataset has categorical and non-categorical attributes. This non-categorical attribute will be normalized. These attributes include

S_AD_KBRIG: Systolic blood pressure (mmHg)
D_AD_KBRIG: diastolic blood pressure (mmHg)
S_AD_ORIT: Systolic blood pressure from ICU (mmHg)
D_AD_ORIT: Diastolic Blood Pressure from ICU (mmHg)
K_BLOOD: the amount of serum potassium in the patient's blood (mmol/L)
NA_BLOOD: total serum sodium in the patient's blood (mmol/L)
ALT_BLOOD: amount of AIAT serum in the patient's blood (IU/L)
AST_BLOOD: amount of serum AsAT in the patient's blood (IU/L)
KFK_BLOOD: amount of serum CPK in the patient's blood (IU/L)
L_BLOOD: the number of white blood cells present in the patient
ROE: the amount of ESR or high blood sedimentation rate experienced by the patient

The last attribute, namely "LET_IS," is a lethal outcome that displays the classification of heart disease suffered by the patient based on the symptoms and complications experienced by the patient. There are 8 classes of disease classification results, which are represented by numbers, namely:

0 indicates the patient is healthy.
1 indicates the patient has cardiogenic shock.
2 indicates the patient has pulmonary edema.
3 indicates the patient is classified as having a myocardial rupture.
4 indicates the patient has congestive heart failure.
5 indicates the patient has thromboembolism.
6 indicates the patient has asystole.
7 indicates that the patient has ventricular fibrillation.

Preprocessing

The first step is to do the initial data processing, which involves some initial steps such as checking the data to see if there is a missing value.

import pandas as pd
import numpy as np
!wget https://s3-eu-west-1.amazonaws.com/pstorage-leicester-213265548798/23581310/MyocardialinfarctioncomplicationsDatabase.csv
data = pd.read_csv('MyocardialinfarctioncomplicationsDatabase.csv')
data

def cetak_rentang(df_input):
  list_fitur=df_input.columns[:-1]
  for fitur in list_fitur:
    max=df_input[fitur].max()
    min=df_input[fitur].min()
    print("Rentang fitur", fitur, "Adalah", max-min)

As seen in the two outputs above, if there is a missing value marked with NaN, then the feature range itself is not the same for all features. Thus, it is necessary to impute and normalize the data.

To perform imputation of data, use the following equation and then call :

def imputasi(df_input):
  list_columns = df_input.columns
  class_column = list_columns[-1]  
  for column in list_columns[:-1] :
    df_input[column] = df_input[column].fillna(round(df_input.groupby(class_column)[column].transform('mean'),0))
    df_input[column] = df_input[column].fillna(df_input[column].mean())
  return df_input

As seen in the picture above, there are no more missing values. So the next step is to normalize the min-max data so that the data has a range between 0 and 1. However, normalized features are only continuous or non-categorical features; discrete features such as age do not need to be normalized. For that, normalization can be done by using the following function:

kolomfitur = ['S_AD_KBRIG','D_AD_KBRIG','S_AD_ORIT','D_AD_ORIT','K_BLOOD','NA_BLOOD',
              'ALT_BLOOD','AST_BLOOD','KFK_BLOOD','L_BLOOD','ROE']
from sklearn.preprocessing import MinMaxScaler
normalisasi = MinMaxScaler()
data[kolomfitur] = normalisasi.fit_transform(data[kolomfitur])

Because the data no longer contains missing values and non-categorical features have been normalized. So the next step is to carry out the classification process on the dataset.

Classification

Model Building

The algorithm model in this project was made using sklearn. The code below serves to build an algorithm model for Naive Bayes, Decision Tree, and SVM.

from sklearn.svm import SVC
from sklearn.naive_bayes import GaussianNB
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import make_classification
from sklearn.metrics import classification_report

models = []
models.append(('NB', GaussianNB()))
models.append(('DT', DecisionTreeClassifier()))
models.append(('SVM', SVC(gamma='auto')))

results = []
names = []

Training Data and Validation data

The next stage is to determine the distribution of training and test data. The division used for training data and test data is 80:20.

array = data.values
X = array[:,0:123]
y = array[:,123]

X_train, X_validation, Y_train, Y_validation = train_test_split(X, y, test_size=0.20, random_state=1, shuffle=True)

from sklearn.model_selection import cross_val_score
from sklearn.model_selection import StratifiedKFold
import warnings
warnings.filterwarnings("ignore")

for name, model in models:
    kfold = StratifiedKFold(n_splits=10, random_state=1, shuffle=True)
    cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring='accuracy')
    results.append(cv_results)
    names.append(name)
    print('%s: %f (%f)' % (name, cv_results.mean(), cv_results.std()))

Model Prediction

Naive Bayes

model = GaussianNB()
model.fit(X_train, Y_train)
predictions = model.predict(X_validation)
X, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=1)
print(classification_report(Y_validation, predictions))

From the picture above, it can be seen that Naive Bayes has an accuracy of 0.48.This accuracy can be considered low compared to other methods. And from these results on 4, 6, and 7 get the value of precision, recall, and very low f1-score of 0.0.

Decision Tree

model = DecisionTreeClassifier()
model.fit(X_train, Y_train)
predictions = model.predict(X_validation)
X, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=1)
print(classification_report(Y_validation, predictions))

From the picture above, it can be seen that the decision tree has an accuracy of about 0.99. The accuracy level can be considered high compared to other methods. And from those results, 7 still
get the value of precision, recall, and very low f1-score of 0.0.

SVM

from imblearn.over_sampling import SMOTE
sm = SMOTE(random_state=42)
X_res, y_res = sm.fit_resample(X_train, Y_train)
model = SVC(gamma='auto')
model.fit(X_train, Y_train)
predictions = model.predict(X_validation)
X, y = make_classification(n_samples=340, n_features=20, n_informative=15, n_redundant=5, random_state=1)
print(classification_report(Y_validation, predictions))

From the picture above, it can be seen that SVM gets an accuracy of 0.92. This accuracy can be said to be high, but under the Decision Tree method. Thus, this method has accuracy in the middle of the other methods. And from these results, 2, 5, and 6 get very low values of precision, recall, and the f1-score, namely 0.0.

Conclusion

MIC, or Myocardial Infarction Complication, is a dataset that contains patients who experience heart disease and possible complications. The need for pre-processing on this dataset is very crucial, such as for missing values, feature values that have not been normalized, and other deficiencies that need to be corrected so that the prediction results on the data can be obtained optimally. then finally, after modeling using three different algorithms, namely Naive Bayes, Decision Trees, and SVM, It is known that Decision Tree is the best algorithm to use in this MIC dataset because it can get higher accuracy compared to other algorithms.

Insight Findings from A Case Study of Customers Leaving Credit Card Services

Aldi Fianda Putra — Mon, 27 Feb 2023 18:23:26 +0000

In this post, I will share my Rakamin Academy Virtual Internship final project regarding a case study related to the problem of a bank manager.

Project Description

In this project, we are expected to solve the manager's problem by finding insights from case study where customer leaving credit card service from several datasets through data engineering and processing. This project mainly used SQL for querying the data and Tableau for data visualization.

Case Study:

A manager at a bank is annoyed by the increasing number of customers leaving their credit card service. They would really appreciate it if someone could figure out a customer profile so they could know where each customer is going so they could proactively go to the customer to provide better service and turn the customer's decision in the opposite direction.

From the case study above, form three types of data processing:

Business Objectives
Data Exploration
Insights Presentation
Conclusion and Solution

Business Objectives

From my perspective, here are some business objectives that need to be carried out to solve the problems in the case study.

Identify the profiles of customers or credit card customers.

The first business objective created is to identify the profiles of our customers or credit card customers. The profile of credit card users is very important to know. This is due to the exploration and search for insights on customer profiles so we can find out about the conditions of consumers and know the background of consumers who are starting to leave credit card services. For this purpose, finding out the profile of customers is done by exploring data and also visualizing data related to customer history.

Data exploration and visualization carried out are:

Customer based on income and average credit limit, the first exploration is related to income and average credit limit. As we know, the higher a person's income, of course, the higher the credit limit that will be given, so this needs to be explored.
Customers based on marital status and customer dependents, the second exploration is related to marriage and the customer's own responsibilities. This is because someone who is at least single and does not have many dependents has the possibility of not using a credit card, especially if they have a fairly high income. Therefore, it is worth exploring.
Customer based on education and income category, the third exploration is the exploration of education level and income category. Education does not directly affect our credit card limit, but education can affect one's income. So this is worth exploring.
Customer based on credit card and income, the final exploration related to the customer profile is the type of credit card the customer has and what is his level of income. From the existing dataset, there are blue, gold, silver, and platinum credit cards. Exploration was carried out to find out the relationship between the customer's credit card and his income.

Identify the amount of shopping or transactions made by the customer.

The second business objective is to identify purchases or transactions made by customers. This is done to see the transaction activity of customers based on the type of credit card and their income and will be measured through the ratio of credit card usage. Thus, the steps that will be carried out to achieve this business objective are to explore by utilizing the features or information needed to identify customer transactions. These features are credit limits, average shopping with a credit card, number of transactions, transaction frequency, and credit card usage ratio. Then, exploration is carried out using the type of credit card and income category as benchmarks.

Identify the relationship between the customer and the bank.

The last business objective is to identify customers' relationships with the bank. Like the problems in the problem, namely the number of customers who have started leaving credit card services. Therefore, it is necessary to identify the relationship between the customer and the bank, which is divided into two parts as before, namely based on the type of credit card and income. In this business objective, the features used are the period of contact with the bank, the number of months of inactivity, the total contact with the bank, and the ratio of credit card usage.

Data Exploration

The following is data exploration using SQL to find some insights that will be visualized later.

Customer based on their income and credit limit

SELECT Income_Category, COUNT(CLIENTNUM) as Total_Customer, 
ROUND(AVG(Credit_Limit),2) as Avg_Credit_Limit
FROM customer_data_history
GROUP BY Income_Category
ORDER BY FIELD(Income_Category,"Unknown","Less than $40K","$40K - $60K","$60K - $80K","$80K - $120K","$120K +");

Customer based on their marital status and dependent

SELECT m_db.marital_status, COUNT(*) as Status_Count,
ROUND(AVG(cdh.Dependent_count)) as AVG_Dependent,
cdh.Income_Category FROM customer_data_history cdh
LEFT JOIN marital_db m_db
ON cdh.Maritalid = m_db.id
GROUP BY m_db.marital_status, cdh.Income_Category
ORDER BY FIELD(Income_Category,"Unknown","Less than $40K","$40K - $60K","$60K - $80K","$80K - $120K","$120K +"), 
marital_status ASC;

Customer based on their education status and income category

SELECT e_db.education_level, COUNT(*) as Education_Count,
cdh.Income_Category FROM customer_data_history cdh
LEFT JOIN education_db e_db
ON cdh.Educationid = e_db.id
GROUP BY e_db.education_level, cdh.Income_Category
ORDER BY FIELD(Income_Category,"Unknown","Less than $40K","$40K - $60K","$60K - $80K","$80K - $120K","$120K +"), 
education_level ASC;

Customer based on their credit card category and income category

SELECT c_db.card_category, COUNT(*) as card_owner_total, cdh.Income_Category
FROM customer_data_history cdh
LEFT JOIN category_db c_db
ON cdh.card_categoryid = c_db.id
GROUP BY c_db.card_category,cdh.Income_Category
ORDER BY FIELD(c_db.card_category,"Blue","Gold","Silver","Platinum"),
FIELD(Income_Category,"Unknown","Less than $40K","$40K - $60K","$60K - $80K","$80K - $120K","$120K +");

Customer based on their credit card category and income category

SELECT Income_Category, COUNT(*) as Total_Consumer,
ROUND(AVG(Credit_Limit),2) as AVG_Credit_Limit, 
ROUND(AVG(Avg_Open_To_Buy),2) as AVG_Buying, 
ROUND(AVG(Total_Trans_Amt),2) as AVG_Transactions, 
ROUND(AVG(Total_Trans_Ct),2) as AVG_Transac_Freq, 
ROUND(AVG(Avg_Utilization_Ratio),2) as AVG_Credit_Util_Ratio 
FROM customer_data_history
GROUP BY Income_Category
ORDER BY FIELD(Income_Category,"Unknown","Less than $40K","$40K - $60K","$60K - $80K","$80K - $120K","$120K +");

Customer based on their credit card category and income category

SELECT c_db.card_category, COUNT(*) as Card_owner,
ROUND(AVG(cdh.Credit_Limit),2) as AVG_Credit_Limit, 
ROUND(AVG(cdh.Avg_Open_To_Buy),2) as AVG_Buying, 
ROUND(AVG(cdh.Total_Trans_Amt),2) as AVG_Transactions, 
ROUND(AVG(cdh.Total_Trans_Ct),2) as AVG_Transac_Freq, 
ROUND(AVG(cdh.Avg_Utilization_Ratio),2) as AVG_Credit_Util_Ratio 
FROM customer_data_history cdh
LEFT JOIN category_db c_db
ON cdh.card_categoryid = c_db.id
GROUP BY c_db.card_category
ORDER BY FIELD(c_db.card_category,"Blue","Gold","Silver","Platinum");

Customer based on their contact and relation with bank

SELECT cdh.Income_Category, c_db.card_category, COUNT(*) as card_owner_total,  
AVG(cdh.Months_on_book) as AVG_Months_on_book,
ROUND(AVG(cdh.Total_Relationship_Count)) as AVG_Total_Rel_Count,
ROUND(AVG(cdh.Months_Inactive_12_mon)) as AVG_Months_Inactive,
ROUND(AVG(cdh.Contacts_Count_12_mon)) as AVG_Contact_Count,
ROUND(AVG(cdh.Avg_Utilization_Ratio),2) as AVG_Credit_Util_Ratio 
FROM customer_data_history cdh
LEFT JOIN category_db c_db
ON cdh.card_categoryid = c_db.id
GROUP BY c_db.card_category,cdh.Income_Category
ORDER BY FIELD(c_db.card_category,"Blue","Gold","Silver","Platinum"),
FIELD(Income_Category,"Unknown","Less than $40K","$40K - $60K","$60K - $80K","$80K - $120K","$120K +");

Insight Presentation

The following is an explanation of the insight presentation that was encountered during the previous data exploration. I also made an interactive dashboard, which was made using Tableau and can be accessed at the following page: Tableau Dashboard .

From the graph above, it is known that the majority of customers have an income of under $ 40K. While the average credit limit is the most for those whose income is above $ 120K +. Based on this visualization, the majority of customers have a fairly low income, so the credit limit that can be given is also small for customers who have an income below $ 40K, namely those with a credit limit below $ 5K.

From the graph above, it is known that the majority of customers are undergraduates, followed by high school graduates, unknown, uneducated, still in college, postgraduates, and doctorates. Based on this visualization, it is also known that customers who are graduates are also the largest group of customers who have an income of less than $40K. In addition, it can be seen that education does not guarantee how much income a person can earn; even doctors themselves have an income of under $40K.

Marital status, average dependents, and customer income are visualized in the treemap on the side. Based on this visualization, it turns out that customers who are married, have at least 2 dependents, and earn under $40K are the largest group of customers. Followed by customers who are single and have an income under $ 40K and 2 dependents.

Customers are divided by credit card and income through the visualization on the side. Based on this visualization, the Blue type of credit card is the most popular, where all income groups of customers use this type of credit card even including those with income above $ 120K. Then followed the types of silver and gold. The Platinum card is the one with the fewest fans, so few that it doesn't even show up on the visualization.

The average number of transactions made by customers based on their type of income can be seen in the image above. Based on this visualization, the average transactions made by customers are more or less close for all income categories, with a significant difference only found in the number of customers.

The average transaction made by a customer based on the type of credit card can be seen in the image above. Based on this visualization, the average transaction is made by customers who have blue credit cards, but the average number of transactions made is fewer than those made by customers who have silver, gold, and platinum credit cards.

The average customer spending can be seen from the graph above. Based on this visualization, customers with the highest income certainly have a fairly high average spending compared to customers with other income categories. From this visualization, it can also be seen that all customer categories tend to spend within the credit limit provided.

The average customer spending by credit card can be seen from the graph above. Like the previous visualization, based on this one it can be seen that customers with a platinum type credit card have the largest average spending compared to other types of credit cards, even higher than the blue type, where previously the majority of customers with an income of $120K+ used this type of card. In addition, customers also tend not to spend beyond their credit limit.

The ratio of credit card use to total products held can be seen from the graph above. Based on this visualization, it can be seen that customers with blue credit cards are far more likely to use their credit cards, the majority of which are used by customers with incomes below $40K. Then, in terms of products held, it is also dominated by customers with incomes below $40K, followed by customers with incomes above $120K.

The average contact between the bank and the customer can be seen in the graph above. Based on this visualization, it can be seen that customers with platinum-type credit cards are the most frequently contacted by the bank. Then in terms of the period of contact with the bank, customers with an income of $ 80K-120K are the most frequent. Finally, the customers with the most inactivity are those with incomes below $40K.

Conclusion and Solution

From the business objectives, data exploration, and insight presentations that were carried out, several things and solutions can be concluded to overcome the problems in the case studies.

Conclusion

The majority of customers have an income below $40K, where they are also the majority are married or single and have at least 2 dependents. The credit limit for this category of customers is also the lowest compared to other categories.
The majority of customers still use blue credit cards; even customers with high incomes, such as those with incomes above $120K+, use this type of credit card instead of using higher types such as silver, gold, and platinum.
The average transactions made are more or less close for all income categories. However, when using the credit card approach, platinum credit cards have the highest average transactions, followed by gold and silver.
In terms of credit card usage ratio, customers with blue credit cards are the ones who use credit cards the most, which are dominated by customers with incomes below $40K. Unfortunately, customers in this income category are also the most product owners, the least contacted by the bank, and also the most inactive.

Solution

By increasing the credit limit for customers with lower middle income, it is expected to be able to attract the desire of these customers to use credit cards again.
Because many customers still use Blue type credit cards, especially since they also have middle to upper income, we should provide more benefits to credit cards at higher levels so that these customers want to switch to using Gold, Silver and Platinum credit cards.
Because blue credit cards are the most common but have the fewest average transactions, we can provide rewards or promos to customers with blue credit cards in order to increase the customer's desire to make transactions with credit cards.
Because customers with incomes below $40K are the largest group and also the largest product owners, we as banks should make frequent contact with these customers, such as by conveying promos, benefits, and other things related to our credit cards.

Deploying Multicontainer Application on Openshift

Aldi Fianda Putra — Sun, 18 Sep 2022 15:15:09 +0000

In this post I will share my Kominfo's Fresh Graduate Academy project about Deploying Multicontainer Applications on Openshift

What is Openshift?

Basicly, Openshift is an orchestration tool owned by Red Hat, which provides a service for cloud development platforms. According to Red Hat Company, Kubernetes is the kernel of a distributed system and Openshift is its distribution. Openshift will make it easier for developers to add the capabilities of PaaS.

Project Description

This is a simple project to deploy PHP applications with MySQL as it database on Openshift. The source code was actually my assignment during my 4th semester at college. The deployment method that I will use is source-to-image, or popularly known as S2I.

S2I Example, source: KubeSphere

Deploy Procedure

PHP Application

The first thing to do is create a new project. In this case, you should login to your Openshift Web UI and choose the Developer role. Then click topology and in the project dropdown, choose Create Project and fill it with your project name.

After the topology is created, then the topology page would look like this where you could choose a deployment source, like from Git, Dockerfile, YAML, etc. In this case, I will use deployment from Git.

After clicking deployment source from Git, the first thing that appears is the text box to input our GitHub repository link. Openshift will choose the suitable builder image based on our source code. Since most of my code is written in PHP, Openshift chose PHP as my builder image. Then choose the PHP version that you want to be used.

Lastly, give the application a unique name, select Deployment Config as the resource type, check the "Create route to the application" checkbox, and then click Create. After this, Openshift will deploy the PHP application for us.

MySQL Application

The next application that will be deployed is the database. To deploy a database easily, I will use the same method as PHP before. First click the +Add button on the side menu, then choose Database.

After clicking the database, on the next page we need to choose a database template. In this case, I choose MySQL, and then a new page will appear where we need to fill in our desired values like for database storage, namespace, service name, username, password, database name, volume capacity, and database version. After that, click Create and Openshift will create a MySQL Application.

Connecting both application

Both the PHP and MySQL applications were not directly connected after we deployed them, so we need to connect them both. The topology of both applications will be as shown in the image above. In my source code, I created a PHP connection file with the following code:

$host=getenv("databaseip");
$port=getenv("databaseport");
$user=getenv("databaseuser");
$pass=getenv("databasepassword");
$db=getenv("databasename");

$mysqli=mysqli_connect($host,$user,$pass,$db);

That block of code is used to fetch the database IP, port, user, password, and database name from the database application that was listed in the PHP environment variable. Hence, we need to input those variables inside the PHP's pod environment variable (you need to know your database pod ip; in this case, I logged in to Red Hat VM and then used get svc to get the database pod ip). The result will look like the image below.

After filling in those database variables, click save and rollback the PHP application. After that, click the "Open URL" button in the top right of the PHP application and you will be redirected to the PHP application. The result will look like this in my project.

There are some problems that still occur in my project. The problems are the authentication method for the database and that no table named "Proker" exists in the database. To fix the first problem, we need to login with the root account (usually by using mysql -u root and then input your password) and then enter the code below.

ALTER USER 'username'@'%' IDENTIFIED WITH mysql_native_password BY 'password';

Why use % and not localhost? because it won't work since using username@localhost will only allow access from localhost. But using username@% will access all locations that are available, hence we can connect to our database IP.

Next is to create a table for the database. This one is simple. We can solve this by creating a simple table like this.

CREATE TABLE proker(
    nomorProgram INT NOT NULL,
    namaProgram VARCHAR(32) NOT NULL,
    suratKeterangan VARCHAR(32) NOT NULL,
    PRIMARY KEY (nomorProgram)
);

When the table is still empty

The table after adding some new rows

After that, we can try logging in again to my page and the main page will generate this kind of table, where the table is empty but we can add new value to the database just like in the image above. And by that way, my project ends, and thank you for taking the time to read my article.

Sorry for the bad image resolution. Since I can't log in anymore to my Red Hat account because it can only be accessed during the course, I have to crop the image from my final project report.

Streaming Twitter Data and Scheduling

Aldi Fianda Putra — Fri, 25 Feb 2022 12:14:55 +0000

In this post, I will share my data engineering course final project from the previous semester. The task was to create an ETL process for streaming data. The streamed data sources are from Twitter and fetched using Kafka. There are some additional tasks, like using Airflow for scheduling all tasks and creating a dashboard. in this project we stream data to fetch tweet about Windows 11. So the output of this project after Stream and ETL process is a sentiment analysis about Windows 11.

Installing Kafka

Before install Kafka, make sure that you have Java version 1.8.0 and scala. Then head to https://kafka.apache.org/downloads to download the installation file and choose the file that has the same scala version that has been installed in your computer, because my laptop has Scala with 2.12 version so we going to download Kafka with version 3.0.0. Installing Kafka is pretty easy, you just need to extract the file somewhere like in home directory. After extracting the file, change the snapshot directory in zookepeer.properties and log directory in server.properties to your kafka installation like this.

for snapshot directory

dataDir=/home/USER/kafka/data/zookeeper

for log directory

log.dirs=/home/USER/kafka/data/kafka

After adding both directories, you can use Kafka right away. You can add Kafka to your path environment variable for easier use. Additionally, if your Kafka broker won't run, remove the listener that is being commented in the server.properties like the image below.

All Kafka installation is now completed, now you can run both Zookeeper Server and Kafka Server by using command below (make sure you run zookeeper first).

zookeeper-server-start.sh ../../config/zookeeper.properties

kafka-server-start.sh ../../config/server.properties

Installing Airflow

The reason we used Linux for this project is that installing Airflow is somehow harder on Windows. To install Airflow, first make sure you already have Python-pip. After that, you can install Airflow right away by running "pip install apache-airflow" and waiting until the installation is completed. You can add airflow to your environment variable like Kafka did before.

After installation is completed, create a new database by using the command airflow db init. This command is only used once, the first time we use Airflow.

Finally, we can use airflow. To use Airflow, you need to open two different consoles and run airflow webserver-p 8080 and airflow scheduler in each console, like in the image below.

Now you can access Airflow at localhost:8080, but you need credentials first to access Airflow. To do this, you can create a new account by running airflow users create [-h]". -e EMAIL-f FIRSTNAME-l LASTNAME [-p PASSWORD] -r ROLE [--random-password] -u USERNAME. Then login to Airflow with the account you have just created. After you login, the homepage will look like the image below.

Streaming data and Simple ETL

Now we are on the main subject, streaming data and ETL process. First you need to create a topic by running the command kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic windows_11. Next, we need to create several python files for the simple ETL process since we are going to schedule every python program by using Airflow.

Stream

To stream data, we create a python file. Since we are going to fetch data from Twitter, make sure you have a Twitter developer account to get access tokens and keys. To stream data, you can use the blocks of code below (change access, consumer key and secret to yours). After that, save the code into a python file named stream.py.

# import library
import tweepy
from kafka import KafkaProducer
from datetime import datetime, timedelta
import csv
import pandas as pd

# Twitter Consumer and Access
consumer_key = YOUR_KEY
consumer_secret = YOUR_KEY_SECRET
access_token = YOUR_ACCESS_TOKEN
access_token_secret = YOUR_ACCESS_TOKEN_SECRET

# setup authentication
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

# instantiation API
api = tweepy.API(auth, wait_on_rate_limit=True)

# Adjust time to local time (GMT +7 / UTC +7)
def normalize_time(time):
    mytime = datetime.strptime(time, "%Y-%m-%d %H:%M:%S")
    mytime += timedelta(hours=7)
    return (mytime.strftime("%Y-%m-%d %H:%M:%S"))

# instantiation of Kafka Producer
producer = KafkaProducer(bootstrap_servers=['localhost:9092'], api_version=(0, 10, 1))

# Topic initialization, keyword, and max query
topic_name = 'windows_11'
search_key = "windows 11"
maxId = -1
maxTweets = 3000
tweetCount = 0
tweetsPerQuery = 500

# Csv declaration, change path to yours
csvFile = open("/home/stndb01/Documents/Data_Engineering/Proyek/"+search_key+".csv", "a+", newline="", encoding="utf-8")
csvWriter = csv.writer(csvFile)

# List declaration
tweet_id = []
tweet_username = []
tweet_text = []

# Looping to fetch tweet by using twitter API until specified limit
while tweetCount < maxTweets:
    # getting first tweet data
    if maxId <= 0:
        # newTweets = api.search_tweets(q=search_key, lang="en", count=tweetCount, max_id=maxId)
        newTweets = api.search_tweets(q=search_key, lang="en", count=tweetCount)
    # getting data from second tweet kedua and forth
    newTweets = api.search_tweets(q=search_key, lang="en", count=tweetsPerQuery)

    # getting certain attribute from tweet
    for i in newTweets:
        record = str(i.user.id_str)
        record += ';'
        record = str(i.user.name)
        record += ';'
        # record += str(normalize_timestamp(str(i.created_at)))
        # record += ';'
        # record += str(i.full_text.encode('utf-8'))
        record += str(i.text.encode('utf-8'))
        record += ';'
        print(str.encode(record))
        producer.send(topic_name, str.encode(record))

        tweet_id.append(str(i.user.id_str))
        tweet_username.append(str(i.user.name))
        tweet_text.append(str(i.text.encode('utf-8')))
        tweets = [str(i.user.id_str), str(i.user.name), str(i.text.encode('utf-8'))]
        csvWriter.writerow(tweets)

    # adding TweetCount and MaxId
    tweetCount += len(newTweets)
    maxId = newTweets[-1].id

dictTweets = {"id":tweet_id, "username":tweet_username, "text":tweet_text}
df = pd.DataFrame(dictTweets)

Extract

The next step is to extract the data from the csv file we've just created. In extract step we extract data from csv to a new temporary csv with prefix _e, this csv later will be deleted in future step. Save the code below into new python file named extract.py.

import pandas as pd
kolom = ['id', 'username', 'tweet']
df_windows = pd.read_csv('/home/stndb01/Documents/Data_Engineering/Proyek/windows 11.csv', names = kolom)

df_windows.to_csv('/home/stndb01/Documents/Data_Engineering/Proyek/windows_11_e.csv')

Transform

The csv file that we've just extracted contains duplicated data. Then, in this step, we will transform the data by removing duplicated data. The code below will use the previous windows_11_e.csv to delete duplicated data and save it to a new csv called windows_11_t.csv. Save the code below into a new python file named transform.py.

import pandas as pd
kolom = ['id', 'username', 'tweet']
df_windows = pd.read_csv('/home/stndb01/Documents/Data_Engineering/Proyek/windows_11_e.csv', names = kolom)
#df_windows.duplicated().value_counts()


df_windows.drop_duplicates(inplace=True, ignore_index=True)
df_windows
df_windows.to_csv('/home/stndb01/Documents/Data_Engineering/Proyek/windows_11_t.csv')

Load

After we transform the data, we load the data into a local database. For this project, we used SQLite3 for the database. The code below is used to load data. This code also deletes both the temporary CSV files that were created in the extract and transform steps. Save the code below into a new python file named load.py.

import sqlite3 as s3
import pandas as pd
import os

# creating connection to database
conn = s3.connect('/home/stndb01/Documents/Data_Engineering/Proyek/windows11_data.db')

# Object cursor to run SQL query
cur = conn.cursor()

# load dataframe to datamart
df_windows = pd.read_csv('/home/stndb01/Documents/Data_Engineering/Proyek/windows_11_t.csv')
df_windows.to_sql('windows11_table',conn,if_exists='replace',index=False)
os.remove("/home/stndb01/Documents/Data_Engineering/Proyek/windows_11_e.csv")
os.remove("/home/stndb01/Documents/Data_Engineering/Proyek/windows_11_t.csv")

Scheduling

The files for the Stream and ETL processes have been created. Then we can create a schedule for those tasks. By creating a new Python file, we are going to create a DAG, or Directed Acyclic Graph, for every task. The DAG code looks like the code below. Save the code below into a new python file named scheduling-proyek.py.

from datetime import timedelta
from airflow.operators.bash import BashOperator
from airflow.utils.dates import days_ago
from airflow import DAG

# insert your airflow account 
default_args = {
    'owner': 'admin',
    'depends_on_past': False,
    'email': ['airflow@example.com'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
}
with DAG(
    'Proyek_Akhir',
    default_args=default_args,
    description='Scheduling untuk task proyek akhir',
    schedule_interval=timedelta(days=1),
    start_date=days_ago(2),
    tags=['proyek'],
) as dag:

    stream        = BashOperator(
                    task_id='Stream_data_twitter',
                    bash_command='python /home/stndb01/Documents/Data_Engineering/Proyek/stream.py'
                    #bash_command='python stream.py',
                    )

    extract        = BashOperator(
                    task_id='Extract_data_twitter',
                    bash_command='python /home/stndb01/Documents/Data_Engineering/Proyek/extract.py'
                    #bash_command='python extract.py',
                    )

    transform        = BashOperator(
                    task_id='Transform_data_twitter',
                    bash_command='python /home/stndb01/Documents/Data_Engineering/Proyek/transform.py'
                    #bash_command='python transform.py',
                    )

    load        = BashOperator(
                    task_id='Load_data_twitter',
                    bash_command='python /home/stndb01/Documents/Data_Engineering/Proyek/load.py'
                    #bash_command='python load.py',
                    )

    # create the execute order, set_downstream mean we execute the 
    # left one and then the right one
    stream.set_downstream(extract)
    extract.set_downstream(transform)
    transform.set_downstream(load)

After you save the file, move the scheduling file to your Airflow installation folder. Inside you will see a DAG folder. Move the scheduling file to that folder and run python scheduling-proyek.py. Then return to the Airflow homepage and you will see a new DAG for your project, like in the image below.

Finally, click the DAG and you will be redirected to a new page that contains a tree graph based on how you queued all your tasks. Click the play button and then the task will run as you set the interval. Since the start date was set 2 days ago and timedelta was set to 1, all tasks will be run in order and every task that has been completed will be marked with green. The result will look like the image below.

Sentiment Analysis

To do a sentiment analysis, we load the data that has been saved to SQLite database. The code and result would be like these.

# Creating connection to database
conn = s3.connect('windows11_data.db')

# creating cursor
cur = conn.cursor()
query = '''select * from windows11_table'''
df = pd.read_sql(query, conn)
df

Since we only deleted duplicated tweets in the transform process before, there are still some other things that are not really fixed that can cause problems in analysis, such as emoji, url links, symbols, and etc. Because those items are saved in byte format, they can interfere with analysis.So we used the regex function to delete them.

def preprocess_text(review):
    review = codecs.decode(review, 'unicode_escape') #remove escape character
    review = review[2:-1]
    review = re.sub('((www\.[^\s]+)|(https?://[^\s]+))','URL', review)
    review = re.sub('[^\x00-\x7f]', '', review) 
    review = re.sub('@[^\s]+','USER', review)
    review = review.lower().replace("ё", "е")
    review = re.sub('[^a-zA-Zа-яА-Я1-9]+', ' ', review)
    review = re.sub(' +',' ', review)
    return review.strip()
df['tweet'] = [preprocess_text(review) for review in df['tweet']]

After that, the data should be okay to be analyzed. Next, we created a new function by using the textblob library to find subjectivity, polarity, and analysis. The result will be added to the new column in the previous dataframe and from that we will know which tweet can be considered a positive, negative, or even neutral review of the Windows 11 launch. The code and result would be as below.

from textblob import TextBlob

def sentiment_analysis(data):
  def getSubjectivity(text):
    return TextBlob(text).sentiment.subjectivity

  #Create a function to get the polarity
  def getPolarity(text):
    return TextBlob(text).sentiment.polarity

  def getAnalysis(score):
    if score < 0:
      return "Negative"
    elif score == 0:
      return "Neutral"
    else:
      return "Positive"

  #Create two new columns 'Subjectivity' & 'Polarity'
  data["TextBlob_Subjectivity"] = data["tweet"].apply(getSubjectivity)
  data["TextBlob_Polarity"] = data["tweet"].apply(getPolarity)

  data["TextBlob_Analysis"] = data["TextBlob_Polarity"].apply(getAnalysis)
  return data

The next code is an optional step. We count the top 20 words that appear frequently, and that would also be displayed with the sentiment analysis result later.

from collections import Counter

def getMostFrequentWord(df):
    most_freq = Counter(" ".join(df["tweet"]).split()).most_common(30)

    most_freq_filtered = {}
    for i in most_freq:
        if(i[0] == 'user' or i[0] == 'url' or i[0] == 'rt'):
            continue
        else:
            most_freq_filtered[i[0]] = i[1]
            if(len(most_freq_filtered) == 20): # ambil 20 kata paling banyak muncul
                break

    return most_freq_filtered

#Calling the function
most_freq = getMostFrequentWord(df)
most_freq

Dashboard

The last step in this project is creating a dashboard that will display the sentiment analysis results in a pie chart and also show the top 20 words that appear frequently. The code and result would look like below.

f, axes = plt.subplots(1, 2, figsize = (15, 7))
# sns.set_style("white")
colors = sns.color_palette('pastel')[0:5]
# colors = sns.color_palette('bright')[0:5]
f.suptitle("{} Twitter Topic Analysis".format(TOPIC), fontsize=20)
f.tight_layout(pad=3.0)

# figure 1
ax1 = plt.subplot2grid((1,2),(0,0))
ax1.title.set_text("{} Tweet Sentiment".format(TOPIC))
fig1 = plt.pie(df.groupby("TextBlob_Analysis").size(), labels = df.groupby("TextBlob_Analysis").size().index, colors = colors, autopct='%.0f%%')

# figure 2
ax2 = plt.subplot2grid((1,2),(0,1))
ax2.title.set_text("{} Tweet Most Common Words".format(TOPIC))
fig2 = sns.barplot(x=list(most_freq.keys()), y=list(most_freq.values()))
fig2.set(xlabel='Most Common Words', ylabel='Frequency')
for item in fig2.get_xticklabels():
    item.set_rotation(90)

Reference

How to install Apache Airflow
Working with streaming Twitter data using Kafka
How to test your data with Great Expectations
Sentiment Analysis using TextBlob
Exploratory data analysis for text data: EDA using python

Wiki.js untuk Organisasi

Aldi Fianda Putra — Sun, 29 Aug 2021 12:05:05 +0000

Apa itu Wiki.js

Wiki.js adalah sebuah aplikasi wiki yang open source yang berjalan pada Node.js dan ditulis dengan bahasa javascript. Pada wiki.js konten ditulis dalam file markdown. Wiki.js menggunakan git untuk menyimpan konten yang akan dipublish.

System Requirement untuk Wiki.js

Pada postingan ini, saya akan menampilkan prosedur instalasi wiki.js versi 2.x. Dengan demikian berikut adalah service yang harus diinstall terlebih dahulu

Node.js versi 10.12 atau yang lebih baru
Database seperti Postgresql, mariadb, mysql dan lain-lain

Prosedur Instalasi

Pada proses instalasi wiki.js kali ini, saya akan memasang wiki.js pada perangkat linux dengan menggunakan docker compose, sehingga Node.js dan database tidak perlu diinstall terlebih dahulu. Berikut konfigurasi file docker-compose.yml untuk penginstalasian wiki.js pada linux.

version: "3"
services:

  db:
    image: postgres:11-alpine
    environment:
      POSTGRES_DB: wiki
      POSTGRES_PASSWORD: wikijsrocks
      POSTGRES_USER: wikijs
    logging:
      driver: "none"
    restart: unless-stopped
    volumes:
      - db-data:/var/lib/postgresql/data

  wiki:
    image: requarks/wiki:2
    depends_on:
      - db
    environment:
      DB_TYPE: postgres
      DB_HOST: db
      DB_PORT: 5432
      DB_USER: wikijs
      DB_PASS: wikijsrocks
      DB_NAME: wiki
    restart: unless-stopped
    ports:
      - "3000:3000"

volumes:
  db-data:

Pada konfigurasi file docker-compose.yml diatas adalah konfigurasi default yang dapat diperoleh pada halaman wiki.js, namun pada untuk port saya ubah menjadi 3000:3000 karena port 80 telah saya gunakan. Setelah konfigurasi dirasa sudah cukup maka kita dapat menjalankan perintah berikut pada direktori yang sama dengan file yml untuk menjalankan servis wikijs.

docker-compose up

Setelah menjalankan perintah diatas, maka proses pemasangan wiki.js akan dimulai dan dapat ditunggu untuk beberapa saat. kemudian apabila tampilan berikut sudah muncul maka wiki.js dapat kita akses melalui browser.

Berdasarkan tampilan terminal sebelumnya, kita diminta untuk mengakses halaman http://YOUR-SERVER-IP:3000/ untuk melanjutkan proses setup. Karena wiki.js dipasang pada komputer sendiri maka alamat yang diakses adalah https://127.0.0.1:3000 atau localhost:3000 dan diperoleh tampilan sebagai berikut.

Ketika tampilan localhost:3000 sudah seperti diatas maka proses pemasangan wikijs sudah berhasil. Berikutnya kita dapat memulai melakukan registrasi akun pada halaman tersebut. Terdapat juga fitur untuk membuat site url pada halaman registrasi.

Tampilan diatas akan muncul ketika kita sudah melakukan registrasi, akan muncul halaman untuk login dengan menggunakan kredensial yang telah kita buat sebelumnya.

Tampilan diatas akan muncul ketika kita pertama kali login dan tidak ada konten sama sekali. Sehingga kita dapat menekan tombol "Create Home Page" untuk membuat halaman baru.

Pada pembuatan konten pada wiki.js, terdapat berbagai cara seperti dengan menggunakan markdown layaknya konten pada halaman dev.to ini ataupun dengan menggunakan html dan rich text format. Pada pembuatan konten pertama kali kita diminta juga untuk menentukan direktori penyimpanan konten yang mana secara default berada pada direktori /en/. Selain itu ketika kita membuat konten, kita dapat juga membuat tag sehingga pembaca dapat mencari konten lain yang relevan dengan konten yang sekarang dibuat melalui tag yang diberikan.

Pada postingan ini saya mencontohkan membuat konten dengan menggunakan markdown. Caranya kurang lebih sama dengan pembuatan konten pada halaman dev.to seperti tampilan diatas. Tampilan sebelah kiri adalah bagian Edit untuk menuliskan konten dan sebelah kanan adalah tampilan Preview dari konten yang kita tulis. Setelah konten sudah selesai dibuat maka kita dapat menekan tombol Create pada bagian atas untuk membuat halaman baru tersebut.

Tampilan diatas adalah tampilan halaman yang telah kita buat sebelumnya. Halaman yang kita buat tersebut adalah halaman utama atau main page. Apabila kita ingin membuat halaman baru maka cukup menekan tombol "New Page" pada sudut kanan atas dan melakukan prosedur yang sama seperti pembuatan konten pertama kali, dengan demikian halaman wiki.js sudah dapat digunakan dan dimanfaatkan untuk pembuatan konten organisasi.

Demikian artikel mengenai pembuatan konten untuk organisasi dengan wiki.js ini saya sampaikan. Mohon koreksi apabila terdapat kesalahan dan semoga konten yang saya sampaikan ini bermanfaat. Terima kasih.