Building Chagu: An AI-Driven Solution for Real-Time Cybersecurity Threat Detection and Logging

The Challenge:

Participating in a hackathon is always an exciting and intense experience, but this time, the challenge was particularly engaging. I was tasked with developing an AI-driven solution that could autonomously detect, analyze, and respond to cybersecurity threats in real time at the end-user level.

In this article, I’ll walk you through how I designed Chagu. This solution detects anomalies and seamlessly integrates with logging systems, allowing anomalies to be logged as critical events in real time. A key part of this project involved securing logs using blockchain, which provides a tamper-proof audit trail of all log entries. This solution forms the foundation for future PhD research.

Why Use Blockchain for Anomaly Detection?

In today’s cybersecurity landscape, organizations are constantly under threat from malicious actors, and logs are the first line of defense. However, traditional log management systems are vulnerable to tampering, data corruption, and other security breaches. This is where blockchain technology comes in.

Blockchain provides a tamper-proof, immutable ledger that ensures every log entry is securely stored. By adding each log entry to a chain of blocks, Chagu makes it impossible to alter historical logs without affecting the entire chain’s integrity. This is critical for detecting security breaches after they occur — preserving evidence is just as important as real-time detection.

But why merge blockchain with anomaly detection?

Trust and Transparency : In traditional systems, log integrity can be questioned. With blockchain, logs are cryptographically secure and transparent, ensuring that even if an attacker tries to alter logs to hide their tracks, the blockchain’s structure will reveal the tampering.
Immutable Evidence : Blockchain secures the logs, and when Chagu detects an anomaly, it flags the log. This flagged log cannot be tampered with, providing reliable data for post-incident investigation and forensic analysis.
Advanced Threat Protection : By securing logs in real time, Chagu is pioneering a proactive approach to cyber defense. Rather than responding to threats after they’ve caused damage, Chagu integrates real-time AI anomaly detection with a tamper-proof log system, mitigating potential attacks before they can escalate.

The Current Cybersecurity Landscape and Chagu’s Role as a Pioneer

In the evolving world of cybersecurity, the volume and complexity of cyber threats continue to grow. From nation-state actors to organized crime , attackers are employing increasingly sophisticated techniques to breach security systems. They exploit vulnerabilities not only in infrastructure but also in the logging and monitoring systems that are supposed to detect them.

Problem 1: Tampering with Logs Logs are vital for detecting security breaches, but in many cases, after an attacker gains access to a system, they erase or manipulate logs to cover their tracks. Traditional logging systems, while useful, don’t offer strong guarantees of integrity, leaving organizations exposed.
Problem 2: Anomalies Go Unnoticed Many existing systems produce an overwhelming amount of data, making it hard to spot unusual behavior that could indicate a breach. By the time suspicious patterns are recognized, it may be too late.
Problem 3: Lack of Automated Response Even when anomalies are detected, many systems rely on manual intervention, delaying the response to threats. This delay gives attackers enough time to exploit vulnerabilities.

Why the World Needs Chagu:

Chagu was built with a vision to tackle these issues head-on by merging blockchain’s transparency and security with AI-driven anomaly detection. Here’s why it stands out as a pioneer in the cybersecurity field:

Merging AI and Blockchain for Tamper-Proof Anomaly Detection

Chagu is one of the first solutions to combine the security of blockchain with the predictive power of AI. Logs are secured in real-time, and any suspicious activity is immediately flagged. The immutable nature of blockchain ensures that these flagged logs can be trusted as reliable evidence of anomalies, making Chagu a trustworthy forensic tool.

Automated Response to Threats:

When Chagu detects an anomaly, it doesn’t just log the event — it can trigger automated security responses, such as raising alerts or even quarantining affected systems. This immediate response is key to preventing threats from escalating.

Real-Time Security in an Overloaded World:

In the age of big data, where companies are generating enormous amounts of log data every second, Chagu cuts through the noise to identify genuine threats. By analyzing logs in real-time and securing them using blockchain, Chagu offers a solution that is scalable, efficient, and secure.

The Problem: Real-Time Cybersecurity at Scale

The challenge was to build a solution capable of detecting unusual patterns in logs and automatically responding to potential cybersecurity threats. However, a key requirement was that the solution must:

Secure logs using a tamper-proof system.
Detect real-time anomalies in system logs.
Log and flag critical anomalies for immediate attention.

The Approach: Introducing Chagu.

The AI-Driven Solution

Chagu is an AI-based cybersecurity solution designed to securely log data, detect anomalies in real-time, and automatically flag critical threats. Let’s take a look at the core components of the project.

One of the unique aspects of Chagu is the way logs are secured using blockchain technology. This ensures that each log entry is stored in a tamper-proof chain of blocks. Below is the blockchain class, which was designed to securely store logs and validate their integrity over time.

import hashlib
import time

class Block:
    def __init__ (self, index, previous_hash, timestamp, data, hash):
        self.index = index
        self.previous_hash = previous_hash
        self.timestamp = timestamp
        self.data = data
        self.hash = hash
class Blockchain:
    def __init__ (self):
        self.chain = [self.create_genesis_block()]
    def create_genesis_block(self):
        """
        Creates the first block in the blockchain, known as the genesis block.
        """
        genesis_block = Block(0, "0", int(time.time()), "Genesis Block", "")
        genesis_block.hash = self.calculate_hash(genesis_block)
        return genesis_block
    def calculate_hash(self, block):
        """
        Calculates the hash of a block by hashing together its index, previous hash,
        timestamp, and data.
        """
        block_string = f"{block.index}{block.previous_hash}{block.timestamp}{block.data}".encode()
        return hashlib.sha256(block_string).hexdigest()
    def get_latest_block(self):
        """
        Returns the latest block in the blockchain.
        """
        return self.chain[-1]
    def add_block(self, data):
        """
        Adds a new block to the blockchain.
        - The block is created with the current time, data, and hash of the previous block.
        """
        latest_block = self.get_latest_block()
        new_block = Block(len(self.chain), latest_block.hash, int(time.time()), data, "")
        new_block.hash = self.calculate_hash(new_block)
        self.chain.append(new_block)
        print(f"Block added: {new_block.hash}")
        return new_block
    def is_chain_valid(self):
        """
        Validates the blockchain by ensuring each block's hash is correct and
        that the chain has not been tampered with.
        """
        for i in range(1, len(self.chain)):
            current_block = self.chain[i]
            previous_block = self.chain[i - 1]
            # Check if the current block's hash is correct
            if current_block.hash != self.calculate_hash(current_block):
                print(f"Invalid block hash at index {i}")
                return False
            # Check if the previous block's hash matches the current block's previous hash
            if current_block.previous_hash != previous_block.hash:
                print(f"Invalid previous hash at index {i}")
                return False
        print("Blockchain is valid.")
        return True

This modified blockchain class provides several benefits:

Tamper-Proof Audit Trail : Every log entry is securely stored in a block, and each block’s hash is calculated based on the previous one, ensuring data integrity.
Validation : The is_chain_valid method validates the entire chain, ensuring that no block has been altered and that the log entries remain intact.

How It Works:

Genesis Block : The blockchain starts with a genesis block, the first block in the chain, which serves as the foundation. This block has a predefined value and a "0" previous hash.
Adding Blocks : Each log entry is securely transformed into a block that includes a timestamp, the previous block’s hash, and the data itself. This creates a chain where each block is cryptographically linked to the previous one.
Chain Validation : The chain is periodically validated to ensure that it hasn’t been tampered with. If any block’s data is altered, the validation process will detect it by comparing the recalculated hash with the stored hash.

Anomaly Detection in Real-Time Logging:

After securing the logs with blockchain, Chagu’s machine learning model analyzes system logs to detect anomalies in real-time. Here’s the logging wrapper that integrates anomaly detection with the blockchain-based logging system.

import logging
import numpy as np
from chainguard import data_transformer
from anomaly_detection_tool import AnomalyDetectionModel

class LoggingWrapper:
    def __init__ (self, log_file='chagu.log'):
        self.logger = logging.getLogger('ChaguLogger')
        self.logger.setLevel(logging.INFO)
        # Initialize Chagu's anomaly detection model and data transformer
        self.anomaly_model = AnomalyDetectionModel(input_shape=10)
        self.data_transformer = data_transformer.DataTransformer()
    def log_event(self, message):
        # Securely transform the log
        secure_log = self.data_transformer.secure_transform(message)
        log_values = self.preprocess_log(secure_log['data'])
        # Detect anomaly in the log
        log_array = np.array([log_values], dtype=np.float32)
        is_anomaly = self.anomaly_model.model.predict(log_array)[0]
        # Log as critical if an anomaly is detected
        if is_anomaly > 0.5:
            self.logger.critical(f"Anomaly detected! Log: {message}")
        else:
            self.logger.info(f"Log is normal. Log: {message}")
    def preprocess_log(self, log_message):
        # Extract numerical values from log data
        import re
        numbers = re.findall(r'\d+', log_message)
        values = [float(num) for num in numbers]
        return values[:10]

With this logging wrapper:

Critical Anomalies : If an anomaly is detected in a log entry, it is immediately flagged and logged as a critical event.
Blockchain-Secured Logs : All logs are securely transformed using blockchain before being processed by the anomaly detection model.

Example in Action:

Let’s consider two real-life logging events processed by Chagu.

Normal Log Event: A user accessed the system at 09:30 AM.

curl -X POST http://127.0.0.1:5000/process_log -H "Content-Type: application/json" -d '{"log": "User JohnDoe accessed the system at 2024-10-01 09:30:00"}'

Log Output:

2024-10-01 09:30:26 - INFO - Log is normal. Log: User JohnDoe accessed the system at 2024-10-01 09:30:00

2. Anomalous Log

Event: A user accessed the system outside normal hours (2:30 AM).

curl -X POST http://127.0.0.1:5000/process_log -H "Content-Type: application/json" -d '{"log": "User JohnDoe accessed the system at 2024-10-01 02:30:00"}'

Log Output:

2024-10-01 02:30:42 - CRITICAL - Anomaly detected! Log: User JohnDoe accessed the system at 2024-10-01 02:30:00

Conclusion:

The hackathon challenge presented an opportunity to build a solution that detects and logs cybersecurity threats in real time. Chagu uses AI-driven anomaly detection combined with blockchain-secured log transformation to flag critical events that require immediate attention.

This project forms a foundation for my upcoming PhD research, where I will explore more advanced AI techniques for cybersecurity. You can watch the Chagu demo video here.

Stay tuned for more updates as Chagu evolves!