DEV Community: Tahmid Ul Muntakim

MariaDB Administration Cheat Sheet: Practical Use Cases and Examples

Tahmid Ul Muntakim — Fri, 10 Jan 2025 14:00:58 +0000

MariaDB Administration Cheat Sheet: Practical Use Cases and Examples

Managing MariaDB effectively requires an in-depth understanding of its core configurations, commands, and features. This cheat sheet, enriched with interactive use cases, provides a practical guide for database administrators to handle MariaDB efficiently.

Core Configurations

MariaDB Configuration File:

Location: /etc/mysql/mariadb.conf.d/50-server.cnf

Key sections:

[mysqld]: Define server options like port, socket, and datadir.

port = 3306
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql

Logging Options: Enable logs for monitoring.

log_error = /var/log/mysql/error.log
slow_query_log = 1
slow_query_log_file = /var/log/mysql/mariadb-slow.log
long_query_time = 1

Performance Tuning: Optimize performance with parameters like:

innodb_buffer_pool_size = 512M
query_cache_size = 32M
max_connections = 200

Interactive Use Case:
Imagine your database performance degrades during peak traffic. By tweaking innodb_buffer_pool_size, you can allocate more memory for caching data, significantly improving read speeds.

Basic Commands

1. Starting MariaDB Service:

sudo systemctl start mariadb

2. Accessing MariaDB Shell:

sudo mysql -u root -p

3. Creating a Database:

CREATE DATABASE sales_data;

4. Granting Privileges:

CREATE USER 'lab_user'@'localhost' IDENTIFIED BY 'test123';
GRANT ALL PRIVILEGES ON sales_data.* TO 'lab_user'@'localhost';
FLUSH PRIVILEGES;

5. Exiting MariaDB Shell:

QUIT;

Use Case: Recovering from Common Issues

Problem: Anonymous user conflicts with root login.
Solution:

Start MariaDB in safe mode:

   sudo mysqld_safe --skip-grant-tables &

Update root user password:

   UPDATE mysql.user SET authentication_string = PASSWORD('newpassword') WHERE User='root' AND Host='localhost';
   FLUSH PRIVILEGES;

Restart MariaDB:

   sudo systemctl restart mariadb

Hands-On: Creating and Managing a Table

Scenario: Creating a table for student records in a new database lab_db.

Steps:

Create the database:

   CREATE DATABASE lab_db;

Use the database:

   USE lab_db;

Create a students table:

   CREATE TABLE students (
       id INT AUTO_INCREMENT PRIMARY KEY,
       name VARCHAR(100) NOT NULL,
       age INT NOT NULL
   );

Insert a record:

   INSERT INTO students (name, age) VALUES ('Alice', 21);

Update the record:

   UPDATE students SET age = 22 WHERE name = 'Alice';

Delete the record:

   DELETE FROM students WHERE name = 'Alice';

Drop the table and database:

   DROP TABLE students;
   DROP DATABASE lab_db;

Performance Monitoring

Enable Slow Query Log:

Edit the configuration file:

   sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf

Add the following:

   slow_query_log = 1
   slow_query_log_file = /var/log/mysql/mariadb-slow.log
   long_query_time = 1

Restart MariaDB:

   sudo systemctl restart mariadb

Check Logs:

sudo tail -f /var/log/mysql/mariadb-slow.log

Advanced Concepts: Role Management

Creating and Assigning Roles:

Create a role:

   CREATE ROLE 'developer_role';

Grant privileges to the role:

   GRANT SELECT, INSERT, UPDATE ON project_alpha.* TO 'developer_role';

Assign the role to a user:

   GRANT 'developer_role' TO 'bob'@'192.168.121.%';

Revoking Roles:

REVOKE 'developer_role' FROM 'bob'@'192.168.121.%';

MariaDB in Docker

Docker Setup for MariaDB:

Create a docker-compose.yaml:

   version: '3.8'
   services:
     mariadb:
       image: mariadb:10.5
       environment:
         MYSQL_ROOT_PASSWORD: rootpassword
         MYSQL_DATABASE: testdb
       ports:
         - "3306:3306"

Start the container:

   docker-compose up -d

Access MariaDB:

   docker exec -it <container_name> mariadb -u root -p

MyISAM and InnoDB Storage Engines

Key Differences:

MyISAM:

Simplicity and Speed: Optimized for read-heavy operations.
No Transactions: Lacks ACID compliance.
Table-Level Locking: Can lead to contention in write-heavy environments.

InnoDB:

ACID Compliance: Supports transactions for data integrity.
Row-Level Locking: Higher concurrency for write-heavy applications.
Foreign Key Support: Enforces referential integrity between tables.

When to Use:

Use MyISAM for read-heavy operations requiring full-text indexing.

Use InnoDB for transactional applications needing high reliability and data integrity.

Table Partitioning Cheat Sheet

Why Needed:

Distribute large tables into smaller, manageable pieces.

Improve query performance by scanning only relevant partitions.

Enable parallel processing for better throughput.

Common Syntax:

CREATE TABLE sales (
    id INT,
    sale_date DATE
) PARTITION BY RANGE (YEAR(sale_date)) (
    PARTITION p0 VALUES LESS THAN (2000),
    PARTITION p1 VALUES LESS THAN (2010),
    PARTITION p2 VALUES LESS THAN MAXVALUE
);

Partition Types:

Range Partitioning: Based on column value ranges.
List Partitioning: Uses discrete column values.
Hash Partitioning: Distributes data based on a hash function.

**Table Indexing and Why Needed

Why Indexing Matters:**

Speed up data retrieval for large datasets.

Optimize query performance, especially for SELECT statements.

Enforce uniqueness of column values.

Common Index Types:

Primary Index: Automatically created for the primary key.
Unique Index: Ensures column values are distinct.
Full-Text Index: Used for text-based searches.

Composite Index: Combines multiple columns for advanced queries.

Creating an Index:

CREATE INDEX idx_name ON students (name);

Best Practices:

Use indexes sparingly to avoid performance overhead during INSERT and UPDATE.

Analyze query patterns to determine the most effective columns to index.

Closing Thoughts

This cheat sheet encapsulates essential MariaDB commands, configurations, and troubleshooting techniques. By combining these practices with real-world scenarios, you’ll be well-equipped to manage MariaDB databases with confidence. Dive into these use cases and bring efficiency to your database administration tasks!

MariaDB is a powerful and open-source #DatabaseManagement system widely used for efficient #DataStorage and #SQL operations. With its compatibility with Docker, #MariaDBDocker offers seamless containerization, making it a favorite in #DevOps workflows. By leveraging #DatabaseOptimization techniques like #TablePartitioning and #Indexing, administrators can achieve better performance and scalability. Understanding the difference between #InnoDB and #MyISAM storage engines is crucial for making informed decisions in #DatabaseAdministration. For those passionate about #LearningSQL and #TechTips, MariaDB serves as an excellent platform to enhance #ITSupport skills and create a robust #OpenSource environment.

Understanding Java Build and Packaging: A Beginner's Guide to DevOps

Tahmid Ul Muntakim — Sat, 09 Nov 2024 14:44:33 +0000

Understanding Java Build and Packaging: A Beginner's Guide to DevOps#

Hey there, future DevOps engineers! 👋 If you're just starting your journey into the world of Java build processes, you're in the right place. Today, we're going to break down everything you need to know about building and packaging Java applications – no previous experience required!

First Things First: What's Java, Anyway?

Before we dive into the build process, let's get our basics straight. Java is like a universal language for computers. When developers write Java code, it gets transformed into something called "bytecode" that can run on any device with a Java Virtual Machine (JVM). Pretty neat, right?

Here's a super simple Java program to get us started:

public class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello, DevOps world!");
    }
}

This little program might look simple, but there's actually quite a bit happening behind the scenes when we want to run it. Let's explore that!

The Build Process: From Code to Running Application

Step 1: Compilation

Remember when I mentioned bytecode? That's what we get when we compile our Java code. Think of it like translating a recipe from English to a universal cooking language that any chef (or in our case, any computer) can understand.

To compile our HelloWorld program, we'd use:

javac HelloWorld.java

This creates a HelloWorld.class file – that's our bytecode!

Step 2: Understanding Dependencies

Real-world applications aren't as simple as our HelloWorld example. They usually need external libraries (we call these dependencies) to work. It's like building a car – you need parts from different manufacturers to make everything work together.

Let's look at a more realistic project structure:

my-java-app/
├── src/
│   └── main/
│       └── java/
│           └── com/
│               └── mycompany/
│                   ├── App.java
│                   └── Utils.java
├── pom.xml
└── target/

Step 3: Enter Maven (Your New Best Friend)

Maven is like your personal assistant for building Java applications. It handles all the tedious stuff like:

Downloading dependencies
Compiling your code
Running tests
Packaging everything together

Here's a basic pom.xml file (Maven's configuration file):

<project>
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.mycompany</groupId>
    <artifactId>my-java-app</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13.2</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

Step 4: Building Your Application

With Maven, building your application is as simple as running:

mvn clean package

This command:

Cleans up any previous builds (clean)
Compiles your code
Runs your tests
Creates a JAR file (Java ARchive) in the target directory

Packaging: Making Your Application Portable

The final step is packaging your application. The most common package types are:

JAR Files

Think of a JAR file as a zip file containing your compiled code and resources. It's perfect for libraries or simple applications.

WAR Files

Web Application aRchive files are used for web applications. They contain everything needed to run your web app, including HTML, CSS, and JavaScript files.

Spring Boot Executable JARs

If you're using Spring Boot (a popular Java framework), you can create a special type of JAR that includes an embedded web server. It's like a complete meal deal – everything you need in one package!

Pro Tips From the Trenches 🛠️

Always Use a Build Tool: Don't try to manage dependencies manually. Maven or Gradle will save you countless hours of work.
Version Everything: Use semantic versioning for your applications (e.g., 1.0.0). It helps track changes and manage deployments.
Keep Your Dependencies Updated: Regular updates help prevent security vulnerabilities and ensure you're using the latest features.
Automate Everything: Learn to use continuous integration tools like Jenkins or GitHub Actions. They can automate your build process and catch issues early.

Let's Put It All Together

Here's a complete example of building and packaging a simple Spring Boot application:

# Clone the project
git clone https://github.com/mycompany/java-app

# Navigate to project directory
cd java-app

# Build the application
mvn clean package

# Run the application
java -jar target/my-java-app-1.0-SNAPSHOT.jar

Wrapping Up

Building and packaging Java applications might seem overwhelming at first, but it's really just a series of logical steps. Start with the basics we covered here, and gradually explore more advanced concepts as you get comfortable.

Remember: everyone started somewhere, and DevOps is all about continuous learning. Don't be afraid to experiment and make mistakes – that's how we all learn!

Have questions? Feel free to drop them in the comments below. Happy building! 🚀

P.S. Want to dive deeper? Check out the official Maven documentation and Spring Boot guides. They're fantastic resources for taking your Java DevOps skills to the next level!

Cilium vs. Calico Comparison

Tahmid Ul Muntakim — Sat, 12 Oct 2024 15:37:37 +0000

In Kubernetes network policy management, Cilium and Calico have emerged as two leading contenders. Both tools offer robust solutions for securing and controlling network traffic within Kubernetes clusters. This blog post will clarify Cilium and Calico, highlighting their key features, advantages, and potential use cases.

Choosing Between Cilium and Calico
The best choice between Cilium and Calico depends on your specific requirements and preferences. Consider the following factors when making your decision:

Performance: If high performance is a critical factor, Cilium's eBPF-based approach may be the better choice.
Complexity: If you prefer a simpler solution, Calico might be easier to manage.
Features: Evaluate the specific features offered by each tool to determine if they align with your needs.
Integration: Consider the level of integration with other tools or technologies in your environment.

Go to this link for more details -
https://www.linkedin.com/pulse/cilium-vs-calico-comparison-tahmid-ul-muntakim-tpqcc/?trackingId=NhBNGODwTZmiypZfuOiCRQ%3D%3D

Both Cilium and Calico are powerful tools for managing network policies in Kubernetes clusters. The choice between them depends on your specific needs and priorities. By evaluating their features, advantages, and potential use cases carefully, you can select the tool that best suits your Kubernetes environment.

Kubernetes #Cilium #Calico #networking

NetworkPolicy #NetworkSecurity #CloudNative #DevOps #Cilium #Calico #eBPF #BGP #KubernetesNetworking #ContainerSecurity #Microservices #DevSecOps #CloudSecurity #ITSecurity

Do You Know /proc at Linux Does Not Contain Real Files?

Tahmid Ul Muntakim — Fri, 06 Sep 2024 17:22:08 +0000

Do You Know /proc at Linux Does Not Contain Real Files?

The proc file system, often referred to as /proc, is a pseudo file system that provides a dynamic interface to the kernel's data structures. Unlike traditional file systems, /proc doesn't contain real files. Instead, it holds virtual files that are created on-the-fly, reflecting the current state of the system. This file system is created upon system boot and disappears when the system shuts down. By navigating /proc, you can gain deep insights into various kernel components and processes, making it a powerful tool for system administrators.

Navigating the `/proc` Directory

Within /proc, you'll find numerous subdirectories and files, each offering specific information about the system. Each process running on the system has its own directory in /proc, identified by its process ID (PID). For example, /proc/[PID]/ contains detailed information about a particular process, including its status, command line arguments, and file descriptors.

Some files within /proc are read-only, providing static details like the operating system version or CPU information. Others are writable, allowing you to configure kernel tunables and settings.

Commonly Used Files and Subdirectories

Here are some essential files and subdirectories within /proc:

/proc/cpuinfo: Provides detailed information about the CPU, including its model, clock speed, and cache size.
/proc/meminfo: Displays details about the system's memory usage, including total, free, and used memory.
/proc/swaps: Shows information about the swap space, including its total, free, and used capacity.
/proc/mounts: Lists the currently mounted file systems.
/proc/self/stat: Provides statistics about the current process, including its process ID, CPU time, memory usage, and more.
/proc/sys/: Contains subdirectories with various kernel tunables that can be modified to fine-tune system behavior.

Configuring Kernel Tunables

To modify kernel tunables, you'll typically write to specific files within the /proc/sys/ hierarchy. For example, to adjust the TCP buffer size, you might write to /proc/sys/net/ipv4/tcp_mem:

echo "4096 65536 16777216" > /proc/sys/net/ipv4/tcp_mem

Persistent vs. Temporary Changes

Changes made directly to /proc files are typically temporary and will be lost upon reboot. To make changes persistent, you should use the sysctl tool and configure settings in /etc/sysctl.conf or create custom configuration files in /etc/sysctl.d/:

echo "net.ipv4.tcp_mem = 4096 65536 16777216" >> /etc/sysctl.conf
sysctl -p  # Apply the changes

Modifying Kernel Tunables Persistently

In this blog, I will talk about modifying kernel tunables persistently. The kernel loads tunable settings from configuration files found in the following directories during boot. The kernel tunable configuration files must end with .conf to be loaded automatically.

When modifying kernel tunables, always create a file inside /etc/sysctl.d. Never modify files inside /usr/lib/sysctl.d, as this directory contains configurations set by the vendor in the package distribution.

Steps to Modify Kernel Tunables:

Find the parameter you want to change using the command:

   sysctl -n [parameter_name]

For example, if the current value of the parameter is 30 and you want to change it, use the sysctl command:

   sysctl -w [parameter_name]=[new_value]

Making the Change Permanent: To make the change permanent, create a configuration file in /etc/sysctl.d/ with the .conf extension. The name of the file doesn't matter, but the extension must be .conf. Inside the file, mention the parameter with its desired value:

   echo "[parameter_name] = [new_value]" > /etc/sysctl.d/my_custom.conf

Applying the Configuration Without Reboot: Use the following command to apply the configuration without rebooting:

   sysctl -p /etc/sysctl.d/my_custom.conf

After executing this command, the parameter value will be updated.

Verifying the Change: Re-run the sysctl -n [parameter_name] command to verify the new value.

Now, when the system reboots, it will automatically load the configuration files from the mentioned locations, ensuring the kernel tunables are set according to your custom configuration.

Security Considerations

While /proc provides valuable information, it also exposes sensitive data, such as process details and system configurations. To enhance security, you can restrict access to /proc for non-privileged users. For example, adding the following option to /etc/fstab will hide process information from non-root users:

proc /proc proc defaults,hidepid=2 0 0

Practical Examples

Here are some useful commands for exploring /proc:

View CPU information:

  cat /proc/cpuinfo

Enable IP forwarding:

  echo 1 > /proc/sys/net/ipv4/ip_forward

Additional Considerations

Static vs. Dynamic Kernel Tunables: Some tunables can be modified at runtime (e.g., TCP settings), while others require a kernel rebuild.
Impact of Changes: Be cautious when modifying kernel tunables. Changes can have unintended consequences, such as system instability or degraded performance. Always test changes in a controlled environment before applying them to production systems.
Persistent Changes: Use sysctl for persistent configurations. Direct modifications to /proc will not survive a reboot.

Conclusion

The proc file system is a versatile tool that offers deep insights into the Linux kernel. By exploring /proc and understanding the available kernel tunables, you can optimize system performance and gain greater control over your environment. However, with great power comes great responsibility—always proceed with caution when making changes, and ensure you understand the potential impact.

Comparing RabbitMQ and Kafka: Choosing the Right Message Broker for Your Application

Tahmid Ul Muntakim — Fri, 26 May 2023 16:49:31 +0000

When it comes to handling large volumes of data and building robust messaging systems, RabbitMQ and Kafka are two popular choices. While both offer high performance, they have different design principles and features that make them suitable for different use cases. In this article, we'll compare RabbitMQ and Kafka, exploring their key differences and providing real-life examples to help you make an informed decision.

Performance and Scalability:
RabbitMQ and ActiveMQ provide high-performance messaging capabilities, but Kafka shines in scenarios that demand high throughput and horizontal scalability. For example, let's consider a real-life use case of a social media platform processing millions of user interactions per second. Kafka's distributed log-based model and built-in scalability features make it an excellent choice for handling such high data volumes efficiently.

Message Prioritization:
One important aspect of message brokers is the ability to prioritize messages. RabbitMQ and ActiveMQ support message prioritization, enabling the processing of higher-priority messages before lower-priority ones. However, Kafka does not offer built-in message priority support. If your application relies heavily on message prioritization, RabbitMQ or ActiveMQ might be a better fit.

Message Ordering:
Ensuring message ordering is crucial in many applications. RabbitMQ and ActiveMQ guarantee message ordering within a single queue or topic, respectively. On the other hand, Kafka ensures ordering within a partition but not across partitions within a topic. This makes Kafka suitable for scenarios where message order within a partition is critical, such as event sourcing or log processing systems.

Message Model:
RabbitMQ follows the Advanced Message Queuing Protocol (AMQP) and utilizes a queue-based message model. Kafka, on the other hand, employs a distributed log-based model, which allows for high-throughput and fault-tolerant data streaming. ActiveMQ is built on the Java Message Service (JMS) standard and also uses a queue-based message model. Depending on your application requirements, you can choose the message model that aligns best with your use case.

Durability and Replication:
All three message brokers support durable messaging to prevent message loss during failures. RabbitMQ and ActiveMQ offer configurable durability options, allowing you to fine-tune the level of persistence based on your needs. Kafka, on the other hand, provides built-in durability through log replication, ensuring data redundancy and fault tolerance. For example, a financial application handling critical transactions may benefit from Kafka's built-in durability features.

*Real-Life Example *-

Microservices Communication:
Consider a microservices architecture where different services communicate with each other. RabbitMQ can be an excellent choice here due to its support for AMQP and its flexible messaging patterns. It enables reliable communication between services, ensuring that messages are not lost even when services experience downtime.

Real-Life Example - Real-Time Data Processing:
Suppose you're building a real-time analytics system that requires processing and analyzing a continuous stream of data. Kafka's distributed and scalable nature makes it a natural fit for such use cases. Its ability to handle high data volumes and provide fault-tolerant data streaming ensures that you can process and analyze the incoming data in real-time with minimal delays.

Choosing the right message broker for your application depends on several factors such as performance requirements, message prioritization, ordering guarantees, and durability needs. RabbitMQ is well-suited for prioritized message handling and guaranteed ordering within a single queue or topic. Kafka shines in scenarios demanding high throughput, horizontal scalability, and fault-tolerant data streaming. Consider your specific use case and requirements to make an informed decision and build a robust messaging system that meets your application's needs.

DEV Community: Tahmid Ul Muntakim

MariaDB Administration Cheat Sheet: Practical Use Cases and Examples

Core Configurations

Basic Commands

Use Case: Recovering from Common Issues

Hands-On: Creating and Managing a Table

Performance Monitoring

Advanced Concepts: Role Management

MariaDB in Docker

Closing Thoughts

Understanding Java Build and Packaging: A Beginner's Guide to DevOps

Understanding Java Build and Packaging: A Beginner's Guide to DevOps#

First Things First: What's Java, Anyway?

The Build Process: From Code to Running Application

Step 1: Compilation

Step 2: Understanding Dependencies

Step 3: Enter Maven (Your New Best Friend)

Step 4: Building Your Application

Packaging: Making Your Application Portable

JAR Files

WAR Files

Spring Boot Executable JARs

Pro Tips From the Trenches 🛠️

Let's Put It All Together

Wrapping Up

Cilium vs. Calico Comparison

Kubernetes #Cilium #Calico #networking

NetworkPolicy #NetworkSecurity #CloudNative #DevOps #Cilium #Calico #eBPF #BGP #KubernetesNetworking #ContainerSecurity #Microservices #DevSecOps #CloudSecurity #ITSecurity

Do You Know /proc at Linux Does Not Contain Real Files?

Do You Know /proc at Linux Does Not Contain Real Files?

Navigating the /proc Directory

Commonly Used Files and Subdirectories

Configuring Kernel Tunables

Persistent vs. Temporary Changes

Modifying Kernel Tunables Persistently

Security Considerations

Practical Examples

Additional Considerations

Conclusion

Comparing RabbitMQ and Kafka: Choosing the Right Message Broker for Your Application

Navigating the `/proc` Directory