DEV Community: Rohit

Committed a Secret to GitHub Repo

Rohit — Thu, 22 Aug 2024 19:05:29 +0000

Oops, I just committed my API key to GitHub—every developer's rite of passage! If you're a newbie who’s just experienced this, welcome to the club! Don't worry, it happens to the best of us, but now it's time to fix that little slip-up before you start polishing your resume. It happened to me when I started working on my first real project called Tapestry Pooling. I had committed a SendGrid API key to a GitHub repo.

Step one: Panic. Just kidding—no panicking! What you actually need to do first is ensure that the secret is useless to anyone who might have seen it. If it’s an API key, delete or regenerate it. If it’s a password, change it faster than you can say, "Oh no!"

Next, it’s time to erase all evidence from the scene of the crime—aka your repo history. The BFG Repo-Cleaner is your new best friend. It’s like a time machine that helps you rewrite history, removing those secrets from your Git history as if they never existed. You can learn how to wield this powerful tool here: BFG Repo-Cleaner.

If your code is on GitHub, there's one more thing you need to do. GitHub is like an elephant—it never forgets, especially when it comes to pull request refs. You’ll need to reach out to GitHub support and politely ask them to scrub those PR refs clean. Be nice; they hold the keys to your repo’s past!

Remember, it's not the mistake that defines you—it's how quickly you can clean up after it ><. And hey, after this, you'll have a great story to share with fellow developers.

Next may be you want to learn how to handler secrets.

Fast Packet IO

Rohit — Mon, 19 Aug 2024 17:35:23 +0000

netmap is a framework for fast packet I/O from userspace. Lets first try to understand why do we need it. OS kernel implements TCP/IP stack protocols up to the transport layer. While the applications layer protocols (HTTP, FPT, SSH, SMTP etc) are implemented in userspace. Per packet dynamic memory allocation, system calls overhead and memory allocation make traditional Linux network stack inefficient. netmap tries to solve this problem make the packet data-path efficient.

Netmap uses these techniques to get it's high performance

a lightweight metadata representation, processing of large number of packets in each system call, thus amortizing its cost;
preallocated, linear, fixed size packet buffers
removal of data-copy costs by granting applications direct, protected access to the packet buffers of packets between interfaces;

Netmap API

netmap has been implemented as kernel module for FreeBSD and Linux.
ioctl(.., NIOCREG, arg)
The argument contains the interface name, and optionally the indication of which rings
we want to control through this file descriptor.

Other fast packet I/O solutions

XDP(express data path) is an high-performance data-path used to send and receive packets by bypassing os kernel networking stack. It uses e-BPF(extended Berkeley Packet Filter), it is an in-kernel virtual machine, ability to run user-supplied program inside kernel. In short, e-BPF allows us to safely extends the functionalities of kernel without changing the kernel source code or loading kernel module

References

chroot system call

Rohit — Sun, 18 Aug 2024 13:28:14 +0000

While doing a literature survey on Serverless architecture also known as Function-as-a-Service(FaaS). It is yet another type of cloud service provided by public cloud service providers such as AWS (lambda), GCP(Google function) and AWS(Azure functions).

I came across chroot system call while reading this paper SOCK: Rapid Task Provisioning with Serverless-Optimized Containers by Edward Oakes et al. It has proposed lightweight isolation as opposed to namespace isolation to solve the problem of coldstart (this is another topic may be will discuss in a seperate blog). A part of solution is to use chroot (latency < 1μs) to provide isolation as instead to namespace isolation (IPC and mount namespace latency > 10ms). At the time though I was aware of chroot syscall but I didn't know how it works under the hood. So, I started looking into the implementation of chroot and it's working.

what is chroot?

chroot changes the apparent root directory for a process and its children. It creates an isolated environment, often used for testing, development, or containing potentially untrusted programs.

#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
    // Change root directory
    if (chroot("/isolate-dir") != 0) {
        perror("chroot");
        exit(EXIT_FAILURE);
    }

    // Change to the new root directory
    if (chdir("/") != 0) {
        perror("chdir");
        exit(EXIT_FAILURE);
    }

    // Execute a new shell
    char *shell = "/bin/bash";
    char *args[] = {shell, NULL};

    execvp(shell, args);

    // If execvp returns, an error occurred
    perror("execvp");
    exit(EXIT_FAILURE);
}

how it works ?

The chroot is implemented in the Linux kernel source code in the file fs/open.c. You can look it on the github linuc repository. Let's dive into it's working

So there is struct fs_struct, a data structure in the Linux kernel that holds the filesystem-related context of a process. This includes information such as the current working directory, the root directory, and the umask. Each process in the kernel has a pointer to an fs_struct, which encapsulates this context. The chroot syscall bsically update the root directory of the process by changing this fs_struct inside task_struct. That's it, it's as simple as that.

struct fs_struct {
    spinlock_t lock;             // Protects the structure
    int users;                   // Reference count
    seqcount_t seq;              // Sequence counter for path walking
    struct path root;            // Root directory
    struct path pwd;             // Current working directory
    umode_t umask;               // File creation mode mask
};

struct task_struct {
    // Process state
    volatile long state;          // Current state of the process (running, sleeping, etc.)
    void *stack;                  // Process kernel stack

    // Scheduling information
    pid_t pid;                    // Process ID
    pid_t tgid;                   // Thread group ID
    struct task_struct *parent;   // Pointer to the parent process
    struct list_head children;    // List of child processes
    struct list_head sibling;     // List of sibling processes
    struct list_head tasks;       // List of all processes

    // Memory management
    struct mm_struct *mm;         // Address space of this process
    struct mm_struct *active_mm;  // Active address space

    // File system
    struct fs_struct *fs;         // Filesystem information
    struct files_struct *files;   // Open file descriptors

    // Signal handling
    struct signal_struct *signal; // Shared signal handlers
    struct sighand_struct *sighand;// Signal handlers

    // Timers
    struct list_head cpu_timers[3]; // CPU-specific timers

    // Debugging
    unsigned int flags;           // Various flags (e.g., for debugging)
    unsigned int ptrace;          // Ptrace flags

    // More fields...
};