DEV Community: Saurav Shah

RCU (Read, Copy, Update) in the Linux Kernel

Saurav Shah — Wed, 24 Dec 2025 18:24:10 +0000

RCU is a mechanism to synchronize between readers and writers and it is very efficient when there are many readers or the data to read is large. It is a lock free synchronization mechanism.

Topics Covered

Why do we need a synchronization mechanism?
Mutex to synchronize
Reader‑mostly workloads
Introducing RCU
RCU with Linked List
A little about how RCU works
General guidelines to using RCU

Why do we need a synchronization mechanism?

Let's consider this simple example:

#include <linux/module.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/debugfs.h>
#include <linux/mm.h>


#define MODULE_NAME "kernel_rcu_example"
#define FILENAME    "val"

struct data {
    int a;
    int b;
};

struct data *ptr = NULL;
static struct dentry *gparent = NULL;

void update_data(struct data *old, int a, int b, u32 ms)
{
    if(!old)
        return;
    old->a = a;
    mdelay(ms);
    old->b = b;
    pr_info(MODULE_NAME ": Data updated\n");
}

ssize_t dbg_write(struct file *file, const char __user *buffer, size_t count, loff_t *off)
{
    char kbuf[100];
    int a, b;
    u32 ms;
    (void)off; //Ignore offset for simplification
    if(count >= sizeof(kbuf))
        return -EINVAL;

    if (copy_from_user(kbuf, buffer, count))
        return -EFAULT;

    kbuf[count] = '\0';  // Null-terminate
    if(sscanf(kbuf, "%d:%d:%u", &a, &b, &ms) != 3)
        return -EINVAL;
    update_data(ptr, a, b, ms);
    return count;
}

ssize_t dbg_read(struct file *file, char __user *buffer, size_t count, loff_t *off)
{
    char kbuf[100];
    size_t len;

    if (*off > 0)
        return 0;

    if(!ptr)
        return -EINVAL;
    if(sizeof(kbuf) >= count)
        return -EINVAL;
    len = scnprintf(kbuf, sizeof(kbuf), "a=%d,b=%d\n", ptr->a, ptr->b);
    if(len > count)
        return -EINVAL;
    if (copy_to_user(buffer, kbuf, len))
        return -EFAULT;
    *off += len;
    return len;
}

static const struct file_operations dbg_fops = {
    .write = dbg_write,
    .read = dbg_read,
};

static int debug_init(void)
{
    if (!IS_ENABLED(CONFIG_DEBUG_FS)) {
        pr_warn(MODULE_NAME": debugfs unsupported! Aborting ...\n");
        return -EINVAL;
    }
    gparent = debugfs_create_dir(MODULE_NAME, NULL);
    if (!gparent) {
        pr_err(MODULE_NAME": debugfs_create_dir failed, aborting...\n");
        return -EINVAL;
    }

    if(!debugfs_create_file(FILENAME, 0660, gparent, NULL, &dbg_fops))
    {
        pr_err(MODULE_NAME": debugfs_create_file failed, aborting...\n");
        goto out_fail;
    }
    return 0;
out_fail:
    debugfs_remove_recursive(gparent);
    return -EINVAL;
}

static int __init kernelrcu_init_module(void)
{
    pr_info(MODULE_NAME ": %s\n",__func__);
    debug_init();
    ptr = (struct data*)kzalloc(sizeof(struct data), GFP_KERNEL);
    update_data(ptr, 1, 2, 0);
    return 0;
}

static void __exit kernelrcu_cleanup_module(void)
{
    pr_info(MODULE_NAME ": %s\n",__func__);
    if(ptr)
        kfree(ptr);
    if(gparent)
        debugfs_remove_recursive(gparent);
}
module_init(kernelrcu_init_module);
module_exit(kernelrcu_cleanup_module);
MODULE_LICENSE("GPL");

The code example has a pointer ptr of type struct data * which has two members a and b. update_data updates this structure and it takes an argument ms, which is the time in milliseconds to sleep before updating the field b. We also create a debugfs entry /sys/kernel/debug/kernel_rcu_example/val which can be read to get the current value of ptr. You can also write to this file in the format a:b:ms to update the value and sleep ms before updating field b.

Now let's deploy this module.

$ insmod kernel_rcu_example.ko
$ dmesg
[1398227.954799] kernel_rcu_example: kernelrcu_init_module
[1398227.954804] kernel_rcu_example: Data updated

Let's print the value of ptr.

$ cat /sys/kernel/debug/kernel_rcu_example/val
a=1,b=2

Let's update the value now. Add a sleep of 5 seconds before updating the field b.

$ echo "5:6:5000" > /sys/kernel/debug/kernel_rcu_example/val

Before this completes, open another terminal and print the value.

$ cat /sys/kernel/debug/kernel_rcu_example/val
a=5,b=2

You see! The value of ptr is not updated completely, only a is updated. This is a typical race condition. We have used mdelay here to simulate the behaviour, but this usually occurs when two threads update and read the same data simultaneously. Since the update operation is not atomic, the writer thread might be preempted before the update completes, and the reader thread sees a partial update.

Mutex to synchronize

The classical approach to solve this issue is to use a mutex. Take the mutex before updating the value and release it once update is done. Similarly, take it before reading the value.

DEFINE_MUTEX(lock);
...
void update_data(struct data *old, int a, int b, u32 ms)
{
    if(!old)
        return;
    mutex_lock(&lock);
    old->a = a;
    mdelay(ms);
    old->b = b;
    mutex_unlock(&lock);
    pr_info(MODULE_NAME ": Data updated\n");
}
...
ssize_t dbg_read(struct file *file, char __user *buffer, size_t count, loff_t *off)
{
    ...
    mutex_lock(&lock);
    len = scnprintf(kbuf, sizeof(kbuf), "a=%d,b=%d\n", ptr->a, ptr->b);
    mutex_unlock(&lock);
    ...
}

Now, if you compile and deploy this module and run the same experiment, you will see that in the second terminal, while you try to read val, it is blocked (for 5 seconds) until the update is done and the proper value is displayed.

$ cat /sys/kernel/debug/kernel_rcu_example/val #Blocks for sometime
a=5,b=2

Reader‑mostly workloads

Consider this: you have readers in the order of 100s or 1000s but few writers, in the order of 1s or 10s. If you use a mutex (or spinlock), the reader needs to wait until the other reader is done. Similarly, the writer may need to wait for a long time before it gets the mutex to update the data. This is very inefficient and not scalable for a large number of readers. What if we allow multiple readers to read the data simultaneously, while still ensuring the data read is consistent (not a partial update)? This is where reader–writer locks or RCU come into the picture. They allow multiple readers to read simultaneously. I will not go into reader–writer locks in this post, but the general idea is: readers take the read lock (multiple readers can take it) which can only be taken if the writer lock is not taken (no writer), and the writer takes the writer lock, which can only be taken when the reader lock count is 0 (no readers).

Introducing RCU

RCU allows multiple readers to read simultaneously, and a single writer can update while readers are reading. The writer need not wait for the readers to finish reading. If a writer tries to write while readers are reading, the writer writes the value to a new location, while the old location is still valid. Once all the readers are done reading, the old location is discarded and the new location is kept.

The idea is, when a writer want to write a data:

Read the current data (RCU)
Copy the current data to a new location (R*C*U)
Update data in the new location (RC*U*)
Publish the new data (update the current location to point to the new location)
Defer the freeing of the old location until all the reader are done

Let's modify the code to use RCU technique.

#include <linux/module.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/debugfs.h>
#include <linux/mm.h>


#define MODULE_NAME "kernel_rcu_example"
#define FILENAME    "val"

struct data {
    int a;
    int b;
};

struct data __rcu *ptr = NULL;
static struct dentry *gparent = NULL;

DEFINE_MUTEX(writer_lock);

void update_data(int a, int b, u32 ms)
{
    struct data *newloc = NULL;
    struct data *curr = NULL;
    if(!ptr)
        return;

    //Create a new location to store the updated data
    newloc = (struct data*)kzalloc(sizeof(struct data), GFP_KERNEL);
    if(!newloc)
        return;


    mutex_lock(&writer_lock);
    //Read the current data
    curr = rcu_dereference(ptr);
    //Copy the data to the new location
    memcpy(newloc, curr, sizeof(struct data));

    //Update the data in the new location
    newloc->a = a;
    mdelay(ms);
    newloc->b = b;

    //Publish the new data
    rcu_assign_pointer(ptr, newloc);
    mutex_unlock(&writer_lock);

    //Free the previous location once all the readers are done
    synchronize_rcu();
    kfree(curr);
    pr_info(MODULE_NAME ": Data updated\n");
}

ssize_t dbg_write(struct file *file, const char __user *buffer, size_t count, loff_t *off)
{
    char kbuf[100];
    int a, b;
    u32 ms;
    (void)off; //Ignore offset for simplification
    if(count >= sizeof(kbuf))
        return -EINVAL;

    if (copy_from_user(kbuf, buffer, count))
        return -EFAULT;

    kbuf[count] = '\0';  // Null-terminate
    if(sscanf(kbuf, "%d:%d:%u", &a, &b, &ms) != 3)
        return -EINVAL;
    update_data(a, b, ms);
    return count;
}

ssize_t dbg_read(struct file *file, char __user *buffer, size_t count, loff_t *off)
{
    char kbuf[100];
    size_t len;
    struct data *curr = NULL;

    if (*off > 0)
        return 0;

    if(!ptr)
        return -EINVAL;
    if(sizeof(kbuf) >= count)
        return -EINVAL;

    rcu_read_lock();
    curr = rcu_dereference(ptr);
    len = scnprintf(kbuf, sizeof(kbuf), "a=%d,b=%d\n", curr->a, curr->b);
    rcu_read_unlock();
    if(len > count)
        return -EINVAL;
    if (copy_to_user(buffer, kbuf, len))
        return -EFAULT;
    *off += len;
    return len;
}

static const struct file_operations dbg_fops = {
    .write = dbg_write,
    .read = dbg_read,
};

static int debug_init(void)
{
    if (!IS_ENABLED(CONFIG_DEBUG_FS)) {
        pr_warn(MODULE_NAME": debugfs unsupported! Aborting ...\n");
        return -EINVAL;
    }
    gparent = debugfs_create_dir(MODULE_NAME, NULL);
    if (!gparent) {
        pr_err(MODULE_NAME": debugfs_create_dir failed, aborting...\n");
        return -EINVAL;
    }

    if(!debugfs_create_file(FILENAME, 0660, gparent, NULL, &dbg_fops))
    {
        pr_err(MODULE_NAME": debugfs_create_file failed, aborting...\n");
        goto out_fail;
    }
    return 0;
out_fail:
    debugfs_remove_recursive(gparent);
    return -EINVAL;
}

static int __init kernelrcu_init_module(void)
{
    pr_info(MODULE_NAME ": %s\n",__func__);
    debug_init();
    ptr = (struct data*)kzalloc(sizeof(struct data), GFP_KERNEL);
    update_data(1, 2, 0);
    return 0;
}

static void __exit kernelrcu_cleanup_module(void)
{
    pr_info(MODULE_NAME ": %s\n",__func__);
    if(ptr)
        kfree(ptr);
    if(gparent)
        debugfs_remove_recursive(gparent);
}
module_init(kernelrcu_init_module);
module_exit(kernelrcu_cleanup_module);
MODULE_LICENSE("GPL");

Now if you deploy this module and run the previous experiment, you will see the old data until the new data is published and becomes visible to readers.

$ echo "5:6:5000" > /sys/kernel/debug/kernel_rcu_example/val
# Blocks
# in another terminal
$ cat val
a=1,b=2
$ cat val
a=1,b=2
$ cat val
a=1,b=2
$ cat val
a=1,b=2
$ cat val  #Finally the updated value
a=5,b=6

First we declared ptr as RCU-protected, struct data __rcu *ptr. Whenever we want to dereference the RCU pointer, we use rcu_dereference(ptr); this ensures memory ordering and correctness under concurrency. Similarly, we use rcu_assign_pointer(old, new) to assign a new location to the existing pointer.

Let's look at the readers’ logic in dbg_read. We use rcu_read_lock() before reading and rcu_read_unlock() to indicate that the reader is done. On the writer side, in update_data:

newloc is allocated a new location.
A lock, writer_lock, is used to synchronize writers. Note how writer_lock is different from RCU and not implicitly embedded in the RCU paradigm. If you are sure that the number of writers is small and the write time is deterministic, it's often better to use a spinlock rather than a mutex.
The writer takes writer_lock, dereferences the current pointer using rcu_dereference, and copies the data to the new location using memcpy.
The writer then updates the data in the new location. Note that the pointer ptr is still pointing to the old data, and the writer has only modified the new location.
The writer then assigns the new location to the existing ptr using rcu_assign_pointer. The old pointer is still valid; if any reader is still referring to that pointer, it will still see the old data.
The writer uses synchronize_rcu() to wait for all pre-existing readers to finish. Once the readers are done, this call returns.
The writer can then safely free the memory pointed to by the old pointer. Any new reader coming in at this point will be referring to the new pointer.

synchronize_rcu is a blocking call and it blocks untill all the readers are done. You might not want to block just to free the old ptr. You can use call_rcu, which is async API.

RCU with Linked List

There are several kernel data structures designed with built-in support for RCU (Read-Copy-Update) to allow safe concurrent reads without locking.

In this example, we use kernel linked list to use of rcu. If you are not familiar with the kernel's linked list implementation, have a look at Linked List in the Linux Kernel.

#include <linux/module.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/debugfs.h>
#include <linux/mm.h>
#include <linux/spinlock.h>


#define MODULE_NAME "kernel_rcu_ll_example"

struct data {
    int seqno;
    int val;
    struct list_head node;
    struct rcu_head rcu;
};

static LIST_HEAD(data_head);
static spinlock_t writer_lock;

static void free_cb(struct rcu_head *rcu) {
    struct data *node = container_of(rcu, struct data, rcu);
    kfree(node);
}

static int add_data(int val) {
    struct data *ptr;

    ptr = kzalloc(sizeof(struct data), GFP_KERNEL);
    if(!ptr)
        return -ENOMEM;

    ptr->val = val;

    spin_lock(&writer_lock);
    if(list_empty(&data_head))
        ptr->seqno = 1;
    else
        ptr->seqno = container_of(data_head.prev, struct data, node)->seqno + 1; //tail seqno + 1
    list_add_tail_rcu(&ptr->node, &data_head);
    spin_unlock(&writer_lock);
    return 0;
}

static int update_node(int seqno, int val) {
    struct data *curr = NULL;
    struct data *old = NULL;
    struct data *new = NULL;
    rcu_read_lock();
    list_for_each_entry_rcu(curr, &data_head, node) {
        if(curr->seqno == seqno){
            old = curr;
            break;
        }
    }
    if(!old) {
        rcu_read_unlock();
        return -EINVAL;
    }
    new = kzalloc(sizeof(struct data), GFP_ATOMIC);
    if(!new) 
    {
        rcu_read_unlock();
        return -ENOMEM;
    }
    memcpy(new, old, sizeof(struct data));
    new->val = val;

    spin_lock(&writer_lock);
    list_replace_rcu(&old->node, &new->node);
    spin_unlock(&writer_lock);
    rcu_read_unlock();

    //Async callback to free the old node
    call_rcu(&old->rcu, free_cb);
    return 0;
}

static void delete_node(int seqno) {
    struct data *curr = NULL;
    spin_lock(&writer_lock);
    list_for_each_entry_rcu(curr, &data_head, node) {
        if(curr->seqno == seqno){
            list_del_rcu(&curr->node);
            spin_unlock(&writer_lock);
            call_rcu(&curr->rcu, free_cb);
            return;
        }
    }
    spin_unlock(&writer_lock);
    return;
}

static void print_ll(void)
{
    struct data *curr = NULL;
    rcu_read_lock();
    list_for_each_entry_rcu(curr, &data_head, node) {
        pr_info("\tseqno:%d , val:%d\n", curr->seqno, curr->val);
    }
    rcu_read_unlock();
}


static int __init kernelrcull_init_module(void)
{
    pr_info(MODULE_NAME ": %s\n",__func__);
    add_data(23);
    add_data(24);
    add_data(25);
    pr_info(MODULE_NAME ": After adding 23, 24, 25 :\n");
    print_ll();
    update_node(3, 26);
    pr_info(MODULE_NAME ": After updating 25 to 26 :\n");
    print_ll();
    delete_node(2);
    pr_info(MODULE_NAME ": After removing 24 :\n");
    print_ll();
    return -EINVAL; //Don't deploy module, only for demo
}

module_init(kernelrcull_init_module);
MODULE_LICENSE("GPL");

This is the dmesg output

[1426406.079680] kernel_rcu_ll_example: kernelrcull_init_module
[1426406.079682] kernel_rcu_ll_example: After adding 23, 24, 25 :
[1426406.079683]        seqno:1 , val:23
[1426406.079683]        seqno:2 , val:24
[1426406.079683]        seqno:3 , val:25
[1426406.079684] kernel_rcu_ll_example: After updating 25 to 26 :
[1426406.079684]        seqno:1 , val:23
[1426406.079684]        seqno:2 , val:24
[1426406.079685]        seqno:3 , val:26
[1426406.079685] kernel_rcu_ll_example: After removing 24 :
[1426406.079685]        seqno:1 , val:23
[1426406.079686]        seqno:3 , val:26

This example is only to demonstrate how to use rcu with linked list. I have called the reader and writer call from the same context (which would ideally not require any synchronization mechanism at all), but the same will work no matter how many concerent readers or writers are there.

Let's have a look at the code example.
First, notice we have a struct rcu_head rcu field in the struct data. I will explain the use of this in sometime. writer_lock is a spin locked used to synchronize the writers.
Let's look at the add_data function. Given a val, it adds a node to the tail of the linked list. the seqno is basically seqno of the tail + 1. Note we need to take the writer_lock before calculating the sequence number, as well as adding the node the linked list. list_add_tail_rcu is used to add the rcu node to the tail of the linked list.
Now, let's look at the update_node function. We take rcu_read_lock before iterating through the list to search for the node to update (based on seqno). If the node is not found, we unlock the rcu reader's lock and return. If node is found, we create a new pointer new and copy the data from the old pointer. We then take the writer_lock and replace the current node with the update node using list_replace_rcu.
I have use a new API call_rcu here. In the previous example, we used synchronize_rcu to wait for the readers to finish, before we free the old ptr. But this would block the caller. We dont want that. call_rcu registers an async callback that will be called later when all the readers are done and doesn't block the caller. Remember the struct rcu_head rcu we added, it is for this usecase. call_rcu takes in pointer to the struct rcu_head reference (which can be used to get the actual container using container_of macro) and the callback function to call. free_cb is called when the readers are done.

delete_node is also similar. Take the writer_lock, iterate through the list and when the node is found use list_del_rcu to remove from the list.

One thing to node is, once you take rcu_read_lock, you shouldn't be calling any blocking function inside the critical section. Notice how I used GPF_ATOMIC in new = kzalloc(sizeof(struct data), GFP_ATOMIC); of the update_node function as GFP_KENRL might block (voluntary preemption).

A little about how RCU works

Now that we know what rcu are, how to use them the the best practices, let use dive a little dep into the implementation of RCU.
You might have noticed there is no reference to a traditional “lock” when calling rcu_read_lock() and rcu_read_unlock(). So what does it lock exactly?
It doesn’t acquire a spinlock or mutex. Instead, it marks the beginning of an RCU read-side critical section. The purpose is to prevent the current CPU from being preempted in a way that would break RCU’s guarantee.

On a non-preemptible kernel, rcu_read_lock() and rcu_read_unlock() generate almost no code! Why? Because if the kernel is non-preemptible and we know we cannot call any blocking function inside an RCU read-side section, the current thread will not be preempted until the section completes. If another reader runs on another CPU, the same applies - no preemption until the read is done
On a preemptible kernel, rcu_read_lock() disables preemption or increments a per-CPU counter so the kernel knows this thread is inside an RCU read-side section. This allows the RCU subsystem to track active readers.

When you call call_rcu() or synchronize_rcu(), the kernel waits for a grace period - the time it takes for all CPUs to exit their RCU read-side critical sections. In other words, the per-CPU counters for all CPUs must drop to zero before old data can be safely freed.

General guidelines to using RCU

Use RCU when there are readers in the order of hundreds or thousands compared to writers. RCU shines in read-mostly workloads because readers are extremely fast and non-blocking.
Never use RCU for frequent updates. Writers pay the cost of waiting for grace periods, so if updates dominate, traditional locking may be better.
Readers must not block. Inside an rcu_read_lock() section
- Do not sleep, call schedule(), or perform blocking I/O.
- Keep the critical section short
Always pair rcu_read_lock() with rcu_read_unlock(). Nesting is allowed but must be balanced.
Use proper update primitives
- list_add_rcu(), list_del_rcu(), list_replace_rcu() for linked lists.
- Use call_rcu() or kfree_rcu() to free old data after a grace period
Synchronize writers correctly
- Use synchronize_rcu() if you need to wait until all readers finish before proceeding.
- Or use call_rcu() for deferred freeing without blocking the writer.
Protect writers with locks if needed. RCU does not serialize writers; you still need spinlocks or mutexes for concurrent updates.
Check context: RCU read-side sections are safe in interrupt context, but updates often require sleeping, so they must run in process context.
Avoid misuse: RCU is not a general-purpose lock replacement. It’s for scenarios where
- Reads are frequent and must scale.
- Updates are rare and can tolerate deferred cleanup.

Cheers!!

Linked List in the Linux Kernel

Saurav Shah — Fri, 19 Dec 2025 10:50:03 +0000

The traditional way of implementing a linked list is to add a pointer to the next node (and a previous node in the case of a doubly linked list). The Linux kernel’s implementation is somewhat different. This post explores the implementation of the doubly linked list in the Linux kernel. 🧵

Topics Covered

Example Code
container_of Macro
struct list_head type
Adding / Deleting a node
Traversing a node
List APIs

Example Code

Let’s start with an example. This example creates a circular doubly linked list and shows how to add/delete nodes and loop through the entries.

#include <linux/module.h>
#include <linux/list.h>

#define MODULE_NAME "KERNEL_LL "

struct data {
    u32 count;
    char *name;
    struct list_head node;
};

static LIST_HEAD(data_head);

static int __init kernelLL_init_module(void)
{
    struct data v1 = {
        .count = 1,
        .name = "data1"
    };
    struct data v2 = {
        .count = 2,
        .name = "data2"
    };
    struct data v3 = {
        .count = 3,
        .name = "data3"
    };
    struct data *curr;
    pr_info(MODULE_NAME"init\n");

    list_add_tail(&v1.node, &data_head);
    list_add(&v2.node, &data_head);
    list_add(&v3.node, &data_head);

    pr_info(MODULE_NAME"After Adding:\n");
    list_for_each_entry(curr, &data_head, node)
        pr_info(MODULE_NAME"%d: %s\n", 
                    curr->count, curr->name);

    pr_info(MODULE_NAME"Invalid access:\n");
    list_for_each_entry(curr, &v2.node, node)
        pr_info(MODULE_NAME"%d: %s\n", 
                    curr->count, curr->name);

    list_del(&v2.node);
    pr_info(MODULE_NAME"After Deleting:\n");
    list_for_each_entry(curr, &data_head, node)
        pr_info(MODULE_NAME"%d: %s\n", 
                    curr->count, curr->name);

    return -EINVAL; //Don't load the module
}
module_init(kernelLL_init_module);
MODULE_LICENSE("GPL");

This is the output

$ dmesg
[962638.228544] KERNEL_LL init
[962638.228545] KERNEL_LL After Adding:
[962638.228545] KERNEL_LL 3: data3
[962638.228546] KERNEL_LL 2: data2
[962638.228546] KERNEL_LL 1: data1
[962638.228546] KERNEL_LL After Deleting:
[962638.228546] KERNEL_LL 3: data3
[962638.228546] KERNEL_LL 3: data1

struct data is just like any custom container with an additional field named node of type struct list_head. Isn’t that sleek? 😄 You can add a member of type struct list_head in any container to make it a list node.

container_of Macro

Before that, let me talk about container_of(ptr, type, member) macro in the kernel.

struct data *pval = (struct data*) kzalloc(sizeof(struct data), GFP_KERNEL);
struct data *pcontainer = container_of(&pval->count, struct data, count);
//pval and pcontainer points to same address

Say you have a pointer to a member in a container (a struct), and you want to get the address of the container; you can use container_of to get the container’s address. The kernel’s linked list uses this container_of pattern heavily. 💡

struct list_head type

LIST_HEAD(name) declares and initializes a list head of type struct list_head. Any node you add to the list must have a member of type struct list_head, and that member is what actually links the nodes together (via prev/next). The container structs (struct data in our case) are never linked directly; only their struct list_head members are.

Adding / Deleting a node

You can use list_add(new, head) to add the node to the front of the linked list. Use list_add_tail(new, head) to add the node to the end. Note how you use list_add(&v1.node, &data_head) and not list_add(&v1, &data_head), essentially adding the struct list_head and not the struct data type.

You start with an empty list.

static LIST_HEAD(data_head);

Add a node to the tail of the list

list_add_tail(&v1.node, &data_head);

Add a node to the head of the list

list_add(&v2.node, &data_head);

Delete a node

list_del(&v2.node);

Traversing a node

You can use the macros list_for_each(pos, head) and list_for_each_entry(pos, head, member) to traverse the list.

list_for_each loops through the list starting from head->next and gives you a struct list_head *pos pointing to the node member.
list_for_each_entry does essentially the same traversal, but additionally uses container_of to give you the actual container (struct data * in this case) at each position.

Note that traversing a list doesn’t do any safety checks. It will start from head->next, follow next pointers, and stop when it reaches head again in this circular list implementation.

Say you have a linked list 3 <-> 2 <-> 1 (head is data_head) and you run this code:

list_for_each_entry(curr, &v2.node, node)
    pr_info(MODULE_NAME"%d: %s\n", 
                curr->count, curr->name);

This is the output.

[966562.045245] KERNEL_LL 3: data1
[966562.045245] KERNEL_LL 0: (null)
[966562.045245] KERNEL_LL 3: data3

You see! We gave v2.node as the “head”. So while traversing, it starts with head->next, which is 1, so it prints 1. Next iteration is 1->next, which is the real head (data_head). When you call container_of on the real head (which is just a struct list_head, not embedded in any struct), it doesn’t have a valid container, so members like count and name are garbage/uninitialized. That’s what we see as 0 and null. Finally it continues from there and prints 3 again.

So:

list_for_each_entry assumes the head argument is a proper list head (not a list node embedded inside a struct).
It is a good practice to name the head in a way that is distinct from the node members and to use that head consistently. In our example, we should always use data_head as the head. ✅

Start using the Linux kernel linked list now! 🚀

Please note that it’s always a good practice to use arrays wherever possible when your data is naturally contiguous, since contiguous data is faster to access and more cache–friendly. Linked lists are great when you need frequent insertions/removals in the middle of the sequence, or when you don’t know the size in advance.

List APIs

These are some of the APIs you can use to manipulate/access linked lists in the kernel. Please refer to the kernel documentation for more detailed information. 📚

LIST_HEAD_INIT(name) : Initialize the list during initial assignment
INIT_LIST_HEAD(*list) : Initialize the list that already exists
list_add(*new, *head) : Add a node to the head of the list
list_add_tail(*new, *head) : Add a node to the tail of the list
list_for_each(pos, head) : Traver through the list, pos point to the current struct list_head while traversing
list_entry(ptr, type, member) : Get the cotnainer for the node pointed by ptr of type struct list_head
list_for_each_entry(pos, head, member) : Iterate over the list to get the container of current position
list_del(*entry) : Remove the entry from the list. The node removed will have the list pointers set to poision value.
list_del_init(*entry) : Remove the entry from the list and reinitialize the removed node (to point to itself)
list_for_each_safe(pos, head) / list_for_each_entry_safe(pos, head, member) : Same as 5 and 6 with additional logic to handle the case where you delete the node while traversing the list
list_cut_position(*list, *head, *entry) : Removes all the entries from head till (including) entry and put it in the list
list_cut_before(*list, *head, *entry): Removes all the entries from head till (exclusing) entry and put it in the list
list_move(*list, *head): Removes one list entry head and adds it to the front (head) of another list list
list_move_tail(*list, *head) : Removes one list entry head and adds it to the back (tail) of another list list
list_bulk_move_tail(*head, *first, *last): Moves a bulk entry starting from first till last to the tail of another list head
list_rotate_left(*head): Rotates the circular list counter-clockwise (left) by one entry
list_rotate_to_front(*list, *head): Rotates the circular list counter-clockwise (left) until head comes to the front
list_swap(*entry1, *entry2): Swaps the nodes pointed by entry1 and entry2
list_splice(*list, *head): Takes the list pointed by list and puts it just after the node pointed by head. Note the list pointed by list will still point to the next node.
list_splice_init(*list, *head): Same as 18, nut reinitializes the list poited by list (it will point to itself).

There are more APIs; you can refer to the Kernel Documentation for the full list and detailed explanations. 🔍

Cheers!!

Product idea (Feedback Required): hubhandle.com

Saurav Shah — Tue, 25 Jun 2024 19:05:32 +0000

Hey Folks,

In my previous post, I shared about nightsweekends s5.

This is for you, let's build some cool stuff! 🚀✨

Saurav Shah ・ Jun 11 '24

#codenewbie #webdev #discuss #learning

This post is about what I am building. I would love to hear from you guys and appreciate your feedback.

hubhandle.com is an easy-to-learn and simple-to-use tool to manage all your social media needs.
The initial version of this tool will allow to publish a post across four social media platform: Instagram, Facebook, LinkedIn, X. Over time, I plan to add more features to make hubhandle.com a one-stop solution to for all your social media needs.

HubXStream will help you schedule all your posts across social media platforms.

HubXSecure will help you collaborate with other team members without sharing passwords.

HubXIntellect will help you create AI-generated posts tailored to your audience in a single click.

HubXAnalytica will provide real-time analytics of your posts and help you develop strategies.

HubXAutomate will help you automate your social media tasks with ease.

There are tools already available for these tasks, but they can be a little difficult to use. I want to make a tool that is simple to use so you can become an expert on day one.

Please share your thoughts and feedback. It will help me create a better product.

If any of you have also joined Nights and Weekends Season 5, comment on what you are building. Let's build some cool stuff together!

Cheers🎉 ~ sauravshah31

Lock / Mutex to a post graduate CS student (Difficulty 4)

Saurav Shah — Thu, 20 Jun 2024 02:00:31 +0000

This is a submission for DEV Computer Science Challenge v24.06.12: One Byte Explainer.

Explainer

A mutex uses architecture-specific atomic instructions (uninterruptible) until successful, allowing thread access to a critical section. For non-atomic architectures, complex logic involving disabling interrupts and continuously checking flags is used.

Additional Context

I am planning to post 5 submissions explaining "Lock/Mutex" at 5 levels of difficulty. This is Difficulty 4.

A post graduate Computer Science student or an experienced software engineer might have used mutex multiple times. Adding a context on how mutex is implemented to achieve mutual exclusion is something people might find interesting.

For more about explaining the term in 5 Levels of difficulty, refer to the below post. It's interesting!

Computer Science challenge, let's make it interesting!!

Saurav Shah ・ Jun 13 '24

#computerscience #discuss #softwareengineering #learning

Previous explanation for Difficulty 3

Cheers🎉 ~ sauravshah31

Lock / Mutex to a software engineer (Difficulty 3)

Saurav Shah — Tue, 18 Jun 2024 19:44:32 +0000

This is a submission for DEV Computer Science Challenge v24.06.12: One Byte Explainer.

Explainer

A mutex blocks access to a critical section until the current thread is done, preventing race conditions but potentially causing performance hits. The GIL in CPython restricts access to shared resources to one thread at a time, impacting multi-threading.

Additional Context

I am planning to post 5 submissions explaining "Lock/Mutex" at 5 levels of difficulty. This is Difficulty 3.

A Computer Science graduate or a software engineer might have heard and used mutex. Mutex explanation and some interesting facts is fun.

For more about explaining the term in 5 Levels of difficulty, refer to the below post. It's interesting!

Computer Science challenge, let's make it interesting!!

Saurav Shah ・ Jun 13 '24

#computerscience #discuss #softwareengineering #learning

Previous explanation for Difficulty 2
Next explanation for Difficulty 4

Cheers🎉 ~ sauravshah31

Lock / Mutex to a CS undergraduate (Difficulty 2)

Saurav Shah — Tue, 18 Jun 2024 13:04:25 +0000

This is a submission for DEV Computer Science Challenge v24.06.12: One Byte Explainer.

Explainer

Locks prevent issues when multiple entities access a resource simultaneously (like multiple apps writing to a file), which can cause unexpected behavior (like overwritten writes). Google Docs uses it to avoid overwritten edits during collaboration.

Additional Context

I am planning to post 5 submissions explaining "Lock/Mutex" at 5 levels of difficulty. This is Difficulty 2.

A Computer Science first year student, probably have not heard of mutex, but might have heard about some problems involving race condition. An explanation that explains what mutex is along with its usage and example is useful.

For more about explaining the term in 5 Levels of difficulty, refer to the below post. It's interesting!

Computer Science challenge, let's make it interesting!!

Saurav Shah ・ Jun 13 '24

#computerscience #discuss #softwareengineering #learning

Previous explanation for Difficulty 1
Next explanation for Difficulty 3

Cheers🎉 ~ sauravshah31

Lock / Mutex to an 8th grader (Difficulty 1)

Saurav Shah — Sun, 16 Jun 2024 11:25:04 +0000

This is a submission for DEV Computer Science Challenge v24.06.12: One Byte Explainer.

Explainer

Imagine you're in an exam hall where only one student can go to the restroom at a time using a pass. When they return, the pass goes to the next student. Similarly, computers use locks to let only one app access a resource at a time while others wait.

Additional Context

I am planning to post 5 submissions explaining "Lock/Mutex" at 5 levels of difficulty. This is Difficulty 1.

An 8th grader probably hasn't heard of the term "mutex." An explanation that relates to something they have seen and explains how computers use a similar concept can be helpful.

For more about explaining the term in 5 Levels of difficulty, refer to the below post. It's interesting!

Computer Science challenge, let's make it interesting!!

Saurav Shah ・ Jun 13 '24

#computerscience #discuss #softwareengineering #learning

Next explanation for Difficulty 2

Cheers🎉 ~ sauravshah31

Computer Science challenge, let's make it interesting!!

Saurav Shah — Thu, 13 Jun 2024 19:11:32 +0000

If you haven’t checked it out yet, dev.to has introduced First Computer Science Challenge in honor of Alan Turing. The challenge is to explain a computer science concept in 256 characters or less.

Let's make it even more exciting!

It's really interesting to look at these videos where an expert explains a concept at five levels of difficulty.

Let's do the same thing in this challenge too. As per the challenge rules, it is allowed to "submit multiple submissions per prompt but you’ll need to publish a separate post for each submission".

Why not submit 5 submissions, for explanations at 5 levels of difficulty? It's fun. And to make it more interesting, let's pitch our explanations to people with varying levels of expertise and see how they interpret and re-explain our explanations. We'll then submit their interpretations in 256 characters or less.

To start with, I explained how to use Lock in Google App Script. Here’s my attempt to explain "Lock / Mutex" at five different levels of difficulty:

Level 1:

My Explanation
Imagine you're in an exam hall where only one student can go to the restroom at a time using a pass. When they return, the pass goes to the next student. Similarly, computers use locks to let only one app access a resource at a time while others wait.
People's Interpretation
(will update)

Level 2:

My Explanation
Locks prevent issues when multiple entities access a resource simultaneously (like multiple apps writing to a file), which can cause unexpected behavior (like overwritten writes). Google Docs uses it to avoid overwritten edits during collaboration.
People's Interpretation
(will update)

Level 3:

My Explanation
A mutex blocks access to a critical section until the current thread is done, preventing race conditions but potentially causing performance hits. The GIL in CPython restricts access to shared resources to one thread at a time, impacting multi-threading.
People's Interpretation
(will update)

Level 4:

My Explanation
A mutex uses architecture-specific atomic instructions (uninterruptible) until successful, allowing thread access to a critical section. For non-atomic architectures, complex logic involving disabling interrupts and continuously checking flags is used.
People's Interpretation
(will update)

Level 5:

My Explanation

Damn! I’m not a mutex maestro, and I’ve dug through every corner of my brain and read more articles than a bored cat on the internet. But guess what? I still can't explain mutexes to an expert without turning into one myself.

One day, this section will be so expertly explained it’ll make your head spin!

Any experts here? Drop your wisdom in the comments, and I'll update this section.

People's Interpretation (will update)

Now, my challenge is to present this to individuals at these five levels and share their takes on it. I'll update with the most relevant responses as I receive them.

Join me in this challenge! If you have a better explanation, comment below. Or, explain the concept to those around you and share their interpretations. I will include the top three dev handles with most appropriate response in my submission (you can work on teams of up to four people).

Or, go ahead and participate on your own. Explain any computer science topic in 256 characters or less. Remember, no plagiarism.

By the way, I still have some buddy passes to nights and weekends s5. Click on this invitation link and you will be directly accepted to the s5 cohort. First kick off starts on 15th June. Let’s build something amazing together!

Cheers🎉 ~ sauravshah31

This is for you, let's build some cool stuff! 🚀✨

Saurav Shah — Tue, 11 Jun 2024 19:36:33 +0000

Hey Folks, 🎉

It's time to bring your ideas to life! 💡 Whether you're a designer, developer, content creator, or just have a passion to create something amazing, this is your chance. 🚀

No matter your skill level—beginner or pro, student or full-time professional—your dream project awaits. 🌟 Use your nights and weekends to build something you've always wanted. 🔨✨

You might know me from my previous blog on "Using Google Apps Script." 📚 Now, it's time to use those skills to create an actual product. 🛠️

And here's the fun part: I'm diving into this journey too, building something I'm passionate about. ❤️ I'll be documenting the process with plenty of technical blogs along the way. 📖🔧

For you, I have buddy passes to Buildspace's @_nightsweekends S5. 🎟️ You will be accepted automatically. Click on this invitation link and join nightsweekends s5. Let's create something incredible together with loads of other excited folks @_nightsweekends! 🌍👥

@sauravshah31 - Feel free to connect, let's share the journey together

Using Google Apps Script: Create a todo web app

Saurav Shah — Wed, 01 Nov 2023 13:43:22 +0000

I have added links to the terms I've mentioned in this blog. There's no need to hastily click on each link as you read along (I understand how overwhelming that can be). Instead, I suggest you navigate through the entire tutorial first without clicking any links. You can revisit the links at your leisure to delve deeper into the topics.

Use this link to view the final todo app we will be building. If you want to deploy "CoolTodoApp" on your own account and use it personally, use this link. You will be asked for some permissions, review and approve it by following this. This is the final source code.

Topics covered

Why to use Apps Script
Getting started with google apps script
Create a todo app: Getting started
Google script: Server side GET and POST endpoints
Rendering client-side UI using HTMLService
Communicating with the server
Templated HTML: Dynamic HTML rendering
Refactoring code into html, css, js and gs files
Building client-side UI
Using Drive Service
Using Spreadsheet Service
Using Charts Service
Generating PDF report
Sending Email
Triggering events
Using Lock Service
Meta tag and client side metadata
Building server-side API
Deploying final CoolTodoApp

Why to use Apps Script

Google Apps Script is a development platform provided by Google that can be used to create applications that integrate with Google Workspace. Using app services you can easily access Google apps like Docs, sheets, drive, etc. and automate stuff in JavaScript. You can use app script to create add-ons, create a client-facing web app, automate stuff, use it as a backend server for your simple apps or create fun projects. The app script is based on JavaScript, so knowing the basics of JS should be enough to get started.

Getting started with google apps script

Let's get started. Go to script.new to create a new Google app script. Let's give the project a name CoolTodoApp.

Google App script allows two kinds of files

Script file: This is the .gs file (eg: code.gs) that contains server-side logic
HTML: This is the HTML file that can be used to build UI. The JS and CSS logic also goes in this file

You can add multiple files to your project. For the Script file, all the objects are exposed globally. This means you can use the objects defined/declared in one file from any file without needing to import them. For HTML file, the server GET endpoint renders a single HTML file. You can then use server-side functions to "import" / "include" other HTML files. You will have clarity once we start using these later.

Now, let's try printing something in the console. Add some console log statements in myFunction function and try running the function.

You can see the "Execution log". This method of running can be used to debug your code or test some functions. To the left of the Run button, there is a Debug Button. You can add breakpoints by clicking the line no, then run the function step by step. This is very helpful to debug code.

Create a todo app: Getting started

Now we can start creating our app. In this tutorial, we will be creating a todo app with a custom client-facing UI. We will be storing the todo list of a day in a spreadsheet. The spreadsheets for the entire month will be stored inside the month folder in Google Drive. We will also set up a notification event, that will send an email on the first of every month with the summary of todos for the previous month.

Before starting to code, we need to think of the requirements of our app. We will have a client-facing UI that lists the todos for the day and allows us to add/remove todos and a server that stores the data. Our requirement is simple:

Client-facing UI shall have to display the following
- List of todos for the day, and a button to delete it
- An input field that allows the user to add new todo task
The server shall handle the following
- Return list of todos for a day
- Store any new todo task
- Send email on the first of every month with the summary of the previous month

Now that we have the product requirements clear, we can start with the technical requirements. This includes the UI design, the server architecture and choices, etc. We will start with the UI design. I have created this simple UI in Figma for our web app. A rough sketch of the UI in the paper should also be enough to get started.

Now we can start thinking about the how server should handle the requests. We know using app scripts we can interact with spreadsheet, drive, docs, etc. We will use Google spreadsheets to store the user todos. To keep things organized, we create a new spreadsheet for each day. We also create a folder for each month and store all the spreadsheets for a month in the respective folders. For the monthly report, we will summarize the data from all the spreadsheets and create a PDF file. We will then send this pdf file to the email as an attachment. We know these operations are possible using app script services, but don't know how to do it yet. Throughout this tutorial, you will learn different aspects of Google app script and app services, and use it to build a complete todo app.

Google script: Server side GET and POST endpoints

Before starting to code, let's see how gs (Google script) can be used as a server. Two common methods to talk to a server are GET and POST. So, we need a way to handle these two requests in gs. App Scripts provides two functions to handle these requests: doGet(e), doPost(e). The e is the request parameter and contains the request data. Refer to the documentation to know more.

So, we have to define these functions in our gs script file, and the return value will be returned as a response to the user. The return value should be of type HtmlOutput or TextOutput, otherwise, you will get an error. To return a value of this type, you can use HtmlService or Content Service. A service in the Google apps script is a utility class that contains useful methods that you can use to interact with Google apps.

Let's modify the Code.gs file. First, let's clean up by removing the myFunction function. Then, add doGet function and return some string using HtmlService.createHtmlOutput(html) function

function doGet(e)
{
  return HtmlService.createHtmlOutput("This is a cool web todo web app made using google app script");
}

Now, you need to deploy your web app. Click Deploy (on the top right) and select Test deployments. Select the type as Web app, Execute as "Me", Who has access "Anyone" and then click "Deploy". You will get an URL. Open it in another tab. You should see the output.

Rendering client-side UI using HTMLService

We can use HTMLService to serve html file to the client. Basically, we need to return a HTML file from doGet function. You can return a html file using createHtmlOutputFromFile(filename).

Let's try out an example. First, Add a new file (HTML). Let's name it index. Now, let's add some html.

<!DOCTYPE html>
<html>
  <head>
    <base target="_top">
  </head>
  <body>
    <p>This is a cool todo app made using google app script.</p>
  </body>
</html>

Now, in return this file in the doGet function in Code.gs

function doGet(e)
{
  return HtmlService.createHtmlOutputFromFile("index.html")
}

You can either use index.html or index as a filename. Google app script doesn't allow you to add more than one file with the same name, even if it is a script and HTML file. If you open the URL (from the test deployment), you should see this html file being served.

Communicating with the server

There is a way to call server-side functions, from your client-side js or even html. This is the easiest way to talk to your server. You can use Client-side API to call the server-side API. Basically, you use google.script.run.yourFunction(), where yourFunction() is defined in the script file (server-side). You can pass most of the datatypes as an argument to the function, and the same will be available on the server. Similarly, yourFunction() can return values, which will be available to your client side.

Let's start with a simple example. The client-side sends the client's current date to the server, and the server logs it to the console. Note that Date datatype is not allowed as an argument, so we convert it to string.
In the server-side code Code.gs, add the logUserDate function.

function logUserDate(date)
{
  Logger.log(`user_date : ${date}`);
}

Now we can call this logUserDate from client-side js. Modify index.html

<!DOCTYPE html>
<html>
  <head>
    <base target="_top">
  </head>
  <body>
    <p>This is a cool todo app made using google app script.</p>
    <script>
      let userdate = new Date();
      google.script.run.logUserDate(userdate.toString());
    </script>
  </body>
</html>

You can now refresh the webpage URL. Go to the script editor, and open the Executions tab. All the server logs will be displayed here. You can see that there are two execution logs, first for the doGet that returns the web page to the client, and then for the logUserDate, which is called from the client side js.

How does it work, you might be wondering. Well, the utility, provided by google.script.run, does the HTTP call for you. It will send a POST request to the server. The server parses the request, calls the correct function and returns the value as a response to the client. This is called Remote procedure call. If you look at your browser network tab, you will see a POST request sent to the server. The request data is a form data in this format request '["logUserDate","[\\"Your date string"]",null,[0],null,null,1,0]'.

You can use the Client-side API to get back the data from the server using withSuccessHandler(function) callback. Similarly, you can use withFailureHandler(function) callback to handle the errors while making the request.

Let's see an example. Say you have a server function getSum(a,b) that calculates the sum of two numbers a and b.

const getSum = (a,b) => {
  return a+b;
}

On the client side js, we can get back the results using the withSuccessHandler(function) callback. The parameter passed to the callback function is the return value from the server (the sum in this case).

<script>
const a = 3;
const b = 4;

const printSum = (result) => {
  console.log(`The sum of ${a},${b} is ${result}`);
};

google.script.run.withSuccessHandler(printSum).getSum(a,b);
</script>

Note that the functions starting with underscore('_') are not exposed to the client. If you want to write a "private" function that can't be called directly by the client, you can prepend the function name with "_". For example, the following _getFileDataFromDrive can't be called by the user directly. However, the client can call the getData function which will then call the _getFileDataFromDrive. This is a common pattern for internal utility functions and exposing them through an interface.

const APP_ROOT_FOLDER = "CoolTodoApp";


/*This function is not exposed to the client*/
const _getFileDataFromDrive = (fileId) => {
  let file = DriveApp.getFileById(fileId);
  let data = file.getBlob();
  return data
}

/*This is the interface exposed to the client to get the data from a file*/
const getData = (filename) => {
  let fileId = _searchFile(CONSTANTS.APP_ROOT_FOLDER, filename); //_searchFile is another private function that searches for the filename and returns it's id
  let data = "";
  if(fileId !== "")
  {
    data = _getFileDataFromDrive(fileId);
  }
  return data;
}

Templated HTML: Dynamic HTML rendering

Templated HTML allows you to render dynamic HTML pages. It allows you to dynamically add sections to your html, before sending it to the client. If you are familiar with other template engines like Jinja, EJS, etc, it is similar to those. You can embed code inside your HTML document, and the server renders those to get a complete HTML, before sending to the client.
To add dynamic logic to your HTML you can use these syntaxes:

To execute code without outputting any content to the template.

<? your logic ?>
eg: <? if(some dondition) ?>

Use it if you trust the source the data is coming from, and the data to be rendered is some script. In the above example, data.user_script is rendered by the server and replaced with a js code.

To render output using contextual escaping.

<?= data / variable ?>
eg: <p><?= data.user_name ?> logged in</p>

To render output without using contextual escaping.

<?!= data / variable?>
eg:  <p><?!= data.static_data ?> logged in</p>

Use this if the data source is not trusted, or anytime you don't need to dynamically add some script. Escaped output means the output is the literal string as provided in the input. It is achieved by adding escape characters instead of evaluating them.

To create a templated HTML, you can use HTMLService.createTemplateFromFile(filename). This will return a template file from the file you have given. To render it (evaluate the template code), use evaluate(). You can also pass data to the template, by setting the template HTML's data variable. Let's try an example. We will show the visitor count to the user.

Let's start with the template html index.html

<!DOCTYPE html>
<html>
  <head>
    <base target="_top">
  </head>
  <body>
    <p>You are the <?= data.visitor_count?> visitor</p>
  </body>
</html>

Here, data.visitor_count will return the visitor count.

For the server side, we will keep track of the visitor in a global variable, and return that in the template. Please note that global variables are not persistent by default, meaning each time you run the script, the global variables will be allocated again and they will get initialized again. So, you can't simply keep track of the visitors using global count. We will be using Properties Service to create a "persistent" property. Let's modify code.gs.

function doGet(e)
{
  let scriptProperties = PropertiesService.getScriptProperties();
  if(!scriptProperties.getProperty('visitor_count'))
  {
    //initialize the visitor_count
    scriptProperties.setProperty('visitor_count', 0);
  }

  //increament the visitor_count for each user
  let visitor_count = scriptProperties.getProperty('visitor_count');
  visitor_count++;
  scriptProperties.setProperty('visitor_count', visitor_count);

  //Render the htl template by passing the visitor_count data
  let template = HtmlService.createTemplateFromFile("index.html");
  template.data = {
    visitor_count
  };
  visitor_count += 1;
  const rendered_html = template.evaluate();
  return rendered_html;
}

If you open the test deployment URL, each time you refresh the browser, you should see the count increase.

Refactoring code into html, css, js and gs files

Now that we know the basics of how Google app script works, we can start coding our application. But, let's see how to structure our code first. As you might have seen, you can only add a "Script" or "HTML" file in your script. However, you can put your HTML, CSS and JS into multiple HTML files and then render it as a single file to the user. You can also have multiple gs files.
Let's see how you can refactor your frontend HTML, CSS and JS. You create a template HTML, call the server-side function to "include" other files and render that HTML template to get the entire page.

You can refer to this for the complete source code and have a look at this for the final web app for the complete source code. index.html is a template HTML that "includes" other HTML files.

<!DOCTYPE html>
<html>
  <head>
    <base target="_top">
    <?!= include('stylesheet'); ?>
  </head>
  <body>
    <div id="root">
      <?!= include('header'); ?>
      <div class="line"></div>
      <?!= include('body'); ?>
      <div class="line"></div>
      <?!= include('footer'); ?>
    <div>
      <?!= include('javascript'); ?>
  </body>
</html>

On the server side, we have a function include(filename) that returns HTML output from the file. index.html template code is calling this server-side function.

function include(filename) {
  return HtmlService.createHtmlOutputFromFile(filename)
      .getContent();
}

Building client-side UI

We should now start with client-side UI. We already have the UI design for our web app. I will put the design again here.

Now, looking at the UI, we structure our client-side HTML into three files. We will also have one file each for CSS and JS. index.html is the template HTML file, that includes all other files. Here are the client-side files:

├── style.html
├── client-script.html
├── header.html
├── pending-tasks.html
├── completed-tasks.html
├── index.html

I will skip the client-side HTML code explanation. You can follow the design to code your client-side app. First start with a static web page, with dummy data, that has all the blocks from the design. Then you can start integrating the backend, and fetch data dynamically. You can refer to this for the source code of the static web page and this for the final web page. One thing to note is the header part. I have linked Bootstrap CDN, just like you would have done normally. Note that meta tags included directly in an Apps Script HTML file are ignored. Also, note how I have loaded the icon (in header.html) from the file stored in Google Drive. You can use this URL (shared with all) to include images from Google Drive: https://drive.google.com/uc?id=drive_id&export=download&format=img_format.

Using Drive Service

Google app script provides some built-in services to interact with Google services like Google Drive, Google Admin Console, etc. Services are objects that expose some APIs which can be used to talk to Google apps. Let's build an app that allows the user to download files stored in Google Drive. We will use the Drive Service.

const doGet = (e) => 
{
  let filename = "";
  if("filename" in e.parameter)
  {
    filename = e.parameter["filename"];
  }

  return HtmlService.createHtmlOutput(`filename : ${filename}`);
}

The code above shows how you can get the query string the user passes. You can refer to this to learn more about other data contained in the e parameter of doGet function. The user will pass the filename as a query parameter, and the server will return the file to the user. An example of a query string passed by the user is https://script.google.com/macros/s/app_id/dev?filename=test.


const APP_ROOT_FOLDER = "CoolTodoApp";

/*
This function will create folder with name "foldername"
in the Google Drive's root folder
*/
const _createFolder = (foldername) => {
  let folderObj = null;
  let folderExists = DriveApp.getFoldersByName(foldername);
  if(folderExists.hasNext() === true)
  {
    //Folder already exists, get the id for that folder
    folderObj = folderExists.next();
  }
  else
  {
    //Folder doesn't exists, create new folder
    folderObj = DriveApp.createFolder(foldername);
  }
  return folderObj;
}

/*
This function will create a dummy file with name "filename" 
under folder (folderObj object) with size "filesz" bytes.
*/
const _createDummyFile = (filename, folderObj, filesz) => {
  let fileExists = folderObj.getFilesByName(filename);
  if(fileExists.hasNext())
  {
    console.log(`file (${filename}) already exists`);
    return null;
  }
  //create dummy data
  let content = "";
  for(let i=0; i<filesz; i++)
  {
    content += "e";
  }
  return folderObj.createFile(filename, content);
}

/*
This function will create bunch of files for testing our application
*/
const _testCreateFile = () => {
  // This will create "CoolTodoApp" folder
  // We will be using this folder for any testing
  let rootFolderObj = _createFolder(APP_ROOT_FOLDER);

  _createDummyFile("file1.txt", rootFolderObj, 1024);
  _createDummyFile("file2.txt", rootFolderObj, 1024);
  _createDummyFile("file3.txt", rootFolderObj, 1024);
}

The code above will create some dummy files under the folder "CoolTodoApp". DriveApp.getFoldersByName will search the entire Google drive (not just the root folder). The return value is a FolderIterator which can be navigated like a linked list. It has a method named hasNext which returns true if there is another entry with that name. The next method increments the iterator to point to the next object. You should always check if there is another object in the iterator using hasNext before calling next. You can call next sequentially to iterator through all the entries. DriveApp.createFolder can be used to create a folder in the root of the user's Drive. Similarly, Folder.createFolder can be used to create a folder in a particular folder pointed by the Folder object. Folder.createFile can be used to create a file in a particular folder.

You can now run the _testCreateFile function using the script editor's run button. If you are running it for the first time, you will be asked to review permission.

Click on the "Review permissions". Then click on "Advanced" and then continue to CoolTodoApp.

After reviewing the permissions, you should be able to run the function. It will create 3 files under "CoolTodoApp" Folder.


const APP_ROOT_FOLDER = "CoolTodoApp";

/*
This function will search for file named "filename" under folder "folder"
and return the fileid
*/
const _searchFile = (filename, folder) => {
  let fileId = "";
  let folderExists = DriveApp.getFoldersByName(folder);
  if(folderExists.hasNext() === true)
  {
    let folderObj = folderExists.next();
    let fileExists = folderObj.getFilesByName(filename);
    if(fileExists.hasNext())
    {
      fileId = fileExists.next().getId();
    }
  }
  return fileId;
}

const doGet = (e) => 
{
  let filename = "";
  if("filename" in e.parameter)
  {
    filename = e.parameter["filename"];
  }

  let fileId = _searchFile(filename, APP_ROOT_FOLDER);

  if(fileId === "")
  {
    return HtmlService.createHtmlOutput(`${filename} not found`);
  }

  let fileObj = DriveApp.getFileById(fileId);
  let downloadUrl = fileObj.getDownloadUrl();
  return HtmlService.createHtmlOutput(`<script>window.location.replace("${downloadUrl}")</script>`);
}

_searchFile will check if a file exists in a folder and return the ID of the file using File.getId. The doGet function then gets the file by id using DriveApp.getFileById and gets the download link using File.getDownloadUrl. Note how I have prepended the _searchFile function with an underscore (_) to prevent exposing it to the client.

The return value of doGet function is a HtmlOutput with a script that redirects the user to the download link. As we can't return File or Blob directly, I am redirecting the user to the download endpoint. You can also display the download URL instead of redirecting.

Using Spreadsheet Service

Like the Drive Service, you can use the Spreadsheet Service to interact with Google Sheets. Let's see how to create a new spreadsheet and save it in the drive.

const APP_ROOT_FOLDER = "CoolTodoApp";

/*
This function creates a Google Sheet named "tasks"
and moves it to the App's folder
*/
const _createSheet = () => 
{
  //Get the app folder
  let appFolder = DriveApp.getFoldersByName(APP_ROOT_FOLDER);
  if(appFolder.hasNext())
  {
    appFolder = appFolder.next();
  }
  else
  {
    console.log(`${APP_ROOT_FOLDER} not created`);
    return;
  }

  //create a new spreadsheet
  let sheetFile = SpreadsheetApp.create("tasks");
  //Move the spreadsheet to the app folder
  DriveApp.getFileById(sheetFile.getId()).moveTo(appFolder);
}

If you run this function, you will be asked to review the permissions again as you are not using another service. After execution, you should see a spreadsheet named "tasks" in the app's folder.

SpreadsheetApp.create can be used to create a new spreadsheet. File.moveTo is used to move a file to a different folder.

Now, we add some rows to the sheet.

const APP_ROOT_FOLDER = "CoolTodoApp";

/*
This function adds some tasks to Google Spreadsheet
*/ 
const _addRows = () => {
  //Get the app folder
  let appFolder = DriveApp.getFoldersByName(APP_ROOT_FOLDER);
  if(appFolder.hasNext())
  {
    appFolder = appFolder.next();
  }
  else
  {
    console.log(`${APP_ROOT_FOLDER} not created`);
  }

  //search the sheet
  let sheetFile = appFolder.getFilesByName("tasks");
  if(sheetFile.hasNext())
  {
    sheetFile = sheetFile.next();
  }
  else
  {
    console.log("spreadsheet tasks not created");
    return;
  }

  //open the spreadsheet using SpreadSheetApp class
  let spreadsheet = SpreadsheetApp.open(sheetFile);
  //get the first sheet
  let sheet = spreadsheet.getSheets()[0];

  //Add header
  sheet.appendRow(["task_id", "task_description", "completed", "createad_on"])

  //Add some dummy tasks
  let today = new Date;
  sheet.appendRow(["1", "This is some task for testing", false, today.toUTCString()])
  sheet.appendRow(["2", "This is one more task", false, today.toUTCString()])
  sheet.appendRow(["3", "This is a completed task", true, today.toUTCString()])
  sheet.appendRow(["4", "This is another completed tasks", true, today.toUTCString()])
}

First, we open the file as a spreadsheet using SpreadsheetApp.open. This will return a Spreadsheet object. Each spreadsheet has 1 or more sheets. We can get all the sheets using getSheets. This will return an array of all sheets, indexed in the same order as they were stored. We use the first sheet by getting the 0th index element in the array. Finally, we add a header and some tasks using Sheet.appendRow. If you run the _addRows function, you will see that the sheet is populated with some rows.

Now, we will learn how to read the data from the sheets.

const APP_ROOT_FOLDER = "CoolTodoApp";

/*
This function prints the completed tasks
*/
const _getCompletedTasks = () => {
  //Get the app folder
  let appFolder = DriveApp.getFoldersByName(APP_ROOT_FOLDER);
  if(appFolder.hasNext())
  {
    appFolder = appFolder.next();
  }
  else
  {
    console.log(`${APP_ROOT_FOLDER} not created`);
  }

  //search the sheet
  let sheetFile = appFolder.getFilesByName("tasks");
  if(sheetFile.hasNext())
  {
    sheetFile = sheetFile.next();
  }
  else
  {
    console.log("spreadsheet tasks not created");
    return;
  }

  //open the spreadsheet using SpreadSheetApp class
  let spreadsheet = SpreadsheetApp.open(sheetFile);
  //get the first sheet
  let sheet = spreadsheet.getSheets()[0];
  //get the tasks
  let tasks = sheet.getDataRange().getValues();
  //Firt row is the header, remove it
  tasks = tasks.slice(1);

  const completed_col_idx = 2;
  let completedTasks = tasks.filter(data => data[completed_col_idx] === true);
  console.log(completedTasks);
}

Sheet.getDataRange() gets the Range of cells that the sheet has. Range.getValues get the values in the selected range. Then, we filter the rows which are completed.

To update a cell, you should first get the row and col index to update. Then use Sheet.getRange to get the Range object corresponding to a row and col (both row and col start with 1 for the first entry). Then use Range.setValue to update the cell value.

const APP_ROOT_FOLDER = "CoolTodoApp";

const _updateRow = () => {
  //Get the app folder
  let appFolder = DriveApp.getFoldersByName(APP_ROOT_FOLDER);
  if(appFolder.hasNext())
  {
    appFolder = appFolder.next();
  }
  else
  {
    console.log(`${APP_ROOT_FOLDER} not created`);
  }

  //search the sheet
  let sheetFile = appFolder.getFilesByName("tasks");
  if(sheetFile.hasNext())
  {
    sheetFile = sheetFile.next();
  }
  else
  {
    console.log("spreadsheet tasks not created");
    return;
  }

  //open the spreadsheet using SpreadSheetApp class
  let spreadsheet = SpreadsheetApp.open(sheetFile);
  //get the first sheet
  let sheet = spreadsheet.getSheets()[0];
  //get the tasks
  let tasks = sheet.getDataRange().getValues();
  //Firt row is the header, remove it
  tasks = tasks.slice(1);

  const id_col_idx = 0;
  const task_desc_col_idx = 1;
  let updateTaskIdx = tasks.findIndex(data => data[id_col_idx] === 3) + 1; //+1 as slice removed the header
  let updateTaskCell = sheet.getRange(updateTaskIdx + 1, task_desc_col_idx + 1 ); //+1 as the A1 notation starts with 1 for the first row/col
  updateTaskCell.setValue("This task was update"); //update the value
}

Using Charts Service

Charts Service can be used to create charts. Let's see an example to create Pie Chart.

const APP_ROOT_FOLDER = "CoolTodoApp";

const _generateReport = () => {
  let appFolder = DriveApp.getFoldersByName(APP_ROOT_FOLDER);
  if(appFolder.hasNext())
  {
    appFolder = appFolder.next();
  }
  else
  {
    console.log(`${APP_ROOT_FOLDER} not created`);
    return;
  }

  const ntasksCompleted = 10;
  const ntasksPending = 15;
  //Prepare data to create data table
  let data = Charts.newDataTable()
                .addColumn(Charts.ColumnType.STRING, "task_type") //add a column to describe type of task
                .addColumn(Charts.ColumnType.NUMBER, "value"); //add a column to describe value for that task type
  //add rows to the data table for completed and pending tasks
  data.addRow(["completed", ntasksCompleted]);
  data.addRow(["pending", ntasksPending]);
  //Build te data table
  let dataTable = data.build();

  //create a new chart
  let chartBuilder = Charts.newPieChart()
                      .setDataTable(dataTable)  //set the data source
                      .setDimensions(675,435)   //set the dimension
                      .set3D()                  //set 3D option
                      .setOption('chartArea',{left:10,top:10,width:`${675 - 20}`,height:`${435 - 20}`}) //add 10px padding on all side
                      .setOption('legend', {alignment:'center', position:'labeled'}) //set label option
                      .setOption('pieSliceText', 'none')  //don't display any content in the slice
  let chart = chartBuilder.build(); //build the chart with the given options
  let blob = chart.getBlob(); //get the Blob for the chart
  blob.setName("task-chart"); //give name to the blob

  appFolder.createFile(blob); //save the blob in the app's folder
  return blob;
}

If you run the _generateReport function, the chart created will be stored in the Drive's app folder (CoolTodoApp).

First, you create a data source for the pie chart. Charts.newDataTable creates an empty DataTableBuilder. Using addColumn you can add a column of a type and give it a label. Then, using addRow you can add rows to the data table. As we have added two columns in the data table, the rows are also two columns. You can chain multiple operations in a single statement like I have done with .addColumn. After setting the table data and options, you can use DataTableBuilder.build to get the table.

Now, you create a new chart, a pie chart in this case. Charts.newPieChart creates a new chart of type "pie chart". You can add the data source using setDataTable. setDimensions, set3D, setOption can be used for controlling the look of the chart. You can get the list of all available options here. After setting the data and options of the chart, you can use build to build the chart. It will create a Chart object.

Generating PDF report

We will need to create a report of tasks for a month for our "CoolTodoApp". There is no service or API to create PDFs. We can however use template HTML to create a HTML report and export it as "PDF". PDFs should be designed such that it is optimized for printing. We can use CSS options to tailor our HTML for printing. How to Create Printer-friendly Pages with CSS is a good reference. In this section, we will not focus on CSS optimization for printers. We will learn more about it when we create the report for our app.

Let's create a simple HTML template for a report

<!DOCTYPE html>
<html>
  <head>
    <title>Creating PDF in Google app script</title>
    <style>
      body {
        box-sizing: border-box;
        font-size: 16pt;
      }

      @page {
        size: A4;
      }
      .pdf-page {
        width: 595pt !important;
        height: 842pt !important;
        margin: 0pt !important;
        padding: 0pt !important;
      }
    </style>
  </head>
  <body>
    <div class="pdf-page">
      <h2>Creating PDF using Google app script</h2>
      <figure>
        <img src="<?!= data.todo_chart ?>"/>
        <figcaption>Tasks completed</figcaption>
      </figure>
      <div>
        This is the pdf page. Added a image, title and some tasks.
      </div>
    </div>
  </body>
</html>

For the image source, I used base64 data as the image source. The base64 data is passed from the server and included in the template html. Also, note that I have used unescaped output syntax <?!= data.todo_chart ?> so that there is no contextual escaping.

function doGet(e)
{
  let report_chart = _generateReport();
  let template_data = {
    todo_chart : `data:image/png;base64,${Utilities.base64Encode(report_chart.getBytes())}` //encode into base64 from the blob bytes
  };
  //Render the htl template by passing the visitor_count data
  let report_templete = HtmlService.createTemplateFromFile("report-template.html");


  report_templete.data = template_data;
  const report_html = report_templete.evaluate();
  report_html.setTitle("Creating PDF in Google app script");
  return report_html;
}

Here, the report_chart is the blob returned from the _generateReport function from the Charts tutorial. The blob is encoded to base64 data using Utilities.base64Encode and passed to the HTML template.

const APP_ROOT_FOLDER = "CoolTodoApp";

function _generatePDF()
{
  let report_chart = _generateReport(); //get the pie chart blob
  let template_data = {
    todo_chart : `data:image/png;base64,${Utilities.base64Encode(report_chart.getBytes())}` //encode into base64 from the blob bytes
  };
  //Render the htl template by passing the visitor_count data
  let report_templete = HtmlService.createTemplateFromFile("report-template.html");
  report_templete.data = template_data; //pass the data to the template file
  const report_html = report_templete.evaluate();
  report_html.setTitle("Creating PDF in Google app script")
  const pdfReport = report_html.getBlob().getAs("application/pdf"); //convert the html to pdf
  let appFolder = DriveApp.getFoldersByName(APP_ROOT_FOLDER).next();
  appFolder.createFile(pdfReport);
  return pdfReport;
}

To convert the HTML report to PDF, you can get the blob using getBlob and then convert it to PDF using getAs. _generatePDF function will create the PDF report and save it into the app folder.

Sending Email

Gmail Service can be used to interact with Gmail service and send emails. Let's see an example of sending an email, with an attachment, to the current user.

const _sendEmail = () => {
  const subject = "Sending email using app script";
  let pdfReport = _generatePDF();
  let template_data = {
    todo_chart : `cid:todo_chart` //cid format for embedding image in email
  };
  let report_chart = _generateReport(); //get the pie chart blob for embedding into email

  let report_templete = HtmlService.createTemplateFromFile("report-template.html");
  report_templete.data = template_data; //pass the data to the template file
  const report_html = report_templete.evaluate();

  MailApp.sendEmail(Session.getActiveUser().getEmail(), subject, "",  //compose email to current user's email 
                        {
                          htmlBody : report_html,  //adding email body from the html template
                          inlineImages : {"todo_chart":report_chart} //adding cid format inline images
                          attachments:[pdfReport], //add pdfReport as attachment
                        })
}

To send the email, we use MailApp Service. You can also use GmailApp Service but GmailApp will have full access to your inbox. Using MailApp, you have full control of the email, so for just sending emails, it is better to use MailApp.

MailApp.sendEmail is the most flexible way to send an email. First, we set the receiver address to the current user's email address using Session.getActiveUser().getEmail(). Then, we set the subject and leave the body blank. For the email body, we will use HTML format. In the options, we add htmlBody to the HTML report generated from the template. Note that to add images to the email HTML body, you will need to use CID format. For the image src, I have used cid:todo_chart. The prefix cid: indicates it is the CID format inline image, and todo_chart is the identifier. In the options, I have set inlineImages to a key-value pair, with the key being the cid identifier and the value being the Blob for the image. I have also added the pdfReport as the attachments options, to add the attachment.

Triggering events

There are a couple of event-based triggers that can be used to automate things. Events are some actions or some conditions like the client sending GET, the user installing an app script, time-driven events like the first of every month, every day, every hour, etc. The doGet and doPost are also triggered when the GET / POST request (event) is sent to the server.

Our "CoolTodoApp" will send a monthly report of tasks to the user's email. This is basically calling the _sendEmail on the first of every month. Let's set a monthly trigger manually. Click on the "Triggers" option.

Click on "Add Trigger" and fill in the parameters for a monthly trigger. Set the function to _sendEmail, deployment as "HEAD", "Time-driven", "Month timer", and 1st 7-8 am, then save it. Now, _sendEmail will automatically be called on the first or every month between 7-8 am in the user's timezone. To delete the trigger, go the the "Triggers" tab and delete the trigger.

We can also add triggers programmatically.


//install trigger that run on the first of every
//month and calls teh _sendEmail function
const _installMonthlyReportTrigger = () => {
  let scriptProperties = PropertiesService.getScriptProperties();
  if(!scriptProperties.getProperty('monthly_report_trigger_id'))
  {
    //initialize the visitor_count
    scriptProperties.setProperty('monthly_report_trigger_id', -1);
  }

  //get the trigger id stored
  let saved_trigger_id = scriptProperties.getProperty('monthly_report_trigger_id');
  let trigger_installed = false;
  const allTriggers = ScriptApp.getProjectTriggers();
  //check if the trigger with the saved id is already present
  for (let index = 0; index < allTriggers.length; index++) {
    // If the current trigger is the correct one, delete it.
    if (allTriggers[index].getUniqueId() === saved_trigger_id) {
      trigger_installed = true;
      break;
    }
  }

  //install the trigger only once
  if(trigger_installed === false)
  {
    let triggerBuilder = ScriptApp.newTrigger('_sendEmail') //create a trigger that calls _sendEmail
                        .timeBased()    //create a tiem based trigger builder
                        .onMonthDay(1)  //trigger on first of every month
                        .atHour(8)      //trigger on 8am - 9am
    let trigger = triggerBuilder.create(); //install the trigger
    //update the saved trigger id so that the trigger is not installed next time this function is called
    scriptProperties.setProperty('monthly_report_trigger_id', trigger.getUniqueId());
  }
}

The function _installMonthlyReportTrigger, creates a new trigger using ScriptApp.newTrigger, adds a timer-based trigger using TriggerBuilder.timeBased and configures it to run on the first of every month using ClockTriggerBuilder.onMonthDay and at 8-9 am using ClockTriggerBuilder.atHour. The trigger is then installed using ClockTriggerBuilder.create. If you run this function, it will install the trigger automatically. Note how I have used Properties Service to make sure the trigger is installed only once.

Using Lock Service

Lock or mutex are Synchronization primitives in computer science used to restrict access to a particular object to only one process or task even when multiple processes are trying to access it.

Let's consider a scenario in our "CoolTodoApp". _getRootFolder is a function that creates the app's root folder if it doesn't exist and returns the id. Let's say we have two endpoints GetPendingTask and GetCompletedTask that return the pending and completed tasks resp. We know that both of these functions will be calling the createRootFolder at some point.

//Code.gs
const APP_ROOT_FOLDER = "CoolTodoAppTest";

/*
Cretes the app's root folder if it doesn't exists
and returns the folder id
*/
const _getRootFolder = () => {
  let folder = DriveApp.getFoldersByName(APP_ROOT_FOLDER);
  if(folder.hasNext() === false)
  {
    //doesn't exists, create folder
    folder = DriveApp.createFolder(APP_ROOT_FOLDER);
  }
  else
  {
    //folder already exists
    folder = folder.next();
  }
  return folder.getId();
}

const GetCompletedTask = () => {
  let appFolderId = _getRootFolder();
  //some more logic to get the data from spreadsheet
  return []; //returning dummy data
}

const GetPendingTask = () => {
  let appFolderId = _getRootFolder();
  //some more logic to get the data from spreadsheet
  return []; //returning dummy data
}

function doGet(e)
{
  return HtmlService.createHtmlOutputFromFile("index.html")
}

On the client side, we will need to fetch both the completed and pending tasks.

<!-- index.html -->
<html>
  <body>
    <script>
      const handleCompletedTasks = (tasks) => {
        console.log("got completed tasks");
      }
      const handlePeningTasks = (tasks) => {
        console.log("got pending tasks");
      }
      const getTasks = () => {
        //call both GetCompletedTask and GetPendingTask
        google.script.run.withSuccessHandler(handleCompletedTasks).GetCompletedTask();
        google.script.run.withSuccessHandler(handlePeningTasks).GetPendingTask();
      }

      window.onload = getTasks;
    </script>
  </body>
</html>

Now let's open the client-side app.

If you look in your Google Drive, you will notice that there are 2 folders named "CoolTodoAppTest" created.

But we have added checks to check if the folder already exists in the _getRootFolder function. So why does this happen? This is a synchronization issue. Since the client side google.script.run is not synchronous (blocking call), both GetCompletedTask and GetPendingTask are called at the same time. So, there arises a situation where the first call has checked that the folder doesn't exist and goes in the condition where the folder is created, but before the folder is created, the second call to GetPendingTask is already made and in that call also the folder doesn't exist. So, both calls create a new folder.

Lock Service can be used to solve this problem. Using a lock ensures a part of the code is executed only by one process at a time, even if multiple processes try to execute the same code at the same time. Let's update the _getRootFolder function.

const _getRootFolder = () => {
  let lock = LockService.getScriptLock(); //Get the script lock
  let status = lock.tryLock(5000); //Acquire the lock (only one process can get it at a time)

  let folder = DriveApp.getFoldersByName(APP_ROOT_FOLDER);
  if(folder.hasNext() === false)
  {
    //doesn't exists, create folder
    folder = DriveApp.createFolder(APP_ROOT_FOLDER);
  }
  else
  {
    //folder already exists
    folder = folder.next();
  }

  if(status)
  {
    lock.releaseLock(); //release the lock so that other process can acquire it
  }
  return folder.getId();
}

Now delete the two "CoolTodoAppTest" folders from your Google Drive and load the client UI again. This time you will see that only one folder is created. You can get the lock (only one lock is available per script) using LockService.getScriptLock, acquire (lock the section of code) using Lock.tryLock and then release (unlock the section of code) using Lock.releaseLock.

Meta tag and client side metadata

Meta tags included directly in an Apps Script HTML file are ignored. If you want to add the meta tag to HTML, you can use server-side addMetaTag. Let's add viewport metadata to make our app responsive on mobile devices.

function doGet(e)
{
  let html = HtmlService.createHtmlOutputFromFile(filename);
  //add metatag
  html.rendered_html.addMetaTag('viewport', 'width=device-width, initial-scale=1');
}

HtmlOutput provides a bunch of other methods to control html metadata.


function doGet(e)
{
  let html = HtmlService.createHtmlOutputFromFile(filename);
  //set the title of client-side HTML
  html.setTitle("CoolTodoApp")
  //set the favicon
  html.setFaviconUrl("https://drive.google.com/uc?id=1twzpy63EbJV4b9U91UtgSZw7Zi8Ou5Lr&export=download&format=png");
}

You can set the title using setTitle. You can add a favicon using setFaviconUrl.

Building server-side API

The Server side will support the following functionalities (Functional requirements):

Client shall be able to fetch completed and pending tasks
Client shall be able to add new tasks, mark a task as complete/incomplete
Tasks shall be stored in Google Sheets
Tasks for each day shall be stored in separate google sheets
Tasks sheets shall be stored in google drive
Google Drive shall contain folders for each month
A summarized report for the tasks for a month shall be sent as an attachment

So, we will expose some endpoints (server-side functions) to the users. All the exposed APIs are kept in Code.gs. We will need some utility functions that talk to the Google services (like Google Sheets and Google Drive). We will keep these in gapi.gs. We will also need to generate a report from the tasks for a month. We will keep this in report.gs. Let's keep all other utility functions in utility.gs. constants.gs will contain the server configurable values. We will also need a HTML template for our report, let's keep it in report-template.html So the file structure for the server-side is as follows:

├── Code.gs
├── gapi.gs
├── report.gs
├── utility.gs
├── constants.gs
├── report-template.html

Have a look at this for the final source code. We will have a look at the files one by one.

Let's start with the APIs to interact with Google services gapi.gs. There is a class named gAPI that encapsulates all the methods needed to interact with the Google service for our use case. We know that we will have three hierarchies of files. All the app-related stuff will go in the App's folder. It is represented by the member variable this.rootFolder. this.rootFolder contains info about the root folder's id and Folder object. To create the folder name, there is a method named setRootAppFolder. This method creates the app's root folder if it doesn't exist and then sets this.rootFolder. This method is synchronized using Lock. Similarly, there are setMonthFolder and setTodoSheet methods to set the month folder and todo spreadsheet resp. checkSheetExists checks if the sheet already exists for the day. There are methods to get tasks, add tasks update tasks, etc. Global instance GAPI can be used to call these methods.

constants.gs contains a const variable named CONSTANTS that stores some server-side configurable values like the app's folder name, sheet headers, etc.

utility.gs contains utility functions like a function to extract the month name from a date, input validation functions, a function to get the last day of the previous month, etc.

Now, we can start exposing our APIs. Code.gs exposes the APIs to add tasks, update tasks as completed or pending, and get completed or pending tasks. These APIs validate the input and call the GAPI methods. doGet constructs HTML output from the template, sets some meta tags and returns the output.

We also need to change the client-side HTML files. Instead of adding data statically, we need to fetch the tasks from the server. client-script.html contains the logic to fetch the task from the server, add a task when the user submits the task form, and update a task when a user checks or unchecks a task.

Our "CoolTodoApp" is complete for the most part. The only thing remaining is the report generation feature. Let's create the template for the report. We need to decide on what to put in the report. The report should contain enough data to give an idea of tasks for the previous month, but it should not be overwhelming and mention every task. I have divided the report into two parts. The first page contains a pie chart and some data points summarizing the tasks for the previous month. The second page and after contain a table with summarized data for each day. I have also added a link to the task spreadsheet for each day in the table. report-template.html is the HTML template for the report. Have a look at this for the report sample.

report.gs contains the functions needed to generate and send the report. _testReportTemplate is the test function used to generate a dummy report from the template. _getTasksSummary returns rows of summarized tasks (date, #completed_tasks, #pending_tasks, #total_tasks, spreadsheet_link) for days between start_date and end_date (both inclusive). _generateTasksChart generates the pie chart summarizing the tasks. _getReportData returns the data for the report-template.html template. _sendMonthlyEmailReport generates the report and sends it to the current user. _installMonthlyReportTrigger installs the monthly trigger to call _sendMonthlyReportTriggerCallback which calculates the start_date and end_date for the previous month and calls the _sendMonthlyEmailReport.

Deploying final CoolTodoApp

We are done with coding the "CoolTodoApp". We have been using "Test Deployment" till now. We need to create a new deployment. Click on "Deploy", then "New Deployment" and add a description for the app. Change "Execute as" to "User accessing the web app" so that when you share the app link with others, the app will run on that user's context, not yours. Change "Who has access" to "Anyone". Note how I am calling _installMonthlyReportTrigger in doGet. This way the trigger is installed in the other user's account as well, when you share the app link.

Open the deployed link and start sharing it with your friends. They can deploy an instance of this app in their own Google account and start tracking the tasks. Here is the final source code and this is the todo app we have built.

Wowza! 🚀 We've created a mind-blowing web app with the power of Google Apps Script! 🌟 Learning has never been this thrilling! 🤓📚 Time to dive into the awesomeness of your CoolTodoApp and stay on top of your tasks like a pro. 📅💪

Just think about how this journey began – understanding the mystical world of Apps Script, brainstorming the app features, diving into APIs, building small features and then layering on more cool stuff until we had our magnificent web app masterpiece. 🎨💼

Now, you're not just a user; you're an explorer of endless possibilities! 🌠🌈 You can even unleash your creativity by publishing your very own extensions or more sensational Cool Apps. 🚀🎉

And don't forget to share your spectacular creations with me on X – I can't wait to see what you've cooked up! 👀 @sauravshah31

GitHub Profile README: A Different Perspective

Saurav Shah — Mon, 03 Aug 2020 07:16:13 +0000

So you have heard about Github Profile README (It's a new feature github added recently, you might wanna google it if you don't know what I am talking about). I have come across couple of posts and articles on how to create a great README for github profile. They are great at explaning how markdown works and also in making a great profile. But is this only the way to use it? I mean there already are many platforms for your profile (LinkedIn, Twitter, Instagram and many more) right? Of course these platform might not seem as much for techies as github is, But you could also host your profile as a static website with github-pages for free. And yes, github-pages supports markdown too!! With a bit of javaScript you could also make your portfolio much more dynamic. The link to your personal portfolio can be place just below you display picture, what a perfect fit!

🤔 (Me): "Sooo what could this README possibly used for???"

This feature is available for all the users. So the README must be personal to a particular user...?

Me: Opens a web browser and starts searching for user profiles.

This guy has a good profile. Ten repos! not bad.
Wow! Thirty repos 😲!! This girl is amazing! Oh wait, there are many forked repos too without any contributions.

(Me): Uses github search filters to display only the non forked repos.

Now that we have filtered out the irrelavant repos using github search filters, we still need to go open their repositories and go through the README file to literally figure out what this users is actually into. This is where Github Profile README can be usefull. It can act as a index for your repositories.

You could use the README to list out your favourite contributions and with a link to the repository alongside a brief about that repo, you could make a useful index for listing out your open source achievements. Rather than telling people who you are, listing out what you have done is a great way to showcase your skills.

Have a look at my github profile (I am kinda novice). You can also keep track of how many users visited your github profile. Have a look at this hitcounter if you want the counting thing. It bascally works by incrementing a count in the database whenever a page requests for an image, and then sends back the corresponding image for that count.

Cheers!