DEV Community: Kabilan Muthusamy

Transactions

Kabilan Muthusamy — Sat, 03 Feb 2024 08:36:30 +0000

In Software development, it's important to prevent an application from getting into an inconsistent state. Let's get into this by looking at an example,

Transferring of money, which involves deducting money from User A, and adding the money to User B.

Case 1:
What happens if the app crashes after deducting money from User A? This is an inconsistent state where User A sent the money and User B didn't receive the money.

Case 2:
What happens when User B also transfer money at the same time? There is a possibility that a transfer may override others. (race condition in a multi-threaded application). The balance of User B would be incorrect, which is also inconsistent.

This can be solved using Database Transactions, a sequence of operations that is considered a single unit of work. A Transactions provides ACID properties, which stands for

▶ Atomicity, all operations should succeed completely or fail i.e. If one operation fails, everything fails.

▶ Consistency, all the writes should be valid.

▶ Isolation, guarantees the correct execution of concurrent transactions, with isolation levels.

▶ Durability, If a transaction succeeds, it should persist even in case of system crashes.

The transaction's Atomicity property can solve case 1. The database locks can be used to prevent a single resource from being concurrently accessed by multiple threads/processes which will solve case 2.

Database Locks

Kabilan Muthusamy — Sun, 16 Apr 2023 03:57:58 +0000

What are Locks in Database?

In programming, Locks are used to having controlled access to resources i.e to prevent multiple threads from accessing the resources at the same time. Likewise, Database Locks provide mechanisms to access shared resources such as tables and rows in a multi-user environment.

Why do we need locks?

Let's take an example of a money transfer operation where user A transfers money to user B and user C transfers money to user A:

The above example produces an inconsistent result thus User A is unaware of the User C updates.

In this case, User A acquired a lock 🔒, so User C was not able to access the rows.

Lock Modes

Shared Locks: Shared locks, also known as read locks, are used to allow multiple users to read a resource simultaneously while preventing any user from modifying it. Shared locks are useful when several users need to access the same data for reading purposes.

Exclusive Locks: Exclusive locks, also known as write locks, are used to grant exclusive access to a resource to a single user or application, preventing any other users from accessing or modifying it. Exclusive locks are useful when a user needs to modify a resource without interference from other users.

Update Locks: Update locks are a combination of shared and exclusive locks. They allow multiple users to read a resource simultaneously but require exclusive access when a user needs to modify the resource. Update locks are useful when a resource needs to be frequently read but modified less frequently.

Intent Locks: Intent locks are used to indicate that a transaction or user intends to acquire a lock on a resource. Intent locks can be used to coordinate the acquisition of shared or exclusive locks between multiple resources to prevent deadlocks.

Schema Locks: Schema locks are used to control access to the schema of a database, including tables, views, and stored procedures. Schema locks prevent other users from modifying the schema while a user is making changes to it.

Range Locks: Range locks are used to lock a range of values in an index, preventing other users from modifying the same range of values concurrently. Range locks are useful when multiple users need to access the same range of values in an index.

Problems associated with locks

Deadlocks: Deadlocks occur when two or more transactions are waiting for each other to release locks on shared resources, leading to a situation where none of the transactions can proceed. Deadlocks can cause a system to become unresponsive, and can only be resolved by aborting one or more of the transactions involved.

Lock Contention: Lock contention occurs when multiple transactions are waiting to acquire the same lock on a shared resource, leading to delays and reduced system performance. Lock contention can be mitigated by using lock escalation, where a database system converts multiple low-level locks into a higher-level lock.

Locking Overhead: Locking overhead occurs when a database system needs to manage a large number of locks, leading to increased memory and processing requirements. Locking overhead can be reduced by using lock-free data structures, where multiple users or applications can access shared resources concurrently without the need for locks.

Reduced Concurrency: Locks can reduce concurrency by preventing multiple users or applications from accessing shared resources simultaneously. This can lead to delays and decreased system performance, especially in scenarios where concurrent access is frequent.

Understanding NGINX Core Contexts

Kabilan Muthusamy — Mon, 31 May 2021 03:42:33 +0000

What is NGINX ?

Nginx [engine x] is a free and opensource high performance web-server that can be used as reverse proxy, load balancer, SMTP proxy and a generic TCP/UDP proxy server. Nginx originally developed by Igor Sysoev in 2002 to solve C10k problem.

NGINX Configuration file

NGINX can be configured with nginx.conf file, By default this file is named as nginx.conf and placed under /usr/local/nginx/conf, /etc/nginx, or /usr/local/etc/nginx.

What are Directives and Contexts?

NGINX configuration consists of key-value pairs called directives. Directives decides which configuration to apply.They can be organized and grouped into Blocks known as Contexts. Contexts are tree like structures that can be nested within one another and Directives can only been used within Contexts.

Directives looks like:

user nginx;
worker_processes 5;

Contexts look like:

# Global Context
....
....

# http Context 
http {
      ....
      ....
      # server Context
      server {
             ....
             ....
      }

}

Global context

The "global" context is placed at the beginning of the core NGINX configuration file and is used to set the configuration for NGINX globally.

The global context allows module writer to configure user, no.of works, file to save the main process's PID, worker CPU affinity and niceness of worker processes.

#global context

# global directive
user nobody;
worker_processes 1;
....
....

Events context

The "events" context is contained within "global" context and can be used to set global options for NGINX connection processing.There can only be a single events context defined within the NGINX configuration.

#global context

# global directive
user nobody;
worker_processes 1;
....
....

#events context
events {
       # events directive
       ....
       ....
}

HTTP context

The "http" context holds directives for handling HTTP and HTTPS traffic. The Server context can be placed inside the HTTP context.

#global context

# global directive
user nobody;
worker_processes 1;
....
....

#events context
events {
       # events directive
       ....
       ....
}
# http context
http {
      # http directive
      ....
      ....
}

Server context

The “server” context is declared within the "http" context. There can be multiple instance of server context within http context. Each server context instance is a virtual server to handle client request.

The listen and server_name directives within the server context will be used to determine which server context can be used to respond the request. The directives in this context can override the directives that may defined in the http context.

# http context
http {
      # server context
      server {
             # server directive
             listen: 80;
             server_name: example.com, www.example.com
             ....
             ....
      }      
      ....
}

Location Context

The "location" contexts are used to define directives to handle client request. When a request for resource arrives at NGINX, it will try to match the URI to one of the locations and handle it accordingly. A location context can be nested within server context and also within another location context.

# http context
http {
      # server context
      server {
             listen : 80;
             # first location context
             location parameter {
                      ....
                      ....
             }
             # second location context
             location parameter {
                      # nested location context
                      location parameter {
                               ....
                               ....
                      }
                  ....
             }
      }      
      ....
}

There are two types of parameter to the location directive: prefix strings (path names) and regular expressions. For a request URI to match a prefix string, it must start with the prefix string.A regular expression is preceded with the tilde (~) for case-sensitive matching, or the tilde-asterisk (~*) for case-insensitive matching.

# location with string (path parameter)
location /home/user/ {
         # URI starting with /home/user/ will match 
         # but /some/home/user/ won't 
         ....
}

# location with regular expression as parameter
location ~ /.html? {
         # URI that  has .html or .htm string
         # in it will match
         ....
}

Upstream context

The "upstream" context is used to define and configure “upstream” servers. Basically, this context defines a named pool of servers that Nginx can then proxy requests to. This context will likely be used when you are configuring proxies of various types. The upstream context should be defined within http context and outside the server context, to be used.

Once the upstream servers have been defined, the name of the same is available within the server context to pass the request to the pool of back-end servers.

http {
    # upstream context
    upstream backend {
        server backend1.example.com;
        server backend2.example.com;
    }
    # server context
    server {
           location / {
                    proxy_pass http://backend;
           }
    }

}

Here the upstream group has been named as backend, which consists of two server configurations.To pass requests to a server group, the name of the group(backend) has been specified in the proxy_pass.

Mail context

NGINX can also be used a mail proxy. The "mail" context provides the ability to implement mail proxy.The mail context is defined in global context and outside of http context.

# global  context
....
....
# mail context
mail {
     # mail directive
     ....
     ....
}

If context

The "if" context provides conditional execution just like if in other programming languages. The if context is provided by the rewrite module and is the primary use of the if context. Since NGINX will test conditions of a request with many other valid directives, the if context should not be used for most forms of conditional execution thus might result in unexpected execution.

http  {
    server {
           # server context
           ....
           ....
           location location_match {
                    # location context
                    if (test_condition) {
                       # if context
                       ....
                    }
           }
    }
}

Limit_except Context

The "limit_except" context is used to restrict the use of certain HTTP methods within a location context.

....
# location context
location /restricted-write {
         # location context
         limit_except GET HEAD {
                      # limit_except context
                      deny all;
         }
}

Here client requests with GET and HEAD methods from any client will be allowed, but client requests with other methods are denied.

Some Best Practices

1.Declare directives in the highest context to which they are applicable, and overriding them in lower contexts to avoid code replication.
2.Use Multiple Sibling Contexts Instead of If Logic for Processing.

Summary

Now, you have a basic knowledge of nginx.conf file , its structure and core context to create your own conf file. Read NGINX's documentaion for more information about NGINX context.