DEV Community: Denis Magda

Geo-Distributed Microservices and Their Database: Fighting the High Latency

Denis Magda — Mon, 07 Nov 2022 16:57:01 +0000

Ahoy, mateys!

My development journey of the first version of the geo-distributed messenger has ended. So in this last article of the series, I’d like to talk about multi-region database deployment options that were validated for the messenger.

The high-level architecture of the geo-messenger is represented below:

The microservice instances of the application layer are scattered across the globe in cloud regions of choice. The API layer, powered by Kong Gateway, lets the microservices communicate with each other via simple REST endpoints.

The global load balancer intercepts user requests at the nearest PoP (point-of-presence) and forwards the requests to microservice instances that are geographically closest to the user.

Once the microservice instance receives a request from the load balancer, it’s very likely that the service will need to read data from the database or write changes to it. And this step can become a painful bottleneck—an issue that you will need to solve if the data (database) is located far away from the microservice instance and the user.

In this article, I’ll select a few multi-region database deployment options and demonstrate how to keep the read and write latency for database queries low regardless of the user’s location.

So, if you’re still with me on this journey, then, as the pirates used to say, “Weigh anchor and hoist the mizzen!” which means, “Pull up the anchor and get this ship sailing!”

Multi-Region Database Deployment Options

There is no silver bullet for multi-region database deployments. It’s all about tradeoffs. Each option provides advantages, while others cost you something.

YugabyteDB, my distributed SQL database of choice, supports four multi-region database deployment options that are used by geo-distributed applications:

Single Stretched Cluster: The database cluster is “stretched” across multiple regions. The regions are usually located in relatively close proximity (e.g., US Midwest and East regions).
Single Stretched Cluster With Read Replicas: As with the previous option, the cluster is deployed in one geographic location (e.g., North America) across cloud regions in relatively close proximity (e.g., US Midwest and East regions). But with this option, you can add read replica nodes to distant geographic locations (e.g., Europe and Asia) to improve performance for read workloads.
Single Geo-Partitioned Cluster: The database cluster is spread across multiple distant geographic locations (e.g. North America, Europe, and Asia). Each geographic location has its own group of nodes deployed across one or more local regions in close proximity (e.g., US Midwest and East regions). Data is automatically pinned to a specific group of nodes based on the value of a geo-partitioning column. With this deployment option, you achieve low latency for both read and write workloads across the globe.
Multiple Clusters With Async Replication: Multiple standalone clusters are deployed in various regions. The regions can be located in relatively close proximity (e.g., US Midwest and East regions) or in distant locations (e.g., US East and Asia South regions). The changes are replicated asynchronously between the clusters. You will achieve low latency for both read and write workloads across the globe, the same as with the previous option, but you will deal with multiple standalone clusters that exchange changes asynchronously.

Alright, matey, let’s move forward and review the first three deployment options for the geo-messenger’s use case. I will skip the fourth one since it doesn’t fit the messenger’s architecture which requires a single database cluster.

Single Stretched Cluster in the USA

The first cluster spans three regions in the USA—US West, Central, and East.

Application/microservice instances and the API servers (not shown in the picture) are running in those same locations plus in the Europe West and Asia East regions.

Database Read/Write Latency for the US Traffic

Suppose Ms. Blue is working from Iowa, US this week. She opens the geo-messenger to send a note to a corporate channel. Her traffic will be processed by the microservice instance deployed in the US Central region.

Before Ms. Blue can send the message, the US Central microservice instance has to load the channel’s message history. Which database node will be serving that read request?

In my case, the US Central region is configured as a preferred one for YugabyteDB, meaning a database node from that region will handle all the read requests from the application layer. It takes 10-15 ms to load the channel’s message history from that US Central database node to the application instance from the same region. Here is the output of my Spring Boot log with the last line showing the query execution time:

Hibernate: select message0_.country_code as country_1_1_, message0_.id as id2_1_, message0_.attachment as attachme3_1_, message0_.channel_id as channel_4_1_, message0_.message as message5_1_, message0_.sender_country_code as sender_c6_1_, message0_.sender_id as sender_i7_1_, message0_.sent_at as sent_at8_1_ from message message0_ where message0_.channel_id=? order by message0_.id asc

INFO 11744 --- [-nio-80-exec-10] i.StatisticalLoggingSessionEventListener : Session Metrics {
1337790 nanoseconds spent acquiring 1 JDBC connections;
0 nanoseconds spent releasing 0 JDBC connections;
1413081 nanoseconds spent preparing 1 JDBC statements;
14788369 nanoseconds spent executing 1 JDBC statements; (14 ms!)

Next, when Ms. Blue sends the message into the channel, the latency between the microservice and database will be around 90 ms. It takes more time than the previous operation because Hibernate generates multiple SQL queries for my JpaRepository.save(message) method call (which certainly can be optimized), and then YugabyteDB needs to store a copy of the message on all nodes running across the US West, Central, and East. Here is what the output and latency look like:

Hibernate: select message0_.country_code as country_1_1_0_, message0_.id as id2_1_0_, message0_.attachment as attachme3_1_0_, message0_.channel_id as channel_4_1_0_, message0_.message as message5_1_0_, message0_.sender_country_code as sender_c6_1_0_, message0_.sender_id as sender_i7_1_0_, message0_.sent_at as sent_at8_1_0_ from message message0_ where message0_.country_code=? and message0_.id=?
Hibernate: select nextval ('message_id_seq')
Hibernate: insert into message (attachment, channel_id, message, sender_country_code, sender_id, sent_at, country_code, id) values (?, ?, ?, ?, ?, ?, ?, ?)

31908 nanoseconds spent acquiring 1 JDBC connections;
0 nanoseconds spent releasing 0 JDBC connections;
461058 nanoseconds spent preparing 3 JDBC statements;
91272173 nanoseconds spent executing 3 JDBC statements; (90 ms)

Database Read/Write Latency for the APAC Traffic

Remember that with every multi-region database deployment there will be some tradeoffs. With the current database deployment, the read/write latency is low for the US-based traffic but high for the traffic originating from other, more distant locations. Let’s look at an example.

Imagine Mr. Red, a colleague of Ms. Blue, receives a push notification about the latest message from Ms. Blue. Since Mr. Red is on a business trip in Taiwan, his traffic will be handled by the instance of the app deployed on that island.

However, there is no database node deployed in or near Taiwan, so the microservice instance has to query the database nodes running in the US. This is why it takes 165 ms on average to load the entire channel’s history before Mr. Red sees the message of Ms. Blue:

Hibernate: select message0_.country_code as country_1_1_, message0_.id as id2_1_, message0_.attachment as attachme3_1_, message0_.channel_id as channel_4_1_, message0_.message as message5_1_, message0_.sender_country_code as sender_c6_1_, message0_.sender_id as sender_i7_1_, message0_.sent_at as sent_at8_1_ from message message0_ where message0_.channel_id=? order by message0_.id asc

[p-nio-80-exec-8] i.StatisticalLoggingSessionEventListener : Session Metrics {
153152267 nanoseconds spent acquiring 1 JDBC connections;
0 nanoseconds spent releasing 0 JDBC connections;
153217915 nanoseconds spent preparing 1 JDBC statements;
165798894 nanoseconds spent executing 1 JDBC statements; (165 ms)

When Mr. Red responds to Ms. Blue in the same channel, it takes about 450 ms for Hibernate to prepare, send, and store the message in the database. Well, thanks to the laws of physics, the packet(s) with the message has to travel through the Pacific Ocean from Taiwan, and then the message copy has to be stored across US West, Central, and East:

Hibernate: select message0_.country_code as country_1_1_0_, message0_.id as id2_1_0_, message0_.attachment as attachme3_1_0_, message0_.channel_id as channel_4_1_0_, message0_.message as message5_1_0_, message0_.sender_country_code as sender_c6_1_0_, message0_.sender_id as sender_i7_1_0_, message0_.sent_at as sent_at8_1_0_ from message message0_ where message0_.country_code=? and message0_.id=?
select nextval ('message_id_seq')
insert into message (attachment, channel_id, message, sender_country_code, sender_id, sent_at, country_code, id) values (?, ?, ?, ?, ?, ?, ?, ?)

 i.StatisticalLoggingSessionEventListener : Session Metrics 
    23488 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    137247 nanoseconds spent preparing 3 JDBC statements;
    454281135 nanoseconds spent executing 3 JDBC statements (450 ms);

Now high latency for APAC-based traffic might not be a big deal for applications that don’t target users from the region, but that’s not so in my case. My geo-messenger has to function smoothly across the globe. Let’s fight this high latency, matey! We’ll start with the reads!

Deploying Read Replicas in Distant Locations

The most straightforward way to improve the latency for read workloads in YugabyteDB is by deploying read replica nodes in distant locations. This is a purely operational task that can be performed on a live cluster.

So, I “attached” a replica node to my live US-based database cluster, and that replica was placed in the Asia East region nearby the microservice instance that serves requests for Mr.Red.

I then requested the application instance from Taiwan to use that replica node for database queries. The latency time for the channel’s history preloading for Mr.Red dropped from 165 ms to 10-15 ms! This is as fast as for Ms. Blue, who is based in the USA.

Hibernate: select message0_.country_code as country_1_1_, message0_.id as id2_1_, message0_.attachment as attachme3_1_, message0_.channel_id as channel_4_1_, message0_.message as message5_1_, message0_.sender_country_code as sender_c6_1_, message0_.sender_id as sender_i7_1_, message0_.sent_at as sent_at8_1_ from message message0_ where message0_.channel_id=? order by message0_.id asc

 i.StatisticalLoggingSessionEventListener : Session Metrics 
    1210615 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    1255989 nanoseconds spent preparing 1 JDBC statements;
    12772870 nanoseconds spent executing 1 JDBC statements; (12 mseconds)

As a result, with read replicas, my geo-messenger can serve read requests at low latency regardless of the user’s location!

But it’s too soon for a celebration. The writes are still way too slow.

Imagine that Mr.Red sends another message to the channel. The microservice instance from Taiwan will ask the replica node to execute the query. And the replica will forward almost all requests generated by Hibernate to the US-based nodes that store primary copies of records. Thus, the latency still can be as high as 640 ms for the APAC traffic:

Hibernate: set transaction read write;
Hibernate: select message0_.country_code as country_1_1_0_, message0_.id as id2_1_0_, message0_.attachment as attachme3_1_0_, message0_.channel_id as channel_4_1_0_, message0_.message as message5_1_0_, message0_.sender_country_code as sender_c6_1_0_, message0_.sender_id as sender_i7_1_0_, message0_.sent_at as sent_at8_1_0_ from message message0_ where message0_.country_code=? and message0_.id=?
Hibernate: select nextval ('message_id_seq')
Hibernate: insert into message (attachment, channel_id, message, sender_country_code, sender_id, sent_at, country_code, id) values (?, ?, ?, ?, ?, ?, ?, ?)

 i.StatisticalLoggingSessionEventListener : Session Metrics 
    23215 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    141888 nanoseconds spent preparing 4 JDBC statements;
    640199316 nanoseconds spent executing 4 JDBC statements; (640 ms)

At last, matey, let’s solve this issue once and for all!

Switching to Global Geo-Partitioned Cluster

A global geo-partitioned cluster can deliver fast reads and writes across distant locations but requires you to introduce a special geo-partitioning column to the database schema. Based on the value of the column, the database will automatically decide what database node in what geography a record belongs to.

Database Schema Changes

In a nutshell, my tables, such as the Message one, define the country_code column:

CREATE TABLE Message(
    id integer NOT NULL DEFAULT nextval('message_id_seq'),
    channel_id integer,
    sender_id integer NOT NULL,
    message text NOT NULL,
    attachment boolean NOT NULL DEFAULT FALSE,
    sent_at TIMESTAMP NOT NULL DEFAULT NOW(),
    country_code varchar(3) NOT NULL
) PARTITION BY LIST (country_code);

Depending on the value of that column, a record can be placed in one of the following database partitions:

CREATE TABLE Message_USA
    PARTITION OF Message
    (id, channel_id, sender_id, message, sent_at, country_code, sender_country_code,
    PRIMARY KEY(id, country_code))
    FOR VALUES IN ('USA') TABLESPACE us_central1_ts;

CREATE TABLE Message_EU
    PARTITION OF Message
    (id, channel_id, sender_id, message, sent_at, country_code, sender_country_code,
    PRIMARY KEY(id, country_code))
    FOR VALUES IN ('DEU') TABLESPACE europe_west3_ts;

CREATE TABLE Message_APAC
    PARTITION OF Message
    (id, channel_id, sender_id, message, sent_at, country_code, sender_country_code,
    PRIMARY KEY(id, country_code))
    FOR VALUES IN ('TWN') TABLESPACE asia_east1_ts;

Each partition is mapped to one of the tablespaces:

CREATE TABLESPACE us_central1_ts WITH (
  replica_placement='{"num_replicas": 1, "placement_blocks":
  [{"cloud":"gcp","region":"us-central1","zone":"us-central1-b","min_num_replicas":1}]}'
);

CREATE TABLESPACE europe_west3_ts WITH (
  replica_placement='{"num_replicas": 1, "placement_blocks":
  [{"cloud":"gcp","region":"europe-west3","zone":"europe-west3-b","min_num_replicas":1}]}'
);

CREATE TABLESPACE asia_east1_ts WITH (
  replica_placement='{"num_replicas": 1, "placement_blocks":
  [{"cloud":"gcp","region":"asia-east1","zone":"asia-east1-b","min_num_replicas":1}]}'
);

And each tablespace belongs to a group of database nodes from a particular geography. For instance, all the records with the country code of Taiwan (country_code=TWN) will be stored on cluster nodes from the Asia East cloud region because those nodes hold the partition and tablespace for the APAC data. Check out the following article if you want to learn the details of geo-partitioning.

Low Read/Write Latency Across Continents

So, I deployed a three-node geo-partitioned cluster across the US Central, Europe West, and Asia East regions.

Now, let's ensure that the read latency for Mr. Red's requests remains intact. As with the read replica deployment, it still takes 5-15 ms to load the channel's message history (for the channels and messages belonging to the APAC region):

Hibernate: select message0_.country_code as country_1_1_, message0_.id as id2_1_, message0_.attachment as attachme3_1_, message0_.channel_id as channel_4_1_, message0_.message as message5_1_, message0_.sender_country_code as sender_c6_1_, message0_.sender_id as sender_i7_1_, message0_.sent_at as sent_at8_1_ from message message0_ where message0_.channel_id=? and message0_.country_code=? order by message0_.id asc

 i.StatisticalLoggingSessionEventListener : Session Metrics 
1516450 nanoseconds spent acquiring 1 JDBC connections;
  0 nanoseconds spent releasing 0 JDBC connections;
  1640860 nanoseconds spent preparing 1 JDBC statements;
  7495719 nanoseconds spent executing 1 JDBC statements; (7 ms)

And…drumroll please….when Mr. Red posts a message into an APAC channel, the writes latency has dropped from 400-650 ms to 6 ms on average!

Hibernate: select message0_.country_code as country_1_1_0_, message0_.id as id2_1_0_, message0_.attachment as attachme3_1_0_, message0_.channel_id as channel_4_1_0_, message0_.message as message5_1_0_, message0_.sender_country_code as sender_c6_1_0_, message0_.sender_id as sender_i7_1_0_, message0_.sent_at as sent_at8_1_0_ from message message0_ where message0_.country_code=? and message0_.id=?
Hibernate: select nextval ('message_id_seq')
Hibernate: insert into message (attachment, channel_id, message, sender_country_code, sender_id, sent_at, country_code, id) values (?, ?, ?, ?, ?, ?, ?, ?)

1123280 nanoseconds spent acquiring 1 JDBC connections;
  0 nanoseconds spent releasing 0 JDBC connections;
  123249 nanoseconds spent preparing 3 JDBC statements;
  6597471 nanoseconds spent executing 3 JDBC statements; (6 ms)

Mission accomplished, matey! Now my geo-messenger’s database can serve reads and writes with low latency across countries and continents. I just need to tell my messenger where to deploy microservice instances and database nodes.

The Case for Cross-Continent Queries

Now for a quick comment on why I skipped the database deployment option with multiple standalone YugabyteDB clusters.

It was important to me to have a single database for the messenger so that:

Users could join discussion channels belonging to any geography.
Microservice instances from any location could access data in any location.

For instance, if Mr. Red joins a discussion channel belonging to the US nodes (country_code=’USA') and sends a message there, the microservice instance from Taiwan will send the request to the Taiwan-based database node and that node will forward the request to the US-based counterpart. The latency for this operation is comprised of three SQL requests and will be around 165 ms:

Hibernate: select message0_.country_code as country_1_1_0_, message0_.id as id2_1_0_, message0_.attachment as attachme3_1_0_, message0_.channel_id as channel_4_1_0_, message0_.message as message5_1_0_, message0_.sender_country_code as sender_c6_1_0_, message0_.sender_id as sender_i7_1_0_, message0_.sent_at as sent_at8_1_0_ from message message0_ where message0_.country_code=? and message0_.id=?
Hibernate: select nextval ('message_id_seq')
Hibernate: insert into message (attachment, channel_id, message, sender_country_code, sender_id, sent_at, country_code, id) values (?, ?, ?, ?, ?, ?, ?, ?)

 i.StatisticalLoggingSessionEventListener : Session Metrics 
1310550 nanoseconds spent acquiring 1 JDBC connections;
  0 nanoseconds spent releasing 0 JDBC connections;
  159080 nanoseconds spent preparing 3 JDBC statements;
  164660288 nanoseconds spent executing 3 JDBC statements; (164 ms)

165 ms is undoubtedly higher than 6 ms (the latency when Mr. Red posts a message into a local APAC-based channel), but what’s important here is the ability to make cross-continent requests via a single database connection when necessary. Plus, as the execution plan shows, there is a lot of room for optimization at the Hibernate level. Presently, Hibernate translates my JpaRepository.save(message) call into 3 JDBC statements. This is what can be optimized further to bring down the latency for the cross-continent requests from 165 ms to a much lower value.

Wrapping Up

Alright, matey!

As you see, distributed databases such as YugabyteDB can function seamlessly across geographies. You need to pick a deployment option that works best for your application. In my case, the geo-partitioned YugabyteDB cluster fits the most for the messenger's requirements.

Well, this article ends the series about my geo-messenger's development journey. The application source code is available on GitHub so that you can explore the logic and run the app in your environment.

Next, I'll take a little break and then start preparing for my upcoming SpringOne session that will feature the geo-messenger's version running on Spring Cloud components. So, if it's relevant to you, follow me to get notified about future articles related to Spring Cloud and geo-distributed apps.

Using Global Cloud Load Balancer to Route User Requests to the Closest App Instance

Denis Magda — Tue, 25 Oct 2022 15:17:12 +0000

Ahoy, Mateys!

In my last two articles, I showed you how to deploy microservice instances across several cloud regions and how to set up a geo-distributed API layer.

All seems good, except that I’ve now got several standalone application instances running across the US West, Central, and East regions, and each instance has its own private IP address.

My next challenge is figuring out how to forward application requests to the instances closest to the user.

I don’t want my Web UI and mobile apps to keep track of all the IPs and decide how and where to route user requests. This is way too complicated and becomes unmanageable. Instead, the Web UI and mobile apps need to connect to a single IP address, which lets other components of my geo-distributed solution decide where to route a particular user request.

What could those components be, matey?

You’ve guessed it: usually, a proxy and load balancer.

In this article, I’ll show you how to configure a global cloud load balancer that serves as both a proxy and a load balancer. This type of load balancer comes with a single IP address that can be accessed from any location on earth and can route a request to the nearest (active!) application instance.

So, if you’re still with me on this journey, then, as the pirates used to say, “Weigh Anchor and Hoist the Mizzen!” which means, “Pull up the anchor and get this ship sailing!”

Explaining Global Cloud Balancer Usage

Before diving into details and technicalities, let’s use a few illustrations to see the value of the load balancer.

Suppose Ms. Blue, one of my application users, is vacationing in the Tampa Bay area of Florida. She promised not to open my corporate Slack-like messenger (the geo-distributed app I’m building) while on vacation.

She failed to keep her promise, but for a good reason. She couldn’t resist sharing a few pictures from the pristine beaches of Tampa Bay.

The diagram above shows what happens when Ms. Blue opens the app and sends a few gorgeous photos.

The app connects to the global load balancer's public IP address, 35.186.215.175. Next, the load balancer determines that the application instance with IP 10.1.12.8 is closest to Ms. Blue (both Ms. Blue and the instance are on the US East coast). Finally, the load balancer asks the East Coast instance to upload the pictures.

Next, Mr. Green, a colleague of Ms. Blue, working from his Seattle-based company office, receives a push notification from the messenger and can’t resist checking what’s been posted.

Mr. Green opens the mobile app and the app connects to the same load balancer using the same public IP address, 35.186.215.175. This time, the load balancer forwards Mr. Green’s request to the application instance from the West Coast that uses 10.1.11.16 IP address.

Also, the load balancer is instrumental in case of cloud outages. Let’s consider a final example.

A few days later, Ms. Blue decides to make her colleagues jealous again and opens the app to send another set of fabulous pictures. The application connects to the load balancer on the same public IP address (35.186.215.175) and…

…this time the load balancer forwards Ms. Blue’s request to the application instance in the US Central with IP address 10.1.10.10.

This is because a storm in the Atlantic impacted the availability of the entire US East Coast cloud region. The US Central region is up and running as normal, and is now the closest for Ms. Blue. So, the request is routed there automatically.

Alright, matey, now let’s move forward and look into the architecture, or the main building blocks, of the global cloud load balancer.

Global Load Balancer Architecture

The global load balancer is comprised of several building blocks. Take a look at the picture below and review the components from left to right.

This picture represents the architecture of the load balancer in Google Cloud. The components might vary depending on your cloud environment. But you can be sure that every major cloud environment supports the global load balancer.

So, moving from left to right in the picture: As you already know, users connect to the public IP address assigned to the load balancer (or DNS name that translates to this address). After that, the load balancer consults with existing forwarding rules to determine which target proxy needs to handle the request.

Once the target proxy is selected, the proxy checks its URL map to decide which backend services are responsible for the request.

Finally, the backend service delegates the request to one of its backends (actual application instances). In our case, during normal operations, requests from Mr. Green, based in Seattle, will be processed by the US West backend instance. The traffic from Ms. Blue will go to the backend in the US East region.

If you wish to learn more about the global load balancer’s architecture, check out this documentation for an extensive overview of all its components.

Global Load Balancer Deployment

At last, matey, let’s go ahead and deploy an instance of the global external load balancer in Google Cloud.

Assuming that applications instances are already running across US West, Central, and East, these are our next steps (explore how to deploy the instances in Google Cloud):

Reserve a static public IP address for the load balancer:

gcloud compute addresses create load-balancer-public-ip \
    --ip-version=IPV4 \
    --network-tier=PREMIUM \
        --global

Configure a health check for a backend service:

gcloud compute health-checks create http load-balancer-http-basic-check \
    --check-interval=20s --timeout=5s \
    --healthy-threshold=2 --unhealthy-threshold=2 \
    --request-path=/login \
        --port 80

Create the backend service for the HTTP traffic and provide the just-created health check to ensure the application instances are up and running:

gcloud compute backend-services create load-balancer-backend-service \
    --load-balancing-scheme=EXTERNAL_MANAGED \
    --protocol=HTTP \
    --port-name=http \
    --health-checks=load-balancer-http-basic-check \
    --global

Provide the backend service with backends (application instances) across several cloud regions:

gcloud compute backend-services add-backend load-balancer-backend-service \
    --balancing-mode=UTILIZATION \
    --max-utilization=0.8 \
    --capacity-scaler=1 \
    --instance-group=ig-us-central \
    --instance-group-zone=us-central1-b \
    --global

gcloud compute backend-services add-backend load-balancer-backend-service \
    --balancing-mode=UTILIZATION \
    --max-utilization=0.8 \
    --capacity-scaler=1 \
    --instance-group=ig-us-east \
    --instance-group-zone=us-east4-b \
    --global

gcloud compute backend-services add-backend load-balancer-backend-service \
    --balancing-mode=UTILIZATION \
    --max-utilization=0.8 \
    --capacity-scaler=1 \
    --instance-group=ig-us-west \
    --instance-group-zone=us-west2-b \
        --global

Finally, configure the default URL map to route all requests to the created backend service:

gcloud compute url-maps create load-balancer-url-map --default-service load-balancer-backend-service

After the backend service is ready, go ahead and configure the frontend part of the load balancer which is the forwarding rule and target proxy.

Create an HTTP proxy providing the URL map:

gcloud compute target-http-proxies create load-balancer-http-frontend \
    --url-map load-balancer-url-map \
        --global

Define a forwarding rule to direct the traffic from the public Internet to the just created proxy:

gcloud compute forwarding-rules create load-balancer-http-frontend-forwarding-rule \
    --load-balancing-scheme=EXTERNAL_MANAGED \
    --network-tier=PREMIUM \
    --address=load-balancer-public-ip  \
    --global \
    --target-http-proxy=load-balancer-http-frontend \
    --ports=80

That’s it! The global external load balancer is fully configured. It might just take a few minutes for your configuration to propagate worldwide.

Testing Load Balancer

Now, it’s testing time!

I’ll play the role of Ms. Blue, who wants to share gorgeous photos from Tampa Bay. A colleague of mine who is based in Seattle will be Mr. Green.

The manual testing is straightforward. I’m opening the messenger in my browser using the load balancer’s public IP address and uploading a few pictures:

How do I ensure those pictures were uploaded through an instance in the US East?

Well, I opened the monitoring tab for the load balancer and confirmed all my requests are delegated to the ig-us-east instance:

After that, my Seattle colleague opens the app to check the pictures. In a few moments, I can see that the load balancer directs his traffic to the ig-us-west!

Job done!

The global load balancer is fully configured and functions globally. This means my geo-distributed messenger can seamlessly process user requests with high performance across the globe and withstand various cloud outages.

What’s on the Horizon?

Alright, matey!

My geo-distributed app now functions across multiple cloud regions and availability zones. But the app, API layer, and database still run within the boundaries of the USA.

My next move would be to scale the messenger to other countries and continents and experiment with several YugabyteDB deployment options (my geo-distributed database).

Follow me to be notified as soon as the next update is published! And check out the previous articles in my geo-distributed messenger application development journey.

Geo-Distributed API Layer With Kong Gateway

Denis Magda — Tue, 18 Oct 2022 19:26:14 +0000

Ahoy, Mateys!

It’s been a while since I published my last article on my geo-distributed application development journey. A hot tech conference season put my development on hold, but now I’m back home and ready to share my experience in building a geo-distributed API layer with Kong Gateway.

Let me take a moment to review where the project had gotten to. This is what my geo-distributed messenger’s architecture looked like:

I managed to automate the deployment of application instances across several cloud regions (to see how the project started and continued, here is a list of all previous articles. However, the application is not a monolith; it’s comprised of several microservices.

The Messaging microservice implements the key functionality that every messenger must possess—the ability to send messages across channels and workspaces. The Attachments microservice uploads pictures and other files. Finally, the Reminders microservice comes in handy when a user wants to ignore a message for now but get back to it later.

So now, mateys, we reach the question, “What do all microservices need?”

Well, they need to talk to each other! Microservices were not born to live in isolation. After following Kong for more than a year, I selected it to create a geo-distributed API layer that can be used as a synchronous communication layer by the microservices.

In this article, we’ll review several ways to build a geo-distributed API layer with Kong. My geo-messenger needs an API layer that can withstand various cloud outages (including regional ones) and service requests from the locations closest to the users.

So, if you’re still with me on this journey, then, as the pirates used to say, “Weigh Anchor and Hoist the Mizzen!” which means, “Pull up the anchor and get this ship sailing!”

Connecting Microservices With Kong Gateway

For the sake of simplicity, I’ll use two microservice as an example.

A user request always goes to the Messaging microservice first, the key one. But, if the user wants to share a picture in a chat channel, for example, the Messaging microservice delegates this job to its Attachments counterpart. In this case, the Kong Gateway is used as a communication channel.

As long as my microservices instances are deployed across multiple cloud regions, I have to go through the same exercise for Kong Gateway.

Let’s discuss the options for running Kong in a geo-distributed way.

Option #1: Deploying Kong as Sidecar Nearby Microservices

The most straightforward way is to deploy Kong Gateway as a sidecar on the same virtual machine (VM) with your microservices. Then you can deploy the VMs with Kong and the microservices across several cloud regions. (NOTE: you can use Kubernetes instead if you like; there are no major architectural differences).

In this example, the geo-messenger uses two VM instances—one in the US East and another in the US West. Each VM runs an instance of the microservice and Kong Gateway. In addition, every VM has its own instance of PostgreSQL which is used solely by Kong to store information about microservices and supported routes.

Google Cloud Storage and YugabyteDB are geo-distributed by definition. Google Cloud Storage keeps attachments that users upload through the Attachments microservice, while YugabyteDB keeps all other application data (messages, channels, reminders, etc.)

The user is connected to the VM closest to their location. In this example, the user is based near New York, so her requests are routed to the VM in the US East region. This is how I achieve the lowest latency possible.

If the US East region becomes unavailable due to a region-level outage, then I’ll lose the VM from that region. But the messenger will continue working with no interruptions because the user requests will now be routed to the VM in the US West.

The Show, Don’t Tell!

Now, let me demonstrate this deployment option.

When I start a VM in Google Cloud, the VM executes a special startup script that installs required libraries, builds and runs the microservices, and configures Kong Gateway. These are the steps for Kong setup:

Download and install Kong Gateway:

  echo "deb [trusted=yes] https://download.konghq.com/gateway-3.x-ubuntu-$(lsb_release -sc)/ \
  default all" | sudo tee /etc/apt/sources.list.d/kong.list
  sudo apt-get update
  sudo apt install --yes --force-yes kong-enterprise-edition=3.0.0.0

Configure Postgres and run Kong migrations:

  sudo -u postgres psql -c "CREATE USER kong WITH PASSWORD 'password'"
  sudo -u postgres psql -c "CREATE DATABASE kong OWNER kong"

  sudo touch /etc/kong/kong.conf
  echo 'pg_user = kong' | sudo tee --append /etc/kong/kong.conf
  echo 'pg_password = password' | sudo tee --append /etc/kong/kong.conf
  echo 'pg_database = kong' | sudo tee --append /etc/kong/kong.conf

  sudo kong migrations bootstrap -c /etc/kong/kong.conf
  sudo kong migrations up -c /etc/kong/kong.conf

Create a Kong route for attachments uploading:

  curl -i -X POST \
          --url http://localhost:8001/services/ \
          --data 'name=attachments-service' \
          --data 'url=http://127.0.0.1:8081'

  curl -i -X POST http://localhost:8001/services/attachments-service/routes \
       -d "name=upload-route" \
       -d "paths[]=/upload" \
       -d "strip_path=false"

Note, all IPs are local because Kong runs on the same VM with my microservices.

Since you have that startup script that prepares a VM with a required configuration, you need to figure out how to deploy such VMs across multiple cloud regions.

Easy! This step can be automated as well:

Call this script to create a VM instance template in the regions of your choice. I create templates for US West, Central, and East.
Provision VMs with Kong and microservices in all those regions:

  gcloud compute instance-groups managed create ig-us-west \
      --template=template-us-west --size=1 --zone=us-west2-b

  gcloud compute instance-groups managed create ig-us-central \
      --template=template-us-central --size=1 --zone=us-central1-b

  gcloud compute instance-groups managed create ig-us-east \
    --template=template-us-east --size=1 --zone=us-east4-b

Next, make sure that the VMs are booted and run normally across the cloud regions:

Finally, pick any of the VMs and test the application by uploading a picture into a chat channel. The Messaging microservice calls the configured Kong route to delegate this task to the Attachments microservice.

Easy, mateys, isn’t it? Now, let’s review two other alternatives for Kong geo-distributed deployments.

Option #2: Kong Standalone Deployment With Shared Database

Another approach is better for applications with a significant number of microservices, or that need to deploy microservices and the API layer on separate VMs (or Kubernetes nodes/pods).

As the diagram shows, we continue running our application across two cloud regions – US East and West. We also continue using Google Cloud Storage and YugabyteDB for application data and attachments. But now, the microservice and Kong Gateway instances are deployed in standalone VMs.

Every microservice and Kong instance is location-aware. For example, when the user from New York wants to share a picture, the request will go to the Messaging instance in the US East (which is closest to the user). Then, that Messaging instance must forward the request to the Attachments service from the same cloud region.

This happens by connecting to the Kong instance from US East and calling the /api/us/east/upload API endpoint. That endpoint is a regional one (see /us/east part in the API URL) and is set up to forward requests to the Attachments microservices from US East.

Finally, Kong stores all the information about routes and services in a shared instance of PostgreSQL. If you use a PostgreSQL-managed service like Google Cloud SQL, it’s possible to configure PostgreSQL replica instances in a different availability zone.

Cloud SQL will failover to the replica if a zone running the primary PostgreSQL instance becomes unavailable. The primary and replica must be in the same cloud region. Presently, Google Cloud SQL doesn’t tolerate region-level outages.

Option #3: Kong Standalone Deployment With Distributed Database

Deployment option #3 is close to deployment option #2. The only exception is the database where Kong Gateway stores details about routes, services, and other system information. Option #2 used PostgreSQL, while this third option uses Apache Cassandra.

Kong supports Apache Cassandra for geo-distributed use cases. A Cassandra cluster can span multiple cloud regions, distributing the load across several distant locations and withstanding region-level outages that are not supported by Google Cloud SQL (managed PostgreSQL).

However, starting with Kong 4.0, PostgreSQL will be the only officially supported database. The Cassandra integration is already deprecated.

The Kong community will likely find a replacement for Cassandra in the foreseeable future. And that replacement might be YugabyteDB, which is built on the PostgreSQL source code and frequently referred to as distributed PostgreSQL.

What’s on the Horizon?

Alright, mateys! Now I’ve got my microservices and API layer working across multiple cloud regions, what’s next?

Well, I need to figure out how to configure the Global Cloud Load Balancer that intercepts user requests at the nearest PoP (point-of-presence) and then routes the requests to one of my application instances (VMs).

Follow me to be notified as soon as the next update is published! And check out the previous articles in my development journey.

Automating Java Application Deployment Across Multiple Cloud Regions

Denis Magda — Fri, 02 Sep 2022 20:10:05 +0000

Ahoy, matey!

What’s good about habits, I hear you ask?

Well, some habits are better (and more socially acceptable) than others! In terms of coding, a habit or routine can help you progress more quickly toward your end goal. It’s better to take daily baby steps than wait for that “perfect” moment.

I developed the habit of coding the geo-distributed messenger two days a week (strictly during business hours, as evenings are for my mischievous sons and dear wife!). So, here is the latest summary, for those of you who are following my development journey.

This is what my end goal looks like:

My geo-messenger has to function across the globe, storing data and serving user requests from multiple cloud regions.

And, how far have I progressed since the beginning of the project? Well, a week ago, my application looked like this:

The app ran in Heroku and used a multi-node YugabyteDB database cluster. The app instance and database were deployed in a single cloud region and could tolerate zone-level outages.

What happened a few days ago? Well, I continued taking baby steps toward my end goal and managed to deploy several geo-messenger instances across multiple cloud regions. So, now my app looks like this:

Several app instances run in the US West, Central, and East regions. I can use any of them to serve user traffic. In the case of regional outages, one instance could go down, but the availability of other instances won’t be disrupted. However, the database still works out of a single cloud region and is vulnerable to region-level incidents, but I’ll handle this later.

So, if you’re still with me on this journey, then, as the pirates used to say, “All Hand Hoy!” which means, “Everyone on deck!”

Let’s review the various connectivity options that I validated with my app. I’ll show you how I deploy those instances across several regions of Google Cloud. As a spoiler, I’ll give you advance notice that I don’t use Google Console (UI).

Creating Google Cloud Project

All of the resources used by my application instances are located in a dedicated Google Cloud project.

Speaking of the deployment automation approach, try to guess what I selected so far…

“Terraform?” No!

“Maybe an Ansible script?” Nope, wrong again.

“Come on, tell me that you at least deploy the app in Kubernetes!” Sorry, matey, nothing like that.

In fact, I chose gcloud CLI, which allows me to create a project, configure the network, and provision VMs via easy-to-follow commands. This is how I began to create my project for the geo-messenger:

gcloud projects create geo-distributed-messenger --name="Geo-Distributed Messenger"

Call me old-fashioned, but I don’t need to worry about Terraform, Ansible, and K8 yet. My priority is to get my app working across geographies in Google Cloud, so I want to go with the fastest approach. Once I reach my initial end goal, I can set another goal to support AWS, Azure, and Oracle Cloud, then use Terraform and other technologies to make the deployment cloud-agnostic.

Defining Firewall Rules

Whenever you create a new project in Google Cloud, you get the default virtual private cloud (VPC). The virtual machines (VMs) that will host my application instances will run within that VPC.

I created a firewall rule to allow SSH, HTTP and HTTPS traffic:

gcloud compute --project=geo-distributed-messenger firewall-rules create geo-messenger-allowed-traffic \
    --direction=INGRESS --priority=1000 --network=default \
    --action=ALLOW --rules=tcp:22,tcp:80,tcp:443 \
    --source-ranges=0.0.0.0/0 --target-tags=geo-messenger-instance

The rule will apply to all the VMs labeled “geo-messenger-instance.” The other parameters are self-explanatory.

BTW, the default network is the Premium one. How is this different from the Standard tier? Check out this illustration:

For my multi-region app, I need the premium network so that user traffic enters the Google global network at the nearest point-of-presence (PoP) and gets to my app instance fast.

Deploying the First App Instance

A gcloud command for the VM provisioning comes with more parameters than the two reviewed above. So, I put together a start_single_app_instance.sh script that starts a VM:

./start_single_app_instance.sh \
    -n {INSTANCE_NAME} \
    -z {CLOUD_ZONE_NAME} \
    -a {APP_HTTP_PORT_NUMBER} \
    -c "{DATABASE_CONNECTION_ENDPOINT}" \
    -u {DATABASE_USER} \
    -p {DATABASE_PWD}

As an example, this is how I start a VM in the US West and connect it to my YugabyteDB cluster in the US East:

./start_single_app_instance.sh \
    -n messenger-us-west-instance \
    -z us-west2-a \
    -a 80 \
    -c "jdbc:postgresql://us-east1.my-instance-id.gcp.ybdb.io:5433/yugabyte?ssl=true&sslmode=require" \
    -u my-user-name \
    -p my-super-complex-password

After the VM is booted, it will execute the startup script that installs the required software and launches a geo-messenger instance on port 80.

Deploying More Instances

The previous step was the most time-consuming. I had to recall how to write bash scripts and advance my knowledge about the gcloud API. But, once I got the script for app instances deployment, it only took a few seconds to start the app instances in other geographic locations.

I used this command to launch an instance in the US Central region:

./start_single_app_instance.sh \
    -n messenger-us-central-instance \
    -z us-central1-a \
    -a 80 \
    -c "jdbc:postgresql://us-east1.my-instance-id.gcp.ybdb.io:5433/yugabyte?ssl=true&sslmode=require" \
    -u my-user-name \
    -p my-super-complex-password

This one launched an instance in the US East region, near Washington, DC:

./start_single_app_instance.sh \
    -n messenger-us-east-instance \
    -z us-east4-a \
    -a 80 \
    -c "jdbc:postgresql://us-east1.my-instance-id.gcp.ybdb.io:5433/yugabyte?ssl=true&sslmode=require" \
    -u my-user-name \
    -p my-super-complex-password

Finally, I confirmed that every instance could serve the user traffic from port 80 using…my own hands and the browser:

What’s on the Horizon?

Alright, matey! Now I’ve got my app working across multiple cloud regions, what’s next?

In my next blog I’ll talk though setting up the Global Cloud Load balancer to intercept user requests at the nearest PoP (point-of-presence) and then route them to one of my application instances. I’ll also provision a multi-region YugabyteDB cluster so that my app can withstand region-level outages.

If you want to read back over my progress check the series here!

How To Connect a Heroku Java App to a Cloud-Native Database

Denis Magda — Thu, 25 Aug 2022 20:11:00 +0000

Ahoy, matey! I'm back from a short vacation and ready to continue my pet project: geo-distributed messenger in Java!

In the last article, I launched the first version of my app, which runs in Heroku and uses YugabyteDB Managed as a cloud-native distributed database. I now feel confident that the geo-messenger can tolerate zone-level outages. Today, I’ll look at several connectivity options for Heroku and YugabyteDB Managed.

You might ask, "What's wrong with the connectivity between those two SaaS products?"

First, your YugabyteDB Managed instance won't be visible to the whole internet once deployed. You must provide the IP addresses of apps, services, VMs, etc. to connect to the database. This is a common and reasonable requirement for cloud-native databases. It's true for MongoDB Atlas, Amazon Aurora, and for any other cloud-native database that treats security seriously.

By default, the YugabyteDB Managed IPs allow list is empty. This means that my geo-messenger's requests will be rejected.

You might smile and say, “Come on, Denis, just find the static IP address of your app in Heroku and add it to YugabyteDB!”

This was my exact thinking, matey! However, Heroku does not provide static IP addresses in the Common Runtime Environment. So I either need to switch to an enterprise plan that includes private spaces or find another option. Being cost-conscious (ie: cheap), I selected the latter.

So if you’re still with me on this journey, then, as the pirates used to say, “All Hand Hoy!” which means, “Everyone on deck!” Let’s review the various connectivity options that I validated with my app.

Allow All Connections

A brute-force solution is to allow all connections to my YugabyteDB Managed instance. I did that by adding the 0.0.0.0/0 address to the IP allow list.

I recommend this option if you’re in early development (and want to focus on coding rather than infrastructure setup) or if you are deploying the complete solution in a VPC network. I was coding nonstop and didn’t want to be distracted, so I used the 0.0.0.0/0 solution as a shortcut.

Use SOCK5 Proxy

Once my coding slowed down, I decided to find a more elegant solution to the Heroku and YugabyteDB Managed connectivity issue. Obviously, I didn’t want my database instance to remain open to the entire internet.

What was my next step? Well, as an experienced engineer with nearly 20 years in the tech industry I knew exactly what to do: I opened a browser and Googled “heroku static ip address java.”

I soon realized that this problem is so widespread that the Heroku marketplace is full of add-ons that can proxy Heroku requests via static IP addresses. (Btw, a good business opportunity if you want to offer your proxy: how about founding a startup?)

I then wasted an hour trying out different proxies. Nothing worked. YugabyteDB rejected the requests from my geo-messenger. I scratched my head, then scratched it again. Before scratching my head for the third time, I realized that all those proxy add-ons work for HTTP traffic only: REST, GraphQL, and other protocols that rely on HTTP. My app uses a JDBC driver that opens a direct socket connection to the database and exchanges messages in the PostgreSQL wire-level protocol.

What was my next step? I’m sure you’ve guessed! I turned to Google again, this time searching for “heroku socks5 proxy for postgres," and finally came across the Fixie Socks add-on that worked for me.

The step-by-step instructions are as follows:

I installed the Fixie Socks add-on heroku addons:create fixie-socks:handlebar -a geo-distributed-messenger. You can swap out “handlebar” for another tier. This one costs me $9 a month for 2,000 requests. There is also a free option for 100 requests a month called “grip.”
I found my static IPs on the dashboard that you can launch with this command: heroku addons:open fixie-socks.
I added those IPs to the YugabyteDB Managed IP allow lists.
Then I introduced the custom environment variable USE_FIXIE_SOCKS which instructs my app to either use or bypass the proxy heroku config:set USE_FIXIE_SOCKS=true -a geo-distributed-messenger.

When USE_FIXIE_SOCKS is set to true, the app configures two JVM-level properties (socksProxyHost and socksProxyPort) asking Java to send all network requests through the proxy:

if (System.getenv("USE_FIXIE_SOCKS") != null) {
    boolean useFixie = Boolean.valueOf(System.getenv("USE_FIXIE_SOCKS"));

   if (useFixie) {
     System.out.println("Setting up Fixie Socks Proxy");

     // The FIXIE_SOCKS_HOST variable is added by the add-on to the environment
     String[] fixieData = System.getenv("FIXIE_SOCKS_HOST").split("@");
     String[] fixieCredentials = fixieData[0].split(":");
     String[] fixieUrl = fixieData[1].split(":");

     String fixieHost = fixieUrl[0];
     String fixiePort = fixieUrl[1];
     String fixieUser = fixieCredentials[0];
     String fixiePassword = fixieCredentials[1];

     System.setProperty("socksProxyHost", fixieHost);
     System.setProperty("socksProxyPort", fixiePort);

     System.out.println("Enabled Fixie Socks Proxy:" + fixieHost);

     Authenticator.setDefault(new ProxyAuthenticator(fixieUser, fixiePassword));
   }
 }

After restarting the app in Heroku, I could connect to YugabyteDB Managed and see application workspaces, channels, and messages.

Use Proxy for YugabyteDB Connections Only

Even though the SOCKS5 proxy method worked, I still was not fully satisfied with my implementation. What’s wrong? Just look at these two lines:

System.setProperty("socksProxyHost", fixieHost);
System.setProperty("socksProxyPort", fixiePort);

Those are JVM-related settings that require every TCP/IP connection to go through my proxy. That’s overkill. I only need the proxy for socket connections to YugabyteDB Managed.

Luckily, this task is easy to solve in Java. You just need to provide an implementation of a custom ProxySelector. This is what I did by introducing my own DatabaseProxySelector class:

public class DatabaseProxySelector extends ProxySelector {

    private String proxyHost;
    private int proxyPort;

    public DatabaseProxySelector(String proxyHost, int proxyPort) {
        this.proxyHost = proxyHost;
        this.proxyPort = proxyPort;
    }

    @Override
    public List<Proxy> select(URI uri) {
        // YugabyteDB Managed host always ends with `ybdb.io`
        if (uri.toString().contains("ybdb.io")) {
            System.out.println("Using the proxy for YugabyteDB Managed: " + uri);

            final InetSocketAddress proxyAddress = InetSocketAddress
                    .createUnresolved(proxyHost, proxyPort);
            return Collections.singletonList(new Proxy(Type.SOCKS, proxyAddress));
        }

        return Collections.singletonList(Proxy.NO_PROXY);
    }

    @Override
    public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
        new IOException("Failed to connect to the proxy", ioe).printStackTrace();
    }

}

As you can see, the DatabaseProxySelector enables the Fixie Socks proxy only for YugabyteDB Managed URIs. Finally, the selector instance is created on an application startup:

if (System.getenv("USE_FIXIE_SOCKS") != null) {
  boolean useFixie = Boolean.valueOf(System.getenv("USE_FIXIE_SOCKS"));

  if (useFixie) {
    System.out.println("Setting up Fixie Socks Proxy");

    String[] fixieData = System.getenv("FIXIE_SOCKS_HOST").split("@");
    String[] fixieCredentials = fixieData[0].split(":");
    String[] fixieUrl = fixieData[1].split(":");

    String fixieHost = fixieUrl[0];
    String fixiePort = fixieUrl[1];
    String fixieUser = fixieCredentials[0];
    String fixiePassword = fixieCredentials[1];

    DatabaseProxySelector proxySelector = new DatabaseProxySelector(fixieHost, Integer.parseInt(fixiePort));
    ProxySelector.setDefault(proxySelector);

    Authenticator.setDefault(new ProxyAuthenticator(fixieUser, fixiePassword));

    System.out.println("Enabled Fixie Socks Proxy:" + fixieHost);
  }
}

Explore Private Spaces

At the start, I mentioned that the Heroku Private Spaces feature should also work (if to believe Heroku’s technical documentation). For the sake of completeness, I’ve added this option to the article. Theoretically, you can set up those private spaces in Heroku and peer the spaces with a YugabyteDB Managed VPC network, but I’ll let you validate that!

What’s on the Horizon?

Alright, matey. This article concludes my thoughts on the current app version running in a single cloud region. Now, let me take a short break before I move on to the next milestone: next, I need the geo-messenger to function across several cloud regions. I’ll be looking at Google Cloud tools to automate the deployment. I’ll keep you posted!

In the meantime, if you’re interested in how my dev journey began (and is going), check out the previous articles in this series.

How to Build a Multi-Zone Java App in One Day

Denis Magda — Thu, 18 Aug 2022 20:09:00 +0000

Ahoy, matey! At last, the time has come to build and launch the first version of my geo-distributed Java application.

It took me around 24 hours in total to create this version. The app currently runs on Vaadin and Spring, it can use PostgreSQL or YugabyteDB as a database, and it either works locally or can be deployed in Heroku.

Don’t have a clue of what I'm talking about? Welcome to my dev journal where I’ve been documenting my experience of building a geo-distributed app in Java from scratch. In my two previous articles, I talked about geo-distributed apps and what their architecture looks like. In this blog, I’ll actually get things done, sharing my first results and any challenges.

So, if you’re with me on this journey, then, as the pirates used to say - “All Hand Hoy!” which means “Everyone on deck!”

My App - The Big Picture

Some of you might have noticed that I keep talking about building a generic geo-distributed Java app. But, what is my app gonna do? Well, I’m building a Slack-like corporate messenger.

Yep, it may sound like I’m reinventing the wheel, but perhaps I have a killer feature in mind, my special secret sauce that will overturn the dominance of Slack!

On a serious note, I’m just curious about how to design and build a geo-distributed corporate messenger (like Slack) that functions with low latency across the globe, tolerates cloud outages, and honors GDPR-like requirements. So, I thought I would try to make my own! Thus, at the end of my journey, I’m expecting to have this:

The messenger application instances and database nodes will be spread worldwide and run in the cloud regions closest to most users. The global load balancer will intercept user traffic and route requests to the application instances closest to the user.

But, that’s the big picture. The finish line, so they say. However, we start with baby steps…

My App - Day 1

While I have an ambitious goal for my geo-distributed messenger, the first version will be much more modest.

I bet that on day 1, my Slack-killer, like most startup companies out there, will not be a big success. In fact, it might be barely noticed. If I get 100 active users daily within the first two months, then it’s already a big win.

So, if I expect to have around 100 active users daily, why should I burn the money on infrastructure that spans the globe? It makes little sense. It’s far more prudent to start small and then grow big. That’s why the first version of my app looks like this:

My messenger will be deployed within a single cloud region in the US. It’s going to be a monolith running in Heroku. There will be a three-node YugabyteDB cluster running across three availability zones. With such architecture, I can withstand zone-level outages and serve the user requests of Ms. Blue and other US-based users at low latency.

Unfortunately, the latency for Mr. Green from Berlin and Mr. Red from Sydney will be high. But, as a startup, I can live with that, especially if I put most of my marketing dollars into growing the US-based user base.

Once the time comes, I can easily scale my current multi-zone geo-distributed app to a multi-region one. At least, I hope that it’s going to be easy. Time will tell…

Let me walk you through the main components of the first version of my messenger application. Here are its GitHub coordinates.

Vaadin: Backend and Frontend

As an engineer, I always look for shortcuts when creating web applications, especially on the frontend side. I can’t be called a web developer, because the Web is not what I’m nerding on daily.

It’s more like this… Every two years I join or start a web project and learn something new. I still remember creating my first web app in raw JavaScript with HTML+CSS, then jQuery was launched and was a big thing. Later, I used GWT for a few projects and during the last project got a chance to work on Angular.

Why did I pick Vaadin for this project? Well, even though I’m a polyglot, Java is my mother tongue. I built a special bond with Java after spending many years at Sun Microsystems and Oracle building JVM and JDK. Thus, as a guy who knows Java inside out, it’s always my first choice.

But, putting my personal attachment to Java aside, what do I like about Vaadin after using it for 4 days in a row? Vaadin is a full-stack framework. You can use it to build both the backend and frontend logic.

Firstly, as with many other frontend frameworks, you create/design pages and views using essential building blocks likes Button, Label, HorizontalLayout, etc. Then the framework translates your code, written in Java, to the JavaScript counterpart with a nice-looking UI. This is what I created:

Not bad for a guy whose design and front-end skills require some work, right? Right?!

Secondly, Vaadin doesn’t require me to run a separate backend instance and implement APIs for the frontend there. Instead, I just create my views (displayed in the browser) and add the server-side logic that the views then ask the backend to execute. For instance, this is the Button that sends a message on a button-click event:

sendMessageButton = new Button("Send");
        sendMessageButton.addClickListener(e -> {
            …. 
                Profile user = userOptional.get();

                Message newMessage = new Message();

                newMessage.setChannelId(currentChannel.getId());
                newMessage.setCountryCode(currentChannel.getCountryCode());
                newMessage.setSenderId(user.getId());
                newMessage.setSenderCountryCode(user.getCountryCode());

                newMessage.setMessage(newMessageArea.getValue().trim());

                newMessage = messagingService.addMessage(newMessage);
}

The listener logic is implemented in my custom UI component - MessageView. The listener gets triggered in the browser end, it’s obvious. Then Vaadin sends this event for processing to the backend side where a new Message is constructed and added to the database via the call to a Spring Framework’s Service entity - messagingService.addMessage(newMessage).

Nice! No need to create a separate REST API layer. Vaadin can do this for me.

PostgreSQL and YugabyteDB: Database

My database choice was driven by past experience and personal preferences.

PostgreSQL needs no introduction. This is my default go-to database for any project for the last 15+ years.

YugabyteDB might still be a dark horse for many. That’s the database I’m nerding on these days. It’s a distributed SQL database built on PostgreSQL. Basically, it’s a distributed PostgreSQL that can work across geographies. Exactly what I need for my geo-distributed app.

As long as YugabyteDB is Postgres-compliant, the app supports both databases out of the box. I use Postgres for my development environment and can always switch to a multi-node YugabyteDB cluster by tweaking the connectivity settings.

Wanna try? No problem, matey, follow these steps:

Launch a Postgres instance locally and create the messenger’s schema following these instructions
Then start the app using a familiar syntax: mvn spring-boot:run
Open http://localhost:8080 and enjoy this view

It takes time to start the app for the first time as the DataGenerator generates hundreds of Workspaces and Channels with thousands of Messages. But, after that, the startup time will be fast.

Next, if you want to use YugabyteDB for the dev environment as well just:

Start YugabyteDB locally
Tweak connectivity settings in the application-dev.properties file
Start the app! Enjoy!

Heroku and YugabyteDB Managed: Deploying to Prod

The deployment to the prod was the most straightforward part for me. I’m a lazy guy who uses cloud services whenever possible (or reasonable). My geo-messenger was no exception.

As you remember, the first version of the app is supposed to work within a single cloud region with database nodes at least in three availability zones. Zones fail frequently and I need to be ready for that. So, what have I done?

First, I deployed a multi-node YugabyteDB Managed cluster in South Carolina (us-east1) region with nodes in three availability zones - us-east1-b, us-east1-c and us-east1-d. Just in case, each node handles both reads and writes, there is no such a thing like “active and standby” nodes:

Next, after creating a Heroku project, I deployed my Java app somewhere in the USA:

Currently, Heroku is not transparent about the cloud region my app is assigned to. I can only hope that the app instance will be running close enough to my YugabyteDB cluster and that Heroku will take care of the app’s availability in case of cloud outages. I probably need to dig into Heroku internals here. Please share your thoughts in the comments if you know Heroku better than I do!

What’s On the Horizon?

The development of the first version of my geo-distributed messenger was a swift and fun experience. It’s nice to get a multi-zone app functioning within just four days of coding.

What do we have coming up next?

Performance in prod sucks. It takes 3-6 seconds to load messages when I switch between channels. I can’t blame Heroku or YugabyteDB for that as I was coding too fast and used Spring Data in a less than optimal way.

So, let me work on the performance issues next and then scale the app across multiple regions. Until I do that, you can deploy the current version of the app in Heroku and see how “fast” it is.

What Makes the Architecture of Geo-Distributed Apps Different?

Denis Magda — Mon, 08 Aug 2022 12:55:00 +0000

Ahoy, matey!

Welcome back to my journal, where I’ve been documenting my experience of building a geo-distributed app in Java from scratch. In the previous article, I broke down the definition of geo-distributed apps. If you missed that part of the journey, turn the “page” back to catch up.

Today, I’ll compare and contrast geo-distributed apps with regular apps (the sort of apps you deploy within a single data center or availability zone).

To understand the difference and find similarities, I need to drill down into the building blocks (architecture) of a typical geo-distributed app. So, if you’re with me on this journey, then, as the pirates used to say - “All Hand Hoy!” which means “Everyone on deck!”

Geo-Distributed App Architecture

What are the main building blocks (architecture) of a geo-distributed app? The answer will surprise you!

Fundamentally, the architecture is the same as with regular applications. A geo-distributed app comes with data and application layers, just as you get with a standard app. It might have a dedicated API layer and consist of several micro-services and it may also use a load balancer to better handle user traffic. It can rely on some middleware and…the list goes on and on!

So, what's the big difference then? Well, the key to the right answer is engraved in the definition of the geo-distributed apps:

A geo-distributed app is an app that spans multiple geographic locations for high availability, resiliency, compliance and performance.

All of those building blocks and components have to possess the characteristics in bold. For the sake of simplicity, let's review those characteristics in relation to the data layer, application layer and load balancer. And some visuals might also help!

The Users of a geo-distributed app live around the world. Depending on the app, you might be dealing with users residing in a single continent (or a large part of it) or, literally anywhere else on planet Earth. In our example (above), we have three users. Let’s call them Ms. Blue, Mr. Green, and Mr. Red.

These users open up their laptops, launch a browser and go to the Internet. They type our app address in the browser window and expect to see the app’s page within a few seconds, if not immediately. If the page doesn’t load within 1-3 seconds they would consider the app slow (and may go elsewhere) So, how does the geo-distributed app handle this?

Global Load Balancer

The geo-distributed app in our illustration relies on a global load balancer that receives requests, figures out the sender's location, and forwards the requests to the application instance closest to the user.

The picture shows that Ms. Blue's request is routed to an application instance running in the US West. Mr. Green's to an instance in Europe. And Mr. Red's to an instance in Australia. The closer the application instance to the user, the faster the app can process their request. This is how the global load balancer contributes to the performance characteristic of this geo-distributed app.

But, how does the load balancer help with high availability and reliability?

Imagine that the US West region becomes unavailable, and the load balancer can no longer forward Ms. Blue's requests there. The balancer won't panic. Instead, it will figure out another closest location for Ms. Blue (which should be US East) and start forwarding her traffic there. Automatically!

Geo-Distributed Application Layer

Alright, let’s move to the application layer of the architecture. This is where the global load balancer forwards a user request.

In the picture you can see that the application layer spans continents. The application can be a monolith or can be split into several micro-services. It doesn’t really matter for now. What’s most important is that multiple instances of the app are running across the globe. I guess it’s already apparent why it’s done this way, so let’s just recap.

Multiple app instances ensure that the geo-distributed app can tolerate various cloud outages, including major incidents. This makes the application layer reliable and highly available. On top of that, with multiple instances, the geo-distributed app can serve user requests at low latency, regardless of the user’s location. This is what makes the app layer performant.

Horizontally Scalable Data Layer

Finally, the application instance selected to serve a user request needs to read data from, or write it to, the data layer (database). With geo-distributed apps, you usually use a distributed database that can scale horizontally. This is why the picture above has multiple database nodes scattered worldwide.

Distributed databases greatly improve the reliability and availability of geo-distributed apps. These databases store redundant copies of data, allowing the app to remain operational during various outages. If a database node becomes unavailable due to a cloud incident, the application requests can be handled by alive and healthy nodes.

From a performance standpoint, the closer the user data to the application instance, the better. You don’t want an application instance from Sydney handling requests for Mr. Red to go to a database node in Asia or South America. Right? Right! With distributed databases, it’s possible to arrange data close to the user’s location, minimizing latencies.

Importantly, the ability to place user data in specific locations also enables the geo-distributed app to comply with data residency requirements. GDPR is a serious thing. So, when the global load balancer forwards Mr. Green’s requests to an app instance in Europe, the instance has to read personal data from and write it to the database node(s) deployed in the European Union.

That’s it. As you see, the architecture of a geo-distributed app comes with the same components you use in regular applications - including the data layer, application layer, and load balancer. The only difference is, in the case of the geo-distributed apps all of those components have to function across several distant locations.

Still confused? Any questions? Nudge me in the comments and we’ll get things sorted together!

What’s on the Horizon?

Alright, with this article I’ve finished discussing the basic concepts related to geo-distributed applications. Now you know what beast a geo-distributed app is, and what components it includes.

What do we have coming up next?

Well, I’ll introduce you to my first geo-distributed app prototype. It’s still raw, but that’s OK as long as there’s a foundation to build upon on.

This is a Java web application built on the Vaadin framework and deployed in Heroku. The database is YugabyteDB. The prototype functions within a single cloud region, but across multiple availability zones.

Geo What? A Quick Introduction to Geo-Distributed Apps

Denis Magda — Wed, 03 Aug 2022 02:02:24 +0000

Have you heard of geo-distributed apps? According to my statistics around 50% of us haven’t!

Microsoft defines a geo-distributed app as an app that spans multiple geographic locations for high availability and resiliency. Geo-distributed is a relatively new term, coined around the time when many of us jumped on the cloud bandwagon and started building cloud-native apps like crazy.

Dealing with geo-distributed apps is now my day duty. While I reside on the data layer side of things - working with YugabyteDB - I'm a proponent of the "eat your own dog food" approach!
Many developers use YugabyteDB as the database component in geo-distributed apps. But, the database is just a piece of the puzzle. There are many more layers to take care of - middleware, backend, API, networking, frontend, and global cloud load balancer, to name just a few.

This article is the first in my journal, where I'll be documenting my experience of building a geo-distributed app in Java from scratch. I know it’s going to be tough, but it’s going to be a completely open and uncensored story. It will include wins, tips & tricks, struggles, and things to avoid!

This story may last a few months, or might even take a bit longer. The one thing I know for sure is that I'll finish the project much faster than Stephen King who worked on the Dark Tower series for 34 years!

So, if you’re with me, then, as the pirates used to say - “All Hand Hoy!” which means “Everyone on deck!”

Clarifying the Definition of Geo-Distributed Apps

Personally, I would expand Microsoft’s definition of geo-distributed apps by adding a couple of extra words - an app that spans multiple geographic locations for high availability, resiliency, compliance and performance. Now that’s too many buzzwords, right? So, let’s break it into pieces because, frankly, the definition is accurate and concise if you know the full context.

The high-availability/resiliency characteristic implies that the app can withstand all types of cloud outages, including major incidents. A typical cloud environment comes with abundant resources and services, but you shouldn’t expect that all those resources to be available 100% of the time. Starting in 2011, AWS alone had a major outage at least once a year. It took an average of four hours to recover. And minor outages happen all the time. The purpose of geo-distributed apps is to remain available and resilient even in the event of major cloud incidents.

So, what does compliance mean in relation to the geo-distributed apps? This one is simpler. Remember GDPR and similar data residency requirements? That’s about it. Suppose you create an app for US-based users, and it becomes a big success. The next milestone is to offer the same service in Europe. Is your architecture ready for that? Or do you need to postpone the launch because the app is not yet GDPR-ready? With a geo-distributed app it happens naturally and as quickly as you like, as long as those apps are capable of keeping users’ personal data in the countries of their origin.

Yo Ho Ho! We’ve got to the performance characteristic of the apps. Let’s use some visuals for this:

Suppose an app’s data resides in a single cloud region, like the US West. Users who live nearby can experience a latency as low as 5 ms (Google Cloud strives to deliver 5ms roundtrip latencies between availability zones within a region). But, if the user interacts with the app from South Asia, latency might be 40x higher (~220ms in Google Cloud because the traffic needs to travel through cables under the ocean surface).

The geo-distributed app’s promise and requirement are to compete against the laws of physics and brutal reality (such as the cables in the ocean) in a way that allows the user from South Asia to experience the same (or comparable) latency as the user from the US:

In the next articles in my developer journal, I’ll dive into how to drop latency from 220ms to 8ms for folks in South Asia. In the meantime, as the picture shows, let me add a database instance to that region, which is a bare minimum.

With those examples in place, the definition of a geo-distributed app should make more sense to you.

A geo-distributed app is an app that spans multiple geographic locations for high availability, resiliency, compliance and performance.

Still confused or do you have any questions on the above? Nudge me in the comments and we’ll get things sorted together!

What’s on the Horizon?

Alright, that’s enough for my opening chapter. What do we have on the horizon?

In the next chapters, I’ll introduce you to the typical building blocks of geo-distributed apps (aka. architecture) and share the results of the prototype I’ve developed over the last four days. The prototype is promising, but, arrrgghh, the performance sucks, and the UI is as ugly as...

Distributing Data Across Distant Locations With Table Geo-Partitioning

Denis Magda — Wed, 20 Jul 2022 13:27:34 +0000

In the first two articles of the table partitioning series, we reviewed how the partition pruning and maintenance capabilities of PostgreSQL can speed up and facilitate the design of our applications. In this final post in the series, we’ll experiment with the table geo-partitioning feature that automatically distributes application data across multiple locations.

We’ll continue using a large pizza chain as an example. This pizza chain has branches in New York, London, and Hong Kong. It also uses a single centralized database cluster to track the orders of its delighted customers.

This time, the geo-distributed database cluster runs on YugabyteDB. For those who are not familiar with this database yet, YugabyteDB is a PostgreSQL-compliant distributed database that distributes data evenly across a cluster of nodes. Additionally, it scales both read and write operations by utilizing all the cluster resources. YugabyteDB supports several multi-region deployments; however, in this article, we’ll focus on the geo-partitioned option.

Geo-Partitioned Table

Let’s take the PizzaOrders table again but now partition it by the Region column. The table tracks the order’s progress (introduced in the first article), and the newly added Region column defines a location of an order:

Orders_US: this partitioned table is for the orders placed in New York and other cities in the US region.
Orders_EU: the branch in London will keep all its orders in this partition. Once the pizza chain opens new locations in Europe, orders from those branches will go to the Orders_EU as well.
Orders_APAC: when Hong Kong customers order a pizza, the order goes to this partition.

As you can see, it’s straightforward to split the PizzaOrders into partitions that will be distributed by the database across distant geographical locations. But, before we get to the step-by-step instructions, let’s provide some rationale for this type of partitioning:

It’s good for performance & user experience — the speed and responsiveness of your pizza application will be similar for all the customers, regardless of their whereabouts. For instance, orders from Hong Kong will go through the Orders_APAC table, where data is stored on the database nodes in APAC.
It’s good for data regulation — all the orders and personal data of European customers won’t leave the boundaries of the EU. The data that belongs to the Orders_EU partition will be located on database nodes in Europe which satisfies the GDPR requirements.
It’s good from a business perspective — the pizza chain headquarters has full control of a single database cluster that can be scaled in specific geographies once the load increases. Also, the app can easily query and join geo-distributed data through a single database endpoint.

Starting the Geo-Partitioned Cluster

Now, let’s experiment with geo-partitioned tables. First, deploy a geo-partitioned instance of YugabyteDB. Here you have two options.

Option #1: You can deploy and configure a geo-partitioned cluster through the YugabyteDB Managed interface:

Option #2: You can simulate a geo-partitioned cluster on your local machine with YugabyteDB open source and Docker.

mkdir ~/yb_docker_data

docker network create yugabytedb_network
# Starting a node in the US
docker run -d --name yugabytedb_node_us --net yugabytedb_network -p 7001:7000 -p 9000:9000 -p 5433:5433 \
  -v ~/yb_docker_data/node_us:/home/yugabyte/yb_data --restart unless-stopped \
  yugabytedb/yugabyte:latest bin/yugabyted start --listen=yugabytedb_node_us \
  --base_dir=/home/yugabyte/yb_data --daemon=false \
  --master_flags="placement_zone=A,placement_region=US,placement_cloud=CLOUD" \
  --tserver_flags="placement_zone=A,placement_region=US,placement_cloud=CLOUD"

# Starting a node in Europe
docker run -d --name yugabytedb_node_eu --net yugabytedb_network \
  -v ~/yb_docker_data/node_eu:/home/yugabyte/yb_data --restart unless-stopped \
  yugabytedb/yugabyte:latest bin/yugabyted start --join=yugabytedb_node_us --listen=yugabytedb_node_eu \
  --base_dir=/home/yugabyte/yb_data --daemon=false \
  --master_flags="placement_zone=A,placement_region=EU,placement_cloud=CLOUD" \
  --tserver_flags="placement_zone=A,placement_region=EU,placement_cloud=CLOUD"

# Starting a node in APAC
docker run -d --name yugabytedb_node_apac --net yugabytedb_network \
  -v ~/yb_docker_data/node_apac:/home/yugabyte/yb_data --restart unless-stopped \
  yugabytedb/yugabyte:latest bin/yugabyted start --join=yugabytedb_node_us --listen=yugabytedb_node_apac \
  --base_dir=/home/yugabyte/yb_data --daemon=false \
 --master_flags="placement_zone=A,placement_region=APAC,placement_cloud=CLOUD" \
  --tserver_flags="placement_zone=A,placement_region=APAC,placement_cloud=CLOUD"

# Updating the nodes’ placement
docker exec -i yugabytedb_node_us \
yb-admin -master_addresses yugabytedb_node_us:7100,yugabytedb_node_eu:7100,yugabytedb_node_apac:7100 \
modify_placement_info CLOUD.US.A,CLOUD.EU.A,CLOUD.APAC.A 3

We’ll use the latter option that starts a three-node cluster with one node in each geographic location:

yugabytedb_node_us: the node is placed in this location placement_region=US and will keep data of the Orders_US partition.
yugabytedb_node_eu: the node is located in Europe (placement_region=EU) and will store orders from the Orders_EU partition.
yugabytedb_node_apac: as you can guess, this node is for the orders of the APAC customers. Thus, it’s placed in the placement_region=APAC region.

Once started, you can connect to the database instance using the following psql command:

psql -h 127.0.0.1 -p 5433 yugabyte -U yugabyte -w

Creating Tablespaces

Tablespaces is a handy PostgreSQL feature. It allows you to define locations on the filesystem where the files representing a database object are stored. As a Postgres-compliant database, YugabyteDB supports this feature and lets you use the tablespaces for geo-partitioning needs.

So, your next step is to create tablespaces for the geo-partitioned PizzaOrders table:

CREATE TABLESPACE us_tablespace WITH (
  replica_placement='{"num_replicas": 1, "placement_blocks":
  [{"cloud":"CLOUD","region":"US","zone":"A","min_num_replicas":1}]}'
);

CREATE TABLESPACE eu_tablespace WITH (
  replica_placement='{"num_replicas": 1, "placement_blocks":
  [{"cloud":"CLOUD","region":"EU","zone":"A","min_num_replicas":1}]}'
);

CREATE TABLESPACE apac_tablespace WITH (
  replica_placement='{"num_replicas": 1, "placement_blocks":
  [{"cloud":"CLOUD","region":"APAC","zone":"A","min_num_replicas":1}]}'
);

These commands create a tablespace for each location: the US, EU, and APAC. Each tablespace gets assigned to the node with similar placement information. For instance, the us_tablespace will be assigned to the yugabytedb_node_us node that you started earlier, as long as the placement info of that node (placement_zone=A,placement_region=US,placement_cloud=CLOUD) corresponds to the placement of us_tablespace.

Creating Partitions

The final configuration step is to get the PizzaOrders table split into three previously discussed partitions: Orders_US, Orders_EU, and Orders_APAC. You can use the commands below to do this:

CREATE TYPE status_t AS ENUM('ordered', 'baking', 'delivering', 'yummy-in-my-tummy');

CREATE TABLE PizzaOrders
 (
   order_id   int,
   order_status   status_t,
   order_time   timestamp,
   region text,
   PRIMARY KEY (order_id, region)
 ) PARTITION BY LIST (region);

CREATE TABLE Orders_US
    PARTITION OF PizzaOrders
    FOR VALUES IN ('US') TABLESPACE us_tablespace;

CREATE TABLE Orders_EU
    PARTITION OF PizzaOrders
    FOR VALUES IN ('EU') TABLESPACE eu_tablespace;

CREATE TABLE Orders_APAC
    PARTITION OF PizzaOrders
    FOR VALUES IN ('APAC') TABLESPACE apac_tablespace;

In fact, a combination of several capabilities enables geo-partitioning in YugabyteDB:

First, the original table (PizzaOrders) is partitioned using the LIST Partitioning method (PARTITION BY LIST (region)).
Second, each partition is assigned to one of the tablespaces. For instance, in the command above, the Orders_APAC partition is assigned to the TABLESPACE apac_tablespace.
Lastly, each tablespace with its partitions is automatically mapped to (i.e., placed on) YugabyteDB nodes from the corresponding geography.

Alright, now let’s run this command just to make sure that the PizzaOrders table was, in fact, partitioned properly:

\d+ PizzaOrders;
                                       Partitioned table "public.pizzaorders"
    Column    |            Type             | Collation | Nullable | Default | Storage  | Stats target | Description 
--------------+-----------------------------+-----------+----------+---------+----------+--------------+-------------
 order_id     | integer                     |           | not null |         | plain    |              | 
 order_status | status_t                    |           |          |         | plain    |              | 
 order_time   | timestamp without time zone |           |          |         | plain    |              | 
 region       | text                        |           | not null |         | extended |              | 
Partition key: LIST (region)
Indexes:
    "pizzaorders_pkey" PRIMARY KEY, lsm (order_id HASH, region ASC)
Partitions: orders_apac FOR VALUES IN ('APAC'),
            orders_eu FOR VALUES IN ('EU'),
            orders_us FOR VALUES IN ('US')

Testing Table Geo-Partitioning

You are now ready to do the final test. Go ahead and add a few orders into the database. As of now, put some data in the US-based pizza chain:

INSERT INTO PizzaOrders VALUES 
(1, 'yummy-in-my-tummy', '2021-12-27 22:00:00', 'US'),
(2, 'yummy-in-my-tummy', '2022-05-15 13:00:00', 'US'),
(6, 'baking', '2022-06-24 8:45:00', 'US'),
(7, 'baking', '2022-06-24 9:00:00', 'US');

Double check the data got placed in the Orders_US partition (refer to the tableoid column in the result):

SELECT tableoid::regclass,* from PizzaOrders 
  ORDER BY order_id;

 tableoid  | order_id |   order_status    |     order_time      | region 
-----------+----------+-------------------+---------------------+--------
 orders_us |        1 | yummy-in-my-tummy | 2021-12-27 22:00:00 | US
 orders_us |        2 | yummy-in-my-tummy | 2022-05-15 13:00:00 | US
 orders_us |        6 | baking            | 2022-06-24 08:45:00 | US
 orders_us |        7 | baking            | 2022-06-24 09:00:00 | US

Next, attempt to put pizza orders in but for customers from London (EU region) and Hong Kong (APAC region):

INSERT INTO PizzaOrders VALUES 
(3, 'yummy-in-my-tummy', '2022-05-23 10:00:00', 'EU'),
(4, 'yummy-in-my-tummy', '2022-06-23 19:00:00', 'APAC'),
(5, 'delivering', '2022-06-24 8:30:00', 'APAC'),
(8, 'ordered', '2022-06-24 10:00:00', 'EU'); 

ERROR:  Illegal state: Nonlocal tablet accessed in local transaction: tablet 49fb2ab17298424096d215f5f1f32515

If you’ve been following these instructions, you will receive the above error message. This happens because our psql session is opened through a YugabyteDB node deployed in the US (yugabytedb_node_us). And, by default, YugabyteDB doesn’t let you perform transactions that span across several geographies. So what are your options? You can connect to the nodes in EU and APAC and insert data from there. Or, you can toggle the force_global_transaction on and insert the data from the US-based node:

SET force_global_transaction = TRUE;

INSERT INTO PizzaOrders VALUES 
(3, 'yummy-in-my-tummy', '2022-05-23 10:00:00', 'EU'),
(4, 'yummy-in-my-tummy', '2022-06-23 19:00:00', 'APAC'),
(5, 'delivering', '2022-06-24 8:30:00', 'APAC'),
(8, 'ordered', '2022-06-24 10:00:00', 'EU');

Once the command succeeds, confirm that the orders are placed properly across the partitions and respective geographies (again refer to the tableoid column in the output):

SELECT tableoid::regclass,* from PizzaOrders 
  ORDER BY order_id;
  tableoid   | order_id |   order_status    |     order_time      | region 
-------------+----------+-------------------+---------------------+--------
 orders_us   |        1 | yummy-in-my-tummy | 2021-12-27 22:00:00 | US
 orders_us   |        2 | yummy-in-my-tummy | 2022-05-15 13:00:00 | US
 orders_eu   |        3 | yummy-in-my-tummy | 2022-05-23 10:00:00 | EU
 orders_apac |        4 | yummy-in-my-tummy | 2022-06-23 19:00:00 | APAC
 orders_apac |        5 | delivering        | 2022-06-24 08:30:00 | APAC
 orders_us   |        6 | baking            | 2022-06-24 08:45:00 | US
 orders_us   |        7 | baking            | 2022-06-24 09:00:00 | US
 orders_eu   |        8 | ordered           | 2022-06-24 10:00:00 | EU

Wrapping Up…

Alright, this is more than enough for starters when it comes to table geo-partitioning. Check out this article if you’d like to learn how to add new regions to a pre-existing geo-partitioned cluster or how to withstand region-level outages.

This article wraps up our series dedicated to the topic of table partitioning for application developers. Have fun building these apps, and stay tuned for some new content related to databases, distributed systems, and Java!

Managing Data Placement With Table Partitioning

Denis Magda — Tue, 12 Jul 2022 20:22:44 +0000

Table partitioning is a very convenient technique supported by several databases including MySQL, Oracle, PostgreSQL, and YugabyteDB. In the first article of this series, we discussed an application that automates the operations of a large pizza chain. We reviewed how PostgreSQL improves the application’s performance with the partition pruning feature by eliminating unnecessary partitions from the query execution plan.

In this article, we’ll examine how PostgreSQL’s partition maintenance capabilities can further influence and simplify the architecture of your apps. We’ll again take the pizza chain app as an example, whose database schema comes with the PizzaOrders table. To remind you, the table tracks the order’s progress (table data is explained in the first article):

Now, pretend you need to separate the orders for the current, previous, and all other earlier months. So, you go ahead and partition the PizzaOrders by the OrderTime column:

As a result, the original table gets split into three partitioned tables or partitions:

Orders_2022_06 - the table keeps all the orders for the current month (June 2022). Suppose that the customer-facing microservices heavily use the data from this partition.
Order_2022_05 - the table stores orders for the previous month (May 2022). Assume internal microservices that facilitate short-term planning regularly query this data in combination with the current month’s data.
OrdersOthers - the remaining historical data used by the BI tools for strategic planning.

Nice, you can partition data by time, and the database will ensure each microservice queries only the data it needs. But, wait, the current and past months are not static notions. Once the calendar page flips to July 1st, July 2020 will become the current month. But how do you reflect this change at the database level? Let’s talk about partition maintenance techniques.

Partition Maintenance

The structure of your partitions might be dynamic. Quite frequently, you might want to remove partitions holding old data and add new partitions with the new data. That’s the case with our pizza chain. And this maintenance task can be easily fulfilled at the database level with no code changes on the application side.

In PostgreSQL, partitions are regular tables that you can query or alter directly. So, whenever necessary you can use standard DDL commands to CREATE, ATTACH, DETACH, and DROP partitions.

Creating Original Partitions

First, let’s create the original partitions that we’ve discussed above:

CREATE TYPE status_t AS ENUM('ordered', 'baking', 'delivering', 'yummy-in-my-tummy');

CREATE TABLE PizzaOrders
 (
   id   int,
   status   status_t,
   ordertime   timestamp
 ) PARTITION BY RANGE (ordertime);

CREATE TABLE orders_2022_06 PARTITION OF PizzaOrders 
  FOR VALUES FROM('2022-06-01') TO ('2022-07-01');

CREATE TABLE orders_2022_05 PARTITION OF PizzaOrders 
  FOR VALUES FROM('2022-05-01') TO ('2022-06-01');

CREATE TABLE orders_others PARTITION OF PizzaOrders DEFAULT;

The PARTITION BY RANGE (order_time) clause requests to split the PizzaOrders table using the Range Partitioning method. The resulting partitions will keep the orders based on the value of the ordertime column. For instance, if the ordertime is between '2022-06-01' (inclusive) and '2022-07-01' (exclusive), then a pizza order goes into the current month’s partition (which is orders_2022_06). The orders_others partition is the DEFAULT one as it will keep all the orders that ordertimevalue doesn’t fit into the range of any other partition.

Second, all the created partitions are regular tables that you can work with using DDL and DML commands. For instance, let’s load sample data and query the current month’s partitioned table directly:

INSERT INTO PizzaOrders VALUES 
(1, 'yummy-in-my-tummy', '2021-12-27 22:00:00'),
(2, 'yummy-in-my-tummy', '2022-05-15 13:00:00'),
(3, 'yummy-in-my-tummy', '2022-05-23 10:00:00'),
(4, 'yummy-in-my-tummy', '2022-06-23 19:00:00'),
(5, 'delivering', '2022-06-24 8:30:00'),
(6, 'baking', '2022-06-24 8:45:00'),
(7, 'baking', '2022-06-24 9:00:00'),
(8, 'ordered', '2022-06-24 10:00:00'); 

SELECT * FROM orders_2022_06 WHERE ordertime BETWEEN '2022_06_20' AND '2022_06_30';

 id |      status       |      ordertime      
----+-------------------+---------------------
  4 | yummy-in-my-tummy | 2022-06-23 19:00:00
  5 | delivering        | 2022-06-24 08:30:00
  6 | baking            | 2022-06-24 08:45:00
  7 | baking            | 2022-06-24 09:00:00
  8 | ordered           | 2022-06-24 10:00:00

It’s certainly handy that we can query partitioned tables directly. However, you don’t want your customer-facing microservices to remember the actual current month and what partition to query. Instead, the microservices will be querying the top-level PizzaOrders table and PostgreSQL will apply the partitioning pruning optimization the following way:

EXPLAIN ANALYZE SELECT * FROM PizzaOrders 
    WHERE ordertime BETWEEN '2022_06_20' AND '2022_06_30';
                                                                     QUERY PLAN                                                                      
-----------------------------------------------------------------------------------------------------------------------------------------------------
 Seq Scan on orders_2022_06 pizzaorders  (cost=0.00..37.75 rows=9 width=16) (actual time=0.010..0.012 rows=5 loops=1)
   Filter: ((ordertime >= '2022-06-20 00:00:00'::timestamp without time zone) AND (ordertime <= '2022-06-30 00:00:00'::timestamp without time zone))
 Planning Time: 0.122 ms
 Execution Time: 0.040 ms

This is the same query but PostgreSQL (and not your application layer) decides which partition keeps the data. The execution plan shows that the query ran against the orders_2022_06 partition, bypassing the others.

However, this ability to work with partitioned tables directly is extremely useful when you need to change the structure of your partitions. Now, assume that tomorrow is July 1st, 2022. You need to add a new partition that will keep the orders for that new current month (July), as well as introduce a few other changes.

Detaching Old Partitions

Let’s first deal with partition orders_2022_05 that presently holds data for the “previous month” (May 2022). You do this because once July becomes the “current month”, June will become the “previous month”, according to the application logic.

First, let’s remove the May partition from the partitions structure using the DETACH command:

ALTER TABLE PizzaOrders DETACH PARTITION orders_2022_05;

Once you do this, attempt to read all the records from the PizzaOrders table to confirm there are no records left for May:

SELECT * FROM PizzaOrders;
 id |      status       |      ordertime      
----+-------------------+---------------------
  4 | yummy-in-my-tummy | 2022-06-23 19:00:00
  5 | delivering        | 2022-06-24 08:30:00
  6 | baking            | 2022-06-24 08:45:00
  7 | baking            | 2022-06-24 09:00:00
  8 | ordered           | 2022-06-24 10:00:00
  1 | yummy-in-my-tummy | 2021-12-27 22:00:00

Don’t get scared, the data for May didn’t evaporate! The data is still in the same partitioned table that you can query directly:

SELECT * FROM orders_2022_05;

 id |      status       |      ordertime      
----+-------------------+---------------------
  2 | yummy-in-my-tummy | 2022-05-15 13:00:00
  3 | yummy-in-my-tummy | 2022-05-23 10:00:00

Next, do you remember that we have the orders_others partition that keeps all the orders that the ordertime column doesn’t fit into the ranges of other partitions? Now go ahead and put the records for May there. You can do this by inserting the data into the top-level PizzaOrders table and letting PostgreSQL arrange records across partitions:

 INSERT INTO PizzaOrders (id,status,ordertime) 
    SELECT detached.id, detached.status, detached.ordertime 
   FROM orders_2022_05 as detached;

Lastly, you can safely drop the orders_2022_05 partition because you already have a copy of the orders for May in the orders_others partition:

DROP TABLE orders_2022_05;

SELECT tableoid::regclass,* from PizzaOrders 
  ORDER BY id;

    tableoid    | id |      status       |      ordertime      
----------------+----+-------------------+---------------------
 orders_others  |  1 | yummy-in-my-tummy | 2021-12-27 22:00:00
 orders_others  |  2 | yummy-in-my-tummy | 2022-05-15 13:00:00
 orders_others  |  3 | yummy-in-my-tummy | 2022-05-23 10:00:00
 orders_2022_06 |  4 | yummy-in-my-tummy | 2022-06-23 19:00:00
 orders_2022_06 |  5 | delivering        | 2022-06-24 08:30:00
 orders_2022_06 |  6 | baking            | 2022-06-24 08:45:00
 orders_2022_06 |  7 | baking            | 2022-06-24 09:00:00
 orders_2022_06 |  8 | ordered           | 2022-06-24 10:00:00

Attaching New Partitions

Finally, let’s create a partition for July that’s about to become the “current month”, in accordance with the application logic.

The most straightforward way to do this is by attaching a new partition to the PizzaOrders table:

CREATE TABLE orders_2022_07 PARTITION OF PizzaOrders 
  FOR VALUES FROM('2022-07-01') TO ('2022-08-01');

The name of the new partition is orders_2022_07 and it’s added to the partitions structure:

\d+ PizzaOrders;
                                            Partitioned table "public.pizzaorders"
  Column   |            Type             | Collation | Nullable | Default | Storage | Compression | Stats target | Description 
-----------+-----------------------------+-----------+----------+---------+---------+-------------+--------------+-------------
 id        | integer                     |           |          |         | plain   |             |              | 
 status    | status_t                    |           |          |         | plain   |             |              | 
 ordertime | timestamp without time zone |           |          |         | plain   |             |              | 
Partition key: RANGE (ordertime)
Partitions: orders_2022_06 FOR VALUES FROM ('2022-06-01 00:00:00') TO ('2022-07-01 00:00:00'),
            orders_2022_07 FOR VALUES FROM ('2022-07-01 00:00:00') TO ('2022-08-01 00:00:00'),
            orders_others DEFAULT

Easy, isn’t it? Let’s test the changes by inserting dummy data for July 2022 and checking what partition those records belong to:

INSERT INTO PizzaOrders VALUES 
(9, 'ordered', '2022-07-02 10:00:00'),
(10, 'baking', '2022-07-02 9:50:00'),
(11, 'yummy-in-my-tummy', '2022-07-01 18:10:00');

SELECT tableoid::regclass,* from PizzaOrders 
  ORDER BY id;
    tableoid    | id |      status       |      ordertime      
----------------+----+-------------------+---------------------
 orders_others  |  1 | yummy-in-my-tummy | 2021-12-27 22:00:00
 orders_others  |  2 | yummy-in-my-tummy | 2022-05-15 13:00:00

Done! You could easily change the structure of the partitions by detaching the partition for May and attaching a new one for July. And no changes were necessary on the application side. Our microservices continued to query the PizzaOrders table directly without bothering underlying partitions.

To Be Continued…

Alright, with this article we finished the review of partition pruning and maintenance capabilities that can improve performance and facilitate the design of your application. Check out this PostgreSQL resource to learn more.

In a follow-up article, you’ll learn how to use geo-partitioning to pin pizza orders to a specific geographic location. After all, we’ve been working on the application for a large pizza chain that feeds and delights customers across countries and continents. Stay tuned!

Optimizing Application Queries With Partition Pruning

Denis Magda — Thu, 07 Jul 2022 16:16:42 +0000

Table partitioning is a very handy feature supported by several databases, including PostgreSQL, MySQL, Oracle, and YugabyteDB. This feature is useful when you need to split a large table into smaller independent pieces called partitioned tables or partitions. If you’re not familiar with this feature yet, consider the following simple example.

Let’s pretend you develop an application that automates operations for a large pizza chain. Your database schema has a PizzaOrders table that tracks the order’s progress.

Each row in the table represents a customer order with a unique identifier (ID), the time the customer put the order in (OrderTime), and the current order’s Status. The status ranges from ORDERED to YUMMY-YUMMY-IN-MY-TUMMY. The latter status means that a pizza has already been delivered to the customer’s doors and, hopefully, consumed.

As a large pizza chain, the application needs to deal with thousands of orders daily. This means the size of the PizzaOrders table is getting out of control. Eventually, you take advantage of the table partitioning feature and split the table into several smaller independent tables called partitions.

The original table gets split by the Status column into three partitioned tables:

The OrdersCompleted table stores all the orders that were handed over to the customer.
The OrdersInDelivery tracks the orders on the road to the customer’s location.
And the OrdersInProgress is for the pizzas that are yet to be baked.

Nice! As a quick win, now you can manage the size and state of each partition independently and control the cost, for instance, by using a cheaper storage medium for historical data (OrdersCompleted). Also, you might see an immediate performance gain after the split for queries that request only pizzas with a specific status. For example, if the app requests all the orders that are in progress, then the request will go to the OrdersInProgress table, bypassing other partitions.

Alright, now you have a basic understanding of table partitioning. Next, let’s dive deeper and see how table partitioning can improve your apps’ performance. In this article, we start with the partition pruning feature.

Partition Pruning

In PostgreSQL, partition pruning is an optimization technique that allows the SQL engine to exclude unnecessary partitions from the execution. As a result, your query will run against a smaller data set, bypassing the excluded partitions. The smaller data set to query the better the performance.

Let’s see what happens when the planner and executor apply this optimization technique.

Creating Partitions

Earlier, we discussed how to use the status column to partition the PizzaOrders table into three tables: OrdersInProgress, OrdersInDelivery and OrdersCompleted. These are the DDL commands:

CREATE TYPE status_t AS ENUM('ordered', 'baking', 'delivering', 'yummy-in-my-tummy');

CREATE TABLE PizzaOrders
 (
   id   int,
   status   status_t,
   ordertime   timestamp,
   PRIMARY KEY (id, status)
 ) PARTITION BY LIST (status);

CREATE TABLE OrdersInProgress PARTITION OF PizzaOrders 
  FOR VALUES IN('ordered','baking');

CREATE TABLE OrdersInDelivery PARTITION OF PizzaOrders 
  FOR VALUES IN('delivering');

CREATE TABLE OrdersCompleted PARTITION OF PizzaOrders 
  FOR VALUES IN('yummy-in-my-tummy');

The PARTITION BY LIST (status) clause requests to split the table using the List Partitioning method. With this method, the table is partitioned by explicitly listing which value(s) appear in each partition. PostgreSQL also supports Hash, Range, and Multilevel partitioning methods that you can review later.

After the split, you can use the command below to see the information about the table with its partitions:

\d+ PizzaOrders;

Partitioned table "public.pizzaorders"
  Column   |            Type             | Collation | Nullable | Default | Storage | Compression | Stats target | Description 
-----------+-----------------------------+-----------+----------+---------+---------+-------------+--------------+-------------
 id        | integer                     |           | not null |         | plain   |             |              | 
 status    | status_t                    |           | not null |         | plain   |             |              | 
 ordertime | timestamp without time zone |           |          |         | plain   |             |              | 
Partition key: LIST (status)
Indexes:
    "pizzaorders_pkey" PRIMARY KEY, btree (id, status)
Partitions: orderscompleted FOR VALUES IN ('yummy-in-my-tummy'),
            ordersindelivery FOR VALUES IN ('delivering'),
            ordersinprogress FOR VALUES IN ('ordered', 'baking')

Next, let’s insert sample data:

INSERT INTO PizzaOrders VALUES 
(1, 'yummy-in-my-tummy', '2021-12-27 22:00:00'),
(2, 'yummy-in-my-tummy', '2022-05-15 13:00:00'),
(3, 'yummy-in-my-tummy', '2022-05-23 10:00:00'),
(4, 'yummy-in-my-tummy', '2022-06-23 19:00:00'),
(5, 'delivering', '2022-06-24 8:30:00'),
(6, 'baking', '2022-06-24 8:45:00'),
(7, 'baking', '2022-06-24 9:00:00'),
(8, 'ordered', '2022-06-24 10:00:00');

And run the query below to see in which partition each record is stored:

SELECT tableoid::regclass,* from PizzaOrders 
  ORDER BY id;

     tableoid     | id |      status       |      ordertime      
------------------+----+-------------------+---------------------
 orderscompleted  |  1 | yummy-in-my-tummy | 2021-12-27 22:00:00
 orderscompleted  |  2 | yummy-in-my-tummy | 2022-05-15 13:00:00
 orderscompleted  |  3 | yummy-in-my-tummy | 2022-05-23 10:00:00
 orderscompleted  |  4 | yummy-in-my-tummy | 2022-06-23 19:00:00
 ordersindelivery |  5 | delivering        | 2022-06-24 08:30:00
 ordersinprogress |  6 | baking            | 2022-06-24 08:45:00
 ordersinprogress |  7 | baking            | 2022-06-24 09:00:00
 ordersinprogress |  8 | ordered           | 2022-06-24 10:00:00

The tableoid column shows the name of a table that stores a respective record. As you can see, all the records were properly arranged across the partitioned tables based on the value of their Status column.

Partition Pruning in Action

Lastly, let’s assume you want to get all the orders that were handed over to customers and, hopefully, consumed by them:

EXPLAIN ANALYZE SELECT * FROM PizzaOrders 
  WHERE status = 'yummy-in-my-tummy';

                                                           QUERY PLAN                                                           
--------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on orderscompleted pizzaorders  (cost=22.03..32.57 rows=9 width=16) (actual time=0.019..0.020 rows=4 loops=1)
   Recheck Cond: (status = 'yummy-in-my-tummy'::status_t)
   Heap Blocks: exact=1
   ->  Bitmap Index Scan on orderscompleted_pkey  (cost=0.00..22.03 rows=9 width=0) (actual time=0.012..0.013 rows=4 loops=1)
         Index Cond: (status = 'yummy-in-my-tummy'::status_t)
 Planning Time: 0.321 ms
 Execution Time: 0.052 ms

The execution plan shows that the query was executed over a single partition (OrdersCompleted) bypassing the others. This is the partition pruning feature in action! And it’s enabled by default in PostgreSQL.

Partition Pruning and Functions

Partition pruning works only if a query filters data by applying constant value or externally supplied parameters. If you try to use a non-immutable function within the WHERE clause section, the planner will skip this optimization and scan the entire table instead. Don’t be surprised if you see the following execution plan:

EXPLAIN ANALYZE SELECT * FROM PizzaOrders 
  WHERE status::text = lower('yummy-in-my-tummy');

                                                           QUERY PLAN                                                           
--------------------------------------------------------------------------------------------------------------------------------
 Append  (cost=0.00..127.26 rows=27 width=16) (actual time=0.074..0.076 rows=4 loops=1)
   ->  Seq Scan on ordersinprogress pizzaorders_1  (cost=0.00..42.38 rows=9 width=16) (actual time=0.040..0.040 rows=0 loops=1)
         Filter: ((status)::text = 'yummy-in-my-tummy'::text)
         Rows Removed by Filter: 3
   ->  Seq Scan on ordersindelivery pizzaorders_2  (cost=0.00..42.38 rows=9 width=16) (actual time=0.019..0.019 rows=0 loops=1)
         Filter: ((status)::text = 'yummy-in-my-tummy'::text)
         Rows Removed by Filter: 1
   ->  Seq Scan on orderscompleted pizzaorders_3  (cost=0.00..42.38 rows=9 width=16) (actual time=0.014..0.015 rows=4 loops=1)
         Filter: ((status)::text = 'yummy-in-my-tummy'::text)
 Planning Time: 11.435 ms
 Execution Time: 0.222 ms

The planner doesn’t like guessing the time it might take to execute the function. Therefore, it queries all the partitions, which might be significantly faster.

To Be Continued…

Alright, that’s enough to understand the basics of table partitioning and how your apps can execute faster if the database applies the partition pruning optimization technique. What’s good about pruning is that you, as a developer, just define your tables and respective partitions, and then the database engine will take care of the optimization.

In the following articles, you’ll learn how partition management can simplify and improve the architecture of event-driven and streaming applications. You’ll also discover how you can use the geo-partitioning technique to pin user data to specific geographies. The latter method comes in handy when an app must comply with data regulatory requirements or serve user requests with low latency, regardless of the user’s location.

How Java Apps Litter Beyond the Heap

Denis Magda — Mon, 27 Jun 2022 20:03:53 +0000

As Java developers, we’re no strangers to the concept of garbage collection. Our apps generate garbage all the time, and that garbage is meticulously cleaned out by CMS, G1, Azul C4, and other types of collectors. Basically, our apps are born to bring value to this world, but, nothing is perfect—including our apps that leave litter in the Java heap.

However, the story doesn’t end with the Java heap. In fact, it only starts there. Let’s take the example of a basic Java application that uses a relational database such as PostgreSQL and solid-state drives (SSDs) as a storage device. From here, we’ll explore how our applications generate garbage beyond the boundaries of the Java runtime.

Filling Up PostgreSQL With Dead Tuples

When your Java application executes a DELETE or UPDATE statement against a PostgreSQL database, a deleted record is not removed immediately nor is an existing record updated in its place. Instead, the deleted record is marked as a dead tuple and will remain in storage. The updated record is, in fact, a brand new record that PostgreSQL inserts by copying the previous version of the record and updating requested columns. The previous version of that updated record is considered deleted and, as with the DELETE operation, marked as a dead tuple.

There is a good reason why the database engine keeps old versions of the deleted and updated records in its storage. For starters, your application can run a bunch of transactions against PostgreSQL in parallel. Some of those transactions do start earlier than others. But if a transaction deletes a record that still might be of interest to a few transactions started earlier, then the record needs to be kept in the database (at least until the point in time when all earlier started transactions finish). This is how PostgreSQL implements MVCC (multi-version concurrency protocol).

It’s clear that PostgreSQL can’t and doesn’t want to keep the dead tuples forever. This is why the database has its own garbage collection process called vacuuming. There are two types of VACUUM — the plain one and the full one. The plain VACUUM works in parallel with your application workloads and doesn’t block your queries. This type of vacuuming marks the space occupied by dead tuples as free, making it available for new data that your app will add to the same table later. The plain VACUUM doesn’t return the space to the operating system so that it can be reused by other tables or 3rd party applications (except in some corner cases when a page includes only dead tuples and the page is in the end of a table).

By contrast, the full VACUUM does reclaim the free space to the operating system, but it blocks application workloads. You can think of it as Java’s “stop-the-world” garbage collection pause. It’s only in PostgreSQL that such a pause can last for hours (or days). Thus, database admins try their best to prevent the full VACUUM from happening at all.

Let me stop here and move down to the next level — SSDs. Check out this demo-driven article if you’d like to develop a much deeper understanding of vacuuming.

Generating Stale Data in SSDs

If you thought garbage collection is just for software then… surprise, surprise! Some hardware devices also need to perform garbage collection routines. SSDs do garbage collection all the time!

Whenever your Java application deletes or updates any data on disk - through PostgreSQL as discussed above or directly via the Java File API - then the app generates garbage on SSDs.

An SSD stores data in pages (usually between 4KB and 16KB in size) and the latter are grouped in blocks. While your data can be written or read at the page level, the stale (deleted) data can be erased only at the block level. The erasure requires more voltage than for reading/writing operations, and it’s hard to target that voltage at the page level without impacting the adjacent cells.

So, if your Java app updates a file, then, in fact, an updated segment will be written to an empty page potentially in a different block. The segment with the old data will be marked as stale and garbage collected later. First, a garbage collector in SSDs traverses blocks of pages with stale data and moves good data to other blocks (similar to the compaction phase in Java’s G1 collector). Second, the collector erases blocks that have only stale data left and makes those blocks available to future data.

Curious how SSD manufacturers prevent or minimize the number of “stop-the-world” pauses? There is a concept of SSD over-provisioning, when each device comes with an extra space that is unavailable to your apps. That space is a sort of a safe buffer that allows apps to continue writing or modifying data while the garbage collector erases stale data concurrently. Read more about the over-provisioning here.

Summary

So, next time someone asks you to explain the internals of Java garbage collection, go ahead and surprise them by expanding the topic to include databases and hardware.

On a serious note, garbage collection is a widespread technique that is used far beyond the Java ecosystem. If implemented properly, garbage collection can simplify the architecture of software and hardware without performance impact. Java, PostgreSQL, and SSDs are all good examples of products that successfully take advantage of garbage collection and still remain among the top products in their categories.