HTTP Load Balancer on Top of WSO2 Gateway — Part 2: Performance Benchmarks

In my previous post, I discussed Load Balancer Engine Architecture and its features. In this post, Ill be discussing on performance bench-marks.

Kindly note that the underlying carbon-gateway-framework is under development. And hence more features will be developed and improvisations will be done to this LB.

Performance Benchmarks

In this performance test, five instances of simple service created using Netty framework were used. Each instance is a fast backend (0s delay) with response of size 1KB.

One Million (1,000,000) requests were sent at different concurrency levels (500 to 12,000) to Netty backend, Nginx (Open source version) and GW-LB using apache bench via automated script.

Benchmarks were conducted in Round-Robin algorithm mode with no persistence policies.

More details can be found here.

Test Bed

These are the configurations used during bench-marking.

VM Details

• **Guest OS :** Ubuntu 64-bit 16.04 VM
• **RAM :** 8 GB
• **CPU cores :** 4
• **JVM Version :** 1.8.0_91
• **Java Runtime :** Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
• **Java HotSpot :** Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

Host Machine Details

• **Host OS :** OS X EI Captain Version 10.11.5 (15F34) MacBook Pro (Mid 2015)
• **Hypervisor :** VMware Fusion Professional Version 8.1.1 (3771013)
• **Processor :** 2.5 GHz Intel Core i7
• **Memory :** 16 GB 1600 MHz DDR3

Note: HTTP Load Balancer on Top of WSO2 Gateway is referred as GW-LB.

Throughput Test

Tests were done twice. Average of Average throughput for each concurrency level is calculated and plotted.

Figure 1 shows throughput comparison between Open Source Nginx and GW-LB and Figure 1.1 shows throughput comparison along with Netty backend.

Figure 1 : Nginx Open Source Version vs GW-LB

Figure 1.1 : Nginx Open Source Version vs GW-LB along with Netty Back-End

Latency Test

Tests were done twice. Average of Mean Latency for each concurrency level is calculated and plotted.

Figure 2: Nginx Open Source Version vs GW-LB along with Netty BE

Memory Test

Java Flight Recorder (JFR) is enabled while starting LB server and recording is stopped after load test ends. The obtained JFR recording has memory usage details.

Figure 3 shows Committed, Reserved and Used Heap values.

Figure 3: Committed vs Reserved vs Used Heap memory values of GW-LB

It took three code reviews and several rounds of performance bench-marking to get these results. Guidance and suggestions from my mentors and community members were very helpful. I enjoyed discussing with my mentors and it was a great learning too! I loved this phase of my GSoC project more than any because our discussions were very good and I could sense that my mentors were also as eager and motivated as me to achieve good performance.

I also kept track of the performance improvisation that we were doing by creating it as an issue so that it might be helpful one day. You can find it here.

Sample Configuration

• You can find a sample config here.
• Also find more samples here.

Kindly note that DSL and underlying Gateway Framework is evolving and it will change over the period of time.

Javadoc

• You can find javadoc here.

Thanks for reading and Happy Coding !

Originally published at HTTP Load Balancer on Top of WSO2 GatewayPart 2: Performance Benchmarks on August 18, 2016.