AMM Performance Testing Report

Gregory Tsipenyuk — Thu, 05 Oct 2023 23:49:41 +0000

Overview

In the rippled 1.12.0 release, the AMM amendment stands out as a significant feature in both size and scope. Since September 2022, the RippleX performance team has collaborated closely with the engineering team responsible for the AMM feature implementation. This report presents a thorough overview of our testing approach, findings, and key takeaways. Based on our analysis, we are confident that activating the AMM amendment will not negatively impact the network's consensus performance or its ability to process other transactions.

Objective

Our primary objective was to assess transaction performance and its effect on consensus processing, particularly concerning the DEX upon implementing the AMM amendment. Specifically, we aimed to ascertain if the AMM amendment could possibly impede the performance of the XRP Ledger network at the worst case scenarios. To achieve this goal, we adopted a new testing methodology compared to the previous performance tests which primarily focused on exploring the maximum software processing capacity. In this testing, our focus was on maintaining a 5-second network consensus processing latency as the capacity threshold.

Testing Methodology

Capacity Criteria

Given the objective to measure the maximum performance impact of integrating AMM transactions in the DEX, we use the 5-seconds consensus latency limit to gauge the consensus capacity while ensuring the network is in its optimal condition. During testing, both the ledger's and consensus's performance and latency were assessed to determine whether the network load surpassed this latency criteria.

Load Modeling

Research was undertaken to understand the distribution of payment transaction path sizes. Our findings showed that over 99.7% of transactions did not specify the Paths field. When the Paths field is not specified in cross currency payments, it is assumed that there is one path and one step. As a result, we came up with the following load scenarios.

Scenario #1: XRP-XRP direct payments
Scenario #2: Payment transactions involving the sending and receiving of two IOU tokens
- 2.1: Using only the AMM-based DEX
- 2.2: Using only the LOB-based DEX
Scenario #3: Mixed load scenario to emulate a heavy load condition
- 3.1: Using only the AMM-based DEX
- 3.2: Using only the LOB-based DEX

For scenario #3, the complexity and the proportions of each condition is laid out in the table below.

AMM TRANSACTION PAYLOAD COMPLEXITY	LOAD PROPORTION
Payment Transaction with 1 path and 1 PathStep between two IOU tokens	73.63%
Payment Transaction with 6 paths and [3,8] PathSteps among IOU tokens	24.72%
Payment Transaction with 3 paths and [1,3] PathSteps among IOU tokens	1.65%

Test Environment

Our testing infrastructure simulated a private XRPL environment with 9 nodes, matching Ripple's MainNet in terms of hardware specifications. The key features of this environment are the following.

Network Setup
- 5 function as validator nodes
- 4 serve as client P2P nodes, interfacing with load generators
Uniform System Specifications: Each system mirrors the hardware specifications found in Ripple’s MainNet rippled infrastructure. Specifically, they are hosted on AWS's EC2 instance type, z1d.2xlarge, boasting 8 CPU cores, 64GB RAM, and 300GB NVMe SSD storage, dedicated to the rippled database.
Network Configuration: All nodes operate within the same AWS region and are interconnected through a shared LAN.
Load Submission: Between 1 and 4 load servers are actively engaged in transmitting transactions to the 4 P2P nodes.
Account Setup: A total of 100K synthetic accounts were established, built atop a public ledger synchronized from the MainNet as of July 3rd, 2023. Provisions were made in the simulation logic to ensure the uniqueness of source and destination accounts for every consensus cycle.
Configuration Continuity: The rippled configuration file was sourced directly from a Ripple MainNet validator. While the core configurations remained unchanged, requisite modifications were made to align with the specific needs of our environment.

Test Data Setup

A total of 100K synthetic accounts were established based on a public ledger synced from the MainNet as of July 3rd, 2023. To maintain the integrity of the simulation, we have implemented logic ensuring that unique accounts are designated as sources and destinations for simultaneous transactions throughout each consensus cycle. The following describes the setup.

Rippling activated across all synthetic accounts
15 IOU Tokens
Designated two synthetic accounts as the “genesis account” and the “liquidity provider”
Accounts have trustlines with the issuers of all the IOU Token in the network
Accounts are sufficiently funded with the IOU Tokens
Created AMM instances to allow intricate payment paths as shown in the “AMM/LOB IOU Tokens Topology” graph
- A bi-directional arrow shows that there is a corresponding AMM instance between the two IOU tokens

Test Results

Here are the test results from the scenarios discussed in the Load Modeling section.

#	TEST SCENARIO	VERSION	TEST DURATION(MIN)	THROUGHPUT(TXN/SEC)
1	XRP-XRP DIRECT PAYMENT	1.12.0	60	708.14
2.1	AMM IOU-IOU /W 1 PATH + 1 PATHSTEP	1.12.0	60	403.50
2.2	LOB IOU-IOU /W 1 PATH + 1 PATH STEP	1.12.0	60	414.74
3.1	AMM - MIXED PATH	1.12.0	60	110.86
3.2	LOB - MIXED PATH	1.12.0	60	214.27

Conclusions

To underscore the scale of the load test, we have juxtaposed the load test throughput with the year-to-date throughput, represented as “Current TPS”, of Payment transactions in the graph below.

From our comprehensive testing and subsequent evaluation, it is clear that, despite the computational intricacies associated with the transactions that interact with the AMM-based DEX, we have made significant advancements in XRPL's performance over the last 18 months. This has propelled the transaction throughput beyond 100 TPS, even under the most challenging conditions. Considering these outcomes, we are confident that incorporating the XLS-30 AMM feature into rippled will not introduce risks to the network in the near term. We remain steadfast in our dedication to rigorous research and development, aiming to continually elevate the performance and adaptability of the XRP Ledger.

A Developer's Guide to Payment Engine System Design

Gregory Tsipenyuk — Tue, 20 Jun 2023 20:32:57 +0000

The RippleX engineering team has published a new whitepaper that provides an overview of the Payment Engine (PE) system, including design concepts and examples. The PE system is a subsystem within the rippled software that is responsible for executing payments, offer crossings, and cash checks in the XRPL network, and is also used as part of the XRP Ledger's path finding algorithm. The full whitepaper demonstrates key concepts of the PE system with several examples.

Synopsis (Super Summary)

The document provides an overview of the design concepts and examples related to the Payment Engine (PE) system, which is a subsystem within the XRPL's rippled software. Here is a summary of the main points covered:

Payment Engine Overview

Payment Engine is responsible for executing payments, offer crossings, and cash checks in the XRPL network.
It converts payment paths into a sequence of specific steps and executes them sequentially.
The steps can involve different types, such as XRPEndpointStep, BookStep, and DirectStep, depending on the nature of the payment.

Design Concepts By Example

Several examples are provided to illustrate different design concepts and scenarios.
Examples include executing steps in reverse and forward directions, re-executing steps based on limitations, consuming offers at the same quality, iterative payment execution, and multi-path payment execution.
The examples showcase how payments are executed, how offers and liquidity are consumed, and how the system handles various scenarios.

Payments via Automated Market Maker (AMM)

The document discusses the integration of Automated Market Maker (AMM) functionality into the BookStep process.
AMM allows tokens to be traded in both directions based on mathematical formulas.
AMM offers are handled within BookStep and can be chosen based on spot price quality (SPQ).
The document explains how AMM offers are generated and resized during payment execution.

Multi-Path Payment Execution

The document explores the execution of payments involving multiple paths, where liquidity can be consumed from different sources.
Paths are sorted based on their quality, and the best quality path is executed first.
The document introduces a Fibonacci sequence-based approach to generating synthetic AMM offers in multi-path payments.
The iterative approach is used to execute multi-path payments until the requested output or input amounts are fulfilled.

The document provides conceptual understanding of the Payment Engine subsystem and helps developers delve into the coding implementation details. It demonstrates various payment scenarios, the interaction between different steps, and the considerations involved in executing payments within the XRPL network.

Read the full whitepaper here.

DEV Community: Gregory Tsipenyuk