DEV Community: Radix

Java Client Library - General Use Code Examples

Radix — Mon, 23 Sep 2019 12:47:20 +0000

Introduction

Let's review some code examples on how to handle generic tasks:

-Initializing a Universe
-Initializing the Application API

Tip: if you're new to our Java library, we suggest you begin with our Get Started guide.

Initializing a Universe

A Universe is an instance of a Radix Distributed Ledger which is defined by a genesis atom and a dynamic set of unpermissioned nodes forming a network.

Available networks

Network	Description
LOCALHOST_SINGLENODE	A locally hosted single node connection.
A single node Betanet connection.	A single node Betanet connection.

A predefined configuration to bootstrap into the LOCALHOST_SINGLENODE network is available:

BootstrapConfig config = Bootstrap.LOCALHOST_SINGLENODE;

Initializing the Application API

The Radix Application API is a client-side API exposing high-level abstractions to make DApp creation easier.

To initialize the API:

RadixApplicationAPI api = RadixApplicationAPI.create(Bootstrap.BETANET, identity);

To continually sync and pull from the network ledger on your account:

Disposable d = api.pull();

To stop syncing:

d.dispose();

Join The Radix Community

Telegram for general chat
Discord for developers chat
Reddit for general discussion
Forum for technical discussion
Twitter for announcements
Email newsletter for weekly updates
Mail to hello@radixdlt.com for general enquiries

Java Client Library - introduction

Radix — Wed, 04 Sep 2019 12:22:57 +0000

Java client library

The developers guide to building on the Radix Network with the Java client library

Introduction

A Java client library for interacting with a Radix Distributed Ledger.

Note: this library and the network itself are currently in Beta development phase. Please report any issues in the GitHub issue tracker.

Features

-Connect to the Betanet test network
-Fee-less transactions for test networks
-Public Key Identity Creation
-Token Creation (ERC-777 style)
-Message sending
-RXJava 2 based
-Utilizes JSON-RPC over WebSockets

Gradle

Include the following Gradle dependency:

repositories {
    maven { url 'https://jitpack.io' }
}

dependencies {
    implementation 'com.radixdlt:radixdlt-java:dbfd5064e5'
}

Join The Radix Community

Scaling DLT to over 1M TPS on Google Cloud

Radix — Fri, 30 Aug 2019 14:07:32 +0000

Radix achieves a record-breaking 1.4 million financial transactions per second using just $700 of Google Cloud infrastructure. The test, which replayed 10 years of Bitcoin transactions history in less than an hour, was done on 1,187 Google servers, spread across 17 countries.

To put this result into context, it’s almost 17x greater than the maximum transactional throughput of the entire Visa network, over 4x greater than Alipay’s highest throughput ever, and more than 2x WhatsApp’s average number of messages sent per second.

This post outlines how we achieved this result, a description of Radix’s underlying technology, how we conducted the test, and how we used Google Cloud Platform (GCP).

If you would like to re-create the test or use any of the tooling, all the relevant code is available here:

https://github.com/radixdlt/mtps

What was the purpose of this test?

The birth of the internet required creating a set of protocols (TCP, UDP, etc.) that allowed computers to be networked together to make a public, permissionless commons for businesses and individuals to connect from anywhere in the world.

The recent explosion of public, permissionless Decentralised Ledger Technologies (e.g. Bitcoin and Facebook Libra) represent the next phase of digitising the global economy: a public infrastructure that can hold, transfer and program money, assets and rights.

However, the internet’s protocols only carry information; they do not store the information once transmitted. Additionally, the protocol doesn’t care if that information is duplicated and sent to thousands, millions or billions of people.

To create digital money however, you must be able to guarantee that the funds cannot be copied — e.g. you’re prohibited from sending the same digital dollar to two different people. To prevent this from happening, the system must have a constantly updated record of who owns what.

Using a centralised service to host such a record is not too tricky. However, making public, permissionless money without using a central server is considerably more challenging. And making this money simultaneously usable for the entire world is an even greater challenge.

The biggest problem that Radix set out to solve: how do you build a protocol for the creation of digital money that could work for over 7.5 billion people and 500 billion devices simultaneously? This test was the first step on the journey to validating that our technology can scale.

What is DLT (Distributed Ledger Technology)?

DLT is a general term that covers blockchain technologies including Bitcoin and Ethereum, DAG (directed acyclic graph) technologies including IOTA and Hashgraph, and new architectures such as Tempo. Tempo is the architecture that Radix developed specifically to achieve the massive scalability and throughput needed for a DLT to be used for mainstream applications.

A DLT network is characterised by a shared record system that can be updated by all the computers in the network without causing conflicting entries in a shared ledger. This is often used for transactional systems, since it’s well-suited for stopping the same funds from being spent twice and it can also enforce more complex systems of rules, often referred to as “smart contracts.”

How is the Radix DLT architecture different?

To build a linearly scalable transactional system, it was first necessary to think carefully about how to shard the Radix ledger. It is often not a good idea to dynamically shard data in distributed, decentralised systems due to the coordination overhead.A neater solution is to create a static shard range and then have a deterministic map of the data to the available shards.

On Radix, your wallet address deterministically maps you to the shard that your funds and transaction information lives on. As a result, there is no need for a global state —just shard level state.

Any computer may join the network and become a “Node.” These Nodes may service as many or as few shards as their local resources (CPU/RAM/Storage) allow. While a Node may vary the number of shards that it serves, the total number of shards in a Radix network is always 18.4 quintillion shards (2^64).

Nodes operating the same shards are redundant to each other, meaning that they validate the same set of transactions and are in consensus with each other. A transaction is typically validated in at least two different shards: the shard of the source address (a.k.a. Sending wallet) and the shard range of the destination address (e.g. receiving wallet).

This illustration above exemplifies the shard range configuration on Radix, with the green vertical lines indicating the static shard space, and the horizontal lines representing the Nodes and the amount of the shard space each is servicing.

To learn more about how this works in practice, please see this short video on sharding on Radix.

How was Radix DLT deployed on Google Cloud?

The Test Scenario

1187 Nodes (VMs running the Radix ledger) are split up over 17 Google Cloud zones (data centers).

During the test, each Node was configured to have a low amount of shard overlap with each other. We did this by cutting the available shards (2^64) into chunks of 2^44 shards. We then allocated Nodes evenly across these chunks. Shards serviced by a Node are called the Node’s “Shard Range.” During the test, each Node Shard Range had approximately 10% overlap with the Node Shard Range of its neighbouring Nodes.

This test scenario was tailored to achieve maximum network throughput as cost-effectively as possible. In a production network, we would expect each shard to be covered by at least 100 different nodes. This would make little difference to the throughput, but would substantially increase testing costs.

During the test, the Nodes ‘gossip’ with each other to exchange transactional information and metadata (over UDP). In a Radix network, each Node is Gossiping (on average) to eight random peers; the dissemination complexity is thus O(8n) as explained here.

Notes on Network Conditions

Where feasible, we wanted to create a test scenario that models a public network. This includes:

-Using only public IP addresses
-Having a geographically distribution of Nodes

The fundamental bottleneck between continents is still the speed-of-light, making packet latencies moderately representative. However, it should also be noted that during the test, the Gossipped transactions never left Google’s high-performance network, and this reduces packet latencies between Nodes.

Under normal operations there is no packet loss within Google’s networks. It is estimated that normal packet loss on the internet is below 3%; which will reduce the efficiency of the Radix main network by about the same amount compared to these tests.

The Explorer Node

The Explorer Node scrapes and aggregates metrics and data from all other Nodes. The most significant service hosted on the Explorer Node is the Explorer Service, which serves the UI and provides a metrics API. Radix maintains a public version of Explorer at: https://test.radixdlt.com/.

The Dataset Preparator Node

The Dataset Preparator Node converts Bitcoin’s transactional data into Radix compatible transactional data so that it can be replayed across the Radix network. The process is very CPU and I/O intensive which is why we chose to host it separately.

This function is made up of two services we built for this purpose:

-bitcoind
-millionaire-dataset-preparator

Tooling

Each optimised test run costs around $700. In order to cut down the run-time of the VMs, we decided to create a custom Ubuntu 16.04 image, with the Radix Core dependencies installed (Docker). We used packer for this, which worked well in GCE.

The remaining provisioning of the Core VMs was done during initial boot through cloudinit. This tool may be less well known to the everyday (Google) Cloud user, but we love it because it enables us to use the same provisioning script for any cloud provider that supports it (which they almost all do).

We used the configuration management tool libcloud in this project. It provides python APIs and is a direct alternative to the Google Cloud SDK (aka “gcloud”). We switched to libcloud when our shell-scripts (using gcloud) became too complex for us to maintain. An additional advantage of using libcloud over gcloud is that we could opt-in the python paramiko SSH client, which —unlike gcloud — supports (indirectly) SSH keys authentication using Yubikeys.

How Google Cloud helped Radix DLT to reach 1M TPS?

To make things simple, we built a custom mtps.py python script that does all the node setup in a single invocation. This is the flowchart of the mtps.py tool. This script is idempotent, meaning that it will skip through the already created components.

The “Boot Node” is an ordinary Core Node. It’s started first, so that it can provide a concrete IP address for the other Nodes to connect to.

Pre-requisites

1.You need to have a Google Cloud account
2.Configure libcloud to use your Google Cloud account
3.Install libcloud and paramiko with:
pip3 install —user apache-libcloud paramiko cryptography
4.Start an interactive python terminal:
python3

>>> gce = ComputeEngine(
"libcloud@m-tps-test.iam.gserviceaccount.com",
"~/m-tps-test.json",
project="m-tps-test"
)

The Explorer Node and Core Nodes need to be reachable from the internet, which is why we need to punch through the firewall:

>>> explorer_ports = [
{"IPProtocol": "tcp", "ports": ["22", "443"]},
{"IPProtocol": "udp", "ports": ["20000"]}
]
>>> core_ports = [
{"IPProtocol": "tcp", "ports": ["22", "443", "20000"]},
{"IPProtocol": "udp", "ports": ["20000"]}
]
>>> gce.ex_create_firewall(
name='explorer-ingress-rules',
allowed=explorer_ports,
description='explorer-ingress-rules')
>>> gce.ex_create_firewall(
name='core-ingress-rules',
allowed=core_ports,
description='core-ingress-rules')

[ doc ] [ source ]

Create Explorer Node

The next step is to create the Explorer node, which hosts the Explorer Service.

NOTE: This command needs a valid explorer_cloud_init configuration (string), which is based on this template. It’s quite hard to generate this template by hand, hence the following code block is not executable as is:

>>> node = gce.create_node(
name = "explorer",
image = "https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1604-xenial-v20190530c",
size = "n1-standard-8",
location = "europe-west3-a",
ex_metadata = {"user-data": explorer_cloud_init},
ex_disks_gce_struct = [
{
'autoDelete': True,
'boot': True,
'type': 'PERSISTENT',
'mode': 'READ_WRITE',
'deviceName': "explorer",
'initializeParams': {
'diskName': "explorer",
'sourceImage': "https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1604-xenial-v20190530c"
}
},
]
)

[ doc ] [ source ]

Create Boot Node / Create Core Nodes

Now this is where things get really complicated as we used 17 different Google Cloud Engine (GCE) Regions.

The Boot node is launched first and the remaining core nodes do not get created until the Boot node is serving on HTTPS port 443 and listening on Gossip (UDP) port 20000.

Creating 1187 Core nodes one-by-one with gce.create_node (as we did with the Explorer node above) is theoretically possible, but very inefficient. That’s why we instead created Instance groups — one in each GCE Region.

NOTE: Similarly to creating the Explorer node, a concrete cloudinit config needs to be generated for the Core nodes (same config used for the Boot node) from this template, which is out-of-scope for this blog.

A Google Cloud Region is a geographical location (City) with multiple data centers (Zones). Typically Google has three Zones in every region. In the Google Cloud Console, it’s possible to create an Instance group in a Region without limiting it to a particular Zone. However, libcloud 2.5.0 only supports the creation of instance groups per Zone. The solution is the following monkey patch, whichwill work until libcloud catches up:

def create_core_group(gce, region, size, template, prefix="cores"):
try:
region = region or gce.region
if not hasattr(region, 'name'):
region = gce.ex_get_region(region)
request = '/regions/%s/instanceGroupManagers' % (region.name)
manager_data = {}

#If the user gave us a name, we fetch the GCEInstanceTemplate for it.
if not hasattr(template, 'name'):
template = gce.ex_get_instancetemplate(template)
manager_data['instanceTemplate'] = template.extra['selfLink']

# If base_instance_name is not set, we use name.
manager_data['name'] = manager_data['baseInstanceName'] = "{0}-{1}".format(prefix, region.name)
manager_data['distributionPolicy'] = dict(
zones = [{"zone": "zones/" + z.name} for z in gce.ex_list_zones() if z.status == "UP" and z.name.startswith(region.name)]
)
manager_data['targetSize'] = size
manager_data['description'] = None

gce.connection.request(request, method='POST', data=manager_data)
except ResourceExistsError:
pass

[ source ]

Final Words

It took our team of five people approximately two months to develop a stable solution for instantiating and running 1M TPS tests in GCE. Overall, our experience with the platform is very positive. However, some features stand out and are worth mentioning in more detail (below).

What we loved about the Google Cloud Platform

Reliability and Performance Consistency

It’s hard to develop, debug and productify a traditional web service. It is even harder for Highly-Available Applications, where a global state needs to be synchronised between a quorum (typically 3 or 5) Nodes. The challenge grows even bigger if you decide to host each Node on the opposite side of the world, as then latency also becomes a significant issue. Our test scenario with 1187 Nodes spread over the majority of the Google Cloud Regions made this even more complex.

Despite this complexity and global spread, the Google Cloud Platform has been rock-solid for us and was one of the few components we did not have problems with. In particular, when analysing performance changes between test runs, GCE was very consistent, allowing us to focus on tweaking RadixCore. To demonstrate this, please see the following two graphs of transactions per second performance over time. Notice it’s the same software, same config, same number of GCE Regions, same amount of nodes/Region (1187); the only difference is that one was run 13.5 hours later than the other:

Stable API and Tooling

Initially we used gcloud. We decided to punt it only at the end of the project when our shell script (using the gcloud CLI) became too big and too complex for us to maintain. Overall we found gcloud to be mature and stable with no timing issues. The only missing feature is the ability to bind it to our YubiKey (SSH Key).

Project based Security

We like how the GCPsecurity boundaries are organised: on the top level you have your Organisation, and within your Organisation you create Projects. In our case, we have business critical services running in one project and the 1M TPS tests running in another. This allowed us to have different levels of access control for our business critical services compared to the 1M TPS tests.

The Google Identity Access Management (IAM) solution is also pretty intuitive. We managed to figure out what occasional permissions we were missing by quick trial and error in the Cloud Console application. Most of the time the tooling actually tells you what permission is missing.

Support is Good and Responsive

We did not need Google Cloud support for anything other than getting quotas raised, but for that we used it a lot.

We got confirmation and response within 24 hours at all times, except the time the Google Cloud Platform went down. The help was substantial, although getting quotas was a very painful and time consuming process.

Areas for Improvement

Quota Process

The wonderful world of GCP quotas! When you are using multiple data centre locations and all of them have different sets of quotas, this process can become frustrating — especially if it’s blocking you from getting what you want: more VMs!

Once you figure out the first quota you need, you can easily request it to be increased through a form in Google Cloud Console. In fact, it’s so easy that it is tempting to fire off individual quota requests as soon as you realise you are missing something. It was later explained to us that some quotas take time, especially when you want to go from eight CPUs to 800 CPUs in one Zone. Google keeps some spare capacity, but a substantial increase is typically a carefully planned process that needs to be approved on many levels of their organisation.

Unfortunately, quotas requests cannot be tracked in the Console – you can only respond to an e-mail if you want to follow up on it. This meant we could not see what quota request our colleagues had already created, which ended up with multiple people creating the same quota request at slightly different times.

Higher Level Quotas

We increase quotas in order to instantiate more VMs. Depending on the VM configuration, we typically need to request more than one quota all at once. For example, the following set of quotas need to be increased together in our test scenario:

-CPUs
-In-use IP Addresses
-Persistent Disk SSD (GB)

As a suggested improvement, instead of exposing each quota to the user, a better UX would allow the user to request quotas from the options menu of a running VM. — e.g. a new quota form would ideally be started from this menu:

In the new quota form the user would have high-level options to:

-Get x more like this in current Zone
-Get x more like this in current Region
-Get x more like this in any Region
-Custom (would take you to the legacy quota form)

This would result in more accurate and complete quota requests, which would both decrease the load on Google Cloud Support and get the end-user unblocked faster.

Conclusion

We are very proud to have gone from an immature idea to a stable product in eight weeks. Without going into detail, there are lots of small decision points during the project that have made these eight weeks very stressful for the team involved, which we could have done better.

One of the bigger decisions we made early in the project was to build our project on Google Cloud Platform. This decision was based on our previous experience with GCP; in fact we had (and still have) substantial amounts of our production infrastructure on GCP at the time this project was kicked off. I’m happy to say that partnering with Google Cloud was the right call for this project as well. Google’s infrastructure is solid, and its support organization can handle unconventional use-cases (1187 ephemeral nodes) like ours and respond within a reasonable amount of time.

Join The Radix Community

The Radix Betanet Emulator - how to start?

Radix — Tue, 27 Aug 2019 09:46:01 +0000

Introduction

This is a quick start guide aimed for non-technical users, to run a Radix DLT Betanet Emulator on your computer. The following steps are covered:

-Installing Docker
-Creating a configuration file
-Launching the Radix DLT Betanet Emulator

Emulator limitations

-The betanet emulator works in a stand-alone mode.
-There is no network or connection between nodes.
-Throughput is limited, as all workload is concentrated on a single node.
-Network API endpoints always return an empty set.

Pre-requisites

Hardware

Your host computer should have at least:

2 CPU cores
8 GB memory
20 GB disk

Note: the actual disk size requirement will grow over time as the ledger grows.

Software

You can run the Radix DLT Betanet Emulator on any operating system that supports Docker and Docker Compose, including:

-Linux
-MacOS X
-Windows 10

Installing Docker

You can download the right Docker Engine (Community Edition) for your system here: https://hub.docker.com/search/?type=edition&offering=community

For a detailed Docker setup guide, please refer to the official installation documentation for Docker CE.

Installing Docker Compose (Linux only)

If you are running Linux, after you completed the Docker setup you need to install Docker Compose. See the official Docker Composite installation guide for download links and additional information.

Tip: Docker Compose is bundled with Docker CE for the Mac and Windows versions.

Creating a configuration file

The Radix DLT software stack is composed of a single docker image, radixdlt/radixdlt-core-emulator:1.0.0-beta.
Your betanet-emulator.yml determines the software components you will run. In particular, the following settings are specified:

-The Docker image to download and start
-Persistent data volumes - that survive restarts and upgrades.

Start with this docker compose file:

version: '2.2'
services:
  core:
    image: radixdlt/radixdlt-core-emulator:1.0.0-beta
    init: true
    ports:
      - "8080:8080"
    environment:
      CORE_NETWORK_SEEDS: core
      CORE_SECURE_RANDOM_SOURCE: /dev/urandom
      CORE_UNIVERSE: v2djcmVhdG9yWCIBA3hanCWf3pmR5E+i+wtWWfKleBrDOQduLb/vcFKOSt9oa2Rlc2NyaXB0aW9ueB5UaGUgUmFkaXggZGV2ZWxvcG1lbnQgVW5pdmVyc2VnZ2VuZXNpc4G/aG1ldGFEYXRhv2l0aW1lc3RhbXBtMTU1MTIyNTYwMDAwMP9ucGFydGljbGVHcm91cHODv2lwYXJ0aWNsZXOBv2hwYXJ0aWNsZb9lYnl0ZXNXAVJhZGl4Li4uIGp1c3QgaW1hZ2luZSFsZGVzdGluYXRpb25zgVECVqurOHBYXwTQFdVa32ALx2Rmcm9tWCcEAgN4Wpwln96ZkeRPovsLVlnypXgawzkHbi2/73BSjkrfaIh5wbllbm9uY2UbAAQdARdF7ABqc2VyaWFsaXplcndyYWRpeC5wYXJ0aWNsZXMubWVzc2FnZWJ0b1gnBAIDeFqcJZ/emZHkT6L7C1ZZ8qV4GsM5B24tv+9wUo5K32iIecG5Z3ZlcnNpb24YZP9qc2VyaWFsaXplcnNyYWRpeC5zcHVuX3BhcnRpY2xlZHNwaW4BZ3ZlcnNpb24YZP9qc2VyaWFsaXplcnRyYWRpeC5wYXJ0aWNsZV9ncm91cGd2ZXJzaW9uGGT/v2lwYXJ0aWNsZXODv2hwYXJ0aWNsZb9sZGVzdGluYXRpb25zgVECVqurOHBYXwTQFdVa32ALx2Vub25jZQBjcnJpWDkGL0pIMVA4ZjN6bmJ5ckRqOEY0UldwaXg3aFJrZ3hxSGpkVzJmTm5LcFIzdjZ1Zlhua25vci9YUkRqc2VyaWFsaXplcnNyYWRpeC5wYXJ0aWNsZXMucnJpZ3ZlcnNpb24YZP9qc2VyaWFsaXplcnNyYWRpeC5zcHVuX3BhcnRpY2xlZHNwaW4gZ3ZlcnNpb24YZP+/aHBhcnRpY2xlv2dhZGRyZXNzWCcEAgN4Wpwln96ZkeRPovsLVlnypXgawzkHbi2/73BSjkrfaIh5wblrZGVzY3JpcHRpb25zUmFkaXggTmF0aXZlIFRva2Vuc2xkZXN0aW5hdGlvbnOBUQJWq6s4cFhfBNAV1VrfYAvHa2dyYW51bGFyaXR5WCEFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFnaWNvblVybHg0aHR0cHM6Ly9hc3NldHMucmFkaXhkbHQuY29tL2ljb25zL2ljb24teHJkLTMyeDMyLnBuZ2RuYW1lZFJhZHNrcGVybWlzc2lvbnO/ZGJ1cm5kbm9uZWRtaW50c3Rva2VuX2NyZWF0aW9uX29ubHn/anNlcmlhbGl6ZXJ4IHJhZGl4LnBhcnRpY2xlcy50b2tlbl9kZWZpbml0aW9uZnN5bWJvbGNYUkRndmVyc2lvbhhk/2pzZXJpYWxpemVyc3JhZGl4LnNwdW5fcGFydGljbGVkc3BpbgFndmVyc2lvbhhk/79ocGFydGljbGW/ZmFtb3VudFghBf//////////////////////////////////////////bGRlc3RpbmF0aW9uc4FRAlarqzhwWF8E0BXVWt9gC8drZ3JhbnVsYXJpdHlYIQUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAWVub25jZRsABB0BF0ZbTmtwZXJtaXNzaW9uc79kYnVybmRub25lZG1pbnRzdG9rZW5fY3JlYXRpb25fb25sef9qc2VyaWFsaXplcngicmFkaXgucGFydGljbGVzLnVuYWxsb2NhdGVkX3Rva2Vuc3gYdG9rZW5EZWZpbml0aW9uUmVmZXJlbmNlWDkGL0pIMVA4ZjN6bmJ5ckRqOEY0UldwaXg3aFJrZ3hxSGpkVzJmTm5LcFIzdjZ1Zlhua25vci9YUkRndmVyc2lvbhhk/2pzZXJpYWxpemVyc3JhZGl4LnNwdW5fcGFydGljbGVkc3BpbgFndmVyc2lvbhhk/2pzZXJpYWxpemVydHJhZGl4LnBhcnRpY2xlX2dyb3VwZ3ZlcnNpb24YZP+/aXBhcnRpY2xlc4O/aHBhcnRpY2xlv2ZhbW91bnRYIQX//////////////////////////////////////////2xkZXN0aW5hdGlvbnOBUQJWq6s4cFhfBNAV1VrfYAvHa2dyYW51bGFyaXR5WCEFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFlbm9uY2UbAAQdARdGW05rcGVybWlzc2lvbnO/ZGJ1cm5kbm9uZWRtaW50c3Rva2VuX2NyZWF0aW9uX29ubHn/anNlcmlhbGl6ZXJ4InJhZGl4LnBhcnRpY2xlcy51bmFsbG9jYXRlZF90b2tlbnN4GHRva2VuRGVmaW5pdGlvblJlZmVyZW5jZVg5Bi9KSDFQOGYzem5ieXJEajhGNFJXcGl4N2hSa2d4cUhqZFcyZk5uS3BSM3Y2dWZYbmtub3IvWFJEZ3ZlcnNpb24YZP9qc2VyaWFsaXplcnNyYWRpeC5zcHVuX3BhcnRpY2xlZHNwaW4gZ3ZlcnNpb24YZP+/aHBhcnRpY2xlv2ZhbW91bnRYIQX///////////////////////////zE0cNgL3/DF////2xkZXN0aW5hdGlvbnOBUQJWq6s4cFhfBNAV1VrfYAvHa2dyYW51bGFyaXR5WCEFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFlbm9uY2UbAAQdARdG1htrcGVybWlzc2lvbnO/ZGJ1cm5kbm9uZWRtaW50c3Rva2VuX2NyZWF0aW9uX29ubHn/anNlcmlhbGl6ZXJ4InJhZGl4LnBhcnRpY2xlcy51bmFsbG9jYXRlZF90b2tlbnN4GHRva2VuRGVmaW5pdGlvblJlZmVyZW5jZVg5Bi9KSDFQOGYzem5ieXJEajhGNFJXcGl4N2hSa2d4cUhqZFcyZk5uS3BSM3Y2dWZYbmtub3IvWFJEZ3ZlcnNpb24YZP9qc2VyaWFsaXplcnNyYWRpeC5zcHVuX3BhcnRpY2xlZHNwaW4BZ3ZlcnNpb24YZP+/aHBhcnRpY2xlv2dhZGRyZXNzWCcEAgN4Wpwln96ZkeRPovsLVlnypXgawzkHbi2/73BSjkrfaIh5wblmYW1vdW50WCEFAAAAAAAAAAAAAAAAAAAAAAAAAAADOy48n9CAPOgAAABsZGVzdGluYXRpb25zgVECVqurOHBYXwTQFdVa32ALx2tncmFudWxhcml0eVghBQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABZW5vbmNlGwAEHQEXRsX0a3Blcm1pc3Npb25zv2RidXJuZG5vbmVkbWludHN0b2tlbl9jcmVhdGlvbl9vbmx5/2ZwbGFuY2saAYp/QGpzZXJpYWxpemVyeCRyYWRpeC5wYXJ0aWNsZXMudHJhbnNmZXJyYWJsZV90b2tlbnN4GHRva2VuRGVmaW5pdGlvblJlZmVyZW5jZVg5Bi9KSDFQOGYzem5ieXJEajhGNFJXcGl4N2hSa2d4cUhqZFcyZk5uS3BSM3Y2dWZYbmtub3IvWFJEZ3ZlcnNpb24YZP9qc2VyaWFsaXplcnNyYWRpeC5zcHVuX3BhcnRpY2xlZHNwaW4BZ3ZlcnNpb24YZP9qc2VyaWFsaXplcnRyYWRpeC5wYXJ0aWNsZV9ncm91cGd2ZXJzaW9uGGT/anNlcmlhbGl6ZXJqcmFkaXguYXRvbWpzaWduYXR1cmVzv3ggNTZhYmFiMzg3MDU4NWYwNGQwMTVkNTVhZGY2MDBiYze/YXJYIQGN+yS2lSWiWm+Qp+RE0XKZZlDCmsYEHqVOgwi5sumVH2FzWCEBUZbY2D5H9/zBPnDtU80qy4A1/EAJyrzw/bOCrHlNgoFqc2VyaWFsaXplcnZjcnlwdG8uZWNkc2Ffc2lnbmF0dXJlZ3ZlcnNpb24YZP//bXRlbXBvcmFsUHJvb2a/Y2FpZFghCEJOW9hqiqRdHgAFw1rMG1kA7xFc89evO1arqzhwWF8EanNlcmlhbGl6ZXJ0dGVtcG8udGVtcG9yYWxfcHJvb2ZndmVyc2lvbhhkaHZlcnRpY2Vzgb9lY2xvY2sBamNvbW1pdG1lbnRYIQMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGVvd25lclgiAQO5kZ9TtqJE8PAqMAkR3IZE413Vm/wmXBPBKjvYm8ddX2hwcmV2aW91c1ECAAAAAAAAAAAAAAAAAAAAAGZyY2xvY2sbAAABaSxAOABqc2VyaWFsaXplcnV0ZW1wby50ZW1wb3JhbF92ZXJ0ZXhpc2lnbmF0dXJlv2FyWCEBPkk2kYaH4ixggyRYaSAYGe80qCxej1B5I+Zi+x/jn4Fhc1ghASXlWsUP3WFDhORmhokZUIVw5sy27TBsZ/Rudr1ueAXkanNlcmlhbGl6ZXJ2Y3J5cHRvLmVjZHNhX3NpZ25hdHVyZWd2ZXJzaW9uGGT/anRpbWVzdGFtcHO/Z2RlZmF1bHQbAAABaSxAOAD/Z3ZlcnNpb24YZP//Z3ZlcnNpb24YZP9lbWFnaWM6T2h//WRuYW1lbFJhZGl4IERldm5ldGZwbGFuY2sZ6mBkcG9ydBl1MGpzZXJpYWxpemVybnJhZGl4LnVuaXZlcnNla3NpZ25hdHVyZS5yWCEBnNHHy2ZNs02RhJ2PaGE8TYjJ6a4JZbs5W5ICDG2FZ4prc2lnbmF0dXJlLnNYIQEzwSczZGOPPINsytlykDyITA51ICNl/GVmTSRwoD4Zeml0aW1lc3RhbXAbAAABaSxAOABkdHlwZQJndmVyc2lvbhhk/w==
      JAVA_OPTS: -server -Xms2g -Xmx2g
    volumes:
      - "core_ledger:/opt/radixdlt/RADIXDB"
      - "core_config:/opt/radixdlt/etc"
    logging:
      options:
        max-size: "10m"
        max-file: "30"
    networks:
      - single_node
  faucet:
    image: radixdlt/faucet:1.0.0-beta
    init: true
    environment:
      FAUCET_DELAY: 0
      FAUCET_TOKEN_RRI: /JH1P8f3znbyrDj8F4RWpix7hRkgxqHjdW2fNnKpR3v6ufXnknor/XRD
      RADIX_IDENTITY_UNENCRYPTED_KEY: JI41zTPZ+DW0JMUBlLciuyrSqT0/Gj9/Oz3+J9MhNL4=
      RADIX_BOOTSTRAP_TRUSTED_NODE: http://core:8080
      JAVA_OPTS: -server
    logging:
      options:
        max-size: "10m"
        max-file: "30"
    networks:
      - single_node
volumes:
  core_ledger:
  core_config:
networks:
  single_node:

1.Create a directory on your computer for storing docker compose files (e.g., radixdlt).
2.Use your favorite text editor to create the betanet-emulator.yml.
3.Copy and paste the content above.

Launching the Betanet Emulator

Open up a Terminal (Mac/Linux) or a Command Prompt (on Windows). Navigate to the directory where you've placed your betanet-emulator.yml file. Launch the Docker containers with:

cd ~/radixdlt
docker-compose -p radixdlt -f betanet-emulator.yml up -d

If successful, it should pull down and look something like this when completed:

Creating network "radixdlt_single_node" with the default driver
Creating volume "radixdlt_core_ledger" with default driver
Creating volume "radixdlt_core_config" with default driver
Pulling core (radixdlt/radixdlt-core-emulator:1.0.0-beta)...
emulator-1.0.0-beta: Pulling from radixdlt/radixdlt-core
c87736221ed0: Pull complete
a0d980c21713: Pull complete
7153696eb942: Pull complete
5a8ad49b35c2: Pull complete
6e8ff60c7fe2: Pull complete
e5bd13ba5d60: Pull complete
dc2a7e34f4b6: Pull complete
9520c49c4dca: Pull complete
Creating radixdlt_core_1 ... done

Check that you have a radixdlt_core_1 Docker container with:

docker ps

You can also check if the Betanet Emulator is up and running at http://localhost:8080/api/system

If running correctly you should get a bunch of metrics - it should look something like this:

{"ledger":{"processed":0,"storingPerShard":0,"storedPerShard":":str:0.0","stored":1,"storing.peak":0,"latency":{"path":0,"persist":0},"checksum":9155064537244249607,"processing":0,"faults":{"tears":0,"assists":0,"stitched":0,"failed":0},"storing":0},"agent":{"protocol":100,"name":":str:/Radix:/2700000","version":2700000},"hid":":uid:8c0976a9ed622c4c0657f57a81c38f85","memory":{"total":2058354688,"max":2058354688,"free":1863937832},"nid":":uid:8cf1ffca3da4bcb3490067ad8e9273e4","serializer":"api.local_system","commitment":":hsh:0100000000000000000000000000000000000000000000000000000000000000","global":{"stored":1,"processing":0,"storing":0},"processors":2,"clock":1,"version":100,"planck":25853760,"shards":{"anchor":-8290564194929689421,"serializer":"radix.shard.space","range":{"high":8796093022207,"low":-8796093022208,"serializer":"radix.shards.range"}},"port":30000,"messages":{"inbound":{"processed":1,"discarded":0,"pending":0,"received":10},"outbound":{"processed":10,"aborted":0,"pending":0,"sent":10}},"events":{"broadcast":75,"processed":{"synchronous":64,"asynchronous":36},"queued":36,"processing":0,"dequeued":36},"key":":byt:A6rybyzH6BO3gp6XVA8NJdqAnL1FQMeCuJ8veFgM2mqQ","timestamp":1551225600000}

Congratulations, you are now successfully running the Betanet Emulator!

Managing the Betanet Emulator

Once you've got the Betanet Emulator running, you can stop or refresh (upgrade) the software using the following commands:

Stop the Betanet Emulator

docker-compose -p radixdlt -f betanet-emulator.yml down

Refresh the Betanet Emulator

docker-compose -p radixdlt -f betanet-emulator.yml pull

Kitematic

This is optional, but if you are running the Betanet Emulator on a Mac or Windows computer, you can download Kitematic to add a UI to your Docker container. If you want access to nice buttons and a live log view this is definitely for you!

Development libraries

Last, but not least, you can start your own DApp development using our Java and JavaScript client libraries.

Join The Radix Community

Radix DLT - Betanet Emulator Live - Start Building Now!

Radix — Fri, 02 Aug 2019 12:35:04 +0000

Radix started as a project to not only be highly scalable but to also make it easy for any developer to build decentralized applications with.

Staying true to this cause, we are releasing a series of developer tools starting with the single node emulator today; so you can start building now before the public betanet goes live.

The single-node emulator behaves like a fully functioning betanet, but on your local machine.

With it you will be able to:

-Create token types

-Mint tokens for token types, subject to permissions

-Burn tokens for token types, subject to permissions

-Transfer tokens for token types, subject to ownership

-Transfer encrypted and unencrypted messages

In addition, there is the ability in the betanet-emulator package to request XRD tokens from a "faucet" to facilitate experiments with the native token.

Follow this quick start guide to install it on your local machine:- https://docs.radixdlt.com/kb/develop/betanet-emulator

Then, use your choice of client library to start building your use-case:

-Java Client Library

-JavaScript Client Library

And, you can use this latest Android wallet release to connect and interact with the betanet emulator.

If you have feedback/questions about the emulator, post them on our Node Runners Forum where one of the team members will help you. If you have library-specific questions, please post them in the Technical Forum.

Lastly, join the conversation with our community on Telegram and tell us what you are building!

Part 1: A Primer on The Scalability Test and Radix

Radix — Fri, 14 Jun 2019 14:13:19 +0000

This is the first part of a two-part series on how we built and deployed a test that pushed the entire transaction history of Bitcoin across the Radix ledger in less than 15 minutes.

What is this test?

These tests replay the entire 10 years of Bitcoin’s transaction history on the Radix ledger, with full transaction and signature validation, on a network of over 1,000 Nodes distributed evenly throughout the world.

For the first time since the creation of public, trustless networks, we have demonstrated a technology that can truly support even the world’s most demanding transactional applications.

What are these tests demonstrating?

That technology for the transfer and ownership of value, without a central authority, can exist at the same scale as the protocol that the internet is based on.

The mission of Radix is to give anyone, anywhere friction free access to the digital economy. To do this, it must be able to service the world without crippling costs or technical bottlenecks.

This essentially means a stateful protocol that can work for over 7.5 billion people and 500 billion devices simultaneously. We built Radix to do exactly this, and to prove it we are running a series of throughput tests starting this week. These tests are built to simulate real-world conditions, with full transaction validation and without cutting any corners.

How does this compare to what has come before?
With the advent of the internet came the advent of digital commerce. Since then the world has needed an increasingly larger transactional throughput just to keep up with the needs of global and increasingly connected citizens:

Early blockchain protocols broke this progression towards platforms that could function for an increasingly interconnected world. Radix provides a platform on which the next generation of digital-first companies can be built, and that can scale to every single person in the world.

What sort of use case requires this kind of throughput?

Few individual use cases require that level of throughput, but as the throughput of a public ledger is shared by every single application built on top of it; the cumulative throughput capacity is key.

The simplest single use case for something of this scale and scope would be the issuance and use (domestic + international, consumer + enterprise + government) of the money of a nation.

Such a system would remove the need for services such as Paypal, Visa, and Mastercard, as well as much of the back end systems that banks use today.

Although the use case of money is only the very simplest of financial applications that can be built on Radix, it also forms the foundation of both economies and financial products, all of which can also be built more easily once money itself is programmable.

To learn more about fiat token/digital cash issuance on Radix, please see our knowledge base.

What dataset are you using to simulate this?

For the first runs, we are testing the throughput of the Radix network using a verifiable data source that we have a lot of love and respect for the Bitcoin ledger transaction history.

We picked the Bitcoin dataset because it is, like Radix, based on the UTXO transaction model, which we can convert to Radix transactional entities (Atoms). For the duration of the test, anyone can search for their accounts and confirm their transaction history matching the real BTC ledger.

We also liked it because there are 460 million bitcoin addresses; which is equivalent in number to the population of a large nation.

Is this the maximum TPS Radix is capable of?
This is by no means the maximum throughput of our platform, but it is definitely stretching it much further than we ever have tried before.

As our scalability is based on sharding, the more shards the higher the transaction throughput possible. As Radix has a [fixed shard space of 18.4 quintillion shards[(https://www.radixdlt.com/post/sharding-in-radix/), the maximum theoretical throughput is way more than could ever be used; even by the entire world.

What does this blog cover?
This blog covers what we did to set up these tests; plus how we got the Radix ledger to do full signature and UTXO validation of the entire Bitcoin transaction history, in less than 30 minutes.

How big is the network?
The first run of these tests concentrates on speed, rather than fault tolerance. As a result, the network consists of approximately 1,000 nodes, with minimal overlap; with each node servicing approximately 1/1,000th of the total ledger.

The Radix consensus does not rely on active global consensus (POW/POS) but a form of shard level passive consensus based on the progression of logical time. The lack of overlap does not mean that transactions are not being correctly validated, but it does prevent the network from being able to deal with significant Node dropout in this configuration.

Should anyone wish to test the fault tolerance of the system by increasing the overlap on our test network, you can spin up your own version of the ledger from our test code on Github. We will also be testing this in the future, but it requires us to continuously request Google to give us enough nodes to test it with!

Radix throughput test code: https://github.com/radixdlt/mtps

On Radix, a node with 8GB of RAM and 4 cores can process approximately 2,000 transactions per second, including full validation and gossiping. For this test, we needed some extra RAM to be able to process and cache the Bitcoin dataset, which raised the RAM requirements to 30GB and 8 cores, but is not representative of main-net requirements.

What are the limitations?
Redundancy in this test is configured using “shard groups” – the network has a fixed shard space of 18.4 quintillion shards and a Node can operate as much or little of the shard space as it likes (assuming it has enough resources). We spread the Nodes out in the shard space using “shard groups” – the smaller the shard groups, the larger the amount of shard space the node is covering. E.g. 1 shard group = 18.4 quintillion shards/100% of the ledger. 2 shard groups = 50% of the ledger per group etc. The more nodes per group, the greater the redundancy – e.g. 100 nodes + 2 shard groups would mean 49 node redundancy per group.

For this test, we are running the network at low redundancy to get the most bang for our buck on Google Cloud. This means approximately 1,000 shard groups for 1,000 nodes. These shard groups overlap a little, but not by a huge amount. Each transaction touches on average 4.2 shards, meaning each transaction is validated and checked for double spends by an average of 4.2 nodes.

In future tests, we will reconfigure the network to have increased redundancy, which will, therefore, have a lower maximum throughput on the network for the same node count. The fundamental limitation is how much money we are willing to spend on running these tests.

Do you detect bad blocks?
There are no blocks or mining on Radix – all Atoms (transactions/on ledger operations) are submitted and checked individually and are determined to be either valid or invalid on a per transaction basis (UTXO double spend check, signature validation, etc.).

Because Radix state sharding has similar properties to Bitcoin’s UTXO model system (with the addition of smart contract like functionality), applying the Bitcoin transaction history with transaction validation and checks for double spends is relatively simple for us to hack into Radix – with the exception of non-standard Bitcoin scripts; where we had to get a bit more inventive – see the millionare-dataset-preparator tool for more details.

How do you stop a double spend?
Transactions are individually validated – this is done using a combination of the Radix consensus layer (Tempo) and the programmable system of constraints that we can add using the Atom Structure and the Constraint Machine. Together these are able to strictly order related transactions (e.g. from the same private key) and drop double spends.

To understand how this works in a bit more detail; please see our [explainer video series here[(https://www.youtube.com/watch?v=sW8nWeUnkK0&list=PLBGHv3uedRNTBeJNq90p-Ph3Yuc7imH-r).

The natively sharded structure of the Radix ledger is essential. Because the shard space is fixed and will never change, even once there are a very large number of people using the network, then it can also be used to help partition transactions and load balance the network.

The main way this is done is via the public key of a wallet. On Radix, the public key of any address also tells you which shard it lives on. This has the very desirable property of automatically grouping together related transactions (all spends from the same key must happen on the same shard) and ungrouping unrelated transactions (two keys have a 1/(2^64) chance of being on the same shard).

This means a node does not need to know about the whole ledger to check the validity of a specific spend; just the shard the key lives on. This is why we can do massively asynchronous processing of everything from application messages to Bitcoin transactions on Radix.

A note on the Bitcoin dataset
The Bitcoin fee model incentivizes grouping together as many transactions as possible in the same block. The Radix fee model will dis-incentivize this (we don’t have blocks). In this regard – although we can achieve a high transaction per second throughput on this data, the bitcoin dataset is not optimized for the Radix data architecture.

For future tests we will be using a more traditional 1-to-1 transactional datasets from financial institutions and crypto-exchanges. This data will produce a friendlier dataset more aligned with the Radix architecture and better represent the vast majority of transactions we will be seeing on the Radix network.

Join The Radix Community

10 years of Bitcoin history, replayed in under 30 minutes

Radix — Wed, 12 Jun 2019 17:13:30 +0000

Today we replayed the entire 10 years of Bitcoin’s transaction history on the Radix ledger, with full transaction and signature validation, on a network of over 1,000 Nodes distributed evenly throughout the world.

For the first time since the creation of public, trustless networks, we have demonstrated a technology that can truly support even the world’s most demanding transactional applications using.

The Result?

The Radix ledger peaked at over 1 million transactions per second, a record for decentralized ledgers. It is even faster than other centralized and established payment processors like Alipay, WeChat and PayPal to name a few. The whole dataset was processed in under 30 minutes.

We were able to achieve this due to Radix’s unique design. Instead of using Blockchains or DAGs, we started from scratch, building a new database and consensus mechanism called Tempo that can scale to support 7.5 billion people and 500 billion devices, simultaneously.

After 7 years of research and development, many dead ends and sleepless nights, today represents a significant milestone in the Radix journey. We can proudly say we’ve built a full state-sharded decentralized ledger that is massively scalable without compromising on security.

To validate our engineering we have decided to run a series of public network throughput tests in an open, transparent manner, that anyone can verify and see happen live.

We decided to use Bitcoin as our data source because it has processed a large number of transactions over the last decade (over 400 million transactions and 460 million addresses) and is an open, fair and transparent data set.

As a comparison, you could run applications like VISA, PayPal, WhatsApp, WeChat and more, simultaneously on the Radix ledger, without breaking a sweat.

The first public test happened today 14.30 London time.

The time and date of our next live test can be seen on our explorer. Make sure you also have your bitcoin addresses ready, as when the test is running you will be able to add in your address and see your transaction history replayed in real time.

These tests will run approximately every week from now! Please sign up to the Radix newsletter and get notified before the next test starts: https://radixdlt.typeform.com/to/x3yotG

Interested in how we did it?

The following technical blog posts explain our methodology and the step-by-step chronology of how we achieved this throughput.

Part 1: A primer on the scalability test and Radix (non-technical)
Part 2: How we actually built the scalability test (technical)

Lastly, if you have any questions, or just want to follow us as we go towards the mainnet, please join the conversation with our community on Telegram or use the #1MTPS on Twitter to share your opinions.

Radix DLT - live scalability test

Radix — Tue, 11 Jun 2019 18:01:14 +0000

Hi everyone!

In case you didn’t know, we are running a scalability test tomorrow, 14:30 #London time, replaying BTC history! Subscribe here to get notified tomorrow - https://radixdlt.typeform.com/to/x3yotG

Radix Universe Visualizer

Radix — Fri, 07 Jun 2019 15:37:24 +0000

Radix Universe Visualizer

At Radix we like to think big, a useful mindset when trying to create a fast and scalable ledger that can support billions of people. With that in mind, Edgars and I (Stephen Thornton) decided to try and build something that would let us view the entire universe. However, since we were limited to just one day, we instead built a Universe Visualizer that would let us view in real-time all of the network activity and atoms in the Radix universe. This had been something Edgar had been wanting to build on Radix for a long time and was something we both agreed would both be useful. Most importantly, we also thought it would be exceptionally cool.

What is it?

The day started inauspiciously, with our hotel serving one of the worst cups of coffee either of us had ever had the misfortune of trying. However, a quick trip into town to get some decent real coffee gave us the time to further brainstorm our ideas. We agreed that we wanted to build something that showed how atoms were being stored across the network in real-time but were unsure of the best way to present it. Some of our initial ideas were quickly discarded due to lacking the physical devices necessary, including using a ‘Palantir’ style model in which we could project a multi-dimensional network view onto a spherical touch-screen display unit, as well as a virtual reality device.

After running through a number of ambitious ideas, we eventually decided to build the visualizer to run on a normal browser, using JavaScript. We had something of an advantage in choosing this route since Edgar was the person who wrote the Radix JavaScript library. We divided the project into three parts:

We used the JavaScript library to twang atoms into the ledger and observe them moving about
Taking this information, we needed to form some graphical representations of what was happening with them
In addition to graphics, we wanted to use sound effects to augment the experience
Edgar took on the first section, while I took the lead on the second. Our inspiration was the idea of a giant pinball table, with atoms rattling madly around the nodes, ringing bells and leaving neon trails like a bad acid trip. Unfortunately for the world, our artists and graphics designers were otherwise engaged so we had to make do with my own talents. I quickly discovered that the two.js graphics library is versatile and easy to use JavaScript library. Using this, I cobbled together some basic graphics that depicted observed events.

We then turned our attention to the sound, which we were both most excited for; supernova explosions when conflicts are detected, choirs of angels singing when a conflict is resolved. If we had more time (and weren’t subject to licensing restrictions) we would have included an R2-D2 voiceover to provide a running commentary.

We were delighted to be awarded third place for our efforts. The only downside was that due to the limited test network we had to work with on the day, we weren’t able to showcase the full visual and audio experience that we had crafted. However, since the hackathon we have been able to trial the Visualizer using a much larger network:

How does it work?

The above describes what we built but for those keen to understand how it works, the basic idea is that:

-The browser app generates a new Radix account for itself
-It then connects to every node in the network using the radixdlt.js library
-When you click on a node, it submits an atom to that node
-Each node then lets the browser app know when it has received the atom, displaying the path it has taken based on the temporal proof in the atom sighting that has been reported
-The path is different every time because the next hop in a temporal proof is determined randomly, eight nodes from the originator, binary branching from then on
-If you click on other nodes, it submits conflicting atoms
-When a node resolves the conflict, it also tells the app that an atom has been rejected due to the conflict
-The aforementioned audio plays when appropriate events are triggered

For the technically minded, we will be publishing the code and do a brief run through of how the code works when the betanet is made public.

Have questions? Message me or Edgars on the Radix discord group (Discord), or you can ask them in our telegram channel (https://t.me/radix_dlt) as well.

Subscribe to our mailing list to get notified when we hit major milestones – https://radixdlt.typeform.com/to/nyKvMV?source=networkvisualizer

JSON-RPC vs REST for distributed platform APIs

Radix — Thu, 06 Jun 2019 12:57:16 +0000

What is REST?

You have probably read about REST a thousand times, but here we go once again:

Let’s start by saying that REST stands for Representational State Transfer, and essentially refers to a style of web architecture that governs the behavior of both client and server.

An API that adheres to the principles of REST doesn’t require the client to know anything about the structure of the API. Rather, the server needs to provide whatever information the client needs to interact with the service. A form in HTML is a clear example of this: the server specifies the location of the resource and the required fields. The browser doesn’t know in advance where to submit the information neither what information to submit. Both forms of information are entirely supplied by the server [1].

REST APIs are everywhere, and distributed ledger technologies are no exception. This is a general example for getting the hash of the latest block added to the Bitcoin blockchain using the Bitcoin Rest API:

// Request
curl -X GET https://blockchain.info/q/latesthash
// Result
0000000000000000002d9c2483cd87685ef41e7b8d1ca718206120e9b1bfee43

So, what about JSON-RPC?

RPC stands for Remote Procedure Call and is a protocol that one program can use to request a service from a program located in another computer on a network without having to understand the network’s details. RPC uses the client-server model. The requesting program is a client and the service providing program is the server. Like a regular or local procedure call, an RPC is a synchronous operation requiring the requesting program to be suspended until the results of the remote procedure are returned. Nevertheless, the use of lightweight processes or threads that share the same address space allows multiple RPCs to be executed concurrently [2][3].

The concept of RPC has been around the software development world for over half a century (yea, time flies), but despite that fact, it is still not as widely known as would be expected for something that enduring.

That being said, it is actually not so surprising, taking into account that most of the current software is web-based, and REST was actually designed with the web-first mindset. When REST came out, RPC APIs were still using XML, and this was problematic. In XML, many attributes are just strings and it’s necessary to ensure the datatypes by adding a metadata layer describing things such as which fields are of which type.

Nowadays, many known decentralised platforms use JSON-RPC based APIs: Ethereum, Ripple and Bitcoin among others.

Here we can see an example of how to use Ethereum’s JSON-RPC API to get the balance of a given account:

// Request
curl -X POST —data ‘{
  “jsonrpc”: “2.0”,
  “method”: “eth_getBalance”,
  “params”: [ "0x407d73d8a49eeb85d32cf465507dd71d507100c1" , "latest" ],
  "id": 1
}’
// Response
{
  "jsonrpc": "2.0",
  "result": "0x0234c8a3397aab58" // 158972490234375000,
  "id": 1
}

But why JSON-RPC for distributed ledger technologies APIs? [5][6]

Agnostic to the protocol

It is hard to emphasise this point enough. If I had to select only one reason why I would choose JSON-RPC over REST is the fact that with JSON-RPC we can have APIs out-of-the-box with which the clients can communicate using any protocol. This has the extra benefit of allowing us to improve the performance of our API using pure TCP (or WebSockets) and removing the HTTP overhead.

REST verbs are limited

In order to execute an operation using REST it’s necessary to define an HTTP method such as GET (default), PUT, POST, PATH or DELETE [4]. But as the name of this methods suggest, REST is CRUD operations oriented. So what happens if this doesn’t meet our needs? An example could be if we have to validate an entity before saving it. Although there’s always a workaround to solve this kind of issues in REST, the truth is that we don’t want to go down that path if we know there’s a better alternative: JSON-RPC.

With the REST approach we will end up having something like this:

// Request
curl -X POST https://localhost:8080/api/entity/validate —data ‘{…}’

But this looks completely counter-intuitive since we are exposing an action as a false resource. On the other hand if we go with JSON-RPC the result will be much more self-explanatory:

// Request
curl -X POST --data '{
  "jsonrpc": "2.0",
  "method": “validateEntity”,
  "params": [ {…} ],
  "id": 1
}’ https://localhost:8080/api

We have a clear winner.

Passing parameters is complicated

There’re four ways of passing arguments using REST: HTTP Headers, HTTP Body, URL path and URL query.

URL Query:

// Request
curl -X PUT https://localhost:8080/api/users?name=Piers

URL Path:

// Request
curl -X PUT https://localhost:8080/api/users/Stephen

HTTP Body:

// Request
curl -X PUT https://localhost:8080/api/users { “name”: “Josh” }

This can be a bit overwhelming, and sometimes we’re not sure which one to use; but JSON-RPC is here to help us sort this out since it offers a simpler way:

// Request
curl -X POST —data ‘[{
  "jsonrpc": "2.0",
  “method”: “addUser”,
  “params”: { “name”: “Angad” },
  “id”: 1
}]’ https://localhost:8080/api

Batch actions

In REST there’s not a simple and nice way of making batch requests, but with JSON-RPC this is a piece of cake:

// Request
curl -X POST—data ‘[{
  "jsonrpc": "2.0",
  “method”: “addUser”,
  “params”: { “name”: “Zalan” },
  “id”: 1
}, {
  "jsonrpc": "2.0",
  “method”: “addUser”,
  “params”: { “name”: “Dan” },
  “id”: 1
}, {
  "jsonrpc": "2.0",
  “method”: “addUser”,
  “params”: { “name”: “Edgars” }
}]’ https://localhost:8080/api

Other benefits of JSON-RPC

In addition to the above, here are a few other benefits of RPC over REST:

-Only one method to send a request – usually POST for HTTP or send for WebSockets.
-The content of the request is completely separated to the transmission mechanism. All errors, warnings, and data are in the payload of the request.
-Use only one response code to confirm the correct reception of a message – usually 200 OK for HTTP.
-Can be easily ported or shared among different transmission channels such as HTTP/S, WebSockets, XMPP, telnet, SFTP, SCP, or SSH.

The separation of message and transmission channel in a JSON-pure API results in an API that is usually faster, and almost always more reliable, easier to use, easier to port, and easier to debug [7].

Summary

The debate of using JSON-RPC over REST is an interesting one, and the usual response is: “it depends”. Both of them are architectural styles for serving content remotely, using a client-server model. There is also no reason for not using RPC and REST at the same time, but you should have a good argument for doing so.

At Radix we think that if you have to serve web content it may be more convenient to go for a REST API approach, since it has been built to work over HTTP, and there’re many available frameworks and libraries out there that will allow you have a minimum viable API set up in a few hours.

But the truth is that if the API you’re building is more complex to model (HTTP verbs are not enough), it requires to work over other transmission channels and/or maintain a connection open (WebSockets), or the performance is critical, then is better to go with JSON-RPC. In the distributed ledger technologies universe the performance is a fundamental factor, and in Radix much more, since we are already needing to handling thousands of transactions per second on our test nets.

[1] https://stackoverflow.com/questions/671118/what-exactly-is-restful-programming

[2] https://en.wikipedia.org/wiki/Remote_procedure_call

[3] https://searchmicroservices.techtarget.com/definition/Remote-Procedure-Call-RPC

[4] https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods

[5] http://wookieb.pl/why-you-should-consider-rpc-for-internal-api

[6] https://www.smashingmagazine.com/2016/09/understanding-rest-and-rpc-for-http-apis/

[7] https://mmikowski.github.io/the_lie/

Join The Radix Community

Radix DLT - Alpha desktop wallet API documentation

Radix — Wed, 05 Jun 2019 13:30:03 +0000

Introduction

The Radix Wallet provides a localhost websocket server which allows any application running either natively or in the browser to perform actions on the ledger through the user's wallet without the user exposing their private keys to the application.

These actions include sending and receiving transactions, messages and custom application data.

The user must explicitly approve any application that wants to do this.

Overview

The server is running at ws://localhost:54345

It is a JsonRPC 2.0 server.

Before making any requests, an application must register it's permissions. This will prompt the user with a dialog to give your app permission to do certain things. If successful, this request will return a token used for authenticating any further request. The lifetime of the token is 30min and it is reset after every successful request.

Afterwards, include the token as token in the request parameters.

API

Register

Register your application. The user will see a popup with your application details and can either approve or deny your access.

method: register

params:

{
name: 'Cashgrab',
description: 'Takes all your money, I don't even know why you would accept this',
permissions: [
  'address',
  'balance',
  'send_transactions'
  ]
}

Successful response:

{
result: { token: 'asdfg' }
...
}

Ping

Check if the Radix Wallet is running and whether a token is still valid.

method: ping

params:

{
token: 'asdfg'
}

Successful response:

{
result: 'pong'
...
}

Send Transaction

Returns the hid of the sent atom

method: send_transaction

permissions: ['send_transactions']

params:

{
token: 'asdfgh',
recipient: '9i9hgAyBQuKvkw7Tg5FEbML59gDmtiwbJwAjBgq5mAU4iaA1ykM',
asset: 'TEST',
quantity: 123.45
}

Successful response:

{
result: {hid: 123}
...
}

Send Message

Send a chat message. Returns the hid of the sent atom.

method: send_message

permissions: ['send_messages']

params:

{
token: 'asdfgh',
recipient: '9i9hgAyBQuKvkw7Tg5FEbML59gDmtiwbJwAjBgq5mAU4iaA1ykM',
message: 'Hi honey, I'm home'
}

Successful response:

{
result: {hid: 123}
...
}

Send application message

Send an application message. Application messages allow storing arbitrary data on the ledger. Returns the hid of the sent atom

method: address

permissions: ['address']

params:

{
token: 'asdfgh',
application_id: 'custom-app',
recipients: [
  '9egJPK7zk2LhPok8VM4noAwVGGS7LKtvTYGJvtF41wrwZAMxjKj',
  '9hFccFfB3Y3A7gToBcD3b9y1ZRnoZfSdHTWVdRPCn3zpKSoVubf',
 ],
payload: {//anything}
}

Successful response:

{
result: {
  result: {hid: 123}
  }
...
}

Get Balance

This is a subscription that will send you an update every time the balance is updated

method: balance

permissions: ['balance']

params:

{
token: 'asdfgh',
}

Successful response:

{
result: 'OK'
...
}

Update notification:

{
jsonrpc: '2.0',
method: 'balance.update',
params: { balance: {TEST: 123.45} }
}

Get Transactions

This is a subscription that will send you an update every time a new transaction occurs

method: transactions

permissions: ['transactions']

params:

{
token: 'asdfgh',
}

Successful response:

{
result: 'OK'
...
}

Update notification:

{
jsonrpc: '2.0',
method: 'transaction.update',
params: { //transaction details }
}

Get Messages

This is a subscription that will send you an update every time a new message is sent or received

method: messages

permissions: ['messages']

params:

{
token: 'asdfgh',
}

Successful response:

{
result: 'OK'
...
}

Update notification:

{
jsonrpc: '2.0',
method: 'message.update',
params: { //message details }
}

Get Application Messages

This is a subscription that will send you an update every time a new message is sent or received

method: application_messages

permissions: ['application_messages']

params:

{
token: 'asdfgh',
application_id: 'custom-app'
}

Successful response:

{
result: 'OK'
...
}

Update notification:

{
jsonrpc: '2.0',
method: 'application_message.update',
params: { 
  applicationId: 'custom-app',
  payload: {//anything}
 }
}

Join The Radix Community

Radix DLT - Android wallet

Radix — Tue, 04 Jun 2019 14:24:47 +0000

The Radix DLT Android Wallet currently works with the live ALPHANET universe and interacts with it by making full use of the latest release of the radixdlt-kotlin library.

The App is a work in progress and will soon be migrated to use the new version of our network.

Android development

-Mostly written in Kotlin (A few classes are in Java but will be converted to keep it 100% Kotlin)
-Uses Architecture Components: Room, LiveData and Lifecycle-components
-Uses dagger-android for dependency injection
-Uses RxJava 2 (Included by default by the radixdlt libs)

Development setup

Use Android Studio 3.2.1 (or newer) to be able to build the app.

The app is currently using the latest release of the radixdlt-kotlin library. It is possible to replace it with the radixdlt-java library but you will need to target newer android devices >= API 24. Also, a few simple changes in the code are required and the dependency to rxkotlin must be included in your gradle file.

implementation(group: 'io.reactivex.rxjava2', name: 'rxkotlin', version: '2.2.0') {
    exclude group: 'io.reactivex.rxjava2', module: 'rxjava'
    exclude group: 'org.jetbrains.kotlin', module: 'kotlin-stdlib'
}

Code style

This project uses ktlint via Gradle dependency. To check code style - gradle ktlint (it's also bound to gradle check).

Contribute

Contributions are welcome, we simply ask to:

-Fork the codebase
-Make changes
-Submit a pull request for review
When contributing to this repository, we recommend to discuss the change you wish to make via issue, email, or any other method with the owners of this repository before making a change.

Please follow our Code of Conduct in all your interactions with the project.

The code is available in our GitHub