How to manage actual Hyperledger Fabric network on your CI pipeline

Atsushin — Wed, 04 Mar 2020 14:12:30 +0000

When you develop an application running on blockchain, do you know how to organize your CI environment with using actual infrastructure? In this article, I'll share how to build a test suite for application running on blockchain network, in particularly, Hyperledger Fabric network that is one of OSS blockchain framework.

Setup environment

You need to follow the steps described in the official document.

In this article, the following environment is assumed:

Ubuntu 18.04.3 LTS
Go 1.13.8
Hyperledger Fabric 1.4.6

Create your workspace

You can clone sample code from https://github.com/nekia/devto1

$ mkdir -p dev/hlf-e2e-test/specs

After initialize go module env, you need to get fabric-test@v1.4.4 go package.

$ pushd dev/hlf-e2e-test/specs
$ go mod init example.com/hlftest
$ go get github.com/hyperledger/fabric-test@v1.4.4
$ popd

fabric-test

fabric-test repository is tool set for testing the Hyperledger Fabric. The repository is officially maintained Hyperledger community. In this article, the following tools are mainly used:

Operator
Operator provides some go package to manage Hyperledger Fabric network with Go program. It also provides CLI as well. In this article, I'll introduce the way how to use this package.
PTE (Performance Traffic Engine)
PTE provides CLI to interact with Hyperledger Fabric networks by sending requests to and receiving responses from the network via Hyperledger Fabric Node.js SDK. We don't cover this tool because in our use case PTE is used by Operator, there is no chance for user to use PTE directly in this scenario.

$ pushd dev/hlf-e2e-test
$ git clone https://github.com/hyperledger/fabric-test.git -b v1.4.4
$ ln -s fabric-test/tools/PTE PTE
$ cd PTE
$ npm install fabric-client@1.4.7
$ npm install fabric-ca-client@1.4.7
$ popd

User need to not only install fabric-test as a go package by using go module function, but also close fabric-test source tree to use PTE (Node.js applicaton) included in the source repository. And PTE seems to be assumed that it is used from component within the fabric-test source tree. Because of that, we still need to keep some relative layout of directory for using PTE outside of the source tree. That's why creating symbolic link above.

$ pushd dev/hlf-e2e-test
$ cp -a fabric-test/tools/operator/templates specs/
$ cp fabric-test/tools/operator/testdata/smoke-*.yml specs/
$ popd

And some files included in templates directory are also important to automatically generate each configuration file such as configtx.yaml, docker-compose.yaml, etc. These template are consumed by ytt tool.

After all the steps up to here, you need to modify network specification file copied from testdata directory. As you can see in the comment, there is special naming rule to switch docker registry. To retrieve container image from Docker Hub, we need to remove -stable postfix from fabric_version. The rest of changes are optional.

--- a/specs/smoke-network-spec.yml
+++ b/specs/smoke-network-spec.yml
@@ -7,7 +7,7 @@
 #! Released images are pulled from docker hub hyperledger/, e.g. 1.4.1 or 2.0.0
 #! Development stream images are pulled from
 #! nexus3.hyperledger.org:10001/hyperledger/, e.g. 1.4.1-stable or 2.0.0-stable
-fabric_version: 1.4.4-stable
+fabric_version: 1.4.6
 #! peer database ledger type (couchdb, goleveldb)
 db_type: goleveldb
 #! This parameter is used to define fabric logging spec in peers
@@ -58,18 +58,18 @@ kafka:
 orderer_organizations:
 - name: ordererorg1
   msp_id: OrdererOrgExampleCom
-  num_orderers: 1
+  num_orderers: 5
   num_ca: 0

 peer_organizations:
 - name: org1
   msp_id: Org1ExampleCom
-  num_peers: 1
+  num_peers: 3
   num_ca: 1

 - name: org2
   msp_id: Org2ExampleCom
-  num_peers: 1
+  num_peers: 3
   num_ca: 1

 #! Capabilites for Orderer, Channel, Application groups

There are 2 type of specification files:

network specification file (ex. smoke-network-spec.yml)
This file defines the specification of Hyperledger Fabric network deployed by using operator tool. This file is mainly consumed by API of networkclient package.
input file (ex. smoke-test-input.yml)
This file defines each action such as creating channels, joining peers to a channel, etc. This file is mainly consumed by API of testclient package.

Prepare test code

package hlftest

import (
    "testing"

    . "github.com/onsi/ginkgo"
    "github.com/onsi/ginkgo/reporters"
    . "github.com/onsi/gomega"

    "github.com/hyperledger/fabric-test/tools/operator/launcher"
    "github.com/hyperledger/fabric-test/tools/operator/testclient"
)

func TestSmoke(t *testing.T) {
    RegisterFailHandler(Fail)
    junitReporter := reporters.NewJUnitReporter("results_smoke-test-suite.xml")
    RunSpecsWithDefaultAndCustomReporters(t, "Smoke Test Suite", []Reporter{junitReporter})
}

// Bringing up network using BeforeSuite
var _ = BeforeSuite(func() {
    networkSpecPath := "smoke-network-spec.yml"
    err := launcher.Launcher("up", "docker", "", networkSpecPath)
    Expect(err).NotTo(HaveOccurred())
})

var _ = Describe("Operator Demo", func() {

    It("starting fabric network", func() {
        inputSpecPath := "smoke-test-input.yml"

        By("1) Creating channel")
        action := "create"
        err := testclient.Testclient(action, inputSpecPath)
        Expect(err).NotTo(HaveOccurred())

        By("2) Joining Peers to channel")
        action = "join"
        err = testclient.Testclient(action, inputSpecPath)
        Expect(err).NotTo(HaveOccurred())
    })
})

// Cleaning up network launched from BeforeSuite and removing all chaincode containers
// and chaincode container images using AfterSuite

// var _ = AfterSuite(func() {
//  networkSpecPath := "smoke-network-spec.yml"
//  err := launcher.Launcher("down", "docker", "", networkSpecPath)
//  Expect(err).NotTo(HaveOccurred())

//  dockerList := []string{"ps", "-aq", "-f", "status=exited"}
//  containerList, _ := networkclient.ExecuteCommand("docker", dockerList, false)
//  if containerList != "" {
//      list := strings.Split(containerList, "\n")
//      containerArgs := []string{"rm", "-f"}
//      containerArgs = append(containerArgs, list...)
//      networkclient.ExecuteCommand("docker", containerArgs, true)
//  }
//  ccimagesList := []string{"images", "-q", "--filter=reference=dev*"}
//  images, _ := networkclient.ExecuteCommand("docker", ccimagesList, false)
//  if images != "" {
//      list := strings.Split(images, "\n")
//      imageArgs := []string{"rmi", "-f"}
//      imageArgs = append(imageArgs, list...)
//      networkclient.ExecuteCommand("docker", imageArgs, true)
//  }
// })

Now, bring up network

$ pushd dev/hlf-e2e-test/specs
$ ginkgo -v
Running Suite: Smoke Test Suite
===============================
Random Seed: 1583242390
Will run 0 of 0 specs

(snip)

Ran 0 of 0 Specs in 29.661 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 30.376068459s
Test Suite Passed

$ docker ps --format '{{.Names}}\t{{.Status}}\t{{.Image}}' | sort
ca0-org1    Up 2 minutes    hyperledger/fabric-ca:1.4.6
ca0-org2    Up 2 minutes    hyperledger/fabric-ca:1.4.6
orderer0-ordererorg1    Up 2 minutes    hyperledger/fabric-orderer:1.4.6
orderer1-ordererorg1    Up 2 minutes    hyperledger/fabric-orderer:1.4.6
orderer2-ordererorg1    Up 2 minutes    hyperledger/fabric-orderer:1.4.6
orderer3-ordererorg1    Up 2 minutes    hyperledger/fabric-orderer:1.4.6
orderer4-ordererorg1    Up 2 minutes    hyperledger/fabric-orderer:1.4.6
peer0-org1  Up 2 minutes    hyperledger/fabric-peer:1.4.6
peer0-org2  Up 2 minutes    hyperledger/fabric-peer:1.4.6
peer1-org1  Up 2 minutes    hyperledger/fabric-peer:1.4.6
peer1-org2  Up 2 minutes    hyperledger/fabric-peer:1.4.6
peer2-org1  Up 2 minutes    hyperledger/fabric-peer:1.4.6
peer2-org2  Up 2 minutes    hyperledger/fabric-peer:1.4.6

$ docker exec peer0-org1 peer channel list
Channels peers has joined: 
testorgschannel0
$ popd