DEV Community: Kofi Oghenerukevwe H.

Minimal Deployment of BTCPayServer in Testnet Mode with Electrum Wallet

Kofi Oghenerukevwe H. — Tue, 16 Feb 2021 21:27:51 +0000

BTCPay Server is a self-hosted opensource solution for receiving Bitcoin and Altcoin payments. BTCPay Server implements the same API as Bitpay so it's easy to switch over from one to the other. There are detailed instructions for setting up the BTCPay Server (for a production environment) on the BTCPay Server documentation available here: BTCPay Server docs. That said, a minimal setup of BTCPay server in testnet mode is a bit tricky and this post is here to document the steps I took when I had to do that a few weeks ago. To be clear The instructions in this post are not for setting BTCPay Server up in production. For production settings, see BTCPay Server's official documentation

The notes here are just to supplement the Minimal Manual Deployment docs on BTCPay Server which can be found here Minimal Manual Deployment

Install Bitcoin

Using these commands

BITCOIN_VERSION="0.19.1"
BITCOIN_URL="https://bitcoin.org/bin/bitcoin-core-0.19.1/bitcoin-0.19.1-x86_64-linux-gnu.tar.gz"
BITCOIN_SHA256="5fcac9416e486d4960e1a946145566350ca670f9aaba99de6542080851122e4c"

# install bitcoin binaries
cd /tmp
wget -O bitcoin.tar.gz "$BITCOIN_URL"
echo "$BITCOIN_SHA256 bitcoin.tar.gz" | sha256sum -c - && \
mkdir bin && \
sudo tar -xzvf bitcoin.tar.gz -C /usr/local/bin --strip-components=2 "bitcoin-$BITCOIN_VERSION/bin/bitcoin-cli" "bitcoin-$BITCOIN_VERSION/bin/bitcoind"
rm bitcoin.tar.gz

Install .NET Core

wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get install apt-transport-https
sudo apt-get update
sudo apt-get install -y dotnet-sdk-3.1

Install NBXplorer

cd ~
git clone https://github.com/dgarage/NBXplorer
cd NBXplorer
git checkout latest
./build.sh

Install Electrum

You can use Electrum to setup either a testnet or live wallet for use with BTCPay Server.

Install Electrum with any of the steps available on their docs Electrum Installation Docs

Install BTCPayServer

cd ~
git clone https://github.com/btcpayserver/btcpayserver
cd btcpayserver
git checkout latest
./build.sh

Long-running processes

A quick note that Bitcoin, NBXplorer and BTCPay are long running processes that should ideally run in the background. So when you run bitcoind for instance, it would take over your ssh session till you stop the process. If you don't want to run them in the background, you can consider using something like tmux to manage different sessions if you plan on just running these commands and leaving them running in multiple windows of the same ssh session.

# create a new session with
tmux new -s BTC

# create a new window in a session with Ctrl + b, c
# detach from session with Ctrl + b, d
# re-attach to session with tmux attach-session -t BTC

Run Bitcoin in Testnet and Background Mode

To run Bitcoind in testnet mode

bitcoind -testnet

Run in the background using

bitciond -daemon -testnet

Run NBXplorer in testnet mode

cd ~/NBXplorer
./run.sh --chains=btc --network=testnet

Setup your testnet wallet with Electrum

# start testnet electrum in the background
electrum daemon -d --testnet 

# create a main wallet
electrum create --testnet 

# load wallet
electrum load_wallet --testnet

# get the master key of the wallet - you'll need this for your BTCPayServer store
electrum getmpk --testnet

Run BTCPay Server in testnet mode

cd ~/btcpayserver
./run.sh --network=testnet --bind=0.0.0.0 --port=8080 --sqlitefile=mydb.db --chains=btc

Unless you're using Tor, you might need to setup a server like Apache or Nginx with an ssl certificate that forwards to BTCPay server on port 8080 because BTCPay server does needs you to access it through HTTPS

Setup your BTCPay Server store.

After BTCPay Server is launched, visit it at its url and create a store. Then integrate that store with your electrum Bitcoin wallet using the master key you got from Electrum earlier.

See Connect BTCPay Server Wallet in the BTCPay Server docs.

Building with IBM Cloud: Setting up a Drupal 8 website

Kofi Oghenerukevwe H. — Mon, 14 Dec 2020 15:04:16 +0000

I recently learned about IBM's Cloud offering and I decided to try it out and write about it. My objective was simple. I would set up a basic Wordpress blog using any of IBM's PaaS offerings. I am familiar with Azure so I was going in with the intention of having a Wordpress blog on IBM's equivalent of Azure's App Service. The Wordpress blog would connect and communicate with a managed MySQL database service.

You've probably guessed from the title of this post that things did not go as planned. IBM does not, at this point in time, offer a Managed MySQL service. And while I could have easily provisioned a virtual server and installed MySQL on it, I don't ever like to deal with the management of databases. I prefer to use managed database offerings because they make it easier for me to handle things like replication, backups, scaling etc.

The Good Parts

Great User Experience

The user experience for IBM's Cloud offering is the best of all the cloud services I have used and tested, and at this point, I have tested all of GCP, AWS and Azure. I immediately liked how easy it was to estimate the costs of the services I was provisioning. I think IBM has done an exceptional job making a platform that is intuitive and easy to navigate. If you have any experience with cloud services, IBM cloud's UI will be easy to get into.

Simple Account Billing

IBM has this concept of a lite account. The lite account is what you get by default when you sign up on IBM cloud and it gives you access to certain free / lite services. These services have very limited capabilities and in fact, I could not provision the managed PostgreSQL service without upgrading my account to a pay-as-you go account.

That said, I liked how clearly distinguished both account types were from each other and how easy it was to filter out services that were available to lite accounts.

The Not-So-Good Parts

The fact that there was no managed MySQL offering easily meant that I was stuck with IBM's Databases for PostgreSQL offering.

At this point, I should mention that there are third-party offerings for schematics that provision MySQL on existing Kubernetes clusters within IBM's cloud environment, but the setup for these offerings required more work than I was willing to put into my test run of IBM's cloud environment for this article. If you're familiar with Kubernetes and don't mind a little hassle, you can definitely use a lot of third-party schematics in IBM Cloud's catalog to provision things like MySQL, Wordpress, Drupal etc. I am not familiar with Kubernetes so this was not an option for me.

Cloud Foundry

Since Drupal works well with PostgreSQL, and IBM has a managed PostgreSQL service, I decided to setup a Drupal site instead. IBM's equivalent of Azure's App Service is called Cloud Foundry. Cloud Foundry has pre-configured environments for running apps written in several languages like PHP, Java, Ruby etc. Some of these environments are managed by IBM, while others are managed by developer communities.

Pushing Drupal's code into Cloud Foundry was easy. But Drupal requires the use of some PHP extensions that are not installed by default in the Cloud Foundry PHP Configuration so I needed a way to get Cloud Foundry to install these extensions. I was happy to find that you can in fact ssh into the Cloud Foundry environment.

You can use either the UI or IBM Cloud's CLI tools to ssh into Cloud Foundry. If you plan to use the CLI tools, you would need to install the Cloud Foundry's cf plugin which is not available by default in the ibmcloud CLI for some reason.

Unfortunately, while I could ssh into Cloud Foundry as a regular user, the user does not have sudo access so I could not install the required PHP extensions. And I learned that there is no way to get sudo access in Cloud Foundry public. I learned that there's a deprecated Cloud Foundry for Enterprise offering that gave you the ability to get sudo access but....like I said, deprecated.

Code Engine

So Cloud Foundry was out. IBM cloud has another PaaS offering called Code Engine which is specifically for running container images. I tried to run the Docker Hub Drupal image with Code Engine but the setup kept failing and I have no idea why it was failing. I visited the public URL that was generated for the Drupal Project I created and the URL just kept loading and loading and.... It's almost like it was stuck in some loop and I don't think the problem was with the container because I was able to run the Drupal docker container on my local machine without any hiccups.

Virtual Server for Classic

I ended up needing to provision a virtual server where I had all the privileges I needed to install PHP, the required PHP extensions, and finally Drupal itself. IBM has several different types of virtual and bare metal server offerings on its cloud. I tried provisioning a couple of them and ended up with the "Virtual Server for Classic" Offering because it was the least complicated to setup and ssh into.

After provisioning the server, I was able to ssh into the server and do whatever I wanted. I was eventually able to install Drupal 8 and Apache on it. But then the weirdest thing kept happening. After a while, say 2 hours, the server would get a disconnected status and while the public IPs and URLs were still actively showing me the Drupal site I installed, I would no longer be able to ssh into the server. I destroyed and created the server several times to try to understand what was happening but, I have not been able to figure it out at this point. The same thing happened each time. After a few hours, the server would get disconnected and I would loose ssh access.

Managed PostgreSQL

IBM Cloud's Managed PostgreSQL offering, ideally should be in the Good Parts section of this article but, that would ruin the structure of my story so...here it is.

IBM's Managed PostgreSQL was a breeze to setup. It was easy to choose the version of PostgreSQL I wanted to install. They have versions 9.5 through 12 available. The database credentials were easy to find and change. The public and private endpoints were easy to find and it was very easy for me to connect to the offering through the public endpoint. Ideally, I should have found a way to connect to the PostgreSQL service through its private endpoint, from within the virtual server I provisioned earlier but I was lazy so...

Concluding Thoughts

I was not able to do what I set out to do when I was preparing to write this article. I ended up setting up a basic Drupal site with PostgreSQL DB instead of a Wordpress site with MySQL DB. That said, my experience of IBM Cloud was actually enjoyable, especially since I have never used it before now. They have a number of services that are still in beta and I will be checking to see how much better they get within the coming months.

Technical Notes and Links to Documentation

If you're interested in using any of the services / tools I used for this process, here are some things to keep in mind.

Drupal 8 will work easily with PostgreSQL 9. You need a bit more extra setup and configurations to get it to work with a higher version of PostgreSQL. I can no longer remember why I opted to use Drupal 8 instead of the latest version - Drupal 9. But I am sure there was a reason. :)
For a complete list of PHP extensions required to run Drupal see the ext-* items in Drupal's composer.json Click Here
If you are installing Drupal behind Apache on a virtual server, you would need to enable Apache's rewrite module and put some rewrite rules in your Apache site configuration file

<Directory /var/www/your_drupal_site>
     RewriteEngine On
     RewriteBase /
     RewriteCond %{REQUEST_FILENAME} !-f
     RewriteCond %{REQUEST_FILENAME} !-d
     RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]
</Directory>

Documentation for installing IBM Cloud CLI tools Click Here
Documentation for installing the Cloud Foundry plugin for IBM Cloud CLI tools Click Here
Documentation for getting started with Cloud Foundry Click Here

Integration Testing with NodeJs, Jest, Knex and SQLite In-Memory Databases

Kofi Oghenerukevwe H. — Mon, 11 May 2020 18:31:52 +0000

If you're using Jest to write integration tests (that hit the database) for your Node application, you'll quickly notice that it's tricky to isolate your tests if you're using only one test database.

While Jest runs all tests in a single testsuite sequentially, it runs your testsuites in parallel (by default). This makes for faster execution times, but it also means that changes to your database in one testsuite could interfere with another test that's currently running in a different testsuite.

An easy solution to this is to configure a new SQLite in-memory database for each of your testsuites or test-files. This way, a testsuite only makes changes to its own database without affecting the database being used by other running testsuites.

In this tutorial, I'll walk through a simple setup for using in-memory databases in your Jest testsuites and running your database migrations on them before each test using Knex. The code for this tutorial is available here: https://github.com/rukykf/jest-sqlite-tutorial

In your knexfile create a new connection for your test file like so:

// knexfile.js

const path = require("path")

module.exports = {
  ...

  test: {
    client: "sqlite3",
    connection: ":memory:",
    useNullAsDefault: true,
    migrations: {
      directory: path.join(__dirname, "migrations")
    },
    seeds: {
      directory: path.join(__dirname, "seeds")
    }
  },

}

When instantiating Knex, specify that you want to use the test config we just created in our knexfile

For instance, in a file db-config.js we could do this

// db-config.js

const knex = require("knex")
const config = require("./knexfile")

let db = null
if (process.env.NODE_ENV === "test") {
  db = knex(config.test)
} else {
  db = knex(config.development)
}

module.exports = db

By default, Jest will set NODE_ENV to test when you are running your tests.

Then in your integration testsuites, you need to run your migrations on the in-memory database before executing any tests.

// users.test.js and db-config.js are in the same directory
const db = require("./db-config")

beforeAll(async () => {
  // run the migrations and do any other setup here
  await db.migrate.latest()
})

test("select users", async () => {
  let users = await db.from("users").select("name")
  expect(users.length).toEqual(0)
})

That's it. Now each of your testsuites will use a separate SQLite database. You still have to keep in mind that all the tests in a given testsuite will use one database.

Setting up PhpStorm to Work with Docker Compose, Xdebug and PHPUnit for Local Development

Kofi Oghenerukevwe H. — Sat, 07 Dec 2019 14:28:46 +0000

Introduction
Docker-compose, Dockerfile and XDebug
Configuring PHPStorm
- Setup PHPStorm Run/Debug Configuration
- Setup PHPStorm PHP CLI
- Setup PHPStorm to run PHPUnit tests within docker container
  - Setup PHPStorm to listen for XDebug
Troubleshooting

Introduction

I decided to write this article because I was frustrated by how difficult it was to find all the information I needed to make these technologies work well together in a single article. I am hoping this resource can be a good reference for myself and for others who might run into the same issues that I ran into.

In this article we'll go through the following:

Setting up the PhpStorm IDE so we can start and stop our docker-compose containers from within the IDE
Debugging our code with PhpStorm and Xdebug (with Xdebug installed in a docker-compose service container)
Running our test-suite with PhpStorm and PHPUnit (with PHP installed in a docker-compose service container)

Note: The information in this article should work well with PhpStorm version 2019.2 and up. Please be aware that some of the configuration options I am about to discuss are not available in certain older versions of PhpStorm.

Part 1: Configurations for docker-compose.yml, Dockerfile and Xdebug

The docker-compose.yml and Dockerfile configuration files I'm using for this article were created by Peter Fisher of How to Code Well and slightly modified by me. You can get access to the original files from his Freecodecamp PHP OOP course Github repository.

In the docker-compose file I'm using for this tutorial I have 3 services (2 database services and one web service) as you can see in this snippet:

docker-compose.yml



version: '3'

volumes:
  mysql_data:
  mysql_data_test:

networks:
  dev_network:

services:
  web:
    build:
      context: .
    ports:
      - 8080:80
    volumes:
      - .:/var/www/html
      - ./dockerconfigs/000-default.conf:/etc/apache2/sites-enabled/000-default.conf
      - ./dockerconfigs/php.ini:/usr/local/etc/php/php.ini
    environment:
      APACHE_DOCUMENT_ROOT: /var/www/html/public
      XDEBUG_CONFIG: remote_host=host.docker.internal
    networks:
      - dev_network
    restart: on-failure

  db:
    image: mysql:5.7
    volumes:
      - mysql_data:/var/lib/mysql
    environment:
      - MYSQL_DATABASE=devDB
      - MYSQL_ROOT_PASSWORD=test
    networks:
      - dev_network
    restart: on-failure


  db_test:
    image: mysql:5.7
    volumes:
      - mysql_data_test:/var/lib/mysql
    environment:
      - MYSQL_DATABASE=devTestDB
      - MYSQL_ROOT_PASSWORD=test
    networks:
      - dev_network
    restart: on-failure

The web service from the docker-compose file above references this Dockerfile in its build context



FROM php:7.3-apache

LABEL maintainer="Peter Fisher"

RUN apt-get update          \
    && apt-get install -y   \
        git                 \
        zlib1g-dev          \
        zip                 \
        unzip               \
        libxml2-dev         \
        libgd-dev           \
        libpng-dev          \
        libfreetype6-dev    \
        libjpeg62-turbo-dev \
        libzip-dev          \
    && pecl install xdebug                                                             \
    && docker-php-ext-install mysqli pdo_mysql iconv simplexml                                      \
    && docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/  \
    && docker-php-ext-configure zip --with-libzip                                                   \
    && docker-php-ext-install gd zip                                                                \
    && docker-php-ext-enable xdebug                                                          \
    && apt-get clean all                                                                            \
    && rm -rvf /var/lib/apt/lists/*                                                                 \
    && a2enmod rewrite headers

RUN curl -sS https://getcomposer.org/installer | php -- --filename=composer --install-dir=/bin

ENV PATH /root/.composer/vendor/bin:$PATH
EXPOSE 9000

WORKDIR /var/www/html

Note: Xdebug, PHP, PHP-CLI and Apache are installed in the Dockerfile for the web service. Also notice that Xdebug's port 9000 is exposed.

Configuring Xdebug

In the web service for the docker-compose file I am binding a php.ini file from my host machine into the web service container.



services:
  web:
    build:
      context: .
    ports:
      - 8080:80
    volumes:
      - .:/var/www/html
      - ./dockerconfigs/000-default.conf:/etc/apache2/sites-enabled/000-default.conf
      - ./dockerconfigs/php.ini:/usr/local/etc/php/php.ini

The php.ini file I am binding to the container has some configuration options for Xdebug added at the end like so:



[XDebug]
xdebug.remote_port=9000
xdebug.remote_enable=1
xdebug.remote_connect_back=1
xdebug.remote_autostart=1

There are other ways to configure xdebug besides modifying your php.ini file. Specifically you could use an xdebug configuration file or you could add the configurations for xdebug as an environment variable right in your docker-compose. In the docker-compose file I'm using for this article, I set the xdebug configuration for remote_host under the web service container like so



 environment:
      APACHE_DOCUMENT_ROOT: /var/www/html/public
      XDEBUG_CONFIG: remote_host=host.docker.internal

So putting everything together my PHP project in PhpStorm currently looks like this (it's a new Laravel project I scaffolded for this tutorial).

Part 2: Configuring PhpStorm

Follow these steps to create a new PhpStorm Run/Debug configuration.

Right click on your docker-compose.yml file and select "Create docker-compose.yml..."

If you don't see this create option, wait a bit and right click again. I observed that it takes a while to show up.

A configuration box will pop up where you can modify some options. I usually change the name of my configuration to something more memorable and I like to check the force builds option. But you can leave the defaults.

Run your new run/debug configuration

Configure PHPStorm's local server to point to your docker service by going to File > Settings > Languages & Frameworks > PHP > Servers. Add a new server and set the host to localhost and the port to whatever port you are publishing your docker-compose web service to.

In my case I am publishing my web service's port 80 to my host container's port 8080 so I'd be filling in 8080 for port.

Select the Use path mappings option since we are working with docker containers and map your project root on your host to your project root in the docker container.

Follow these steps to configure PhpStorm's PHP CLI interpreter

Go to File > Settings > Languages & Frameworks > PHP to configure PHP's CLI. Click the Add a new CLI interpreter from Docker, vagrant... option. Then select docker-compose

Select the docker-compose service that contains your PHP installation. For me, that's the web service.

Ensure to select the Connect to existing container option. This is important if you're working with a separate container for your database and it ensures that PhpStorm has access to the network your containers are joined to. If you select the Always start a new container option, then each time your run PHPUnit within PhpStorm, a new container that is not joined to any of your networks will be created.

Follow these steps to configure PHPUnit in PHPStorm

Go to File > Settings > Languages & Frameworks > PHP > Test Frameworks and add a new PHPUnit by Remote Interpreter configuration. In the dialog that comes up, select the CLI interpreter we created in the previous step from the drop-down.

Now you should be able to run your PHPUnit tests within PhpStorm without doing a docker exec into your service container.

Follow these steps to get PHPStorm to listen for XDebug on the right port

Go to File > Settings > Languages & Frameworks > PHP > Debug and ensure PhpStorm is listening for Xdebug on port 9000 and can accept external connections. Then ensure that PhpStorm is listening for debug connections.

Troubleshooting

If you find that you're not able to receive Xdebug connections then ensure your path mappings are in order see this step
If you are using multiple containers in a network, ensure that your PHP CLI interpreter has the Connect to existing container option set