Building c-lightning plugin with .NET

Update 2022/05/20: Based on the methodology described in this article, I have created a package for building your own plugin. please check out DotNetLightning.ClnRpc

Recently, I have been modifying my own Lapps to work as a c-lightning plugin in order to make them work with not only with LND but also with c-lightning (What are Lapps?).

I couldn't find someone doing it with .NET/C#/F#, so I will leave here a guideline for how to.

I also made a sample application, so if you are more interested in the sample than the explanation, please go there. My application is in F#, but this sample is written in C#. The language is different, but the basic approach is the same in both cases.

In this case, we will assume that there is a single class named MyAwesomeRpcServer, that has method handlers for RPC invocation. It starts out like this

using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using StreamJsonRpc;

namespace HelloWorldPlugin.Server
{
  public class MyAwesomeRpcServerOptions
  {
    public string GreeterName { get; set; } = "World";
    public bool IsInitiated { get; set; }
  }

  public class MyAwesomeRpcServer
  {
    private readonly MyAwesomeRpcServerOptions _opts;

    public MyAwesomeRpcServer(MyAwesomeRpcServerOptions opts)
    {
      _opts = opts;
    }

    [JsonRpcMethod("hello")]
    public async Task<string> HelloAsync(string name)
    {
      return $"hello!! {name}! This is {_opts.GreeterName} !!";
    }
  }
}

The requirements are

1. We want to expose an rpc method called hello.
2. We want to return a message containing GreeterName passed as a command line argument
3. We want to return a message that includes the name passed as an RPC parameter from the User.
4. The number of exposed RPC may increase in the future.

1. Create a JsonRPCServer using StreamJsonRPC.

First, create an application that acts as an ordinary JsonRPC 2.0 server.
The standard way to do this is to use the StreamJsonRpc library authored by MS.

If you want to use it as a normal web server over HTTP, this sample is a good reference.
However, the c-lightning plugin communicates with lightningd itself via standard input/output, so this time we will use Console.In and Console.Out for transport.

It looks like this

var rpcServer = new MyAwesomeRpcServer();
var formatter = new JsonMessageFormatter();
var handler = new NewLineDelimitedMessageHandler(Console.OpenStandardOutput(), Console.OpenStandardInput(), formatter);
var rpc = new JsonRpc(handler);
rpc.AddLocalRpcTarget(rpcServer, new JsonRpcTargetOptions());
rpc.ExceptionStrategy = ExceptionProcessing.CommonErrorData;
rpc.StartListening();

The Json RPC 2.0 specification does not specify the way of separating each messages as well as transports, so we must choose a suitable one.
Since the c-lightning plugin uses newlines to delimit messages, we use NewLineDelimitedMessageHandler as a message handler.
Also, if the communication partner is not C# (which is not in our case), it is a good to specify ExceptionStrategy as ExceptionProcessing.CommonErrorData.

2. Make read/write operations thread-safe.

Normally, the RPC server runs in multi-threaded mode, but writing to stdin/output in parallel will result in message corruption.
The easiest way to deal with this problem is to process requests sequentially.

In this case, the easiest way is to use AsyncSemaphore, as described in the official guidelines.

In the case of plugins, performance issues related to rpc calls are rare, so we decided to limit writes by acquiring the semaphore for all methods.

3. Automatic generation of methods

The c-lightning plugin must expose two RPC methods: getmanifest and init.

The former is to pass information from the plugin to c-lightning, and the latter is to pass information other way around when c-lightning is launched.

Much of the processing logic can be generated automatically from other plugin method definitions, so we use reflection to keep things generic enough.

For example, getmanifest looks like this

    [JsonRpcMethod("getmanifest")]
    public async Task<ManifestDto> GetManifestAsync(bool allowDeprecatedApis = false, object? otherParams = null)
    {
      using var releaser = await _semaphore.EnterAsync();
      var userDefinedMethodInfo =
        this
          .GetType()
          .GetMethods(BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly)
          .Where(m => !m.IsSpecialName && !String.Equals(m.Name, "initasync", StringComparison.OrdinalIgnoreCase) && !String.Equals(m.Name, "getmanifestasync", StringComparison.OrdinalIgnoreCase));
      var methods =
        userDefinedMethodInfo
          .Select(m =>
          {
            var argSpec = m.GetParameters();
            var numDefaults =
              argSpec.Count(s => s.HasDefaultValue);
            var keywordArgsStartIndex = argSpec.Length - numDefaults;
            var args =
              argSpec.Where(s =>
                !string.Equals(s.Name, "plugin", StringComparison.OrdinalIgnoreCase) &&
                !string.Equals(s.Name, "request", StringComparison.OrdinalIgnoreCase))
                .Select((s, i) => i < keywordArgsStartIndex ? s.Name : $"[{s.Name}]");

            var name = m.Name.ToLowerInvariant();
            if (name.EndsWith("async"))
              name = name[..^5];
            return new RPCMethodDTO
            {
              Name = name,
              Usage = string.Join(' ', args),
              Description = _rpcDescriptions[name].Item1,
              LongDescription = _rpcDescriptions[name].Item2
            };
          });

      return new ManifestDto
      {
        Options =
          // translate `System.CommandLine` to c-lightning compatible style.
          CommandLines.GetOptions().Select(CliOptionsToDto).ToArray(),
        RpcMethods = methods.ToArray(),
        Notifications = new NotificationsDTO[]{},
        Subscriptions = new string[]{},
        Hooks = new object[] {},
        Dynamic = true,
        FeatureBits = null
      };

init is as follows. This is more difficult to automate than getmanifest, and the process varies depending on the requirements of the application.

    [JsonRpcMethod("init")]
    public async Task InitAsync(LnInitConfigurationDTO configuration, Dictionary<string, object> options)
    {
      using var releaser = await _semaphore.EnterAsync();
      foreach (var op in options)
      {
        var maybeProp =
          _opts.GetType().GetProperties()
            .FirstOrDefault(p => string.Equals(p.Name, op.Key, StringComparison.OrdinalIgnoreCase));
        maybeProp?.SetValue(_opts, op.Value);
      }
      _opts.IsInitiated = true;
    }

init passes method arguments as objects.
The object has fields named configurations and options, So the method handler on the C# side must have a same name to automatically bind.

In this case, we only bind the startup options givin by the c-lightning side to _opts.
LnInitConfigurationDto's information on the c-lightning side (e.g. is an user using tor? What lightning feature bits are supported? ) can be used to freely perform the initialization process.

We set IsInitiated at the completion of the initialization process.
This is used later to suspend the server startup process until the init message is received.

4. JsonRPCLogger

The fact that c-lightning and the plugin communicate via standard input/output means that logs cannot be sent to standard output at will.

The existing c-lightning plugin sends a message to the c-lightning's log rpc method as an rpc notification and leaves the rest of the processing to the c-lightning side.
In this way, the logs are unified on the c-lightning side, making them easier to read.

.NET is bundled with ConsoleLogger, but this performs the writing process as a background task, so the aforementioned thread race may occur if we implement the logging logic in the same way.

To solve these problem, I have created my own ILoggerProvider.

5. startup process

c-lightning starts the plugin specified by the user at startup, passing the environment variable LIGHTNINGD_PLUGIN=1 when launching.

Thus, the presence or absence of this environment variable can be used to determine in the program whether the application should be run as a plugin or not.

In my case, I refer to this variable in two places
The first is where you configure the host.

Action<IHostBuilder> configureHostBuilder = hostBuilder =>
{
  var isPluginMode = Environment.GetEnvironmentVariable("LIGHTNINGD_PLUGIN") == "1";

  if (isPluginMode)
  {
    hostBuilder
      .ConfigureLogging(builder => { builder.AddJsonRpcNotificationLogger();})
      .ConfigureServices(serviceCollection =>
        serviceCollection
          .AddSingleton<MyAwesomeRpcServerOptions>()
          .AddSingleton<MyAwesomeRpcServer>()
      );
  }
  else
  {
     // configuration for running as a normal webserver.
  }
};

The other is during Host execution.
We delay the execution of IHost.RunAsync() as follows.

  var host = hostBuilder.Build();
  var isPluginMode = Environment.GetEnvironmentVariable("LIGHTNINGD_PLUGIN") == "1";
  if (isPluginMode)
    await host.StartJsonRpcServerForInitAsync(); /// initialize only rpc server and listen to `init` call.
  await host.RunAsync();

This will delay the startup of BackgroundService and log output until the init message is received from lightningd.

6. compile as a single binary

Finally, the plugin must be compiled as an executable single binary.
Therefore, we utilize the Single file Executable Compilation feature of .NET

And that's it!
You can specify the --plugin or --plugin-dir of c-lightning to launch the plugin.

publish as a library?

At first, I thought about converting what I have done this time into a library and releasing it to the public, but since it is not that hard to do it, and the init process is difficult to automate, I will just leave the document and examples here.

If you want to make it into a library, probably the best way is to use the Source Generator