DEV Community: R

Using Discourse to Collaborate on n8n Workflows

Wed, 01 Oct 2025 04:08:50 +0000

This should have been simple to set up, and if it were, I wouldn't be writing this. If you've spent any time on the n8n community forums then you've probably seen a really nifty feature where a forum post will render a workflow, right in the post, as if you had a mini-n8n instance embedded there to let you poke around in the workflow-node settings, copy part of the posted workflow for your own use, zoom in and pan around to take a closer look at things, etc. That's interesting on its own, but you can also set that up for your own internal purposes. For instance, if you wanted to enable a development team to share tips and workflow fragments that cannot (or should not) be shared on a public forum, you could set up a private instance of the same software n8n uses for their community forums.

Enough about the what and why... on to the how...

Discourse

Discourse is the software used by n8n for their community forums, and if you've spent much time searching/reading online discussions lately, you'll recognize Discourse as a VERY common choice. I'll have to limit the coverage of installing Discourse to "pre-requisite" status in this article, because the Discourse install/setup is a convoluted mess that has already been argued to death and it just can't be summarized into a few simple steps. However, you'll find information about how to get Discourse running here (good luck). Once you have that done, and you're logged in as an administrator, you're ready to continue setting up the n8n-demo component and plugin.

n8n-demo component

Before jumping through the hoops to integrate the demo component into your Discourse server, it will probably help to check it out standalone. To do that, you just need to grab the source JSON from a workflow in n8n, and follow the instructions here. Getting that working amounts to adding a few <script> tags and a <n8n-demo> tag to an html page, and loading it in a browser. This isn't a huge challenge, but it will probably help to see that in action to understand better how the component works within Discourse.

Discourse plugin

The discourse-n8n-wf plugin is a wrapper for (part of) the n8n-demo component, which enables it in a Discourse forum post by intercepting the handling of a markdown code-block. Getting the plugin installed and configured in Discourse required quite a bit of guessing and reading between the lines in the Discourse documentation and the plugin documentation (which is a bit sparse). Also, heads up, the plugin is only a wrapper for the <n8n-demo> tag part of the n8n-demo component; the <script> tags need to be configured separately in the Discourse admin dashboard to get everything working.

Steps

I couldn't find anywhere that had all of this in one place, so that's the main point of this article. Hoping this serves to eliminate some head scratching (or pounding) for someone.

Set up Discourse, register the admin login, and be sure you can get to the Admin Dashboard
- Click your Avatar -> Click the Discourse UI -> Avatar -> person icon -> Preferences -> Admin button -> Click Preferences -> Click the Admin button
Add the plugin to the Discourse app.yml (find this block, and add the last line shown here)

hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git
          - git clone https://github.com/n8n-io/discourse-n8n-wf.git

Rebuild the Discourse container
- launcher rebuild app
Verify that the plugin is installed.
- Admin Dashboard -> PLUGINS left-menu -> Installed Plugins sub-menu -> Search "N8n wf"
Configure the plugin in 2 places, neither of which is in the grayed-out "Settings" button on the "Installed Plugins" list item.
- Admin Dashboard -> SECURITY left-menu -> Security sub-menu -> filter: "iframes"
- Add an "Allowed iframes" item with https://my-n8n-server.example.com/workflows/demo
  - the /workflows/demo part is fixed
  - match the protocol (http or https) and host name to your own server.
- Admin Dashboard -> All site settings left-menu -> filter: "N8n-core-url"
- Fill in just the base URL matching the "Allowed iframes" URL above, like https://my-n8n-server.example.com
- Note: Couldn't find a way to navigate directly to this setting, so it's sorta hidden / hard to find.
Add a theme "component" to all themes with the script tags for the n8n-demo component
- Admin Dashboard -> APPEARANCE left-menu -> Themes & compo... sub-menu -> Components "tab"
- Click Install button
- Click + Create new option
- Enter Name: n8n-demo component (or a similar name you like with at least 4 characters apparently)
- Leave "Component" selected as the type.
- Click the Create button
- Click Add all themes and then the green check button.
- Click the Edit Code button
- Switch to the <head> tab
- Paste in the script tags from here
- Click the Save button
Navigate back to the forum
To verify that the rendering component is working, create a post with workflow JSON enclosed between 2 separate lines with the markdown code-block (3-back-tic) delimiters like:

```

{
  "nodes": [
    {
      "parameters": {},
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        0,
        0
      ],
      "id": "ad04e21c-5d86-4463-8f03-0cd375224997",
      "name": "When clicking ‘Test workflow’"
    },
    {
      "parameters": {
        "url": "https://httpbin.org/get",
        "sendQuery": true,
        "queryParameters": {
          "parameters": [
            {
              "name": "pure",
              "value": "magic"
            }
          ]
        },
        "options": {}
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        220,
        0
      ],
      "id": "10394cdf-c137-4cc9-8f17-c1510bb5380b",
      "name": "HTTP Request"
    }
  ],
  "connections": {
    "When clicking ‘Test workflow’": {
      "main": [
        [
          {
            "node": "HTTP Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {},
  "meta": {
    "instanceId": "1234"
  }
}

```

Rendering On Your Own n8n

By default, the n8n-demo tag generated by the n8n-discourse-wf plugin submits the workflow source to https://n8n-preview-service.internal.n8n.cloud/workflows/demo, via iframe, to be rendered in graphical (editor) form. Not only is this a potential "leak" of your workflow data, but it consumes resources that n8n hosts to support their own community forums, and is an external element over which you have no control, for capacity, availability, or security. There is currently no official, documented way to override this, but it should be possible to change the src attribute and point it instead to the URL specified in the N8n-core-url / Allowed iframes settings mentioned above. I'll add an update here when/if I find a way to do that. Please leave a comment if you find a way.

UPDATE: When Discourse installs the plugin, it clones the files from a git repository. One way to override the src attribute in the <n8n-demo> tag is to fork the plugin repo and edit the code to hard code the URL of the n8n instance where rendering should be done.
UPDATE: There is an environment variable that tells n8n to run in "preview mode" here which is necessary to make the /workflows/demo URL path work without requiring a login.

Wemos D1 Mini w/ Waveshare e-Paper 2.13 HAT

Tue, 16 Sep 2025 00:20:37 +0000

AI flat-out lied to me several times while trying to get this working, so now I guess I'm giving AI something to read to (maybe... but I doubt it) get it right when someone else goes looking for this specific info.

The title already tells you this will cover a fairly specific set of circumstances. I imagine it could be expanded to apply to similar things, but I'll bet this is an exact match for what someone is looking for.

So... I was trying to line up all the planets to get a Wemos D1 Mini working with a Waveshare 2.13" e-Paper HAT display. If you found this, you probably already know what a nightmare it can be to figure out which pins to use on a microcontroller / development board, since it is apparently just too much to ask that any 2 given examples of hardware or software reference anything by the same names.

Reconciling Pin Names

For each of the following where a D1 Mini pin connects to a line on the e-Paper HAT, the convention is SPI_Common_Name / D1_Mini_Label ~ D1_Mini_AltLabel (if any) / Raw_Pin / GPIO_Pin_Name / Arduino_Macro_Constant (if any) -> Harness_Wire_Color -> e-Paper HAT Pin/Label

+3.3v -> Gray -> VCC

Supply and signal voltage should be 3.3v for everything.

Ground -> Brown -> GND

Be sure the ground on the D1 Mini and the e-Paper HAT are connected to each other.

MOSI/MOSI~13/13/GPIO13/D7 -> Blue -> DIN (SDI)

The first source of confusion is that the interface for the e-Paper HAT is SPI, but data only flows from the microcontroller to the display, so there is DIN, but no DOUT/SDO to connect to MISO. DIN (Data-IN, sometimes labeled SDI for "Serial Data In") connects to the pin that has traditionally been called MOSI. In fact, the screen print on the D1 Mini labels it as such (and also 13). The D7 designation is from the confusing Arduino pins naming system. This and other pins are sometimes referenced using the "digital pin" number so this applies to some of the other connections too. In code, you should be able to literally specify 13 and you may find that there is a macro named D7 that substitutes to 13.

SCLK/SCK~14/14/GPIO14/D5 -> Yellow -> CLK

This is assumed within the GxEDP2 library. Sketch code doesn't need to reference this, but the hardware needs to be connected correctly.

SS/SS~15/15/GPIO15/D8 -> Orange -> CS

This is another unnecessary name change from the traditional abbreviation. CS supposedly stands for "Chip Select" or some references may say it means "Channel Select" In any case, it's the part of SPI that allows daisy-chaining multiple devices. Oops... shouldn't have mentioned anything involving a chain either. Word/thought police are en-route for sure now.

???/0/0/GPIO0/D3 -> Green -> DC

This is another source of confusion since the "DC" or "Data/Command" pin isn't part of a normal SPI connection. This is apparently part of the protocol for controlling the e-Paper HAT where the driver can tell the display device whether the data being sent to DIN is Data, or a Command.

???/2/2/GPIO2/D4 -> White -> RST

This pin is what the driver uses to reset the e-Paper HAT.

???/4~SDA/4/GPIO4/D2 -> Purple -> BUSY

This pin is what the driver uses to determine if the e-Paper HAT is ready to receive more data or commands.

Hardware Connection Special Considerations

D8/GPIO15 pull-down. Without a 10K resistor connecting this pin to ground, the level converters in the "HAT" board will probably interfere with the D1 Mini boot process.
D4/GPIO2 pull-up. Without a 1K resistor connecting this pin to +3.3v, the level converters in the "HAT" board will probably interfere with the D1 Mini load/programming/reset process.

Code

Once you have the hardware hooked up, you'll need a working example of the code. Instead of duplicating the details here, you can check all that out in this git repository .

The README.md file there has links to libraries and other references that might help too.
Note: The source is set up for use in VSCode with the excellent PlatformIO extension, not the Arduino IDE. If you haven't tried that yet, you don't know what you're missing... give it a go.

Using AWS SDK from n8n Code Node

Sun, 18 May 2025 21:00:43 +0000

HTTP Request vs. JS Client Code

Many of the service-integration nodes in n8n are just customized wrappers for a RESTful, http based API. If the entire API is not supported in a particular service-integration node, it is often possible to use an HTTP Request node to implement other RESTful service API calls. It is even possible to "borrow" the credentials item from the service-integration node and use it in the HTTP Request node.

Some services, like (Amazon) AWS services are (apparently) are too complex to support with a simple http based client, so they generally require the use of an SDK client library, and cannot be implemented within n8n by simply using an HTTP Request node. n8n also has a Code node that can use npm based libraries/modules with a bit of special configuration in the n8n runtime environment. This article shows how to use AWS SDK libraries in an n8n Code node.

Prerequisites

n8n self-hosted, preferably running in Docker / Docker Compose
AWS Account + AWS IAM User + AWS Access Key for the IAM User
Basic knowledge of Javascript and JS Object Notation
Basic knowledge of Docker, docker build, docker compose

Overview / Steps

Get an Access Key & Access Secret for an AWS IAM User.
Assign Permissions (Policies), to the IAM User, for the SDK to be used.
Create an n8n container with the SDK npm libraries installed.
Configure the Docker runtime (compose) with required environment vars.
Use an n8n Code node to run SDK client code in a workflow.

Example SDK: Polly Text-To-Speech

To demonstrate the use of an AWS SDK library, and handling a binary data response, the example use-case here will call the Polly Text-to-Speech (TTS) service to generate an audio file from text input. The general approach to installing, enabling and using AWS SDK libraries should be applicable to other SDKs, but it's sometimes difficult to understand instructions given in generalized/abstract terms, so hopefully it will be easier to understand this concrete example.

SDK Access Policy

Before using any of the various AWS services via SDK, the authenticated user needs to have access to the service. AWS manages this primarily through policies. Access to the Polly API service can be granted by adding the AWSPollyFullAccess policy to the IAM user to which the Access Key/Secret belongs. (Links below to the AWS user/policy management page.)

Setting up the n8n Runtime Environment

This assumes n8n is running self-hosted, in a Docker container. There are 3 elements of the n8n runtime environment that need to be customized in order to get an AWS SDK client to work.

Install the SDK module/library, and its dependencies
Allow n8n to use the SDK module/library (and dependencies)
Provide configuration/environment-variable-values required by the SDK.

Installing (Polly) SDK and AWS CredentialsProviders npm modules/libraries

Use docker build with a Dockerfile like the following to create a customized n8n (Docker) container image with the AWS SDK npm libraries added.

Dockerfile

  ARG tag=latest
  FROM n8nio/n8n:$tag
  USER root
  RUN npm install -g @aws-sdk/credential-providers
  RUN npm install -g @aws-sdk/client-polly
  USER node

Allow n8n to use the AWS SDK modules/libraries

Use the image created in the first step instead of the base, published n8n image, specify (or add) the npm library/module names in the NODE_FUNCTION_ALLOW_EXTERNAL environment variable's csv list

docker-compose.yml

services:
  n8n:
    image: n8n-with-aws-sdk-addons
  ...
    environment:
      - N8N_HOST=n8n
      - N8N_PORT=5678
      ...
      - NODE_FUNCTION_ALLOW_EXTERNAL=@aws-sdk/credential-providers,@aws-sdk/client-polly

Set AWS SDK credentials environment variables

In this case, the Code node will use the fromEnv() credentials provider, so it will need environment variables with the access key and secret.

docker-compose.yml

services:
  n8n:
  ...
    environment:
      - N8N_HOST=n8n
      - N8N_PORT=5678
      ...
      - AWS_ACCESS_KEY_ID={your-aws-access-key}
      - AWS_SECRET_ACCESS_KEY={your-aws-secret}

Create a `Code` node to use the SDK Library/Module

The following example code takes a number of input items and, for each one, calls the Polly AWS service to convert the input text to a generated audio file.

The input item must include the text-to-speech input text in an attribute named ttsTextInput like this:

{
  "otherAttribute": "any other json content",
  "ttsTextInput": "What would a wood chuck chuck if a wood chuck could chuck wood."
}

Code node Javascript

const { PollyClient, SynthesizeSpeechCommand } = require("@aws-sdk/client-polly");
const { fromEnv } = require("@aws-sdk/credential-providers");

/*
 * This gets the credentials from environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
 */
const config = {
  region: "us-east-1",
  credentials: fromEnv()
}

const client = new PollyClient(config);

async function sendPollyRequest(ttsTextInput) {
  const input = {
    OutputFormat: "mp3",
    Text: ttsTextInput,
    VoiceId: "Joanna"
  };
  const command = new SynthesizeSpeechCommand(input);
  const response = await client.send(command);
  const pollyResponseAudioBytes = await response.AudioStream.transformToByteArray();
  // console.log(`Received polly response with ${pollyResponseBytes.length} bytes`);
  return pollyResponseAudioBytes;
}

for (const item of $input.all()) {
  const pollyResponseAudioBytes = await sendPollyRequest(item.json.ttsTextInput);
  const pollyResponseAudioBytesBuffer = Buffer.from(pollyResponseAudioBytes);
  const n8nBinaryDataPollyAudio = {
       data: await this.helpers.prepareBinaryData(pollyResponseAudioBytesBuffer,
            'polly-audio-response.mp3', 'audio/mp3')
  }

  // console.log(`Binary data item: ${JSON.stringify(n8nBinaryDataPollyAudio)}`);
  item.binary = n8nBinaryDataPollyAudio;
}


return $input.all();

Links and References

AWS SDK Polly documentation
AWS SDK credentials providers documentation
AWS Billing Console Page - To Monitor Polly API Charges
AWS Cloudwatch Console- Polly Dashboard - To Monitor Polly API Usage
AWS IAM User Management Dashboard (Note: Users may be defined in a different region depending on your account.) ** AWSPollyFullAccess Policy
n8n Code Node Documentation
n8n Binary Data Buffer Info (Note: This isn't used in the code above, but may provide some help if things change in n8n regarding binary data handling).
n8n AWS S3 Node Documentation - Example of an AWS "built in" node. Others include SES, SQS, Lambda, DynamoDB, etc. (but not Polly as of this writing).

Notes

This article does not cover connectivity issues. If n8n is unable to reach AWS, the answers will not be here.
This article does not offer any advise on how to manage the cost of AWS services. Consuming API services without monitoring usage can lead to unexpected charges.
There are alternatives in n8n that might be easier to implement, like installing the AWS SDK CLI (command line interface) and using an Execute Command node in the workflow, or running a "sidecar" service that wraps AWS SDK functions with a RESTful/http-based API service and using an HTTP Request node to call that service instead of directly accessing AWS.
Docker Compose could be used to run the Docker build and create a container image from the Dockerfile automatically, instead of using docker build separately, but that is another topic that is covered in the Docker documentation.

Watching HTTP Traffic from n8n with mitmproxy

Tue, 22 Apr 2025 16:05:30 +0000

Console logging and error messages in n8n sometimes include very little useful information for troubleshooting http requests that a workflow sends to one of the integrated services. Fortunately, most (if not all) of the http requests n8n sends, use Axios, and Axios can be configured with environment variables to use a proxy for http and/or https. Then, mitmproxy can be run in a docker container to forward those requests and capture/display the details in an easy-to-use web UI.

Prerequisites

n8n running in Docker, preferably started using docker-compose

Goal

Set up mitmproxy to run in Docker, and configure n8n to use mitmproxy for https/http requests.

Steps (overview)

Set up mitmproxy to run in docker (using docker compose)
Adjust the runtime configuration (docker-compose.yml) of n8n to include environment variables that instruct Axios to use mitmproxy.

mitmproxy docker-compose.yml

This docker-compose.yml file was adapted from this example, but...

The DNS parts were removed
The networks were adjusted to actual RFC 1918 private networks.
- 172.201.. and 172.202.. overlap public, "real" address ranges and are blocked by default in mitmproxy.
- Reported here
Some other adjustments (e.g. tty: true) were made so that the current Python-based mitmproxy code will start correctly in the docker container.
A default password is specified for the mitmproxy web UI so it won't be necessary to watch the console for a generated password.

docker-compose.yaml

services:
  mitmproxy:
    image: mitmproxy/mitmproxy
    hostname: mitmproxy
    restart: always
    tty: true
    volumes:
      - ./mitmproxy_data:/home/mitmproxy/.mitmproxy
      # - './cert:/root/.mitmproxy'
    entrypoint: mitmweb
    command: '--web-host 0.0.0.0 -m regular@80 -m regular@443 --set web_password=changeme123'
    networks:
      mitm_internal:
        ipv4_address: 192.168.115.80
      mitm_external:
        ipv4_address: 192.168.116.80
    ports:
      - '38080:80/tcp'
      - '38443:443/tcp'
      - '38081:8081/tcp'

networks:
  mitm_internal:
    name: mitm_internal
    internal: true
    ipam:
      driver: default
      config:
        - subnet: 192.168.115.0/24
          gateway: 192.168.115.1
  mitm_external:
    name: mitm_external
    ipam:
      driver: default
      config:
        - subnet: 192.168.116.0/24
          gateway: 192.168.116.1

Adjustments to the n8n docker-compose.yml

Add the docker network to the n8n service/container

services:
  n8n:
    ...
    networks:
      - mitm_external
...
networks:
  mitm_external:
    name: mitm_external
    external: true

Add environment variables to tell Axios to use mitmproxy
- Note: These are not n8n environment variables, but are used directly by the [Axios] npm library/module.
- Note: https requests could also be routed through the https listener on mitmproxy, but that potentially introduces certificate related issues.

services:
  n8n:
    ...
    environment:
      ...
      - "HTTPS_PROXY=http://mitmproxy:80"
      - "HTTP_PROXY=http://mitmproxy:80"

Using mitmproxy with n8n

After starting the mitmproxy container and restarting the n8n container to pick up changes to networks and environment variables...

Open the n8n editor to a workflow which sends out http/s requests.
Open the mitmproxy web UI in another browser or browser-tab.
- http://localhost:38081
- Log in with the web_password specified in the mitmproxy docker-compose.yml ("changeme123" if you didn't modify it).
Execute the workflow (or http based workflow step), and the mitmproxy web UI should begin showing the requests as they are sent.
Click on one of the request items to see details about the request, response, connection, etc.

Alternative to "Global" Proxy Config

Instead of setting the HTTPS_PROXY and/or HTTP_PROXY environment variables, which sends all Axios traffic through the proxy, some n8n nodes, like the HTTP Request node, have a Proxy option that can be explicitly routed to mitmproxy. For nodes that support this:

Add the Proxy option in the node configuration
Set the target proxy URL to http://mitmproxy

Configuring OpenWRT DDNS for freedns.afraid.org

Sat, 26 Oct 2024 17:57:19 +0000

Overview

freedns.afraid.org is a DNS provider that offers free registration of subdomains on a long list of "public" (shared) top-level domains.
This is a more friendly alternative for use with Dynamic DNS clients, than many similar services, because it does not require a periodic "manual" intervention to renew a registered subdomain just to keep it active. Other such services seem to use the "manual renewal" as a tactic to "inconvenience" people into paying for their service.

Setting up the DDNS client in OpenWRT to monitor for changes in a dynamic IP address, and update freedns.afraid.org, is a little confusing, because the DDNS scripts do NOT use their "Direct URL" approach for IP updates.

Official Docs

The freedns.afraid.org docs describe settings that might work with some OpenWRT/DDNS configurations, but the DDNS client parameters didn't exactly work in my case. The other option, which uses cron and curl, might work, but isn't the usual/proper way to have OpenWRT handle DNS updates, and anything non-standard ends up burning extra time to document it well, or extra time later trying to remember how it is working.

"Direct URL" vs. DDNS Client

"Direct URL"

The biggest source of confusion while setting this up came from what freedns.afraid.org calls the "Direct URL." This URL has the path /dynamic/update.php?{authtoken}. It is intended to be called/fetched from anywhere within a NAT network, and the server extracts the "apparent" originating IP (outside all NAT layers) from the request, as the IP to associate with the subdomain name. The token embeds authentication information, and the reference to the subdomain, so the request requires no other parameters. This URL, and its "token" are NOT used by the OpenWRT DDNS client.

DDNS Client

freedns.afraid.org ALSO supports typical ddns client URLs with the path /nic/update, which accepts various URL parameters like hostname, and myip. THIS is what the OpenWRT DDNS client update script generates and uses. Authentication for this update option at freedns.afraid.org is HTTP basic, and requires the username:password for the freedns.afraid.org account, NOT the token, and NOT the FQDN of the subdomain (as the line option username 'yourhostname' in the docs incorrectly suggests).

DDNS IP resolution within NAT

One important aspect of the OpenWRT setup is whether the OpenWRT router is behind one or more NAT (network address translation) layers. If so, the default wan option for IP address source WON'T WORK, since it will discover an IP address that is still "hidden" within the private network. The DDNS client needs to reach outside the NAT/private network to get the "public facing" IP. See instructions below re: using the URL option instead.

How To

Now, with some of the confusing bits cleared up (let's hope), here are the steps to get things going.

Sign up an account at freedns.afraid.org
Decide on a suitable subdomain name, and add it to the account under one of the public domains controlled by freedns.afraid.org.
- i.e. Click "Add" on https://freedns.afraid.org/subdomain/
- Note: There are other options (transfer domain, etc.) too, but establishing a "registered" FQDN is the goal for this step.
If not already installed, add the DDNS client/service to OpenWRT.
- See: https://openwrt.org/docs/guide-user/services/ddns/client
- Optional: Also install the LuCI components to manage the DDNS service.
Add a DDNS "service" for freedns.afraid.org (using an example FQDN of mychosenname.example.com) (names/values in parens are for direct option entries in /etc/config/ddns without LuCI UI)
- Lookup Hostname (lookup_host): mychosenname.example.com
- DDNS Service Provider (service_name): Choose "afraid.org-basicauth"
- Domain (domain): mychosenname.example.com
- Username (username): freedns.afraid.org account username
- Password (password): freedns.afraid.org account password
Switch the DDNS service's IP address source from Network to URL IP lookup (Advanced tab in LuCI). This uses the checkip.dyndns.com utility to reach all the way out from all internal network layers and echo back the "public" IP, instead of resolving the first-layer private address (i.e. when the "wan" side of an internal router is still within a NAT/private network).
- IP address source (ip_source): Choose "URL" (web)
- URL to detect (ip_url): http://checkip.dyndns.com

Notes

Private IP Error

Either the DDNS scripts, or maybe the freedns.afraid.org update service rejected a private/"non-routable" (i.e. RFC-1918) IP address with:

 WARN : Updating IP at DDNS provider failed...
...
ERROR : No or private or invalid IP '192.168.0.2' given! Please check your configuration

LuCI vs. afraid.org-v2-token

The DDNS client in OpenWRT has another DDNS Service provider option named afraid.org-v2-token. This sounds like it would support the "Direct URL" approach, but...

Best guess for where to fill in the token is:
- Optional Parameter (param_opt)
- Or possibly Optional Encoded Parameter (param_opt_encoded)
- These were generated by LuCI, but not documented here
LuCI still requires 'username' and 'password', which are not used (so, presumably, they could be filled in with anything like notused).
The server response on an update indicates the (local/LAN) IP is still being included in the request, so the IP address source option must still be changed from Network to URL
Whatever request URL this configuration is constructing and sending to freedns.afraid.org, it does NOT seem to match the "Direct URL" format, and does not work.

Self Hosting n8n with Docker and Traefik/LetsEncrypt (for https)

Sun, 13 Oct 2024 19:32:07 +0000

Setting up n8n in Docker, without https, is relatively simple. That's covered here. However, some things (e.g. oauth2 connection to Google Drive) are difficult or impossible to do in n8n unless it is accessed using https (SSL). There are many ways to do that, including configuring n8n itself with a dedicated server certificate, but running Traefik as a reverse proxy, in front of n8n, provides automated integration with LetsEncrypt to obtain and renew a "trusted by default" server certificate, and just let n8n run without https.

Prerequisites

n8n running in Docker, preferably started using docker-compose
Access to start a Traefik container in Docker
A registered (top level) domain name with access to DNS configuration.

Goal

Set up Traefik, in Docker, such that it handles requests for a subdomain, like myn8n.example.com and forwards (reverse-proxies) that traffic to n8n, also running in Docker.

High Level Steps

Register/activate API access to domain registrar
- How to do this varies depending upon which registrar is used.
- The list of supported registrars, and links to specific instructions can be found here
Get Traefik running in a container
- This is documented here.
- Part of this example setup configures Traefik to automatically fetch a server certificate, for https, from LetsEncrypt
- Note: The example docker compose shows env-variables for a DNS/registrar provider named ovh. This part will be different with each different registrar provider. The Traefik plugin specifics for each registrar (like which ENV vars are required) can be found along with the API-access instructions from the previous step.
  - Unless you are actually using ovh as your registrar, change this: - "--certificatesresolvers.myresolver.acme.dnschallenge.provider=ovh" from ovh to the one matching your actual registrar, e.g. ionos, godaddy, namecheap, etc. etc.
  - The environment variables will also be different. For instance, if you are using GoDaddy you would include environment variables GODADDY_API_KEY and GODADDY_API_SECRET instead of all the ones starting with OVH_
Configure DNS so that a domain/subdomain name maps to the Traefik server (from the web browser's perspective)
- If this cannot be done in the public DNS, it will still work with host entries in the /etc/hosts file on one machine, a hostname/DNS feature on a router, or an "internal-network" DNS like pihole.
- Note that it is probably not a good idea to configure the public DNS at your registrar to resolve an internal-only (i.e. private/LAN) Traefik server IP address. Also, securing Traefik for public access is outside the scope of this article.
Add configuration (labels) to the n8n container to tell Traefik to handle the https access.
- The example whoami service shown in the Traefik docs is ok for checking to be sure the reverse proxy is working, but it doesn't make the importance of the "labels" section exactly clear.
- In the n8n container, there would be corresponding labels like these:

  labels:
      - "traefik.enable=true"
      - "traefik.http.routers.n8n.rule=Host(`n8n.example.com`)"
      - "traefik.http.routers.n8n.entrypoints=websecure"
      - "traefik.http.routers.n8n.tls.certresolver=myresolver"

Background Info, How Stuff Works, etc.

Supplemental Note on Docker Networks

When I set up n8n and Postgres in Docker, I put both of the containers in their own docker network for isolation from other stuff running in docker, so there was a network defined in n8n's docker-compose.yaml like:

  networks:
    n8nstack:

And each container "joined" the network like:

    services:
      n8n:
        # ...
        networks:
          - "n8nstack"
      postgres:
        # ...
        networks:
          - "n8nstack"

Traefik was running on the default docker network, not the n8nstack network, so the reverse-proxy wasn't working. To fix this, Traefik was similarly configured with its own (external) docker network, created by docker command like...

docker network create traefikproxy

and, in Traefik's docker-compose.yaml...

  networks:
    traefikproxy:
      external: true

with each container referencing the network like:

  services:
    traefik:
      # ...
      networks:
        - "traefikproxy"
    whoami:
      # ...
      networks:
        - "traefikproxy"

Somewhat obviously, the n8n docker-compose would need to be updated so the n8n service would ALSO join the traefikproxy network with:

  networks:
    n8nstack:
    traefikproxy:
      external: true

and the n8n service (not others) ALSO joins the traefik network with:

    services:
      n8n:
        # ...
        networks:
          - "n8nstack"
          - "traefikproxy"

The part that was not as obvious was that the Traefik-related labels in the n8n container/service would now need an additional label to tell Traefik which docker network to use, like

labels:
  - "traefik.enable=true"
  - "traefik.docker.network=traefikproxy"
  # ...

Note: I think this could also have been reversed such that the traefik service joined the n8nstack network, and any other stack's network too, but it seemed better to me (more encapsulated) to go the other direction.

The Use Case that Made this Necessary

The reason I got motivated to set up n8n behind a reverse proxy, with https/SSL enabled, is that there seems to be no other way to get through the oauth2 sequence for Google API access, which is required for the Google Drive node. Google wants a redirect URL on the oauth2 credentials that maps to a real, public domain name, and uses https. There may be ways to configure n8n such that it generates an https redirect url (possibly using WEBHOOK_URL) but without actually loading the n8n UI using the https endpoint, it seemed like some part of the whole oauth2 sequence would always skip the track and fail to work properly. Setting up Traefik to deal with the SSL certificate hassles was ultimately the best option.

Quick Overview of Traefik

Traefik is (primarily) a reverse proxy service that accepts and routes web traffic (from browsers and other http(s) clients) to one or more "back end services." In this context, n8n is a "back end service."

Note: If the notion of a reverse proxy isn't familiar, reading up on that would probably help.

Traefik + LetsEncrypt - Certificate Automation

This is a simplified overview of how this works...

Traefik is configured with an API key for the registrar that controls DNS for the top level domain and its subdomains.
Traefik is configured to accept https/443 requests on the domain, one or more of its subdomains, or both.
When required (i.e. when the SSL server certificate for a given host name is missing or expired), Traefik automatically sends a request to LetsEncrypt to fetch a server certificate.
LetsEncrypt responds to Traefik with a "DNS Challenge", which is basically a request to create an arbitrary DNS record, temporarily, in order to prove "ownership" of the domain.
Traefik uses the registrar API key (and a corresponding "provider" plugin) to make the DNS change, and then sends a message back to LetsEncrypt saying, "Check DNS again now."
LetsEncrypt checks DNS and, assuming the "challenge" entry/record is found, returns the server certificate for the requested domain/subdomain.
Traefik stores the certificate (for use on the proxy server) and cleans up the temporary DNS records (again using the registrar API functions).

Related n8n Details

N8N_EDITOR_BASE_URL

When n8n is reached through a reverse-proxy like Traefik (or a kubernetes load balancer, or a rule on a dedicated nginx web server, or any number of other things like that), there is an environment-variable (or n8n config) setting named N8N_EDITOR_BASE_URL to tell n8n, internally, what to use (in the UI and elsewhere) as the "external" (browser context) base URL. It may be necessary to set this in the n8n config when using Traefik.

Currently, (around n8n v1.59.0), the credentials dialog for a Google API related credential, and, therefore, the redirect URL sent to Google for oauth2, appears to formulate the redirect URL based on either:
1. how the UI itself is loaded into the browser, or
2. N8N_EDITOR_BASE_URL, if it is specified in the environment or config.

WEBHOOK_URL

Another override that might be necessary when n8n is cloistered behind Traefik, is the WEBHOOK_URL config/environment-variable.

This is used for... big surprise... webhook related things like $execution.resumeUrl
It does not appear that WEBHOOK_URL has an influence over the oauth2 redirect URL, although I think that may have been suggested in a few places. Perhaps WEBHOOK_URL also served the purpose of N8N_EDITOR_BASE_URL in a previous version of n8n.

Developing Custom Nodes for n8n with Docker

Sat, 14 Sep 2024 01:28:59 +0000

The n8n automation / orchestration / workflow tool already covers a massive number of services without adding anything to it, but there may come a time when writing your own "custom-node" makes sense. The custom-node tutorial hits the high points, but doesn't include (as of this writing) any specifics about how to develop a custom node and test it in a self-hosted instance of n8n running in a Docker container. That's the subject of this post.

Prerequisites

n8n running in Docker, preferably started using docker-compose
An n8n custom-node (typescript/npm) project that is already working

Quick Clarification

This is about developing custom-nodes for n8n, which are like plugins for the product, not to be confused with Node.js, even though n8n runs in Node.js. Assuming if you're still reading you probably know the difference, but if you thought this was about coding in Node.js, stop now. This isn't the droid you're looking for.

Goal

Map one or more custom-node projects dist subdirectory directly into an n8n Docker container such that n8n will find the "compiled" files and load the custom-node(s) at startup.

Dev Process Steps

(i.e. "How This Works Once You Have It Set Up")

For each "development cycle"...

Make changes to the custom-node source.
rm -rf dist (when necessary, in the dev project directory, to remove old/renamed file versions)
npm run build - i.e. re-compile
docker-compose restart n8n - i.e. trigger code reload in n8n

Setup

Add one or more docker volumes, mapped to the dev/source directories:
- Edit the n8n docker-compose.yaml
- The important part here is that the volumes map into the container at a path under /Users/node/.n8n/custom

services:
  n8n:
    ...
    volumes:
      - /Users/myuserid/devstuff/custom_node_dev_project_1/dist:/Users/node/.n8n/custom/node_modules/my-awesome-custom-node
      - /Users/myuserid/devstuff/custom_node_dev_project_abc/dist:/Users/node/.n8n/custom/node_modules/my-other-custom-node

That's it. Go write code.

Explanations (...if you care...)

(Read the rest of this [later] if you want to know reasons.)

npm link

The tutorial says to use npm link to map a custom-node project into n8n, but that assumes n8n is running from source, not in Docker.

Notes:
- Getting set up to run n8n from source is more time-consuming and has its own challenges, vs. running n8n in docker.
- Docker volumes don't permit access to symlinked files, so npm link won't work for this anyway.
- Trying to run npm link from the n8n custom node starter project already causes an error because the project (in package.json) is set up to require pnpm in the preinstall script, which runs implicitly from npm link and causes a conflict between pnpm and npm. :(
  - The error message is npm ERR! command sh -c npx only-allow pnpm
  - See: package.json

Effective Directory Structure

The directory structure mapped into the docker volumes mimics what npm link my-awesome-custom-node and npm link my-other-custom-node would have created, if they were run within the n8n runtime's .n8n/custom directory. Within the running Docker container, it ends up looking like this:

├── .n8n
    ├── custom
        ├── node_modules
            ├── my-awesome-custom-node
            │   ├── dist
            │   ├── ...other stuff...
            ├── my-other-custom-node
                ├── dist
                ├── ...other stuff...

Normal Docker install for a Custom-Node

Per the instructions for Install(ing) Private Nodes, you would just "Copy the node and credential folders from within the dist folder into your container's ~/.n8n/custom/ directory."

There are few issues with that strategy though.

Without some additional directory structure, which isn't specified in the instructions, it would only allow one custom-node to be presented into the n8n Docker runtime.
If the project is based on the custom node starter project, from the n8n github repo, you'd be copying the nodes and credentials directories/folders, not node and credential directories... but that could be just something that didn't get corrected/updated in the docs.
More importantly... copying stuff into a container's filesystem is transient. As soon as the container gets rebuilt/reset, all that stuff is gone. Mapping a volume to a directory on the host filesystem survives container resets, and doesn't require any overwriting. After each npm run build of the project, the container (i.e. n8n) just needs to be restarted to load the changes.

Things that might lead here...

DEV Community: R

Using Discourse to Collaborate on n8n Workflows

Discourse

n8n-demo component

Discourse plugin

Steps

Rendering On Your Own n8n

Wemos D1 Mini w/ Waveshare e-Paper 2.13 HAT

Reconciling Pin Names

+3.3v -> Gray -> VCC

Ground -> Brown -> GND

MOSI/MOSI~13/13/GPIO13/D7 -> Blue -> DIN (SDI)

SCLK/SCK~14/14/GPIO14/D5 -> Yellow -> CLK

SS/SS~15/15/GPIO15/D8 -> Orange -> CS

???/0/0/GPIO0/D3 -> Green -> DC

???/2/2/GPIO2/D4 -> White -> RST

???/4~SDA/4/GPIO4/D2 -> Purple -> BUSY

Hardware Connection Special Considerations

Code

Using AWS SDK from n8n Code Node

HTTP Request vs. JS Client Code

Prerequisites

Overview / Steps

Example SDK: Polly Text-To-Speech

SDK Access Policy

Setting up the n8n Runtime Environment

Installing (Polly) SDK and AWS CredentialsProviders npm modules/libraries

Allow n8n to use the AWS SDK modules/libraries

Set AWS SDK credentials environment variables

Create a Code node to use the SDK Library/Module

Links and References

Notes

Watching HTTP Traffic from n8n with mitmproxy

Prerequisites

Goal

Steps (overview)

mitmproxy docker-compose.yml

Adjustments to the n8n docker-compose.yml

Using mitmproxy with n8n

Alternative to "Global" Proxy Config

Links

Configuring OpenWRT DDNS for freedns.afraid.org

Overview

Official Docs

"Direct URL" vs. DDNS Client

"Direct URL"

DDNS Client

DDNS IP resolution within NAT

How To

Notes

Private IP Error

LuCI vs. afraid.org-v2-token

Self Hosting n8n with Docker and Traefik/LetsEncrypt (for https)

Prerequisites

Goal

High Level Steps

Background Info, How Stuff Works, etc.

Supplemental Note on Docker Networks

The Use Case that Made this Necessary

Quick Overview of Traefik

Traefik + LetsEncrypt - Certificate Automation

Related n8n Details

N8N_EDITOR_BASE_URL

WEBHOOK_URL

Links

Developing Custom Nodes for n8n with Docker

Prerequisites

Quick Clarification

Goal

Dev Process Steps

Setup

Explanations (...if you care...)

npm link

Effective Directory Structure

Normal Docker install for a Custom-Node

Links

Things that might lead here...

npx only-allow pnpm

ERR_PNPM_LINK_BAD_PARAMS You must provide a parameter

npm install n8n-nodes-my-custom-nodes

Create a `Code` node to use the SDK Library/Module