Note Taking with Org Roam and Transclusion

Ever Improving my Personal Note Taking Process

This past week, I’ve been migrating my blogging workflow towards an Org-Mode-first workflow. By that I mean I’m writing my blog posts using Org-Mode and exporting the content to Hugo for building Take on Rules.

Why This Added Layer of Effort?

The short answer to “why this added layer of effort” is because of Org-Roam and Org-Transclusion. But those are technical implementations. First I want to talk about the functional issues I’ve encountered.

I write a lot of notes. For work, I keep a daily list of tasks I’ve worked on. I also write meeting notes

And I write notes to think through a technical approach. For all of that, I use Org-mode.
For commit messages and interactions with GitHub, I use Markdown.

For play, I write session reports, poetry, thoughts about games, reviews, and so on. Up until recently, I’ve written most of that with Markdown.

In other words, I had two different formats for my non-programming writing. That alone isn’t a reason to change; and is one reason I previously had not.

Yet this bifurcation sat as a mental irritant, even though it was not quite a problem to solve.
A year ago someone told me that at some point I’d be migrating to Org-Mode for blogging. So credit to them for seeing that future. I chose to sit with this mental friction, to better understand the problem I was experiencing.

What Changed?

A slow moving confluence of moments brought about this change.

When I was facilitating the New Vistas in the Thel Sector campaign, I wrote notes using Org-Mode and Org-Roam and exported my notes to Markdown. At the time, I would then finesse the export. This involved scrubbing links to “private” notes. My GM notes if you will.
I was also new to Emacs and not yet comfortable with emacs-lisp.

As that campaign spun down, I started writing functions to help me compose Markdown blog posts.
These functions lean heavily on yasnippet and later custom emacs-lisp.
In all of this, I wasn’t ready to address the mixture of “public” and “private” notes paired with the idea of yet another content migration.
I have migrated this blog’s page generation from Wordpress

Fast forward to , and I’m playing in two Burning Wheel Gold games. I’m facilitating the Mistimed Scroll campaign.
I am about 5 session reports behind on this campaign
. And I’m playing in Burning Locusts. In both campaigns, one of the fellow players is also an Emacs enthusiast.

When they shared their DOT notation graph representating the relationships in Burning Locusts, I followed with Using PlantUML to Model RPG Relationship Maps. They then wrote Burning Plants.

That exchange was another brain-worm that nudged me revisit my blogging process.

Why? Because Org-Roam’s Org-Roam UI can generate an interactive graph based on nodes. And more importantly, I had found and was playing games with a fellow gamer that was interested in Emacs, relationship graphs, note taking, and many other shared interestes.
The kind of gamer who I’d love to meet for coffee and just talk about games and theory.

Sidebar for a Glossary

Before I go further, I want to briefly work through some graph terms.

Node

a chunk of descriptive information

Edge

a reference from one Node to another Node

In Org-Roam, when I create Node A and add a link to Node B, that creates an edge A -> B. In Org-Roam, Node A references Node B and Node B has a back link to Node A.

Org-Roam also exposes the concept of :ROAM_REFS:. Let’s say I add Node C to my notes. For Node C lets set its :ROAM_REFS: to include “https://orgmode.org”. Now, anytime I link to “https://orgmode.org” in my notes, Node C will have a back link to that reference.

In other words, :ROAM_REFS: let me create proxies for external concepts.

As I learned about that, I started exporting my blog’s posts to my Org-Mode directory.
The original reason for export into Org-Mode was because I have also been writing more technical blog posts that I wanted to reference in my private note-taking; a confluence of labyrinthine moments.

But, I was treating my blog as the primary source of knowledge. And that runs contrary to the actual model. My thoughts are private, and in speaking them, they become public. But I digress a bit.

Back to taking Session Notes

With my blog now “imported” into my Org Mode ecosystem.
Not migrated just imported via some Ruby scripts and Pandoc antics. Critical in this import is that I would generate a node ID and a :ROAM_REF: entry. Thus creating the connection between public URL and private node identity.
I started thinking about the directional flow of information.

This is when I again revisited Ox-Hugo. I was looking to scratch the itch of resolving the directional flow of knowledge, and the Do I need to re-write my whole blog in Org? page gave me confidence to spend a bit of time exploring.

I started with a Literate Programming approach and wrote my takeonrules.org export functions.
I also mentally framed this whole thing as an experiment; something I would test and observe and rollback if necessary.

The main concept being that I wanted to correct the flow of information (e.g., private to public). When exporting a node, I did not want to export links to private nodes.
It is okay to export the text but I don’t want to export broken links.

And my experiment worked.

But Wait, There’s More

As I solve one problem, I become aware of more opportunities that arise with the new state. And I owe you, dear reader, information about Org-Transclusion.

Once I had the export working, I started looking at the graphical structure of my notes in Org-Roam UI. And as structured, each game session node (e.g. the node I publish to my blog) had lots of edges. After all, I was writing all of these notes in one node.
My custom export process assumes that I’m exporting one file which has one Org-Roam node; an implementation detail that has thus far been adequate

Which node has the reference impacts the back links. To make it concrete; my session notes incorporate NPCs, and while back links from the NPC to their sessions is nice, I’m often more interested in the scenes in which they were present.

Around this time, I also got the wild idea of “How might I, a player in Burning Locusts share my notes with the game facilitator, such that they could overlay their notes on my notes. I spent a bit of time thinking through that in my Burning Locusts Campaign Data repository.
You can also see a snapshot of the campaign data.

Which brings me to Org-Transclusion. I started reworking this in my to be published Burning Locusts: Session 7 notes.
At the time of writing this I linked to my internal notes, but you dear reader, will not have such a link until I both re-export this post and publish the session notes.

What I’ve now chosen to do is to create a node for each scene. And transclude those nodes into my session report. In a way, my session report is an index of scenes. Here’s what that session report looks like:

:PROPERTIES:
:ID: 4E332C1F-57FA-47D3-B303-A4B21AF3BA3B
:SESSION_REPORT_DATE: 2022-02-16
:SESSION_REPORT_GAME: BURNING-WHEEL-GOLD
:SESSION_REPORT_LOCATION: via Discord and Roll20
:END:

#+title: Burning Locusts: Session 7
#+SUBTITLE: Arson, Ambush, and Art Sales
#+FILETAGS: :session-report:burning-locusts:rpg:burning-wheel:

* TODO More Happenings at Adriano’s Party

#+transclude: [[file:20220225---adriano-faraldo-s-party.org::*Session 7][Session 7]] :only-contents

* TODO Aftermath of the Ambush at Adriano’s

#+transclude: [[id:1226FDD8-E7D3-4AF1-9958-5DC0ABE721FF][Aftermath of the Ambush at Adriano’s]]

* DONE Background Events Resolved After Adriano’s Party

#+transclude: [[id:6AC158C5-2279-4A80-916F-E087F5B6FF2D][Background Events Resolved After Adriano’s Party]]

* DONE Frederico Meets with the City Clerk

#+transclude: [[id:8761371C-B4C2-4E06-8C13-135BB0780382][Frederico Meets with the City Clerk]]

* DONE Antonius and Maccio Have a Conversation Regarding the Arsonist

#+transclude: [[id:7DABFA0F-5200-46E0-A494-F9EBFD23CBAD][Antonius and Maccio Have a Conversation Regarding the Arsonist]]

* DONE Frederico and Antonius Seek a Fence

#+transclude: [[id:0BEFCA44-8DA4-4167-A727-07F676F6EBD1][Frederico and Antonius Seek a Fence]]

* TODO Closing Scene

I have minimal memory of this, as it was very much a denouement.

For transclusions to “register” as back links, I removed (keyword "transclude") from the org-roam-db-extra-links-exclude-keys variable.
I submitted an issue to org-transclusion describing the behavior without this adjustment

You’ll also note that I repeat the header with the label of the transclusion. I wouldn’t need to do this, if I moved the header into the file. But I prefer this method.

Conclusion

Riffing on Clarke, and as I’ve said before, “Any sufficiently advanced hobby is indistinguishable from work.”

I have a lot of tooling for helping me write Markdown blog posts, and nothing that I’ve done invalidates that. Instead I’ve extended my workflow to allow me to now better take and share session notes. If I’ve learned anything in my 2 years with Emacs, all aspects of my computer life benefit when I experiment with writing and knowledge management.

For example, when I’m writing my daily work activity log and am working with someone on that activity, I add a reference to those coworkers.
Yes, there’s a node for each person I interact with at work
. This way, if I need to recall information, I have the back link available as a tool to assist on that recollection.

Postscript

As I finished writing this post, I realized that another catalyst in this change was adopting a Literate Programming approach. I spent time moving my Emacs configuration to use org-babel-tangle, which allows for a mixture of prose and code. Which is not to confuse the prose with code comments.

It turns out that taking the time to write through an observed problem with prose tools while also having access to coding context all in the same file helps me better think through these problems.

Comments

[RutviOrtiz]

A Web Application to Visualize the Org-Roam Database
spells to harm someone