DEV Community: arc42

How to regain concentration and focus

Dr. Gernot Starke — Wed, 03 Aug 2022 12:34:00 +0000

This article was originally published at INNOQ.

No sooner had I started a task, the next thing I knew I was doing something else. I distracted myself by checking my email inbox every so often, and I was addicted to checking what was going on in the world on various news websites, constantly interrupting the original task.

You're perfectly right: This sounds completely insane. Somehow these bad habits had sneaked silently into my life.

News is to the mind what sugar is to the body.

Rolf Dobelli, Author and entrepreneur

And then two lightning bolts struck almost simultaneously: a book and a blog post.
I significantly changed my (digital) life as a result and within only two weeks I was able to get more done again, sleep better, and am significantly happier.
The short version: news diet and productive smartphone use.

Lightning strikes

How my concentration was lost…

For years, I've been producing content on a fairly regular basis in the form of talks, books, journal articles, blog posts, podcasts, and contributions to open-source projects (To quantify that: 30+ books or book editions and about a hundred articles in various magazines within the last 20 years - that's still one and a half books and five articles on average per year).

In the last few months, I ran out of steam, which made me personally very discontent. In early 2022 I published the last blog post, and in late 2021 the last articles. For the third time, I postponed the new edition of the arc42-in-Action book ...

I had plenty of ideas, but somehow I couldn't put them down on paper. Unfortunately, ideas alone are not enough: the ideas also have to be made into reality, structured, and written down in comprehensible ways.

Moreover, I was always a dedicated textbook reader, and binge-read a new book every few weeks. That, too, became more and more difficult for me - even though I was genuinely interested in the topics of the unread books on the shelf.
Now, I could chalk it all up to the pandemic, my age (don't ask...), or whatever. But finding bogus excuses is just not my thing.

"You've got to do something about that," I told myself. At first, I tried to increase my productivity and creativity by getting up earlier. No such luck. Staying up later in the evening didn't work either.

Then I tried what I've recommended (successfully!) countless times in my job as an IT consultant: A systematic analysis of the situation, a self-review.

The self-analysis

I tried to analyze my own way of working. I quickly noticed that it was becoming increasingly difficult for me to concentrate on individual tasks: No sooner had I started a task than I was already doing something else in between - or distracted myself with emails or various news websites. Likewise, I sat at preparing a lecture - and checked the INNOQ Slack channels in between. Furthermore, I continued to write on a slide - and took a brief look at my smartphone to see what's been happening on WhatsApp or Signal. And since I was holding the phone in my hand anyway, I could read a few news items on BBC-News there too.

Did you know that smartphone users unlock and look at their phones an average of 80-100 times a day (source: Cision Newswire, several other sources giving similar numbers)? I was one of them.
News websites like CNN, WSJ, and BBC had me as a permanent customer.

For me, these context switches already had addiction-like traits.
Even while I was watching a series (yes, I admit that I like to binge on one or two episodes), I would pick up the phone or tablet in between and casually check the news.
Against my better judgment, I also checked my e-mails first thing in the morning before I even drank my first espresso.

And in my inbox, there were always various news summaries, from stock-market news, and international politics up to technology updates.

Unfortunately, all this news has done me more harm than good: For my coaching and consulting engagements, I don't need to know anything about current domestic or foreign politics.
For my software architecture and engineering workshops, neither stock market trends nor the details of international terrorist acts are of any value.
My articles and books deal with software and software engineering, not with climate or politics.

A self-discovery

Constantly switching between (sophisticated) technical work, the news from around the world, and private communication on personal and professional topics, I would argue, cost me a significant amount of my ability to concentrate.

My brain (apparently) doesn't handle frequent context switches well. Moreover, my brain must have unconsciously perceived distractions as something positive - and, like Pavlov's dog, kept wanting more of them, at increasingly shorter intervals.

A (Pavlovian) dog

I noticed the lack of concentration myself. However, my own fainthearted attempts, with Pomodoro timer and calming sounds, clearly failed.
Effective remedies from the outside had to come first, two major sources helped me:

While travelling to an architecture workshop, I sat on the train, not reading the news for a change, but attempting to read a book.

For some time now, I have been using Blinkist to read short summaries of various non-fiction books, to find out for myself whether I is worth for me to read the original of these books in their entirety.

So - I'm sitting on the train and reading one of these Blinks, when I'm struck by a mental lightning bolt: I felt so caught by Rolf Dobelli's explanations, caught in my concentration trap, that I spontaneously decided to get serious. Let me quote from this book:

News is not good for us:
It clouds our mind, distorts our view of what is really important, robs us of time, makes us depressed and paralyzes our willpower.
Rolf Dobelli, Author and entrepreneur

Paralyze my willpower, rob me of time.

That was precisely my problem.

While still sitting on the train, I decided to try a strict break from any news (news, not e-mails or other messages) for a while.

However, that wasn't enough: I remembered a blog post I had read a few years ago - about the productive configuration of smartphones, which has the promising subtitle Configure Your iPhone to Work for You, Not Against You.
I therefore decided to optimize my iPhone for maximum productivity and to reduce or even eliminate distractions.

Drastic, but effective

A (currently very strict) zero-news-diet.
(Re)configure smartphone for high productivity and minimal distractions.
Reduce distractions on tablets and computers. Add to that slight changes I made to my daily habits: I've resolved (and managed for a few weeks now) to stop staring at my phone in the morning, and to stop using my iPhone at all in the evening after 8 PM at the latest (exception: setting an alarm for the next morning).

Zero News Diet

During the aforementioned train ride, I cancelled my subscription to my formerly favorite online news portal (the German "Spiegel-Online") and deleted the associated app from my iPhone. I used the initial motivation to cancel additional news subscriptions and deleted the their apps from my smartphone and iPad so that I wouldn't be tempted.

I have resolved very firmly not to read any news at all for a while. In other words, a zero news diet. That was really hard for me for two or three days. Yet, that alone was a startling realization for me: I was actually addicted to news and variety... and cold withdrawal is hard with any kind of addiction, as you know.

I purposefully invested the hours gained each day.
As a kind of early reward for my zero news diet, I treated myself to an interesting non-fiction book as a PDF (by the way, the infamous "Thinking, Fast and Slow" by the great Daniel Kahnemann).
The result: I was thrilled to have made good progress without any distractions.

To support my news diet, I instructed my browser (Firefox) to not present me with recommendations for supposedly interesting or important topics on new tabs or windows.

Firefox preferences: No more news suggestions

Reconfigure Smartphone

The above quote, "Configure Your iPhone to Work for You, Not Against You," sets the objective:
It's the subtitle of a very long blog post by Tony Stubblebine.
In addition to his many configuration suggestions, Tony also gives various advice for being more attentive and healthy, entirely worth reading in my opinion.

I summarize the things most important for me, which he illustrates in his article with many screenshots and explanations:

Turn off (almost) all notifications – so those red dots aren't constantly vying for your attention or making you feel guilty.
Hide social media apps as well as possible. Facebook, Instagram, Twitter & Co act like drugs. They deserve to be removed from the start/home screen and moved somewhere in the back of your smartphone. I deleted the Twitter app and kept Instagram only because it allows me to catch up a bit with my adult children.
Messaging apps (email, WhatsApp, Signal and co.) go into a folder, and preferably on the second screen of that folder. For me, it looks like this (K11n stands for communication):

Communication apps – unimportant ones need more “swipe” gestures
Turn on do-not-disturb-mode, the longer, the better. At least from evening until morning.
Lifting the phone should not unlock it. This setting is called "Display & Brightness/Activate on Lift" on the iPhone - and should definitely be turned off.
Turn on the screen time widget – this gives me control over all the things I spend my smartphone time on... Meanwhile, I have made a sport out of getting to the smallest value possible.
Turn on content and app restrictions. I only allow myself 15 minutes a day for email and Instagram.
Install a swipe keyboard app, so I can text faster on the phone and not have to awkwardly click each letter. I use Microsoft Swiftkey (yes correct, a Microsoft app on the iPhone) for this, others do better with Google's Gboard.
Wallpapers in muted colors. I chose a completely black home screen, and a reduced color lock screen:

iPhone backgrounds in muted colors

Reduce distractions on tablets and computers

You will find similar preference settings on your tablet and your computer, respectively. I set those to reduce notifications as much as possible:

Notifications settings, almost all turned off

In case you fall off the wagon

In case you want to be a bit stricter with yourself, there are various blockers for popular desktop platforms that can prevent access to certain websites or applications for certain times. These can improve your digital self-discipline. I tried ColdTurkey and like it a lot. However, I hope to become disciplined enough to stick to my new habits without this electronic tether… at least in the near future.

Furthermore...

Besides the aforementioned settings and the zero news diet, I like to listen to soundscapes for (supposedly) better mental focus when working intensively at my desk. I write supposedly because I like this kind of aural background, but can't prove that it helps me in any way. My family, by the way, thinks it's horrible.

If you want to try it out: My two favorites are Endel and brain.fm. Works on both smartphones and desktops (but please don’t say I didn’t warn you…).

I still haven't gotten comfortable with Pomodoro timers, they annoy me more than they help.

And finally, thanx to Łukasz, you might want to restrict your YouTube usage with the unhook extension. I tried that and immediately liked it. You have to tweak the default settings, though. No more wasted hours due to the addictive YouTube proposition algorithm...

Conclusion

Why don't you start a self-experiment, and refrain from reading sports, politics, business, and technology news as much as possible for a while? Reduce your smartphone time drastically for one or two weeks, and enjoy the time gained with a good book, a personal conversation or approach a previously abandoned activity that has been left undone so far (due to a perceived lack of time…)

Good luck – and I look forward to your feedback.

Acknowledgements

Thanks to Jochen Christ, Joachim Praetorius, Jan Seeger and Ben Wolf for reviews and constructive comments. @m and Joy Heron have drastically improved readability and wording.

This article was originally published at INNOQ, achieved a #4 rank and more than 300 comments on Hackernews

Image Sources

Dog
Lightning
Header image by Paul Skorupskas

Enable ssh access to multipass vms

Dr. Gernot Starke — Mon, 01 Aug 2022 15:03:00 +0000

The problem

You are using multipass to create lightweight virtual (Ubuntu) machines.
You want to ssh into those machines, because you cannot or don't want to use the standard shell command multipass shell <name-of-vm>.
The naive approach fails with permission denied:

$ ssh 192.168.205.7
The authenticity of host '192.168.205.7 (192.168.205.7)' can't be established.
ED25519 key fingerprint is SHA256:lWKUbxxxx.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.205.7' (ED25519) to the list of known hosts.
gernotstarke@192.168.205.7: Permission denied (publickey).

Permission denied, although there is a route to this virtual machine available...

Different approaches

Two approaches have been documented elsewhere:

Add an ssh-key to an existing virtual machine (explained by Josue Bustos here). Downside of this approach: You have to repeat it (more or less manually) for all new VMs you create with multipass launch. Therefore, let's look at a solution that will also work for new VMs...
Pass an ssh-key while creating a multipass VM (explained in detail by Ivan Krizsan here). In short: You generate a new ssh-key, and pass the public key into every new VM. I summarize the required steps below.

The solution

(I mention it again, the idea presented here have first been described by Ivan Krizsan):

Create new ssh-key for your multipass VMs
Create a reusable launch configuration for multipass VMs with cloud-init
Launch new multipass VMs with this configuration
Find out IP-address of new multipass VM
ssh into the new VM with this IP-address

Prerequisites: You have:

multipass installed. If not, it's well-documented on their website.
a terminal running a decent shell (bash, zsh).
a decent text editor (VS-Code, nano, vim or the like.)

1.: Create ssh-key

On your host machine (the one with multipass installed), change to the directory from where you will be launching multipass vms. It can be your home directory, but any other will do.

$ ssh-keygen -C vmuser -f multipass-ssh-key

vmuser can be any (dummy) username, it will later be used to log into the VM.
The parameter to -f is the filename for the generated key. You can choose a name of your liking, but there must not be an existing key with the same name in the same directory.

You will be asked to enter a passphrase, leave that empty! I know, it's not as secure as it should be, but multipass VMs are used for development and test only...

Empty passphrases are NOT suited for production environments.

2.: Create cloud-init configuration

Cloud-init is the standardized approach for cross-platform cloud and VM configuration of instance initialization.
Multipass can handle such configuration, we use it to pass the ssh key into the vm.

In the directory where you generated the ssh-key, execute the following:

$ touch cloud-init.yaml
$ code cloud-init.yaml

You can obviously use any text editor of your choice. Nowadays I use VS-Code for most of my edits.

Put the following content into this file:

users:
  - default
  - name: vmuser
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
    - <content of YOUR public key>

The <content of YOUR public key> starts with the letters ssh-rsa and ends with the username you supplied in step 1. Both have to be included!
Remember, it's yaml: Sensitive to spaces and dashes.
In case you generated an ed25519 type of key, it starts with ssh-ed25519. If you don't know what I'm talking about, ignore this last line.

3.: Launch multipass VM with ssh configuration

You will use a slightly different launch command for your VM, by adding the cloud-init parameter:

$ multipass launch -n testvm --cloud-init cloud-init.yaml

testvm is the name of your new vm, you can choose any name or even leave it blank. In the latter case, multipass will create a random name for you.
cloud-init.yaml is the name of the configuration file we created in step 2.

4.: Find IP address of new VM

$ multipass info testvm
Name:           testvm
State:          Running
IPv4:           192.168.205.4
Release:        Ubuntu 20.04.4 LTS
Image hash:     692406940d6a (Ubuntu 20.04 LTS)
Load:           0.00 0.00 0.00
Disk usage:     1.5G out of 4.7G
Memory usage:   140.5M out of 976.9M
Mounts:         --

testvm is the name of the VM you were using in step 3 (launch VM).
The output of the info command contains the IPv4 address of this VM, you need that in the next step.

5.: ssh into new VM

$ ssh vmuser@192.168.205.4 -i multipass-ssh-key -o StrictHostKeyChecking=no

Welcome to Ubuntu 20.04.4 LTS (GNU/Linux 5.4.0-122-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

vmuser is the username for which you created the ssh key
Please replace the IP address by the one you found out in step 4.

And voilá - you made it.
The command itself should be obvious... you pass the name of the ssh-key with the -i commend (short for identity). In addition, you turn strict host checking off.

You might see additional messages, depending on your host machine ssh configuration - but you should be logged in your multipass VM by now.

May the power of the shell be with you.

Acknowledgements

Thanx to Ivan Krizsan) for providing the initial solution and his nice writeup. I took me a while to cut through numerous web-searches to find his post, therefore I decided to write it up again under a slightly different title.

All credits and kudos to him! All faults a clearly mine.

Brief introduction to arc42

Dr. Gernot Starke — Mon, 21 Feb 2022 14:17:56 +0000

Brief Introduction to arc42

arc42 ¹ is a template for architecture communication and documentation.
It is a proven, practical and highly pragmatic approach and takes the pain out of documentation.

arc42 provides everything you ever need to communicate and document your software architecture.
It's tool and technology agnostic, therefore can you use arc42 for arbitrary systems.
arc42 is open source and completely free to use.

This article provides a quick overview of arc42 and its parts.
For more information, please consult the (rather extensive) documentation [docsˆ] available on their documentation website (which, since V8/January 2022, contains several practical examples).

arc42 answers the following two questions in a pragmatic way and can be tailored to your specific needs:

What should you document/communicate about your architecture?
How should you document/communicate?

If you work under time or resource constraints, you will most likely skip certain parts of arc42. Our upcoming post on "lean documentation" will give guidance for such cases.

Before we dive in...

Everything is optional - there is no need to fill in every section of the template. Treat arc42 documentation like the compartments of a cabinet, similar to the image below: The cabinet has a value, even if certain compartments remain empty.

(image by Jan Kolar from Unsplash)

The order of these compartements is optimized for reading and understandability. You create your content in any order you like or your project requires. For example, in many development projects, the context (section 3) is defined and created first, followed by the top quality requirements (section 1.2).
arc42 is completely tool-agnostic. You might create and maintain arc42 in your toolchain of choice: A wiki, a collection of Office documents, plain Markdown or Asciidoc, one of the commercial UML tools or even LaTeX. Whatever your tool of choice is, arc42 has you covered.

The structure of arc42

arc42 proposes a dozen sections (aka compartments) for your documentation.
These are organized top-down to improve understandability.

1. Introduction and goals

Short description of the requirements and driving forces of your system:

Describe what the system delivers for its users in one or two sentences.
Describe the two or three major use cases or features of the system.
You should provide an extract or abstract of requirements.
Restrict yourself to the top three (max five) quality requirements for the architecture, which have highest priority for the major stakeholders.
Include a table of important stakeholders with their expectation regarding architecture or documentation.

2. Constraints

Anything that constrains teams in design and implementation decisions or decision about related processes. Constraints can sometimes go beyond individual systems and might valid for whole organizations and companies (e.g. company-wide technology choices or government regulations).
Time and money are common constraints in many organizations.

3. Context and scope

The context delimits your system from its (external) communication partners (neighboring systems and users).
It specifies or documents the external interfaces.
You should always document the context from a business or domain perspective.
If infrastructure or specific hardware plays an important role, you might also show a technical perspective.

4. Solution strategy

Summary of the fundamental decisions and solution strategies that shape the architecture. Can include technology, top-level decomposition, approaches to achieve top quality goals and relevant organizational decisions.

5. Building block view

Usually, boxes and arrows, showing the high-level code structure of the system. To phrase it a little more formal: The building block view explains the static structure of the system and contains, abstractions of source-code. It refines the context view, where the complete system is depicted as blackbox.

Level 1 contains the major subsystems, components or parts of the system. In some cases, this coarse overview provides enough structural overview.
Level 2 refines one or more elements from level 1: Create a separate whitebox (plus explanations) for every level-1 building block you like to explain in detail.

You can drill down to an arbitrary level of detail, but keep in mind the following:

Recommendation
Try to stick to level-1, as it often gives enough guidance and understanding for most stakeholders.

6. Runtime view

The runtime view explains the behaviour or processing of one or several building blocks.
It serves as companion to the static building block view from section 5 above.
The runtime view might explain important use cases or features, interactions at critical external interfaces, error and exception behaviour.

Recommendation
Use runtime scenarios primarily during development to validate or design building block structures. They help to identify missing or unnecessary dependencies. Keep only a small number of scenarios in your documentation

7. Deployment view

Software needs hardware to execute on, that's where the deployment view comes into play:
It shows the technical infrastructure with environments, computers, processors, networks and network-topologies.
In addition, it maps the software building blocks to those infrastructure elements.

8. Cross-cutting concepts

(aka cross-cutting concerns)

Overall, principal regulations and solution approaches relevant in multiple parts (→ cross-cutting) of the system. Concepts are often related to multiple building blocks. Include different topics like domain models, architecture patterns and -styles, rules for using specific technology and implementation rules.

9. Architecture decisions

Keep a collection of architecturally significant decisions that are important, expensive, critical, large scale or risky including rationales, that are not recorded elsewhere.

Please use your judgement to decide whether an architectural decision should be documented here or whether you better document it locally (e.g. within the building block view or any cross-cutting concept).
You may have already captured the most important decisions of your architecture in the solution strategy.

Recommendation:
Use Architecture Decision Records to capture decisions in a simple, structured yet pragmatic way.

10. Quality requirements

Quality requirements, often described as scenarios.
You may use a quality tree to provide a high-level overview.
The most important quality goals should have been already described in section 1.2. (quality).

Let's look at some examples:

Topic	Scenario
Performance	System shall performs all checks on a 100MByte file in less than 10 seconds.
Operability	The temperature range in which the correct functioning of the device is ensured should range from -25 degrees to +85 degrees
Maintainability	A new additional verification algorithm can be included in less than one day.

11. Risks and technical debt

Known technical risks or technical debt. What potential problems exist within or around the system? What does the development team feel miserable about.

In case somebody reviewed or audited your system, their findings might be included here.

12. Glossary

Important domain and technical terms that stakeholders use when discussing the system.
Please include only specific terms - and avoid explaining REST, HTTPS or other words that have common explanations.

In addition, the glossary can serve as the translation reference if you work in a multi-language (international) environment.

Like what you saw?

Give it a try! Document a few of your architecture decisions, sketch your scope and context and explain a few key technical concepts - and you have your first version of your architecture documentation.

Download the template in a format you like. Unpack and start documenting. In case you have questions or want to see examples, consult the extensive documentation site ².

In case you want to know more, we have you covered:

I wrote about the principles of technical documentation. That deep-dive article explains the fundamental requirements your technical documentation needs to fulfill - and many proven approaches how to achieve these.
Another article on lean documentation is in the making, please stay tuned. Until then, watch the video "Sparsame Dokumentation" (oops, sorry, it's in German)

Comments or questions?

Leave a comment right below - or open an issue on the arc42 site. Otherwise, thanx for reading this article.

https://arc42.org. The template has been created by Dr. Peter Hruschka and Dr. Gernot Starke. They published the first version in 2005, after having used predecessor versions for many years in different large and medium-sized projects. ↩
https://docs.arc42.org. The extensive documentation website, containing more than 140 tips and 30 examples on how to use arc42. ↩

Principles of technical documentation

Dr. Gernot Starke — Sat, 29 Jan 2022 16:12:10 +0000

This article collects fundamental requirements for technical documentation, especially software architecture documentation, together with ideas how to satisfy those.

TL;DR

The content of technical documentation needs to be correct, current, understandable and relevant. It needs to be maintainable, referenceable and use proper language. Furthermore, the whole documentation and its content need to be easy to find. To create and maintain such documentation, you should put everything under version control, prefer established tools and automate certain parts. Finally, you should document incrementally and continuously - and not defer documentation to a later stage of development.

To help you achieve these goals, the article summarizes more than 30 simple practices that will facilitate your documentation task.

0. Introduction

During several years of consulting work, I have noticed that different development teams across organizations in various domains repeated the same documentation mistakes over and over again. Therefore I decided to summarize some fundamental requirements for technical documentation

I deliberately mention modeling and diagramming only briefly as these will be subject of another post.

A counterexample

Imagine you are planning a trip to Scotland with your family. You want to visit the famous castles and legendary landscapes, and walk in the footsteps of the brave Scottish knights.

To best prepare for this wonderful trip, you visit your local bookstore and ask for a suitable guidebook. The clerk proposes the “Adventure Travel Guide to Scotland”.

Fig. 0.1: Your map of Scotland

Back home, you treat yourself to a decent cup of tea, recline in your favorite chair and start reading. After a couple of pages, you start to wonder if you got the right book (aka documentation):

To your disappointment the documentation is not useful, as the authors start in Scotland, but quickly deviate to a lengthy coverage of the Spanish island of Fuerteventura
Next thing, parts of the documentation seem quite outdated to you. The only map you can find in your new book, despite being visually appealing, was printed¹ in 1747, more than 250 years ago (see Fig 0.1). Where the heck are the airports and highways?
It contains some serious mistakes - the city of Edinburgh is misspelled Edembourg²

Hopefully, your technical documentation better fits its title and is
less than 250 years behind the current “version” of the system. But
maybe you get the analogy: Documentation should be there to help, not hinder.

Overview of documentation requirements

Let’s start with an overview of documentation requirements, so you know what to expect. The main part of this article covers both requirements and solution approaches in detail.

Fig. 0.2 Categories of documentation requirements

The most important of these categories cover the content of
documentation:

Requirement	Explanation
Req-1: Correct	Documentation needs to be accurate and free from errors. Wrong documentation is often worse than no documentation.
Req-2: Current	Documentation needs to be correct over time, reflecting changes performed upon code, infrastructure or interfaces of the system.
Req-3: Understandable	Documentation needs to be understood by the target audience.
Req-4: Relevant	With respect to structure, form and content, documentation shall be relevant for the tasks of its audience.

A few of these requirements are covered in the fundamental book book³.

Fig. 0.3: Content requirements

In addition, you should consider a few formal requirements, independent of documentation content:

Requirement	Explanation
Req-5: Referenceable	Use a consistent numbering schema for headings, diagrams and tables.
Req-6: Proper language	Use proper language, correct spelling and grammar, active voice, positive statements and short sentences.
Req-7: Maintainable	Maintainability is key to keeping documentation current.

Fig. 0.4: Formal and content requirements

Finally, a few requirements concern your documentation process and
tooling:

Requirement	Explanation
Req-8: Easy to find	Documentation itself should be easy to find whenever needed. Its content should be easily navigable and searchable.
Req-9: Version controlled	As the system evolves, so will your documentation, without losing its history
Req-10: Appropriate tools	Focus on content, reduce time needed for tool-setup.
Req-11: Continuously updated	Make it a habit to maintain and expand the documentation with every relevant change in your system.

That leaves us with the following overview of documentation requirements:

Fig. 0.5: All documentation requirements

In the upcoming sections I will consider these requirements one-by-one. You will find several tips and proposals on how to achieve them.

Req-1: Correct

Technical documentation absolutely needs to be correct, “free of
mistakes or errors and strictly adhere to fact and truth” (³, p3).

Some examples:

When a diagram shows relationships between certain elements, those relationships have to exist in reality.
On the other hand, if is no relationship between those two elements, there shouldn’t exist one in the underlying code!
If a specific version of any product or framework is mentioned, exactly this version shall be used in reality.

This requirement seems trivial, but correctness has to be achieved even in situations where development teams are constantly working to improve or enhance the system. They change code, update interfaces, frameworks or underlying technology. Important stakeholders will change requirements or constraints.

The issue with constantly changing systems leads to another requirement (“current”), which will be subject of a larger section. Before we dive into this subject, let’s look at some options you have to get your documentation correct.

How to achieve correctness

Have the right people create and maintain the documentation.
1. The exact same people who design and implement specific parts of the system also document these parts. The advantage: No information is lost at some potential interpersonal communication barrier. The disadvantage: These people might not be capable of producing high quality technical documentation, they might not have appropriate tools, motivation or time.
2. Have the architect(s) create that documentation. On one hand they will be able to extract the required knowledge from the responsible stakeholder. On the other hand, they should know the basics of quality technical documentation.
Have all parts of your documentation reviewed by the people who built, decided or implemented that exact thing - as they know the truth.
Prefer abstract information over detailed: Abstract information is still correct, it’s just leaving out some detail. Leaving out details reduces the chance for errors and the need for
constant updates. Have the courage to leave things undocumented (better no documentation than wrong documentation). The next section on current documentation will dive deeper into that subject.

Req-2: Current

Change is normal: Development teams have to implement new features or optimize/fix/change existing ones. They have to adjust the system to cope with changes in infrastructure, updates in frameworks or 3rd-party products. Teams have to update their systems to reflect changes in external infrastructure, data formats or a plethora of other aspects that might change during the lifetime of any software system.

Despite all those changes, the documentation has to fulfill the
correctness requirement. Teams must constantly adapt (maintain) their documentation, so it remains correct.

How to keep documentation current

Regularly review your documentation, especially those parts that relate to recently-changed elements of the system.
Leave out the obvious. Focus on important and specific aspects, elements or decisions in your system.
Document only those parts or aspects of your system and its architecture that are definitely required by stakeholders.
From time to time you should iterate over all parts of the documentation and:
- question whether this part is still required by somebody and
- you are still willing to maintain this part of the documentation in the future.
- If the answer to one of these questions is “no”, then remove the corresponding part of the documentation. As everything is version controlled, it would be available if needed later on.
Optimize your documentation (not only your system!) for maintainability. The section on Easy to find contains numerous approaches that help to optimize maintainability.
Automate the generation of certain parts of your documentation..

Req-3: Understandable

Although understandability seems like a trivial and basic requirement for technical documentation, it is pretty hard to achieve and really easy to get wrong. Let’s dig into some details of what makes documentation understandable:

Terms and symbols need to be clear and unambiguous.
Use abbreviations only after they have been written in full.
The overall organization (aka structure) of the documentation shall be optimized for consuming/reading
People tend to comprehend things better if they understand the reasons
Form, structure, language, tone and symbols should be adequate for the audience.

How to improve understandability

Know your target audience

Understandability depends on the target audience.

What people understand depends on their knowledge, experience, language skills, culture etc. A specific part of your documentation might be fully understandable by certain people, and completely alien to others. Different stakeholders will have different requirements concerning documentation.

The more you know about your stakeholders, the better you will be
prepared to meet their demands. This is closely related to Relevance.

Follow these steps:

Identify all stakeholders: Identify all roles or people that might have an interest in or need for your technical documentation
Identify what your target audience really needs, what are acceptable forms or formats for documentation
Create and maintain a stakeholder table containing that information.

This information about your stakeholders will also be important to
improve the relevance of your documentation.

Document unambiguously

Ensure your documentation is clear, precise and presented in a way that the target audience can understand. Most diagrams leave room for interpretation, especially if naming and/or semantics of elements isn’t exactly clear. Make sure that all symbols in diagrams and all specific term have a defined meaning. Add a legend to clarify visual elements if you are not using a standardized notation.

See also Explain your notation
See also Combine diagrams with text

Place details into context

Your audience shall be able to start with an overview and then “dive in” when they need more information. Therefore, organize your documentation top-down.

Technical documentation will contain technical details that are prerequisite to understanding some decisions or concepts. Ensure people can understand the context of such a detail. Add references to external documents for readers who might not meet your knowledge
prerequisites.

Fig. 3.1: Details with context

Explain specific terms

In case you use specific terms with a special or uncommon meaning or
semantics, you should define or explain these. Especially if the
semantics of your terms slightly deviate from common or standard
definitions.

Provide a glossary of such terms.

Explain reasons

Many decisions taken during development of a system are reflected in its source code. What’s missing though, are the explanations or reasons for these decisions.

Therefore, consider documenting the reasons and explain why the
development team took a particular path. Start with explaining the
requirements and constraints for a specific decision, then explain the reasons for the decision.

Use diagrams to explain structures

Structure: the way that something is built, arranged, or
organized

https://www.merriam-webster.com/dictionary/structure

Diagrams or graphical models are well-suited to convey structural
information, like building-blocks (aka components and their
dependencies, hardware elements and their relationships or even dynamic structures like processes.

Use them with care - as changing or updating diagrams is harder than
updating a short textual description. On the other hand, structural
information is traditionally conveyed via diagrams.

Restrict diagrams to higher abstraction levels. Prefer to omit details from your diagrams, instead of trying to explain low-level structures in diagrams. Stick to a single topic per diagram.

Explain your notation (or use a standard)

Graphical models or diagrams are an elegant way to convey structural
information, they can improve visual effectiveness.

Visual effectiveness is a measure of how the appearance of information and the use of visual elements within it affect the ease with which users can find, understand, and use the information.
³,p.277

Every symbol in a diagram should have a clear semantic or meaning. That makes the diagram unambiguous.

Either you provide a legend which explains the meaning of every symbol type, or you rely on a standard graphical notation, like UML⁴, C4⁵ or ArchiMate⁶;⁷.
If you rely on a standard, reference it in addition to the legend.

Combine diagrams with text

A diagram itself might leave room for interpretation, unless amended by textual or tabular clarification. See the following example, where component names give hints on their actual responsibility, but terms might be misleading. What is a “Reporter”? A journalist? A component creating a report, or delivering it?

Fig. 3.2: An example diagram that needs explanation

A table explaining important elements from a diagram can help to remove
such ambiguities - see for yourself:

Element	Responsibility / Meaning	Details
`Checker`	performs the actual checking, relies on the status- and error messages of the `Parser`	org.aim42.hsc.core
`Parser`	open-source library to parse HTML files	org.jsoup
`Suggester`	suggests solutions to problems found during checking	org.aim42.hsc.core
`Reporter`	creates a hicely formatted html report of all problems found during checking	org.aim42.hsc.core

Use helpful structure

The structure of technical documentation should be organized top-down, beginning with an overview, and gradually adding details.

Approaches like arc42⁸ or C4⁵ propose structures that are highly practical, established and free to use.

Req-4: Relevant (aka task oriented)

Documentation shall serve the purposes of its target audience, see Know your target audience above. They have specific goals or tasks that can be enabled or supported by technical documentation.

Relevance depends on the goals and tasks of the target audience.

Let’s look at some potential stakeholders, their goals and documentation needs:

Stakeholder	Task/Goal	Documentation
New developer	Efficient onboarding	Overview of requirements, solution approaches, building block structure, technical concepts, important decisions
Development team	Guide implementation	technical concepts, decisions, building block structure, some runtime scenarios
—“—	Take decision	e.g. quality requirements
Tester	Test parts of the system	some internal interfaces, technology overview
Neighbour system	Use external interface, e.g. API	context, technology overview, API details
Management	Budget decisions	context, technical decisions, used 3rd party frameworks
Auditor	Check compliance	often: deployment view, technical decisions, high-level building blocks
External reviewer	Due diligence	solution approaches, high-level structure, technical concepts

How to improve relevance

Identify tasks of target audience

You need to know what information the target audience needs from the
documentation, (see above). Create a
stakeholder table to contain information about each stakeholders’ tasks
and/or goals, so you can better estimate their documentation
requirements.

Show examples of existing documentation

It is highly effective to show each stakeholder examples of existing
documentation: Based upon specific examples, stakeholders can point out
what they need in addition, or what parts they are not interested in.

Beware of the “I need everything” trap: When stakeholders see detailed
and well-organized documentation, they might want “everything” - even
though they don’t need parts. Therefore, it might be more efficient if
you show only those parts which you estimate people will really need!

Req-5: Referenceable

Referenceable elements can be uniquely identified in the context of your
documentation. Instead of writing “see the diagram below” or “the table
above”, you should strive for “see diagram 5.2” or “the table 3.1”. You
can chose any numbering or referencing scheme you like, but you should
use it consistently.

How to make documentation referenceable

Number every section and heading of technical documentation, at least to heading level 2.
Number every diagram or image. Prefix diagram numbers by the number/id of the appropriate section. For example, the first diagram in section 3 shall be labelled “Fig. 3.1”.
Number every table, again prefixed by section-id/number.
Provide a table of contents (toc). Electronic or online versions of your documentation should contain hyperlinks within the toc. For paper or printed versions, use page numbers.

You should use appropriate tools to facilitate
referencing.

Req-6: Proper language

Your target audience shall have no reason to complain about spelling or grammar mistakes, but shall instead focus on content. Proper language will also improve readability and understandability.

How to improve documentation language

Language, wording and style of your documentation should be adequate for your target audience. Sentences should be grammatically correct.
Text should be formulated in a factual way and avoid emotional or offensive language.

Some specific advice:

Use positive statements, aka avoid negation. The human brain needs significantly longer to understand negation⁹.
Write concisely, aim for brevity. The average length of a sentence should be 15-20 words.
Use the active voice, as it makes it easier for our brain to comprehend and remember.
Aim for correct spelling and grammar.

¹⁰ contains several examples and explanations of “proper language”.

Req-7: Maintainable

The structure and organization of technical documentation should be
optimized for maintainability - so that keeping the documentation
current despite frequent changes becomes as easy as possible.

As already mentioned above, many aspects of systems change -
requirements, source code, configuration, deployment, operations, external interfaces, etc. These changes shall be easy to incorporate into the technical documentation, therefore maintainability is an important requirement.

Please note: We refer to maintainability of the documentation, not
of the system itself.

How to improve maintainability

Abstract: Omit (some) details

Leave out certain facts: Explicitly decide to not document irrelevant or overly detailed facts. Leaving out some facts reduces the amount of content you have to keep up-to-date in the future. Only document facts that:
- you or your team can promise to maintain in the future.
- important stakeholders need to know to do their work
Document abstractions instead of details. That is a special case of not documenting certain facts, namely the details. But what kind of details could be left out of documentation? Please note that I wrote could be left out, not shall be left out. In some cases you will want or need to document details!

instead of…	try this…
trying to name/show all elements	aggregate elements, especially when you can give explicit cohesion criteria
giving a specific version number	omit that version number or specify a range like “version >2.1”
showing all children of an element	omit children, point to parent
showing all dependencies between two elements	show just one (aggregate) dependency (see below for an example)
explaining interface details like parameters	just name the interface, omit details

You see the underlying principle: Leaving out details facilitates change
and will make your documentation more robust against (at least some)
changes in requirements, code or infrastructure.

Examples of omitting details

Fig. 7.1: Abstractions instead of details

In the first example, we condense three building blocks (A,B,C) into a
single one. At the same time, we reduce the details of explicit
dependencies between foo and A,B,C from 9 different to one. As this
single dependency carries several responsibilities (remember, it serves as an abstraction for nine detailed relationships), it’s a good practice to give such dependencies a name and explain some of the details in text or a table.

Fig. 7.2: Example: Abstraction in runtime view

The second example shows a detailed sequence diagram containing three participants (b1, b2 and b3), having some interactions among themselves and a few with the rest of the system (a and c). In the abstraction I made use of an interaction reference to hide details, see ¹¹ or ¹².

Keep the source of every diagram

Diagrams are created using appropriate tools. Often diagramming or modeling tools use proprietary binary data formats, which are converted and exported to a format like png, jpg, svg or pdf.

To be able to later modify diagrams, you need the original (source) representation.

At best, the source of diagrams is versioned and referenced from the diagram.

Regularly delete outdated parts

If something has become common knowledge, you may delete it from your written documentation.

In one of my projects, I tagged such parts as “to-be deleted”. In case nobody from the team objected within a week or so, that content would be removed from the active documentation and moved to an archive instead. Such tagging works great in some wikis.

Automation won’t improve your content

You can (and should) automate data collection, transformation, or even some visualization, which we will discuss in the section on tools later on.

But: The intellectual process of creating valuable content for technical documentation cannot currently be automated.

Documentation generated from source code will be restricted to
high-level abstractions of this source code. It cannot give reasons or explain things, therefore it will always miss critical information.

See ¹³ for a
coverage of automated (living) documentation.

Req-8: Easy to find

Finding information is essential. The organization, retrievability,
and visual effectiveness of information contribute most to whether
users can find the information that they need.
³,p.213

This requirements relates to both the content of the documentation and
to the documentation itself. In reality, documentation is often located
in access-controlled areas that are not covered by any search engine:
That means you have to know where to look for the documentation.

How to make information easier to find

Ensure the documentation itself can be easily found, for example by providing entry points at well-know locations (i.e. the start- or homepage of the product or team wiki).
Use a template, like arc42 ⁸
Organize your documentation top-down, see place details into context above
Use cross-references and hyperlinks where possible, so readers can jump within your structure
Provide a document overview that explains the structure of your documentation. For standards, like arc42, such overviews already exist.
Facilitate navigation and search, see Req-9: Easy to find

Finally, you should accept some repetition in documentation to make it
more comfortable for stakeholders (See ¹⁴, ARID).

Req-9: Version controlled

As there will be several people actively contributing to technical
documentation, you should version-control its artifacts similar to
source code:

It shall be possible to rollback certain documentation changes.
The ability to work on different versions of the documentation (e.g. in different branches of a version management system) will be useful, especially in larger or distributed development teams.
Documentation shall closely correspond to the identified versions or releases of the system. In case somebody needs to go back in time, for example to fix a bug in an older release of the system, the documentation of that older release shall be available without much effort.

How to version-control documentation

Obviously the straightforward approach is using version control systems
(VCS), like git, subversion or similar tools.

But in reality that’s not always as simple as it sounds:

Organizations might prefer proprietary systems for documentation, for example wikis (e.g. Confluence™).
Documentation authors, especially from non-technical roles, might not be able (or refuse to) use a VCS.

Therefore I propose the following strategy:

Ideally, add all documentation artifacts to a VCS, like git. Your VCS should facilitate branching and merging artifacts. Additionally, it should allow for versioning of binary artifacts.
In case your documentation tools produce binary or proprietary artifacts, which cannot easily be added to your VCS, take the following path:
- Export all release versions of your documentation (those you need to keep) to PDF. Ideally, the result is a single PDF file containing the complete documentation to a specific release or version.
- Add the resulting PDF to version control.
- Automate the preceding two steps (creating and adding the PDF). See automate certain parts.

Req-10: Use appropriate tools

You need a toolchain that satisfies a number of requirements. Let’s start with text processing:

structured text, with different heading levels
lists (numbered and/or bulleted)
cross-references
include images/diagrams created by other tools (at least png/jpg, svg/pdf recommended)
tables
split documents into several parts
support reviews by others than the author(s) with review comments and suggestions
the ability to generate numbers/ids for diagrams, tables and sections/headings

As you should create diagrams or models (see Use
diagrams), your diagramming tool shall support:

static structures, like components, building-blocks or technical infrastructure
dynamic structures, like activities or processes
any standard notation like UML ⁴, C4 ⁵ or ArchiMate ⁷
legends in diagrams, to enable Explain notation
flexible layout, so you can improve visual effectiveness of diagrams (e.g. place important elements in the center, leave appropriate whitespace around elements, use fonts or color to emphasize aspects)
links or cross-references to related diagrams

Prefer established tools

For development teams it might be fun to use the latest and greatest toolchain, with this and that plugin to spice up the otherwise boring task of documenting the architecture.

But: Any toolchain needs a certain amount of maintenance: Updates, backups, user support, and other mundane activities will require somebody’s attention. If you install and operate your own tooling, it might be you who needs to invest this time. If you rely on established tools, your company or organization will provide this support.

I’m a documentation tooling fanboy. I often install and test new tools, so I can develop my opinion and give better advice to my clients. This advice is, more often than not, directed towards the boring toolchains that already exist, where maintenance, backup and support are already sorted out.

In exceptional cases…

If (and only then!) there is no appropriate and usable toolchain available for documentation - you need to introduce one. Such a toolchain requires documentation, maintenance and support effort.

Automate certain parts

You should try to automate certain parts of maintaining your
documentation, to keep it current.

Some candidates for automation are the following:

creating platform-neutral representations (like PDF) of the complete documentation
publishing the documentation, e.g. in a browseable format
testing the sanity or consistency of your documentation, e.g. by checking hyperlinks

For the generation and publishing part, have a look at docToolchain ¹⁵, a lightweight, open-source approach to create and maintain technical documentation. It can read several data formats (AsciiDoc, PowerPoint®, Excel®, a variety of image formats etc) and generate numerous output formats, including static websites.

Req-11: Continuously updated

Deferring documentation to later is like procrastination, or saying “I’ll start healthy living next week.” Start right now, in small chunks. You decided to upgrade that persistence framework? Create an ADR for this decision. The team decided to split that module in two separate parts? Make an appropriate note in the building block view explaining the reason for the split.

How to enable or facilitate continuous documentation

Include documentation task in your Definition-of-Done (DoD)

For many agile teams that work in iterations, the Definition-of-Done “ensures everyone on the team knows exactly what is expected of everything the team delivers” (quoted from ¹⁶)

If something is not contained in the DoD, it won’t be done. Therefore you need to make sure that technical documentation is contained in your DoD.

Reserve small timeboxes for documentation

Invest a few minutes per day, or 15min every Friday to create, maintain or improve your documentation.

Document incrementally

Start with an overview, a birds’ eye perspective. Drill down into details only if somebody is willing to maintain these details.
Start small and add facts when they have gotten robust enough. If certain aspects or elements are still under discussion or investigation, don’t include those in your documentation.
If your team is experimenting with a certain technology, framework, structure or concept, defer the details to a later point in time.
Maintain a documentation backlog, where you collect open issues for your documentation.

Summary

Instead of adding even more text, I provide a graphical summary:

Fig 12.1: Pattern map for technical documentation

You can download a high-res pdf version from arc42.

Acknowledgements

Thanks and praise to my knowledgeable and articulate reviewers Ralf D. Müller, Joachim Praetorius, Marco Stroppel and Peter Hruschka.
Header image by (Retrosupply)[https://unsplash.com/photos/jLwVAUtLOAQ].

References

Map from Wikimedia Commons, provided to Wikimedia by Geographicus Rare Antique Maps, a specialist dealer in rare maps and other cartography. It was first drawn by Daniel de la Feuille in 1706. ↩
Edembourg actually was the correct spelling back in the 18th century... ↩
G. Hargis, Ed., Developing quality technical information: A handbook for writers and editors. Upper Saddle River, N.J: Prentice Hall Professional Technical Reference, 2004. ↩
"UML 2.5 Diagrams Overview." [Online]. https://www.uml-diagrams.org/uml-25-diagrams.html ↩
S. Brown, "The C4 model for visualising software architecture." https://c4model.com. ↩
"ArchiMate Quick Guide." [Online]. Available: https://archimatetool.gitbook.io/project. ↩
VAN HAREN PUBLISHING, ARCHIMATE(R) 3.1 SPECIFICATION. S.l.: VAN HAREN PUBLISHING, 2019. ↩
"Docs.Arc42.org." https://docs.arc42.org/home/. ↩
M. A. Sherman, "Bound to be easier? The negative prefix and sentence comprehension," Journal of Verbal Learning and Verbal Behavior, vol. 12, no. 1, pp. 76–84, Feb. 1973. ↩
C. Hibbard, "Six Principles of Technical Writing." ↩
"Sequence Diagram with References." https://sparxsystems.com/resources/gallery/diagrams/software/sw-sequence_diagram_with_references.html. ↩
"UML Sequence Diagrams - Graphical Notation Reference." https://www.uml-diagrams.org/sequence-diagrams-reference.html. ↩
C. Martraire, Living documentation: Continuous knowledge sharing by design, 1st edition. Boston, MA: Pearson Education, Inc, 2019. ↩
"Documentation Principles — Write the Docs.” https://www.writethedocs.org/guide/writing/docs-principles/. ↩
R. D. Müller, "docToolchain." http://doctoolchain.org/docToolchain/v2.0.x/. ↩
J. Sutherland, "Definition of Done," 07-Aug-2014. https://www.scruminc.com/definition-of done. ↩

Awesome presentations deserve beautiful code

Dr. Gernot Starke — Thu, 20 Jan 2022 11:43:30 +0000

Ever seen ugly source code on a slide?

Welcome to reality. Every now and then we all need to put code samples onto PowerPoint or Keynote presentations.

But wait - presentation programs are made for text, not for code:
They loose all syntax highlighting when you paste your valuable code into their limited textboxes...
Look at this horrible example:

Fig.1 - Slide with ugly code.

I like to introduce carbon - a free website that elegantly solves this problem. On their website the creators describe their goal short and concise:

"Carbon lets you create and share beautiful images of your source code".

You see a possible result below:

Fig.2 - Slide with beautiful code.

What a difference!

Nice'n Easy

Carbon is incredbly easy to use: Simply paste your code in the textbox or drag a source file.

Carbon has a number of interesting configuration options available: Themes, borders, (fake) window controls and more.

Predefines themes

Several themes are available, all except the Darcula-Pro themes are free:

Fig.3 - carbon theme (Monokai).

If you prefer a lighter theme, you are covered too:

Fig.4 - carbon theme (OneLight)

In case you need to, you may even configure your own custom theme. It's a lot of work, and for me the standard themes are way good enough.

Additional Layout Options

You can set padding, border color, shadow, width, (artificial) window controls and a few more options:

Fig.5 - some config options.

Export to png and svg

As expected, carbon can export the image of your source as png or svg:

Fig.6 - export options.

Alternatives

All right, you may also copy/paste source code as RTF (and hope that your presentation tool really preserves formatting).
You could either paste your code to Microsoft-Word (crossing fingers again for preserved formatting), and export it to PowerPoint from Word. Might work, but is less fun. And in contrast to carbon.sh, it does no good to our atmosphere :-).

But: Accessibility Issues

When you convert source code to images, you have to (manually) take care of accessibility.
Images like the ones generated by carbon.sh are not suitable for screen readers.
If you want to adhere to accessibility good practices, you should therefore include the source code shown in the alt-tag of the generated images.

Enjoy!

You never again need to include source code as plain and ugly text in any of your presentations!

One final remark: The name "carbon" was chosen by the authors because they want to reduce CO2 in the atmosphere, read the small print (...offsets) on their website.

Thanx

Kudos to Joachim Praetorius for improving my English and Lars Hupel for pointing out the accessibility issues!

Thanx to Miguel Henriques
for the header image.

Authoring Markdown with Zotero - My Workflow

Dr. Gernot Starke — Sun, 16 Jan 2022 06:58:27 +0000

This post describes an authoring workflow that combines the simplicity of markdown (for writing) with the power of a reference manager (for citing and generation of a bibliography).

This is my current writing workflow:

Collect references, articles, books etc. via Zotero, my citation manager. You might call this the “research phase”.
Edit article/text/book using markdown editor.
Zotero exports your references to a file (continuously in background)
Use make to compile markdown to final result:

4a. pandoc reads your markdown source file
4b. pandoc reads the bibliography file
4c. pandoc formats the bibliography entries following the desired citation style, described in a .csl file
4d. pandoc creates desired output format(s), like docx or html.

You like more details? Then follow along…

Why Markdown (sometimes) isn’t enough

I don’t need to advertise the advantages of markdown as free,
lightweight, simple yet powerful-enough approach to authoring arbitrary content like articles, blogposts (like this one!) or even books.

But: Markdown is optimized for writing, not for citing. If you refer to other sources, like websites, blogs, books or articles in your writing, you want to systematically reference these sources. An example:

See the original article about Markdown
[1] for further info.

In markdown source, this would be the following:

See the original article about Markdown ([@gruberDaringFireballMarkdown]) for further info.

For a blog post, one could simply include the hyperlink, but for any
printed publication a real reference is more appropriate, often
required by publishers. Such references in your bibliography will look similar to the following example:

[1] J. Gruber, “Daring Fireball: Markdown.” https://daringfireball.net/projects/markdown/.

In case you write just a single article, this isn't a big problem: Manually add your sources at the end of your text, and you’re done. But if you (like myself) publish regularly, you might want to have a central repository of your references including publication dates, publisher info, links etc. That’s where Zotero comes into play:

Zotero

Zotero is a free, open-source and user-friendly tool to help you collect, organize and cite references,
like articles, books or research papers (see [2])

Zotero is available for all major platforms (Win, Mac, Linux).

You can sync your reference library across multiple devices and add new references either by ISBN, DOI or similar ways. For online publications, powerful browser plugins are available to add any webpage to your library, together with a snapshot for offline viewing!

Required Zotero Plugins

You need to add BetterBibTeX (short bbt, see [3]) to your Zotero installation.
bbt can export your Zotero library (or a subset of it) to an arbitrary file system location - and keep this export updated! Wheneve you add a new reference to Zotero, this exported file gets updated too!

Adding References to a Text

I like the editing experience of VS-Code, mainly for its rich plugin universe. One of these plugins is the Zotero-Citation-Picker:
It gives you a search box and retrieves results directly from your Zotero library.

When you have found the correct reference, the plugin enters the unique citation key at the current cursor location.

Compile the Bibliography

Now you have a markdown text containing several of these “citation keys” - but still no list of references, the so called bibliography*.

That’s where the next player enters the field: pandoc (see [4] ).

Compile Output with Pandoc

“If you need to convert files from one markup format into another,
pandoc is your swiss-army knife.”

Although you could use pandoc directly from the command line, I prefer good old make, as I cannot remember parameters and options. Look at my Makefile:

OUTPUT=out
SOURCE=content.md # name of your markdown source file

CITATIONSTYLE=ieee.csl
BIBLIO=~/Dropbox/zotero-export/zorg-biblio.bib
TARGET=$(OUTPUT).docx $(OUTPUT).md

PANDOC=pandoc \
   --from=markdown \
   -s --bibliography $(BIBLIO) \
   --citeproc \
   --csl=$(CITATIONSTYLE) \
   --metadata link-citations=true \
   $(SOURCE)

# create docx
$(OUTPUT).docx: $(SOURCE) $(METADATA)
   $(PANDOC) --output=$(OUTPUT).docx


# create github flavored markdown
$(OUTPUT).md: $(SOURCE) $(METADATA)
   $(PANDOC) --to=gfm --output=$(OUTPUT).md

all: $(TARGET)

clean:
   rm -rf $(OUTPUT).docx $(OUTPUT).md

.PHONY: all clean $(TARGET)
.DEFAULT_GOAL := all

(This Makefile is available as Github-gist)

Yes, I know: One could wrap pandoc into a tiny Docker container - but pandoc is easy to install and is incredibly fast. A Docker container would remove the dependency to pandoc, but add the overhead of a running container.

When you try this on your own, please fulfill the following
prerequisites:

install pandoc
add the citation-style csl-file to your current directory (I didn’t manage to keep this file in any central location - help or suggestions would be appreciated)
add the Makefile to the current directory
update the BIBLIO variable in the makefile to point to your Zotero export location.

Image Credit

Splash image by Trent Erwin on Unsplash.

Bibliography

This bibliography has been auto-generated by the workflow shown above.

[1] J. Gruber, “Daring Fireball: Markdown.”
https://daringfireball.net/projects/markdown/.

[2] “Zotero | Your personal research assistant.”
https://www.zotero.org/.

[3] E. Heyns, “Better BibTeX for Zotero.” Dec-2021.
https://retorque.re/zotero-better-bibtex/

[4] “Pandoc - About pandoc.” https://pandoc.org/.