DEV Community: Erik Hofer

Introducing scope42 - Improve your software architecture with precision! 🎯✨

Erik Hofer — Tue, 25 Oct 2022 13:00:46 +0000

As software architects, we often face legacy systems that have many architectural issues. Also in greenfield projects, the architecture is developed iteratively and needs to be improved on a regular basis.

_Let's see how architecture improvement can be organized and assisted by tooling_ 🚀

Introduction to aim42
The birth of scope42
- Management of items
- Relationship graphs
- Docs-as-code
State of the project & roadmap
Tech stack
Your feedback is welcome ❤

Introduction to aim42

aim42, the Architecture Improvement Method, is an open, community-driven framework for working on software architectures.

aim42 supports software evolution, maintenance, migration and improvement - in a systematic and pragmatic way

— About aim42

It defines a process consisting of three phases

Analyze
Evaluate
Improve

accompanied by cross-cutting concerns. The method reference collects concepts and patterns that can be used to implement the process. It proposes three main types of entities of special interest:¹

Issue (problem)
Risk (potential future problem)
Improvement (remedy)

Instances of these, I'll call them items, need to be collected, managed, and associated with each other. This is where the question of tooling comes into play.

aim42 is licensed under Creative Commons Attributions Sharealike. For more information refer to:

The birth of scope42

I came into touch with aim42 in the course of my master's thesis which was concerned with improving the software architecture of a legacy system. I approached this with my standard means of note-taking but realized at some point that a dedicated tool would be beneficial. This birthed the idea of scope42 and I started to work on it after finishing my thesis.

[github.com/scope42/scope42](https://github.com/scope42/scope42)

Management of items

One of the first challenges that arise, as the number of items grows, is managing them. This requires structured metadata (e.g. title, status) as well as a way to filter and sort by it.

This is the core functionality of scope42. It is similar to a ticket tracker. An existing generic one would require quite some configuration work to get started - scope42 comes with the aim42 domain model out-of-the-box. It even incorporates some content of the aim42 method reference so you can learn it along the way.

Also, I try to provide a modern and slick UI that is not overloaded but focused on the particular domain. The aim is to make the act of writing documentation as fun as possible. At least in my personal experience, this is already quite successful.

Another notable feature is the integrated full-text search and quick access to items via the Strg + K / ⌘ + K shortcut.

Relationship graphs

While you can link tickets in classical trackers like JIRA, visualization of these relationships is not a common feature.²

The connection of items (e.g. issue is caused by another issue, improvement mitigates risk, etc.) are at the core of the aim42 domain model. It is an important concern and the overview can get lost quickly with a growing number of items.

Because of this, scope42 features relationship graphs for visualization. On an item's details page, you can quickly see its direct connections. Also, when viewing a list of items (e.g. filtered by specific criteria), a graph displays the relationship between these items.

Docs-as-code

The scope42 web app runs in your browser - no need to download anything. However, a vital difference to most such applications is how the data is stored. There is no database.

All items reside as files on your local machine. The app interacts with them via the File System Access API (currently sadly does not work in Firefox, only in Chromium-based browsers). This ensures privacy (data is never sent anywhere) and data ownership - if scope42 disappears tomorrow, you will not lose your data.

Another important aspect of data ownership is an open format. You should, at any point, be able to take your data and use it somewhere else. In fact, scope42 is built around the idea of processing the structured data managed with it and e.g. incorporating it into your existing documentation.

A main driver for the choice of a concrete format is the concept of docs-as-code. It is based on the idea that documentation is stored in a lightweight text-based format. It can then - similar to or even alongside code - be checked into version control, authored with your editor/IDE, and automatically processed and tested. You can read more about this topic at docs-as-co.de.

Applying this approach to more structured data is a bit of a challenge. As a compromise between human-readability and machine-readability, a format based on YAML is used. Example (improvement):

title: Upgrade Spring Boot version
created: 2022-05-02T17:25:16.909Z
modified: 2022-05-02T18:09:42.952Z
status: implemented
tags:
  - backend
  - urgent
  - security
ticket: https://github.com/scope42/scope42/issues/91
description: The current version of Spring Boot upgrades dependencies to secure
  versions of Log4j.
resolves:
  - issue-3
modifies:
  - risk-1
comments:
  - author: Jane Doe
    created: 2022-05-02T17:27:17.630Z
    content: Accepted and placed into the fast lane on the board.

In many places (for example item descriptions and comments), Markdown is supported for formatting. This includes advanced features like GFM extensions, code syntax highlighting, links to scope42 items, Mermaid diagrams, and more to come. For more information see docs.scope42.org/markdown.

For collaboration, you can use your file-syncing technology of choice. It is recommended to check your scope42 data in a Git repository. As for your code, this enables a detailed change history and the ability to merge concurrent modifications.

State of the project & roadmap

I already use scope42 for multiple real architecture projects at work. The experience from these - and iterative improvements based upon it - have led to the first stable release being recently published.

// Detect dark theme var iframe = document.getElementById('tweet-1582788370584940544-135'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1582788370584940544&theme=dark" }

It is still in a relatively early stage in terms of features that I have in mind. But the release of version 1.0.0 means that there will be no breaking changes to the data format in the foreseeable future. You can safely start to use it and will be provided with automatic migrations if necessary at some point.

The project is completely open-source and free software. It is published under GPLv3 on GitHub.

scope42 / scope42

🎯 Improve your software architecture with precision!

Improve your software architecture with precision!

This tool helps you to keep track of issues, arising risks and possible improvements of your existing architecture. The terminology and concepts are based on aim42, the Architecture Improvement Method.

scope42 is a Progressive Web App that runs entirely inside your browser. Click the link below to access the app.

🔗 app.scope42.org

Features

✨ Management of items with a fancy UI

Filtering • Sorting • Full text search • Learn about aim42

💑 Graphs for visualizing the relationships between items

Quick overview • Drag & Drop • Navigate

📝 Docs-as-Code principle and full data ownership

Human-readable • Check into version control • Process programatically • No vendor lock-in

title: Upgrade Spring Boot version
created: 2022-05-02T17:25:16.909Z
modified: 2022-05-02T18:09:42.952Z
status: implemented
tags
  - backend
  - urgent
  - security
ticket: https://github.com/scope42/scope42/issues/91
description: The current version of Spring Boot upgrades

…

View on GitHub

**The app is available at [app.scope42.org](https://app.scope42.org).**

There is a large roadmap of planned features I am eager to implement. These for example include UX improvements like an advanced Markdown editor, more Markdown features, and item management capabilities.

In the long run, scope42 will be expanded from an architecture improvement tool to a general architecture documentation platform. In most projects, these two concerns will naturally play together. For example, if you use the arc42 template for your documentation, the aim42 items would be integrated into chapter 11 "risks and technical debt".

In fact, there is already an additional item type decision (ADR). This falls more into the realm of documentation (arc42 chapter 9). However, it quickly became clear that ADRs are also connected to aim42. Often, proposed improvements need to be assessed by architecture decisions, especially if there are multiple options to resolve an issue.

In addition to this, the focus will be laid on exporting data and integration with other tooling. This includes a convenience library for programmatic processing, static site export, and a set of recipes, e.g. for integrating with things like docToolchain.

Lastly, a great advantage of this application over a static collection of Markdown/AsciiDoc files will be dynamic adaption for specific tasks. It should be possible to view your documentation from different architectural perspectives. This principle is for example implemented for diagrams by Structurizr. It will be applied more universally by scope42. Also, it may be useful to tailor your documentation to different target audiences (e.g. developers, management) as described by arc42.

Tech stack

Code is written in TypeScript
UI is built with React and Ant Design
Relationship graphs are created using Cytoscape.js
Code is organized in a monorepo powered by Turborepo
The CI/CD pipeline is based on GitHub Actions and Changesets
The production version of the app and deployment previews for pull requests are hosted on Netlify

Tech choices and other architecture decisions are documented using scope42 itself in the repository.

Your feedback is welcome ❤

If you think this app could be useful to you, don't hesitate to try it out yourself!

If you have any feedback, feel free to leave a comment here, open an issue/discussion on GitHub or get in touch with @scope42_org on Twitter.

My dream would be to build a community around this tool that shapes it so we can all profit from each other and make our daily work even more fun and effective 🚀

Following for updates on any platform is greatly appreciated - as well as sharing this article with people you think may be interested ❤

In the actual aim42 domain model, risk is not a primary entity type but an extension of issue. In scope42 I decided to make the distinction more sharp. Also, there is another entity type cause in the method reference. I modeled this as a relationship between issues/risks instead. ↩
Obsidian is good at visualizing relationships and would adhere to docs-as-code. However, it is only free for personal use. Software architecture is often worked on in a commercial setting. ↩

Publish to DokuWiki programmatically without any API

Erik Hofer — Tue, 20 Sep 2022 17:16:57 +0000

In a project I am working on we use DokuWiki for our documentation. In addition to the manually written text, I wanted to integrate some automatically generated content that is updated every day.

There is a Plugin that provides a REST API but natively, there is only XML-RPC. Not that fancy, but for simply pushing simple page content, this would be sufficient. So I tried a fetch version of the sample jQuery client.

ℹ Note that the following examples use the native fetch API that required Node 18+.

const WIKI_URL = 'https://example.com/dokuwiki'

const xml=`
  <?xml version="1.0"?>
  <methodCall>
    <methodName>wiki.getRPCVersionSupported</methodName>
    <params></params>
  </methodCall>`

fetch(WIKI_URL + '/lib/exe/xmlrpc.php', {
  method: 'POST',
  body: xml,
  headers: { 'Content-Type': 'text/xml' }
})
  .then(response => response.text())
  .then(text => console.log(text))

It printed the follwing response.

<methodResponse>
  <fault>
    <value>
      <struct>
        <member>
          <name>faultCode</name>
          <value><int>-32605</int></value>
        </member>
        <member>
          <name>faultString</name>
          <value><string>XML-RPC server not enabled.</string></value>
        </member>
      </struct>
    </value>
  </fault>
</methodResponse>

Uh, bummer 😐
Now, the DokuWiki instance is shared among multiple teams. Installing a plugin or enabling the XML-RPC API would involve at least some bureaucracy.

Because of that, I tried to come up with a solution without any API. I mean, if I can do it in my browser, there must be a way for a script to do it, right?

So the first thing I did was looking at what DokuWiki does internally when I edit a page in the browser. In principle, it is a simple HTML form.

The form data shows that crafting a corresponding HTTP request is not that simple. We probably need to read some values from the form or reverse-engineer how to produce them. Especialy sectok and changecheck look suspicious.

To save time on that, my first impulse was to use a UI automation tool like Selenium but luckily, I came up with a way simpler solution.

It is based on cheerio, an "implementation of core jQuery designed specifically for the server". With this, crafting the request becomes as easy as submitting a form with jQuery (almost).

But first, we need to sign in. For this, we have to make fetch support cookies (side note, even with XML-RPC, this would have been the case 🙄). I did it by utilizing the package fetch-cookie. Then, I just looked how the login form of DokuWiki works and came up with the following request.

const makeFetchCookie = require('fetch-cookie')
const fetchCookie = makeFetchCookie(fetch)

const WIKI_URL = 'https://example.com/dokuwiki'
const USERNAME = 'example'
const PASSWORD = 'example'

await fetchCookie(WIKI_URL + '/doku.php?do=login', {
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  body: new URLSearchParams({ u: USERNAME, p: PASSWORD })
})

After this, we can request the editing form for the page we want to update.

const PAGE_PATH = 'path/to/page'

const html = await fetchCookie(`${WIKI_URL}/doku.php/${PAGE_PATH}?do=edit`)
      .then(response => response.text())

We load the HTML into cheerio and look for the text area with the ID wiki__text. Then we can update the content of the text area.

const cheerio = require('cheerio')

const $ = cheerio.load(html)

const textarea =  $('textarea#wiki__text')

textarea.text('This is the new, //generated//, page content.')

Note that if you want to replace parts of the exsiting text, you can get the current content by calling textarea.text() without the argument.

We can now get the content body for our POST request in the following way.

const formData = $('form#dw__editform').serialize() + '&do[save]='

The addition of do[save]= is needed because that would be added when clicking the submit button. Without this, no data will actually be saved.

Finally, we can craft and send the request.

await fetchCookie(`${WIKI_URL}/doku.php/${PAGE_PATH}?do=edit`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  body: formData
})

We successfully published to DokuWiki programmatically without any API 🎉

Full example code:

const cheerio = require('cheerio')
const makeFetchCookie = require('fetch-cookie')
const fetchCookie = makeFetchCookie(fetch)

const WIKI_URL = 'https://example.com/dokuwiki'
const PAGE_PATH = 'path/to/page'
const USERNAME = 'example'
const PASSWORD = 'example'

async function main() {

  await fetchCookie(WIKI_URL + '/doku.php?do=login', {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({ u: USERNAME, p: PASSWORD })
  })

  const html = await fetchCookie(`${WIKI_URL}/doku.php/${PAGE_PATH}?do=edit`)
      .then(response => response.text())

  const $ = cheerio.load(html)

  const textarea =  $('textarea#wiki__text')

  textarea.text('This is the new, //generated//, page content.')

  const formData = $('form#dw__editform').serialize() + '&do[save]='

  await fetchCookie(`${WIKI_URL}/doku.php/${PAGE_PATH}?do=edit`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: formData
  })
}

main()

Build and Publish a Multi-Platform Electron App on GitHub

Erik Hofer — Thu, 31 Dec 2020 15:50:18 +0000

Introduction
Create Electron Application
Build on Linux, Windows and macOS
Publish Release to GitHub
Draft Releases
Outlook

Introduction

I recently dived into Electron for a hobby project. I like the approach of using web technologies to create the UI and shipping a native application to all platforms.

For a Proof of Concept, I wanted to actually set up the process of building and distributing such an application on GitHub. It was a lot of trial and error to finally get there. Because of this, I documented my learnings in this tutorial. The final setup I came up with is actually surprisingly simple.

You can find the complete example here:

erikhofer / electron-publish-example

Example repository for building and publishing a multi-platform Electron app

Create Electron Application

We are going to use Electron Forge for creating the example application.



npx create-electron-app electron-publish-example
cd electron-publish-example
npm start

We can now see our example application in a native window.

The distributable packages can be built with the following command.



npm run make

This will only build package formats that are supported by your operating system. For more information see Electron Forge Makers.

Build on Linux, Windows and macOS

Ok, so how do we build the application for other platforms? Luckily, GitHub provides a free and easy way to do this. We start by creating an empty repository and pushing our example code. I assume that you know the basics of Git. Everything we do from now on needs to be pushed to GitHub.



git init
git add .
git commit -m "Create example app"
git branch -M main
git remote add origin https://github.com/erikhofer/electron-publish-example.git
git push -u origin main

We then create a new file .github/workflows/build.yml with the follwoing content.



name: Build
on: [push, pull_request]

jobs:
  build_on_linux:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@master
      with:
        node-version: 14
    - name: install dependencies
      run: npm install
    - name: build
      run: npm run make

  build_on_mac:
    runs-on: macos-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@master
      with:
        node-version: 14
    - name: install dependencies
      run: npm install
    - name: build
      run: npm run make

  build_on_win:
    runs-on: windows-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@master
      with:
        node-version: 14
    - name: install dependencies
      run: npm install
    - name: build
      run: npm run make

It basically defines the same job three times for different operating systems. The workflow is executed for all branches and for pull requests to verify that the application can still be built after making changes.

After pushing the file, we go to the "Actions" tab of the repository (example). We can see our newly created workflow running. Click on it to see the individual tasks and wait for them to finish.

That's it! 🚀 We are now building a native application on Windows, Linux and macOS.

Publish Release to GitHub

Fine, now how do we get access to the distributable files? We could set up artifact uploading for this. While this is useful for developers, it is not sufficient for providing the application to users. For that, we want to utilize GitHub releases.

A release is based on a Git tag. It has a descriptive text (e.g. changelog) and can have files attached to it. That's exactly what we need.

Electron Forge provides a GitHub publisher that does all the work for us. We need to install it in our example project.



npm install -D @electron-forge/publisher-github

Then we add the following configuration to the package.json file (make sure to adapt this to your repository).



{
...
  "config": {
    "forge": {
    ...
      "publishers": [
        {
          "name": "@electron-forge/publisher-github",
          "config": {
            "repository": {
              "owner": "erikhofer",
              "name": "electron-publish-example"
            }
          }
        }
      ]
    }
  }
}

Finally, we create a second workflow .github/workflows/release.yml with the following content.



name: Release
on:
  release:
    types:
      - created

jobs:

  publish_on_linux:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2       
    - uses: actions/setup-node@master
      with:
        node-version: 14
    - name: install dependencies
      run: npm install
    - name: publish
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: npm run publish

  publish_on_mac:
    runs-on: macos-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@master
      with:
        node-version: 14
    - name: install dependencies
      run: npm install
    - name: publish
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: npm run publish

  publish_on_win:
    runs-on: windows-latest
    steps:
    - uses: actions/checkout@v2       
    - uses: actions/setup-node@master
      with:
        node-version: 14
    - name: install dependencies
      run: npm install
    - name: publish
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: npm run publish

It is basically the same as build.yaml but uses the publish script (make is not needed, this would build the app twice). It also accesses the GITHUB_TOKEN and is only executed for created releases.

After pushing all changes, we can go to the "Releases" section in the "Code" tab of the repository and click "Create a new release". As "Tag version" we chose "v1.0.0" and click "Publish release". In the "Actions" tab we can now see that our newly created workflow is executed.

After it has finished, we go back to our release (example). It should now have the packaged application for all platforms attached.

Draft Releases

So far we pushed a tag and then manually created a release via the GitHub website. With this approach, the release is published immediately and the files are attached later—assuming the action is executed successfully. If something goes wrong, watchers have already been notified about the release via email.

Ideally, we want to draft a release, attach the application files and then publish, if everything looks good. There is, however, a caveat.

The release event is not triggered for draft releases. (Source)

That means that if we create a draft release and then publish it, the created activity is never detected. If we use published instead, we still have the same behavior as before.

We can solve this by letting Electron Forge create the release. For that, we first change the workflow so that it is executed for all version tags.



name: Release
on:
  push:
    tags:
      - 'v*'

This is sufficient for automatically creating releases. Additionally, we can now configure the publisher to create a draft instead.



"publishers": [
  {
    "name": "@electron-forge/publisher-github",
    "config": {
      "repository": {
        "owner": "erikhofer",
        "name": "hello-electron"
      },
      "draft": true
    }
  }
]

To create tags, we can utilize the npm version command. It automatically updates the package.json and package-lock.json. Let's create a new version 1.1.0.



npm version minor
git push --follow-tags

⚠ Make sure to push the created tag to GitHub. By default, Git does not push tags.

After the workflow has finished, we go to the releases page again. We can now see the automatically created draft release.

If everything looks fine, click on "Edit", enter a description and click "Publish release". We now have a proper setup for distributing our multi-platform Electron application. ✨

Outlook

If we actually want to pubslish a software that is used by the public, we need to address code signing as the next step. It is a security mechanism and at least on macOS, there is no practical way around it. It is also needed for auto-updating the application on other platforms.

Electron Forge and other tools have convenient built-in support for the code signing process. But be aware that certificates need to be purchased with an anual fee.

Speaking of auto-updating, this is an interesting topic to look into next. Now that we have a setup to host our binaries via GitHub releases, we can also use this for update distribution. There is even a completely free service for open source applications. For more information, take a look into the Electron Forge docs.

We created an Open Source Programming and Auto Evaluation Platform for CS Education

Erik Hofer — Fri, 28 Aug 2020 12:18:07 +0000

At our university there are still paper-based coding assignments in computer science classes. To make a step into the 21st century, we started developing Programming and Auto Evaluation Platform almost two years ago. We are now happy to anounce it to the public.

Free and open source under AGPL
Programming language agnostic
Flexible auto-evaluation system based on Docker
In-browser IDE for zero-setup coding exercises
Integration with Learning Management Systems via LTI standard

We still have a large roadmap ahead of us but we are already running in production from the MVP on. We continuously field-test with actual students and would love to get other educational institutions on board. If you are interested in trying in trying it out, please get in touch with us!

The application is developed with Kotlin (Spring) in the backend and TypeScript (React) in the frontend. We are looking for contributors to make this a real community-driven project!

Check out codefreak.org or github.com/codefreak/codefreak

codefreak / codefreak

Online Programming Platform and Evaluation/Auto-Feedback System for Coding Assignments

Code FREAK

Code FREAK (Code Feedback, Review & Evaluation Kit) is an online programming platform and evaluation/autograding system for coding assignments. It supports every major programming language (language agnostic) and has a modular evaluation system based on Docker.

Main Features

Support for every major programming language (language agnostic)
Pluggable evaluation system based on Docker (Dynamic Testing, Linting, …)
In-browser IDE based on VSCode/Coder
Modern UI based on React and Ant Design
Integrates with learn management systems (LMS) via LTI 1.3 standard
LDAP authentication
100% free and open source

Installation

We currently only support installation via Docker. The image name is cfreak/codefreak. Check out its tags for the latest version.

Try with Docker 🐋

You can try out Code FREAK locally. The only requirement is a working installation of Docker on your computer.

docker run -it --rm \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 8080:8080 \
    cfreak/codefreak

The UI is accessible…