DEV Community: Niklas Merz

The Cordova community survey

Niklas Merz — Wed, 29 Nov 2023 16:10:00 +0000

If you are reading this you probably use or have used Apache Cordova. The Apache Cordova projects wants to understand the users and contributors better. Therefore, we invite you take part in a short survey for app developers and contributors of Cordova plugins.

Take survey

Thank you very much for taking part in this survey. The goal of this survey is to identify areas that can be improved by the Cordova community together. The survey is open from now on to December 24, 2023. The results will be shared on the Cordova blog.

For me personally the survey is very important, and I am really looking forward for the results. I think we all have a certain impression about the project and the survey should help us get a clear picture of what to do to make this project better for everybody.

Disclaimer: The survey questions are designed to not contain any personal information and can be answered anonymously. Please take note that the survey is using Google Forms to process the responses. The Google privacy policy is applicable for the data processed.

It's all about the community - Open Source Personas

Niklas Merz — Sun, 29 Oct 2023 09:00:00 +0000

After some time working with open source communities you start to recognize names and people with different roles. From my experience there are typical types of open source maintainers. I tried to come up with some personas and made up some names for them. If you have more ideas let me know.

Of course a person can be more than one type of maintainer and people shouldn’t be put in boxes, but I think it’s kind of interesting to see the types of people required to run a project successfully.

Silent hero

The first type we want to have a look at is very important in the projects I’m active in. Let’s call it Silent hero. It’s a person that prefers to stay in the background and does not need attention or fame. They do lots of work and often take care of the thankless tasks like big refactorings, dependency updates, releases or documentation updates.

They just want to work effectively and drive progress. That’s why they usually do a lot of coding work and prefer not to engage in big dicussions if they think they won’t help to move the project forward.

From my experience this is the most important type of maintainer to keep a project alive.

Helping hand

Some projects have maintainers that are very active in engaging with people asking questions, submitting bug reports or PRs. A helping hand is the most important person for the outward perception and new users and contributors of the project. They are the first contact person for the community. By providing answers to questions and triaging issues and PRs they help the project gaining new users and contributors.

It’s very valuable to have people on board that have good communication skills and like to help others.

Advocate

Some projects may have a kind of marketing person. They help to spread the word about the project by doing talks, writing blog or social media posts. They also care about branding or the website to make the project attractive for new users and want to see it grow.

Legend

Sometimes projects are lucky to still have the founders as active maintainers. They have deep knowledge about the internals and know the history behind design decisions. The Legend started the project or is part of community since “the early days”. They may not be the most active contributors, but they are still around for helping other maintainers with their rich experience. They might be a little hard to reach because the moved on to other things after years of working with this project. All maintainers should respect their experience and try to keep them around.

I found a bug in an open source project - What to do next?

Niklas Merz — Sun, 22 Oct 2023 10:19:32 +0000

Practical advise for getting issues with open source projects solved

If you are working with open source dependencies in your projects it’s very likely you will encounter some type of issue or feature requests. In the last few years of working as software developer I got in touch with many open source communities of different shapes and sizes. I talked to a lot of people who say they don’t really know how get started with open source.

Let’s walk through of a typical journey for your first interaction with open source. In this example we will talk about a bug you discovered in an open source dependency in your project. If you want to add a feature to the steps are pretty similar.

Investigate the bug

You really need to have a close look at your problem to determine the next steps. At first, I try to isolate the problem. If you think one of your dependencies is the problem you need to break it down and get a good understanding what the real problem is.

Identify the right project

Your project is most likely compromised of many open source dependencies. You must try to isolate the problem and make sure you find the right project to report and fix your bug.

Check new versions

If you have the right project you should check if there is a new version available and test it if available. You should always keep the versions up-to-date and fix the problem in the latest version if possible. Also, you might be able to try versions that are in development just by checking out the development branch from GitHub. Package managers like npm or pip usually have a feature for adding dependencies from git.

Create minimal reproduction

If a new version does not fix the problem you should start your reporting and fixing journey by creating a minimal reproduction project for this problem. Typically, you start by creating a new project and just add the dependency in question. Then you should add the minimum code required to see and understand your problem. This makes it a lot easier for you and others to understand and test the problem with the dependency.

Report the bug properly

With the minimal reproduction project you can now report the bug precisely. You should provide all the information you can, to help the maintainers to understand and fix the problem.

Try to find a fix yourself

If you find any issues, projects tend to love to get them reported with possible fixes as a pull request. After your investigation you might have an idea what goes wrong in the dependency. Don’t be afraid. Have a look at the code and try making some changes. You might find a solution yourself pretty quickly. You can also test it very easily with your minimal reproduction project and your actual project. This helps a lot with submitting the patch and explaining why it’s needed. If you are struggling to get started with fixing the dependency you can ask in your submitted issue for ideas and help to get started.

Be patient and respectful

Once you submitted an issue or pull request just be patient. You cannot expect the maintainers to respond quickly. Everybody in open source works on their own terms and schedule. Some maintainers have very limited timed and only work on open source in their free time. Keep that in mind if you are eagerly waiting for a response and always be respectful in your communication.

Stay with the community

It’s good to stay around if you care about the project and not only appear if you have issues you need help with.

Participate in the community

Feel free to take part in the communities you care about on GitHub, Slack, Stackoverflow etc.

You can contribute in many ways not just working on code and issues. It’s also important to help users by answering questions or improving the documentation.

Keep your dependencies up to date

It’s a good practice to regularly check your dependencies for new versions and update them. This will help you to avoid debugging problems that are already fixed in newer versions and improves overall security and stability of your project.

The 4-day-workweek is the best change in my career so far

Niklas Merz — Wed, 31 Aug 2022 22:00:00 +0000

As of today I have been working as a programmer for exactly 8 years (I am including my 2,5 years of apprenticeship here). Today is also my first anniversary at my current company. Two months ago I started working 4 days a week and this is the best change in my career so far.

My career might sound boring at first. I worked for seven years at the same company. I learned a lot and really liked the people there. Last year it was finally time for a change and I got to work for the startup of a good friend. My colleagues are awesome and working for a small company is oftentimes very exciting and can be lots of fun.

But let’s be honest, working on software is not always fun. In my career I’ve always had days, weeks, sometimes even months when the tasks sucked and work is just hard. Programming in flow, creating cool things and seeing the results of your work is awesome. It really depends on the clients, projects and many other factors how much of your work is doing cool stuff or just doing dull work that needs to be done.

I’m really grateful to have a boss who cares a lot about how I’m doing and what can we do to improve our work environment.

Recently, my company introduced a model where you switch your full-time job to a part-time position where every Friday is off. We talked about this in our regular one-on-one meeting in spring, and I decided to switch to this model for the upcoming summer months. It’s been two months now, and I would go so far that working part-time is the best thing that happened to me in my career. I’m a lot happier now than I was in the previous months. The gained free time made me more mindful of how I spend my time. To me, it feels like I spend the work hours more efficient and the weekend is a lot more relaxing. I have more time for activities that are very important for me, like spending time with friends and family and my hobbies.

The weekend just feels longer. Not just a bit longer but much longer. On Saturdays, I am more relaxed now because it’s the midpoint (as we Germans call it: Bergfest) of the weekend. It’s similar to vacations. The start feels a bit weird and the middle days are the best and most relaxing part. At the end you are less relaxed because the end of your free time is in sight. On Sundays, I no longer really feel what I used to call “Sunday blues”. I am usually looking forward to Monday. I do a lot of code reviews, and often I can review new code, fixed PRs and new tasks that happened on Friday. Going back to the work I did three days ago also feels good because I get a fresh look at it. I also try to finish stuff on Thursdays and prepare stuff for a good start on Mondays. Somehow a 4-day week is a lot less frightening than a “normal week”, too. It goes by really quickly and the outlook of the next relaxing weekend is really motivating. In short, I would say a three-day weekend does its job of recovering me a lot better than the normal weekend. Working four days is mentally less exhausting and feels more effective.

Efficiency and productivity gains or losses are probably really important for companies implementing the four-day-workweek or thinking about it. I sadly just don’t have any hard metrics to measure it (also productivity in software engineering is especially hard to measure in general). While you are working 20% less, I don’t think you will lose 20% of work getting done. I feel you spend your time more wisely. It’s only been two months for me, and it probably takes a while to see the effects on productivity. I imagine it also depends a lot on the job, company, people and clients if this model works. Every company needs to find ways to make it work. I really like that we discuss this in the team, and it’s not just a management decision.

Maybe this change is so good for me in particular because it helps me pursue my hobby. During the last few years mountain biking has become my favorite free time activity. Going out on long tours consumes quite some time and the extra day on the weekend opens up more possibilities. It makes it a lot easier for me to organize a “big bike day”, getting chores done and visiting family and friends. The weekend feels less crammed and more relaxing.

I like spending time outdoors in the summer. After work, I don’t really have the energy to sit in front of the computer to work on open-source or passion projects. For now my four-day-workweek is planned until the end of October. Then we will discuss and decide if I go back to a full workweek. In the cold and dark winter I usually work more on open-source and private projects. Maybe I will also use the longer weekend for that. But my company has the Open Source Friday where everybody is encouraged to work half a day each week on open-source projects of their choosing. I can image this would be perfect for me as well.

Working with Git without actually getting it

Niklas Merz — Thu, 31 Dec 2020 15:00:00 +0000

An opinionated guide to Git

Nobody actually understands Git but everyone knows exactly what they need to get their work done.

Git is a pretty complex tool with many features and ways to work with it. I always feel I have now idea how to properly use it but I know just enough that I can use it confidently for my daily work.

I recently talked with more people about some basics of Git and how I use it. Getting started with Git can be scary and you might need some time to gather the knowledge you need to work efficiently without messing up every now and then. It’s totally normal to mess up your local repository, copy your changes and start with a fresh clone of the repository.

Just try it out and learn along the way. That’s how I did it over the years and now I thought why not create a document on the aspects of Git that are important for me to make it easier to get on a path. This is a list of Git features I need daily or every now and then with short explanations. It is not an extensive guide or tutorial for Git. You can find many of them in good quality plenty online.

Overview

An opinionated guide to Git
Overview
Terminology
Setup
- Identity
- SSH
- Yubikey
Getting started with a project
- Clone
- Init
- Forking
Committing
- Add & Commit
- Commit Message
- Push
- Pull
Understanding Change
- Git log
- Blame
- Tags & Releases
Workflows & Branches
- Pull requests (Merge requests)
- Rebase
- Updating a feature branch
- Cleaning up commits
Closing words

Terminology

You should understand some terms to work with this guide:

Version control = system responsible for managing changes to computer programs
Git = a version control system
GitHub = provider which offers Git repository hosting and collaboration tools
Repository = data structure that stores metadata (changes) for a set of files or directory structure

Setup

Some steps for getting started with Git for the first time or setting it up on a fresh operating system.

Identity

If you start to use Git for the first time you probably need to set up your identity. Your identity is just a name and an email address that are used for every commit. You typically set up a global identity that is used for all repositories on your device. Make sure you use an email address added to your account on GitHub etc. to allow these service to link your commits to your accounts.

Setting up a global identity is simple. Your type to commands:

$ git config --global user.name "John Doe"
$ git config --global user.email johndoe@example.com

More information

SSH

You should set up SSH keys on your development machine to use them for authentication when pushing to Git providers like GitHub, Gitlab, Bitbucket. SSH keys consist of a private and a public key. You add the public key to your account on GitHub etc. and the private key stays on you machine and you should take care to not publish or loose it. Every time you run commands with Git remotes like git push origin or git fetch origin Git uses the key for your account instead of your password. GitHub for example deprecated the basic authentication.

For setting up SSH with GitHub follow this document.

Yubikey

I personally prefer having my private key for SSH and GPG on a smartcard like a Yubikey instead of my devices. The main benefit is that I can use different machines more easily without worrying about updating and securing my private key. This is just nice to have.

This repository explains this a bit more and has an excellent guide on how to set it up.

Getting started with a project

If you have set up Git on your machine you can start with a project. For existing projects you usually start by cloning the repository from a remote like GitHub. If you have a project folder with source files you want to use init to set up the Git repository for this project.

Clone

With git clone <URL> you create a local copy of the repository on your development device. With this repository you get the files and the complete history with all changes and commits.

If you have write access to the repository you are cloning, you usually want to use the SSH URL like git@github.com:NiklasMerz/git-examples.git instead of the HTTPS URL to use SSH authentication when pushing your changes.

Init

If you start a new project just use git init in the project folder to initialize a new Git repository. You can then use git add * and git commit to add your files. If you want to push your repository to Git servers like GitHub you can use git remote add origin <URL> to add a new remote and git push origin -u master afterwards to push the changes to the remote for the first time.

Documentation for init and remote

Forking

Forking is commonly used on platforms like GitHub to contribute changes to an open source project. If you clone an open source project you don’t have write access to push changes for obvious reasons. If you create a fork on GitHub first, GitHub creates a copy of the project in your account. You have write access to your own fork and can make changes. If you want to contribute your changes back to the original project you can create a pull requests from your fork. More on that later.

Committing

Small commits with good commit messages are crucial for an efficient use of Git.

Add & Commit

Committing with Git is usually a two step process. After you did a change to a file it is an unstaged change. You add it with git add <filename> or all files with git add * to the staging area. Staged changes are prepared for the commit and you can now commit with git commit.

I often use the shortcut git commit -am "<Commit Message>". This command does many things at once. -a adds all changed files to the staging area. Caution: New files won’t be added and you need to add them before. -m sets the commit message directly from the command and it won’t open a text editor like usual. This has some drawbacks because you don’t really see which files get committed and you cannot properly format the commit message. I use this for small commits and simple changes. For large changes I prefer the normal two step git add and git commit process.

Personally I use mix of Git on the command line and the interface in my IDE (VSCode). Some prefer UI Git clients some just the command line. I feel I get better control with the Git CLI but use what fits best your taste and environment.

Commit Message

I think it’s more important to write the why and not the what in a commit message. You can see what a commit does by looking at the code diff but if you write the reason for a change (especially if it’s a strange fix) in the commit message your future self or colleague will thank you for making it easier to understand. It can be helpful to describe where a change was made (like backend/frontend or specific part of a big project), too. This helps if you scan through a large number of commits looking for something specific. You should always add some words about the change and reference issues from your issue tracker if there are any. GitHub and many other tools have special keywords to link and close issues automatically with commit messages.

Just think about what information could you need to understand the change in the future. Ask yourself why and what.

Push

After you did the commit, it’s just done on your local machine. If you want it to be available on GitHub etc you need to push it. You can push one ore more commits with git push origin <branchname>. Usually your default remote server is called ‘origin’. You can have multiple remotes like your own fork, the upstream project or other hosting providers aside from GitHub.

Documentation about working with remotes

Pull

If you need to get the latest changes from a remote you need to use Git pull.

Just a short examples. You are on the main branch locally and somebody else updated the main branch on GitHub. If you type git pull origin main, the latest changes from GitHub get pulled and possibly Git creates a merge commit if it needs to merge your local changes with the remote changes. Alternatively you can use git pull origin main --rebase to use rebase instead of merge. Read the rebase section to understand what happens there and what you need to take into consideration.

Understanding Change

My number one reason to use Git, aside from a backup, is that I love to look at previous commits to understand how a project developed. I even use it for stuff I did myself just a while back.

Git log

I often look at the history of the project or single files to understand when and why changes came in the project. For simple things git log and git diff <commit hash> are sufficient but most of the time I prefer a UI like GitHubs UI or Git clients like the one in VSCode.

Blame

Git Blame is a useful tool to understand changes to lines in a file. GitHub and other Git services and clients use blame to show short information about the latest commit for each line.

Tags & Releases

Tags are a simple way to mark specific snapshots and are typically used for versions & releases. GitHub offers additional features for releases where you store files and a description to tags.

More about tags and releases.

Workflows & Branches

Workflows in Git typically involve working with branches, pull requests and git commands to merge branches like git merge and git rebase. There are many ways to implement a development workflow with Git and you can find lots of examples and advice online.

The most common and simplest workflow often found on open source projects on GitHub etc is that you have a default branch (by default main). Features get developed in their own branches and merged to the default branches via pull requests. External contributors of open source projects create their own fork of the project and submit pull requests from their branches to the upstream projects default branch.

If you come from a central source control management system, like SVN, the main difference may be that branches are more heavily used in Git. With SVN typically everyone works on their features and bug fixes and commits them to a default branch “trunk”. With Git your typically work in branches and merge to the default branch once your change is done, reviewed and tested.

The official Git documentation has a chapter about Git branching workflows, too.

Pull requests (Merge requests)

An important way to integrate changes form a branch or fork back to the default branch or upstream project is a pull request. A pull request (or sometimes merge request) is a feature found on Git platforms which allows you to create request for you or project developers to review and merge your code. You can create pull requests for your feature branches or for merging your changes in a fork back to the original project. More about pull requests on GitHub

My workflow for contributing fixes in open source projects is like this:

Find project on GitHub and fork it into my account
Clone my fork to my PC
Create a new branch in my local repository. (I usually don’t touch the default branch in my work and merge changes from the upstream project if needed.)
Make changes and commit
Push new branch to my fork
Create pull request on GitHub. (GitHub usually even prints a URL to create it in your command line after pushing)

Rebase

I am not really an expert or heavy user of Git rebase. You find plenty useful information online about good uses of rebase. A rebase is really useful and powerful but it rewrites the history in Git. If you rewrite the history in your local repository and you want to push that to a remote it might get rejected if you try to rewrite the remotes history. You can solve this by using a force push which can be dangerous and destructive. Check out the Git documentation and help online to not loose your important data.

More about rewriting history

For me there are two use cases where I typically use rebase:

Updating a feature branch

If you work on a feature in a branch and your or someone else commits changes to the main branch you might need to update your feature branch. The feature branch is one commit ahead and one commit behind the main branch. On GitHub it looks like this:

I often use git rebase main in my feature branches. What Git basically does it takes your commits that are just in the feature branch sets them aside, pulls all commits from main in, and applies your commits again. Now your feature branch is no commits behind the main branch and you can work with the latest code from main.

Cleaning up commits

I commit very often and sometimes with stupid commit messages. An interactive rebase opens a text editor where you can edit commit messages, squash multiple commits to one with a good commit message and much more.

Closing words

This is just how I use Git. Some of this stuff may be bad or wrong. If you got ideas for this post let’s talk

I will update this post as I encounter more features, uses cases and tips. Subscribe to the newsletter to get updates.

Using Github Actions to archive Netlify Analytics data

Niklas Merz — Mon, 12 Oct 2020 19:00:00 +0000

Update November 2021: Netlify introduced a new V2 API and deprecated the V1 API. It's still not official but they announced the deprecation mentioning external users of this API. The action has been updated and released for V2.

I recently started to take a closer look at the traffic this blog gets. Since it runs on Netlify and I really like their solution I have been a happy user of Netlify Analytics for a while now. The big drawback for me is that the UI only shows the last month of analytics. I wanted to obseve trends for a longer so I set out to archive this data somehow.

My first Google Search for an official API to analytics data was not successful since Netlify currently does not offer API access via their official API. But Netlify staff suggested using the unofficial API like others have done. Raymond Camden and Jim Nielsen did a great job explaining how it works. So I took a look at my browsers dev tools and started working with that.

I don’t want to care about servers, databases and stuff for my blog that’s why it is on Github and Netlify. So I thought let’s try a scheduled Github Action for storing that data. So here we go the Github Action: https://github.com/marketplace/actions/netlify-analytics-collector with this workflow:

name: Test run

on:
  schedule:
    - cron: '59 23 * * *'
  workflow_dispatch:


jobs:
  export-run:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v1
    - uses: niklasmerz/netlify-analytics-collector@2.0.0
      with:
        netlify-token: ${{ secrets.NETLIFY_TOKEN }}
        netlify-site-id: ${{ secrets.NETLIFY_SITE }}
    - uses: actions/upload-artifact@v2
      with:
        name: exports
        path: '*.csv'
  sheet-upload:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v1
    - uses: niklasmerz/netlify-analytics-collector@2.0.0
      with:
        netlify-token: ${{ secrets.NETLIFY_TOKEN }}
        netlify-site-id: ${{ secrets.NETLIFY_SITE }}
        days: 0
        disable-header: true
    - uses: niklasmerz/csv-to-google-spreadsheet@master
      with:
        csv_path: pageviews.csv
        spreadsheet_id: ${{ secrets.google_spreadsheet_id }}
        worksheet: 0
        append_content: true
        google_service_account_email: ${{ secrets.google_service_account_email }}
        google_service_account_private_key: ${{ secrets.google_service_account_private_key }}
    - uses: niklasmerz/csv-to-google-spreadsheet@master
      with:
        csv_path: visitors.csv
        spreadsheet_id: ${{ secrets.google_spreadsheet_id }}
        worksheet: 1
        append_content: true
        google_service_account_email: ${{ secrets.google_service_account_email }}
        google_service_account_private_key: ${{ secrets.google_service_account_private_key }}
    - uses: niklasmerz/csv-to-google-spreadsheet@master
      with:
        csv_path: bandwidth.csv
        spreadsheet_id: ${{ secrets.google_spreadsheet_id }}
        worksheet: 2
        append_content: true
        google_service_account_email: ${{ secrets.google_service_account_email }}
        google_service_account_private_key: ${{ secrets.google_service_account_private_key }}

It looks complex at first but is pretty simple to use and customize. This workflow is set to run daily consists of two jobs. One archives the last 30 days of data in a ZIP file with CSV files into this Action run as an artifact. The other jobs updates a Google Sheet if you set it up. I like using a Google Sheet that gets updated daily to create some graphs and have an up-to date source of all data. Look at the secrets to replace and it should be ready to go. More info in the actions repo.

The second workflow creates a release each month with an ZIP file of your analytics data. This way you have a backup history of your data.

These workflows are just an example. You can customize the workflow to your needs via other actions in the workflow. For example I added an action to send a Slack notification every time the workflow fails so I can check it and not loose any data.

I really like the solution to have this action running right in the repo where my blog lives but it can run locally to.

Let’s hope the API does not change or we even get an official API to use some day.

One year at Apache

Niklas Merz — Sat, 02 May 2020 09:45:00 +0000

How I got to understand The Apache Way in the past year

I have been working with Apache software for many years now, but in the past year I really got into how projects at the Apache Software Foundation. I became part of two communities within this huge open source organization. My day job and many of my side projects involve working with Apache Cordova and just end of 2019 I became Cordova project management committee (PMC) member and committer. Although I don’t work with programmable logic controllers (PLCs) or just any industrial IoT I closely follow the Apache PLC4X project and sometimes contribute there. Getting in touch with these folks was pure coincidence and I am proud to be a committer for this project, too.

Apache Cordova

Cordova has been a part of developer toolchain for a long time. I have used it for many projects since I got in touch with it at my job like 4 years ago. Cordova is a tool to develop cross platform apps with web technologies.

Over the years I did quite a few contributions to Cordova for many areas of the project. Especially at my job we experienced some issues with a complex app and major iOS updates or WebKit bugs that needed work done in Cordova. I did my best to debug, fix and report those bugs to Cordova or Apple. I am really no iOS developer but I found my way through the Cordova iOS codebase many times to learn something new. Most of these issues resulted in contributions to Cordova and I am proud for a few PRs that really helped our app and the project from my point of view.

I worked on some Cordova plugins and really like the adventure of going into Cordova and figuring out how to do something. 2019 was an interesting (and sometimes exhausting) year for me with Cordova and iOS 13. I spent a lot of time figuring out how to fix stuff that was suddenly broken. I have been on the dev mailing list for some time but I was more active in issues and e-mails while working more on Cordova itself. I am really happy my employer sees the great value being part of the Cordova community brings us as users of the project. I now have the opportunity to work on Cordova more actively and become part of the decision making that affects the technical groundwork of our products.

In December 2019 I got elected as a PMC member and this was really an awesome surprise and a big achievement for me after this great time of working and interacting with these people who work with Cordova.

One of the main Apache ideas is “community over code” and in the past year I really started to understand how that works

The EU Apache Hackathon

Almost a year ago I fortunately found an invitation online for Apache communities by the European Commission. The European Union has a project called FOSSA 2 that focuses on the security of open source projects used by the EU and general public. They organized a series of hackathons in Brussels to bring communities together to work on their project and exchange ideas with the EUs institutions. For the second of three hackathons they were looking for Apache projects to take part for one weekend of working together. I already blogged about FOSSA 2 and the hackathon back then.

At the event I started to really understand how and why Apache works. The projects are so different but they share some important ideas and work together in interesting ways. Now I got a feeling how open source works best and I can distinguish different approaches all OSS projects take out there.

This event was very special to me. I was already enthusiastic about open source before but speaking to these experienced and cool people was a lot of fun. I learned a lot in these two days and some great new friendships were formed.

PLC4X

The hackathon was the first time I got to know the Industrial Internet of Things and PLCs. I met some of the cool people behind the project, worked with them for a bit and I am still around the project. I contribute when I have the time or think my different skill set can bring something new to the table. The project and the people working on it have such a different environment and background than my daily development life. I really like that we often learn something from each other.

I feel honoured the PLC4X PMC decided to give me committer privileges.

The work of a PMC member

After becoming PMC member I spent a lot of time reading Cordovas and the Apache Software Foundations documentation what a PMC member does. It really takes time to understand what you should and what you shouldn’t do as PMC member. The hardest part is to understand how and why those processes work. Take for example release voting. A release is in simple terms an “official Apache product” and the PMC has ensure it adheres to rules and standards. The PMC member who creates a release is called release manager and has to follow written instructions to create a release. At first I thought that this is overly complex and some stuff is unnecessary but after some time I understood that it is like it is to protect the contributors and foundation from legal issues and create a release that is legally fine.

I am very happy some PMCs took the time to explain everything to me and how they work on Cordova. I am regularly in touch with some people on the PMC team.

Another important task is voting on releases. When release managers create a release they will start an email thread on the mailing list for voting on the release. Everybody can check the release and vote on it, but only PMC members votes are binding. There is a document explaining what to check in a release. Voting basically means checking that the release only contains code intended for the release, all license and notice files exist, the release builds and tests are OK and it is properly tagged. Usually this is done very quickly and I try to vote on every release. The code itself should already be OK since all pull requests are reviewed and you do not need to test everything manually. A release needs three or more positive votes (+1) and more positive than negative votes (-1). With enough time passed and enough votes the release gets officially published and, in Cordovas case, pushed to NPM. The Apache Release guidelines are a long read.

Right before becoming a PMC member I did some work on a Cordova plugin (cordova-plugin-inappbrowser) that I need a lot for work. My contributions were merged but I was waiting for a release. As soon I was an official PMC member I started preparing the release by triaging issues and PRs. I signed up for doing the release and many helped me to get this release done. It felt good after so many years of working with Cordova to finally work on maintaining Cordova and release something people are waiting for.

The future

Nobody knows what the future brings but I feel I will be involved with Apache for while. Some time I ago I was under the impression that Apache is not cutting edge or kind of bureaucratic but now I know why it is like that and I strongly favour that governance model over some OSS projects that are under the strong influence of corporations and are ‘just open source for marketing’.

Experiencing an Hackathon organized by the European Union

Niklas Merz — Tue, 07 May 2019 16:00:00 +0000

I was lucky to be part of a very special hackathon with Apache projects in Brussels organized by the EU.

The second FOSS Hackathon organized by EU FOSSA in May hosted a event were a few Apache Software Foundation communities came together to meet in person. Some of the communities are older and mature projects like Apache Tomcat and others were pretty small, young or lesser know communities. The event was a chance for people who work together but never met in person to come together and for communities to get to know each other.

For more information about EU FOSSA and the context about the event read my post published at the same time. This post is more about my experience at the event and some opinionated thoughts on the event and efforts in open source. If you want to know more about what each community did at the event have a look at the work in progress homepage on Github. This page is still not quite done because the attendees are editing the content via Pull Requests.

I attended not as part of any of these projects but more as an open source enthusiast. I have been watching Apache communities from a little bit apart for some time. I quickly met other attendees completely new to all of these projects as well. I think it was a great idea to invite other people like students and me to get a different view on things, share knowledge and the values of the Apache community.

I have to start with a big thank you to the organizers. Even though my application was done last minute travelling to Brussels and attending turned out perfect for me. It was a pleasant experience to be there, the team was really nice and organized everything. The hotel and the venue at Silversquare Triomphe provided a great environment to talk to people and work on the projects.

After some opening words and an introduction about the EU and FOSSA the attendees started discussing and working on their projects. Attendees not affiliated directly to their projects joined the groups and participated in many ways.

It quickly came clear that we won’t follow the agenda that was used at the previous hackathon. I think the organizers did a great job by going out of the way letting the community decide what to do on their own pace, “The Apache Way”. I and probably others got the impression that working with open source communities does not work with a schedule and clear goal but more organically. We just did what we wanted to do and came together in the conference room when time felt right to present what we are doing. We really got the chance to make it our event and it turned out to be fun and productive.

I did not write any code myself at this weekend. I did write some text on the website that was created collaboratively and the posts on my blog. I usually prefer coding over meetings, documentation etc. but this time was different. Talking to everybody was really interesting and inspiring and I could work on my communication and writing skills. This really is the spirit of the event since sharing knowledge and the community is more important than producing code. The projects at the event are mostly written in Java and I am familiar with Java development to some degree, but personally I need to become familiar with a codebase for some time to write useful code. I did read a lot of code of the projects. I also built some projects myself from source code and got an introduction of interesting parts of the projects. Other participants did some impressive work done in code in a short period of time. I think talking to people, learning new technologies I never heard of before, helping with documentation and getting spirit of Apache was even more important to me at the event.

Experienced Apache committers suggested to have a key signing. Some attendees including me and members of the Commission never did such thing and it was another opportunity so see and talk about how open source works. Key signing and the Web of trust is an important practice for open source communities to ensure trust and security. So this fitted perfectly into the environment of the hackathon.

Heartbleed, one of the triggers to start EU FOSSA, was a topic of discussion to during the event. Some people remembered what they did at the time and how significant it was for some organization. This showed me how critical smaller software projects can be without anybody really noticing it. That’s why initiatives like FOSSA are so important. Much of the big buzz and attention in open source goes to big projects by large corporations or foundations. They often have lots of money and manage open source projects like commercial products inside a company. Smaller projects like OpenSSL that are used by millions of products look like old unsexy projects nobody notices. But security in these projects is critical for the security of millions of users. Apache has many important projects and they are all community led projects that have their own spirit and way of doing work.

Tomcat is a good example. It is very mature project which has been a trusted solution for many years. The development on Tomcat moves more slowly than on shiny new projects because the focus is on stability. The community is aware of its duties towards Tomcat users and produces reliable releases. That’s why decision making and talking about changes as community is more important than working on code. It really surprised me to hear that for some releases only three people are making a decision for releasing a such significant project. Everybody can build Tomcat from source very easily and vote for releases. Just a little bit more participation could improve the Tomcat releases and spot bugs.

Open source like it is executed at Apache is really different from open source done by company. I think activities like this one really help organizations like the EU to understand it and find ways to support it.

Another important thing on my point of view is to build awareness about open source. I think that’s another purpose of the hackathon to raise awareness inside the EU institutions what is all about. But for me as an open source developer it is also important that people in the open source community are involved in the conversation with public institutions like the EU what they can do to collaborate. EU FOSSA is becoming better at communicating what they are doing and what their mission is. Having people directly involved also helps to share it with a broader audience and getting more traction. Writing about it and posting on social media is just a small contribution.

FOSSA is just the start of the journey. The EU needs to figure out which way is the best or most effective way to improve security and quality of open source software by supporting its communities. Hackathons or other kind of events could be one way.

During one of roundtables we shared our thoughts that it is hard to measure success. Metrics like issues closed or security bugs fixed don’t really capture the value of the event. Building the Community is probably the most important thing. Being together in one room really has lasting effects. Because many different projects came together synergies were found that will show achievements in the future.

Another important aspect is capturing what happened at the event. An website for the event was created collaboratively using Github. I think this is a perfect idea because Github is already part of the developers tools and was really helpful to write down what we did together.

You can clearly see that FOSSA 2 is doing many different things to figure out how to approach challenges in open source. I think this is really the best way to do since nobody knows what’s best. I really hope that FOSSA 2 becomes a permanent budget item after its end and the preparatory action is successful. What FOSSA is doing is not only important for the EU institutions but also the public. Spending public money on open source solutions for public institutions really makes sense. The limited time and budget of FOSSA should be spent responsibly to ensure the measures found in that project become a permanent thing on the EUs agenda.

Open source has many aspects. I cannot capture everything I am thinking about to words, but I tried my best to document what happened at the hackathon. I am very positive about what has been done by FOSSA and hopefully will come in the future.

I hoped you enjoyed this post and I am almost certain I am going to write more about FOSSA and open source in general.

Participants for the 2nd edition were invited Apache members and interested people like me. I found out about the event on Twitter by @EU_DIGIT just a few days before. I applied through the online form on the website and got accepted. Attending the event was free and transport and accommodation was covered by the organizers.

More resources

The European Union and Open Source Software

Niklas Merz — Tue, 07 May 2019 13:18:00 +0000

The European Union (specifically the European Commission with the department DIGIT) is becoming more prominent about its efforts in taking part in the open source world.

With the organization of hackathons it is directly starting the conversation with open source communities. Already two of three events were held in Brussels with the most recent one with members of the Apache Software Foundation. The third one will take place at the start of October 2019 and they are looking for a community for this event. For context about what these hackathons are and why the EU is having a conversation with open source communities, it may be good to have a look what the initiative FOSSA by the European Commission is.

The Heartblead bug in OpenSSL in 2014 really started a bigger discussion about security in open source software. This bug showed that much of the internet and software in general is depending on small open source projects. The discussion about the importance of security in open source software was brought to politics back then. This basically became a trigger for FOSSA.

In fact there are two programms under the name FOSSA. Let’s start what FOSSA actually means: Free and Open Source Software Auditing. The first activities of FOSSA were exactly that: auditing open source software. It started in 2015 after the efforts of two members of the European Parliament Julia Reda and Max Andersson. The European Commission basically created a budget for creating a list of open source software used in European institutions and started auditing some of them. The first two open source projects reviewed by FOSSA were Apache HTTP server and the password manager Keepass.

In 2017 Members of the European Parliament achieved a new budget and got the project expanded. This became EU FOSSA 2 which added more activities like bug bounties, conferences and the three hackathons. FOSSA 2 is now offering money in form of bug bounties for researchers who report security issues in critical open source software used in EU institutions. You can find more about that on Julia Redas homepage.

FOSSA 2 is a preparatory action at the moment, which means it is kind of an experimental activity with a limited timeframe and budget. The European Commission approved a budget to find a way to work on this resort of open source. The members of the FOSSA 2 project work on goals and measures that could become a permanent budget item for the European Commission. There is some public information on the work done on the official page and you can expect more communication to come.

It is a good sign that the EU is strongly considering open source software for new applications used in European institutions. Initiatives like FOSSA show that the EU is trying out different ways to support open source. Members of the parliament, commission, DIGIT and more are supporters for that vision.

Open source now exists for some time but in 2019 there is still no clear idea how to do open source in a sustainable way. The activities of FOSSA show different approaches to support open source communities and produce better and more secure software.

If you are interested in open source you should keep an eye on FOSSA activities and their findings.

Resources

This talk from FOSDEM 2018 by Marek Przybyszewski project manager at EU DIGIT working on FOSSA gives a good overview about FOSSA 1 and the first parts of FOSSA 2
Hackathon page
FOSSA publications
@EU_DIGIT on Twitter
Julia Reda FOSSA
Julia Reda on FOSSA 2 Bug Bounties

Go Serverless: Part 2 Google Cloud Run

Niklas Merz — Thu, 11 Apr 2019 15:51:01 +0000

Running Serverless Functions in Containers

What are Cloud Run and Knative?

Google Cloud Run was recently announced at Google Cloud Next after a private alpha period and we are finally able to talk about it. Cloud run basically allows us to deploy stateless HTTP(S) endpoints via containers without managing servers or Kubernetes clusters. Now we have a serverless platform for our Docker containers with pricing and deployment as easy as most serverless platforms. Everything is managed like scaling the container, routing traffic, custom domains and HTTPS certificates.

Cloud Run is built on Knative an open-source Kubernetes based project which gives confidence that our code/containers are still highly portable.

Check out Googles Documentation for concepts behind Cloud Run and Knative.

Building the Docker Container for Go

Cloud Run needs Docker images in our projects registry to deploy services and their revisions.

Like part 1 we are using the project Gistdirect for trying Google Cloud Run. This function serves a simple redirect service.

Go Serverless: Part 1 Google Cloud Functions

Niklas Merz ・ Apr 12 '19

#go #serverless #googlecloud #googlecloudfunctions

NiklasMerz / gistdirect

Redirect to urls from a gist with a hostfile. Ready for serverless and Containers

gistdirect

Gistdirect is a simple URL redirection service / URL shortener.

Imagine bitly.com or goo.gl and your hosts file combined

Create a Github Gist or just any plain file hosted somewhere. Gistdirect will grap that file and redirect the alias to the full url.

Sample "host" file:

gh https://github.com

Test File

The URL to the file is provided as an environment variable: GIST_URL. Make sure to provide an URL that is a raw textfile. If you use a GitHub Gist click on the raw button before grabbing the URL like https://gist.githubusercontent.com/NiklasMerz/a9b5905f742b5863197a0af0465a39f6/raw/.

It is a simple Go function which is easy to run on any serverless platform. You can also run the binary somewhere and it will listen on port 8080. Executables are avaible from releases.

I got it running with Google Cloud Functions and Firebase hosting. You can deploy the function from "function.go" via the web…

View on GitHub

We need a Docker container to run our Go function. Gistdirect has a Dockerfile similar to this Dockerfile with modifications to use Go modules, because Gistdirect is structured to use modules:

FROM golang:alpine as builder

ENV PATH /go/bin:/usr/local/go/bin:$PATH
ENV GOPATH /go

RUN apk --no-cache add gcc g++ make ca-certificates

COPY . /home/gistdirect

RUN set -x \
    && cd /home/gistdirect/main \
    && go build \
    && mv main /usr/bin/gistdirect \
    && rm -rf /go \
    && echo "Build complete."

FROM alpine:latest

RUN apk add --no-cache \
    ca-certificates

COPY --from=builder /usr/bin/gistdirect /usr/bin/gistdirect

EXPOSE 8080
ENTRYPOINT ["gistdirect"]
CMD [""]

This Dockerfile builds the main.go file which uses the redirect handler in the main() function and makes it executable to the final Docker image we will run.

To prepare the Dockerfile for Cloud Run we need to have the gcloud CLI tool and must be signed in to a project with billing enabled (see part 1). If you followed through part 1 use the same project to reuse your domain. We can submit our project to Google Cloud Build which builds the image and puts it into the registry of our project.

gcloud builds submit --tag gcr.io/[PROJECTID]/gistdirect

Deploying to Cloud Run

Creating the service is really easy with the Google Cloud Console. Just go to Cloud Run, “Create new Service”, Select the image, set the environment variable and we are good to go.

When the service is ready we get a URL like this: gistdirect-khsgfjhsg-uc.a.run.app and that’s our function.

Get a nicer domain

If you followed through part 1 you probably already set up and verified a domain to this project. You can go to the Cloud Run main page and select “Manage custom domain”. There you can use your previously added domain, set another custom domain and map it to your service. Once HTTPS is ready and you did the CNAME change in your DNS, you are ready to call your container.

Go Serverless: Part 1 Google Cloud Functions

Niklas Merz — Sun, 07 Apr 2019 15:20:01 +0000

Running Serverless Functions with Golang

Go is the most popular language for cloud computing and serverless is becoming more popular for all kinds of workloads. Let´s have a look on how to run a simple Go function on the different cloud providers.

We will start with with Google Cloud Functions and Firebase. AWS Lambda is the most popular serverless platform today. Some providers resell AWS Lambda with nicer/easier tooling and different pricing. We will have a look at Zeit Now and probably “bare-metal” AWS Lambda in this series.

For testing the different providers we will use my tiny project Gistdirect which is a simple HTTP function that redirects to URLs saved in a Github Gist. This project is simple to build and deploy. The Gistdirect function waits to be called with a path to redirect to the given URL.

The beauty of Go (in functions)

I like to use Go for serverless because basically all platforms are expecting standard Go handler functions like this: func Handler(w http.ResponseWriter, r *http.Request). This means you can build your functions reusable for all kinds of handlers like the standard http package or routers like mux. I don’t like building stuff that only works in a particular (cloud) environment. Gistdirect runs standalone as well thanks to a simple main package.

Google Cloud Functions

Since January 2019 Google Cloud functions support Go and combined with Firebase hosting we can get Gistdirect running on its own domain pretty easily.

Obviously you need a Google account and sign up for Google Cloud (GCP). In the cloud console we create a new project. After creation we switch to the Cloud functions menu item and activate the API. After that we could click a button to create a new function, set the runtime to Go, set the environment variable to the Gist URL and paste or upload the code from function.go. But we will use the gcloud CLI tool which is more convenient for frequent updates and testing:

Setup gcloud CLI: Quick start
Select a project: gcloud config set project NAME
Get the Gistdirect project from Github: git clone https://github.com/niklasmerz/gistdirect && cd gistdirect
Have a look at function.go: The function Handler(..) contains the redirect code and should be called as entry point.
Deploy function for runtime Go 1.11 with the enviroment variable needed for Gistdirect and the entry function Handler CLI command: gcloud functions deploy gistdirect --entry-point Handler --trigger-http --runtime go111 --set-env-vars "GIST_URL=https://gist.github.com/NiklasMerz/a9b5905f742b5863197a0af0465a39f6/raw/"

If everything worked, we will get a result like this:

deploying function (may take a while - up to 2 minutes)...done.                                                                             
availableMemoryMb: 256
entryPoint: Handler
environmentVariables:
  GIST_URL: https://gist.github.com/NiklasMerz/a9b5905f742b5863197a0af0465a39f6/raw/
httpsTrigger:
  url: https://us-central1-PROJECT.cloudfunctions.net/gistdirect
labels:
  deployment-tool: cli-gcloud
name: projects/serverless-blog-236916/locations/us-central1/functions/gistdirect
runtime: go111

Now we can hit https://us-central1-PROJECT.cloudfunctions.net/gistdirect/repo in a browser and we will get redirected to the Gistdirect repo because the test hostfile we set to the environment variable, has this redirect rule.

That’s it our serverless functions is running on GCP!

Bonus get a nicer URL

The whole point of gistdirect to redirect from short URL to longer one. That’s why we want to point our own domain to the function. We can do that with Firebase.

Go to the Firebase console
Go to “Hosting” in the menu
Click “Get started” and install the Firebase CLI tool via NPM
Just before we go on to setup our local project we can add our custom domain or sudomain to firebase. We follow the instructions to verify and add the domain. I had some trouble getting the DNS records right the first time, but it should work pretty straightforward.
cd into the Gistdirect project directory
Use firebase login only and not firebase init
We can switch to our project with firebase use PROJECTID
Have a look at firebase.json

{
  "hosting": {
    "public": "public",
    "ignore": [
      "firebase.json",
      "**/.*",
      " **/node_modules/**"
    ],
    "rewrites": [
        {
            "source": "*",
            "function": "gistdirect"
        }
    ]
  }
}

We define here that all our requests to our Firebase domain (https://PROJECTID.firebaseapp.com/) or custom domain should get redirected to our Cloud Function. We just need to set the name of our function and call firebase deploy to set it up. Because Firebase does not support Go functions right now we need to take the longer route and set up our functions with GCP.
You can find more information here.

Attention : The function and the Firebase project must live in the same region. My projects are set to us-central1. But you could try other regions.

Now we are ready to go to use serverless functions with our domain

Learning

Niklas Merz — Mon, 04 Mar 2019 07:00:01 +0000

Learning new stuff in tech

A blog post by @jessfraz and this tweet by @erikstmartin inspired me to think about how I learn and work on tech.

Erik St. Martin

@erikstmartin

This is exactly the way I learn. Go straight for thing I want to learn till I run into a term or concept I don’t know, research that, if I run into things I don’t know researching that, I stop and research that too, then unwind the stack. Then I keep repeating this process. twitter.com/jessfraz/statu…

16:02 PM - 24 Mar 2019

jessie frazelle 👩🏼‍🚀 @jessfraz
I wrote a blog post about digging into RISC-V and how I learn new things :) https://t.co/DQu0UmriiC

25207

Basically this is how I do it. You could call this the top-down approach to learning. I go straight to the thing I want to do and dig deeper down to learn what I need. This means learning things deeper down the stack or more top-level than my starting point. Basically this is learning by doing, I set myself a goal to achieve something and learn what I need to know along the way. I teached myself most of the stuff I know like Angular that way. I wanted to build an app and picked up everything I needed to know about how you do certain things in Angular while doing it. I learned the basics of Go while reading and going through the book “Writing and interpreter in Go”. (Check it out: https://interpreterbook.com/). Learning this way is most fun and effective to me.

The hard parts

This approach to learning worked many times for me, but it comes with some challenges.

Losing interest

The hardest part is to keep on moving down the right path. It´s easy to get lost in a topic or lose track. I abandoned many projects or topics because it looked like the effort to learn everything is not worth it. Many personal projects I started didn´t come very far because I just did not want to spent the time to learn some complex stuff and don´t enjoy in my own time. I think this is normal for everybody (hello imposter syndrome).

If you are forced to learn something maybe because it´s your job to solve this problem, you don´t really have the problem of losing interest but it can be hard to keep motivation high.

You can also lose track easily if you learn like this. You need to research something to understand the thing you started with and you go down the rabbit hole of information and end up learning completely different things. You may have learned good stuff, but you made little progress on the thing you started with.

Not suitable for every environment

This type of learning doesn´t really fit traditional education. In school or university you may have to learn something you are not really interested in. For me it´s very hard to make progress learning that way.

Ben Lesh 🧢🏋️‍♂️💻🎨

@benlesh

I think it's also important to understand TCP. So I have all beginners send packets to servers by touching stripped network wires to a 9 volt battery *really fast* to send their requests manually. twitter.com/markdalgleish/…

05:55 AM - 26 Mar 2019

Mark Dalgleish @markdalgleish
It's a mistake to learn a framework without first learning the fundamentals. That's why, when dealing with beginners, I always start off by making them build their own CPU, programming language and operating system.

1091

In my experience traditional education often is bottom-up type learning which can be very hard for me to handle. I need the bigger picture and practical use of something to even start learning.

If you choose to learn on your own and don´t get a degree it can become harder to get the interest of potential employers. You may know a broad spectrum but it can be hard to show what you are capable off without a fancy degree or major project or company name at the start of your career.

Learning is all about solving problems to me. Learn what you need to do a job and pick up everything interesting along the way.