DEV Community: axem

Tool Containerization Best Practices For Embedded Software Development

janosmurai — Tue, 24 Sep 2024 12:41:02 +0000

Containerization of development tools in the embedded software industry is still emerging but gaining traction rapidly. More teams are beginning to use containers to standardize and streamline their development environments, improving reproducibility and collaboration. However, the industry still faces challenges in fully integrating containerized workflows.

In this blog post, we will explore the process of containerizing a development environment for embedded software development, using the STM32F1 target as a basic example. Instead of relying on the vendor's integrated IDE, we will assemble the necessary development tools ourselves. This approach allows us to encapsulate the tools and their dependencies in a container image for improved consistency and portability.

We've already discussed the advantages of containerization, but now let's delve into the practical steps to create a containerized development environment. Our example toolkit includes:

Build system: GNU Make
Toolchain: Arm GNU Toolchain (the official compiler toolchain from Arm)
Debugger and deployer: stlink-org (an open-source implementation of ST's STLINK Tools)
Test environment: CppUTest (a C/C++ based test framework)

A container image is essentially a file with executable code that can create a container on a computing system. The main objective of this tutorial is to use containerization to isolate the development tools and their dependencies. To achieve this, we will leverage Docker as our container engine, so we will need to craft a Dockerfile. (See the official documentation to install Docker on your system.)

Before creating the container image, the initial decision revolves around selecting an appropriate base image. For development containers, a Debian-based image is generally sufficient. This base image is relatively compact, stable, and comes with many of the dependencies pre-installed.

The next crucial step is to gather all the necessary tools and their dependencies. Here's a breakdown of how to obtain each component:

Make and stlink-org: You can easily install these tools from the Debian repository using the apt package manager.
Gnu-arm-none-eabi: To enable debugging for the toolchain, you must install the gdb package via the apt package manager. Additionally, you'll need the wget and bzip2 packages to download and install the toolchain's binary, which is available from Arm's official file server.
CppUTest: CppUTest is installed from source, and you can obtain the source files from the project's GitHub repository using git. To build the source files successfully, you'll need the following packages: g++, cmake, libtool, and autoconf.

To ensure that you've gathered all the necessary dependencies, a helpful tip is to attempt to install each tool to a container created from the chosen base image. For example, if you're using a Debian base image, you can run the following command:
docker run \-it debian:bullseye /bin/bash
This should open up a shell in a Debian based container. In this shell try to run the commands you’d like to add to your dockerfile. This approach is much more efficient than troubleshooting the Dockerfile with rebuilding every iteration.

Creating a monocontainer

Now that we know how to obtain our development tools, it's time to dive into containerization. For this example, We've prepared a Dockerfile to encapsulate the toolset and dependencies within a monocontainer.

# Use the Debian base image as our starting point.
FROM debian:bullseye

# Install the required packages.
RUN apt update -y && \
    apt -y install g++=4:10.2.1-1 \
                   cmake=3.18.4-2+deb11u1 \
                   libtool=2.4.6-15 \
                   autoconf=2.69-14 \
                   git=1:2.30.2-1+deb11u2 \
                   gdb=10.1-1.7 \
                   wget=1.21-1+deb11u1 \
                   bzip2=1.0.8-4 \
                   make=4.3-4.1 \
                   stlink-tools=1.6.1+ds-3

# Clone and install CppUTest
RUN git clone https://github.com/cpputest/cpputest
WORKDIR /cpputest
RUN autoreconf . -i && \
    ./configure && \
    make tdd
ENV CPPUTEST_HOME=/cpputest

# Set the working directory for your project
WORKDIR /work

# Download and set up the GNU Arm toolchain
RUN wget https://developer.arm.com/-/media/Files/downloads/gnu-rm/10.3-2021.10/gcc-arm-none-eabi-10.3-2021.10-x86_64-linux.tar.bz2 && \
    tar -xjf gcc-arm-none-eabi-10.3-2021.10-x86_64-linux.tar.bz2 && \
    rm gcc-arm-none-eabi-10.3-2021.10-x86_64-linux.tar.bz2 && \
    mv gcc-arm-none-eabi-10.3-2021.10 /opt/gcc-arm

ENV PATH="/opt/gcc-arm/bin:${PATH}"

Here's a breakdown of the Dockerfile:

We start with a Debian base image.
We update the package list and install the required software dependencies directly from the Debian repository using the apt package manager.
We clone the CppUTest repository from GitHub, build it in the /cpputest directory, and set the CPPUTEST_HOME environment variable to the installation path.
We configure the working directory as /work.
The GNU Arm toolchain is downloaded, unzipped, and placed in the /opt/gcc-arm directory. We also update the PATH environment variable to ensure the toolchain is accessible system-wide.

To create the container image, run the following command:

docker build \-t dev\_env\_image .

On a PC Intel i7-8550U (8) @ 4.000GHz, this process took approximately 5 minutes and 22 seconds.

With the containerized development environment ready, you can experiment with it using a simple demo repository. Clone the repository with:

git clone https://github.com/axem-solutions/example

Navigate to the root of the example directory.

cd example

Run the container.

docker run --privileged --rm -it -v "$(pwd)":/work dev_env_image:latest bash

Here’s a breakdown of the command:

—privileged: Give access to the USB devices. This is not a secure solution, only used for the sake of simplicity.
—rm: Remove the container after stopping it.
-v “$(pwd)":/work: Mount the current working directory to /work.
dev_env_image:latest: Use the image we have just built.
bash: Run bash inside the container.

Inside the container, you can use the tools as you would natively.

Build the project:

make

Run the test cases:

cd /work/app/test
make

Deploy to the target:

cd /work/build
st-flash write tutorial.bin 0x8000000

Another option is to start the tasks alongside the container. When the task finishes the container stops.

Build the project:

docker run \--rm \-v "$(pwd)":/work dev\_env\_image:latest make

Run the test cases:

docker run \--rm \-v "$(pwd)":/work dev\_env\_image:latest /bin/sh \-c "cd app/test; make"

Deploy to the target:

docker run \--privileged \--rm \-v "$(pwd)":/work dev\_env\_image:latest /bin/sh \-c "cd build; st-flash write tutorial.bin 0x8000000"

Problems with the monocontainer approach

While placing each tool in a single container may seem like a straightforward approach, it can lead to several challenges reminiscent of issues faced with traditional Integrated Development Environments (IDEs). In this chapter, we will discuss the downsides of the monocontainer approach and why it may not always be the ideal solution.

Scalability and Maintainability: The monocontainer approach may work well for simple projects, as demonstrated in our example. However, in real-world scenarios, software development often involves a multitude of tools. As more tools are added, the Dockerfiles can become excessively lengthy, potentially stretching to hundreds of lines. This can lead to a lack of scalability and make the environment hard to maintain.

Difficult bug localization: The containerized tools may share resources and have interdependencies. Making modifications to the image can inadvertently result in complex and challenging-to-detect malfunctions. This can make the process of pinpointing the root cause of a problem even more difficult.

Time-Consuming Modifications: Docker builds are structured with a series of ordered build instructions defined by the Dockerfile. Each instruction roughly translates to an image layer. When building an image, Docker attempts to reuse layers from previous builds. However, if a layer has changed since the last build, that layer and all subsequent layers must be rebuilt. Meaning, the lower layer gets modified, the more time it takes for the image to build. Consequently, maintaining large Dockerfiles can quickly become a time-consuming task.

Image Variants: Throughout the development lifecycle, there may be a need for different image variants. For example, when setting up a Continuous Integration/Continuous Deployment (CI/CD) server, some tools may become unnecessary (e.g., the debugger), while new tools (e.g., the CI/CD service) are required to run CI/CD pipelines. Alternatively, changes in the project may necessitate the creation of a new development environment while retaining the old one for compatibility reasons.

As mentioned earlier, Docker images are composed of layers. These layers can be shared among images until the point where the first difference occurs. This means that layers of image variants, starting from the first one that differs from the original, consume additional space on the host storage.

Working on Multiple Projects: In scenarios where developers are simultaneously working on several projects, each project may have its own dedicated image. However, some tools used across these projects may be the same. Just like with image variants, Dockerfiles should be thoughtfully constructed to minimize storage consumption. Once the first differing layer is encountered, subsequent layers are not shared, even if they are identical.

Size constraints: Huge Dockerfiles result in huge container images, which can easily end up hundreds of gigabytes. Developers working on multiple projects can quickly run out of available local storage. The size of the container images can also become problematic when they get pulled over metered connections, especially on CI/CD providers where this can happen quite often.

In the following chapters, we'll explore potential solutions to these challenges and how to leverage dedicated tool images for a more efficient and scalable development environment.

Solution: Dedicated Tool Images

Scalability and Maintenance: Placing each separate tool into its own container resolves the scalability and maintenance issues. With separate containers for each individual tool, the development environment becomes highly modular, making it easy to add or remove tools as needed without affecting the entire setup. This modularity simplifies the development environment's management.

Troubleshooting: Troubleshooting is greatly simplified with dedicated tool images. When an issue arises, you only need to inspect the Dockerfile of the specific malfunctioning tool. This pinpointed approach reduces the complexity of debugging and minimizes the potential for conflicts between tools.

Efficient Build Process: By using separate tool images, you significantly reduce the number of image layers. This means that rebuilding images, even if lower layers were modified, is much faster. The build process becomes more efficient, making it easier to maintain large Dockerfiles and keep development environments up-to-date.

Adaptation and Variants: Changing tools or creating new image variants becomes straightforward. You can swap out a tool image for a new one without affecting other tools. This flexibility allows for the quick creation of new development environments without multiplying storage consumption.

Storage Efficiency: Dedicated tool images optimize storage usage. Only the layers that differ from the original image occupy extra space on the host. This minimizes the storage footprint and is especially useful when working on multiple projects or using variants of the same image.

Overall, this method of utilizing dedicated tool images effectively resolves the limitations and challenges associated with the monocontainer approach. It provides a modular, efficient, and scalable way to manage development environments, making it easier to adapt to changing requirements while simplifying troubleshooting and minimizing storage consumption.

Separating the Tools for the Example Project

In this chapter, we'll revisit the example project and explore how to separate the tools effectively. Understanding the communication between the tools is crucial to decide how they can be distributed into individual containers.

CppUTest

The CppUTest and the GNU Arm toolchain don't rely on each other. They operate on source files independently, making them suitable for separation. The Make tool directly calls CppUTest, so it must be present in the same image. Make is installed via the cmake package. Below is the Dockerfile for CppUTest:

FROM debian:bullseye

# Install the required packages.
RUN apt update -y && \
    apt -y install g++=4:10.2.1-1 \
                   cmake=3.18.4-2+deb11u1 \
                   libtool=2.4.6-15 \
                   autoconf=2.69-14 \
                   git=1:2.30.2-1+deb11u2

RUN git clone https://github.com/cpputest/cpputest

WORKDIR /cpputest

RUN autoreconf . -i && \
    ./configure && \
    make tdd

ENV CPPUTEST_HOME=/cpputest

WORKDIR /work

Build command:

docker build \-t axemsolutions/cpputest .

Build time: 2m 24s

GNU Arm Toolchain

The Make tool directly calls the GNU Arm toolchain, so it must reside in the same image. Here's the Dockerfile:

FROM debian:bullseye

# Install the required packages.
RUN apt update -y && \
    apt -y install gdb=10.1-1.7 \
                   wget=1.21-1+deb11u1 \
                   bzip2=1.0.8-4 \
                   make=4.3-4.1

# Installing the gnu-arm-none-eabi toolchain
WORKDIR /work

RUN wget https://developer.arm.com/-/media/Files/downloads/gnu-rm/10.3-2021.10/gcc-arm-none-eabi-10.3-2021.10-x86_64-linux.tar.bz2 && \
    tar -xjf gcc-arm-none-eabi-10.3-2021.10-x86_64-linux.tar.bz2 && \
    rm gcc-arm-none-eabi-10.3-2021.10-x86_64-linux.tar.bz2 && \
    mv gcc-arm-none-eabi-10.3-2021.10 /opt/gcc-arm

ENV PATH="/opt/gcc-arm/bin:${PATH}"

Build command:

docker build \-t axemsolutions/make\_gnu-arm .

Build time: 2m 14s

Stlink-org

Stlink-org communicates with the GNU Arm toolchain (GDB client) over TCP/IP. This network connection can be established between separated containers, allowing Stlink-org to be placed in a separate container. Here's the Dockerfile for Stlink-org:

FROM debian:bullseye

# Install the required packages.
RUN apt update -y && \
    apt -y install stlink-tools=1.6.1+ds-3

WORKDIR /work

Build command:

docker build \-t axemsolutions/stlink-org .

Build time: 7s

The separated tool images significantly reduce build times, saving valuable time when frequent rebuilds are necessary.

While the usage of these images is not detailed in this blog post, you can find comprehensive instructions in our Tutorial. This tutorial covers setting up and compiling a project for the NUCLEO-F103RB, obtaining tool images from axem’s Docker Hub, and flashing and debugging the application on the target using VS Code and its Dev Containers extension.

The DEM Solution

While separating tools into their individual containers offers numerous advantages, it can lead to a proliferation of container images, which can quickly become unmanageable. It becomes challenging to keep track of which images are needed for the different projects. To address this issue, we introduced a tool called DEM (Development Environment Manager). DEM enables the creation of development environments from tool images and allows them to be assigned to specific projects.

To learn more about DEM and its capabilities, please visit the project's GitHub repository and explore its detailed documentation.

The axem Open Tool Dockerfiles repo

The approach of creating dedicated containers not only simplifies the management of development environments but also encourages the reuse of tool images. At axem, our team is dedicated to developing tool images that can serve as fundamental building blocks for constructing new development environments. That's why we've made our repository of Dockerfiles open to the community.

Open Tool Dockerfiles (OTD) is an open and collaborative repo for storing and sharing Dockerfiles, specifically tailored for embedded software development tools. Our mission is to create a community-driven repository where developers can freely contribute, access, and utilize containerized build systems, debuggers, toolchains and more. We welcome contributions from everyone, so feel free to add new Dockerfiles to the repository.

The generated tool images will be easily accessible from our free and open-source registry called axem Open Registry (aOR). This registry is designed to make the process of building development environments even more efficient, giving you the tools you need to succeed in your projects.

Conclusion: A Containerized Future for Embedded Software Development

In the realm of embedded software development, adapting to the growing complexities of projects is a key challenge. Monocontainers, once appealing, can become unwieldy and inefficient. Troubleshooting, adapting to new requirements, and working on multiple projects all pose challenges.

The solution lies in dedicated tool images. By separating tools into isolated containers, we address scalability, maintenance, and troubleshooting issues. The Development Environment Manager (DEM) streamlines environment management, while the axem Open Registry (aOR) offers a central repository for tool images.

This transition from monocontainers to modular tool images, coupled with DEM and aOR, marks a significant leap in efficient embedded software development. It allows for increased agility, time savings, and seamless project work. Embracing containerization is the future, enabling developers to excel in the ever-evolving field of embedded software.

STM32 Development Without an IDE

janosmurai — Fri, 19 Jul 2024 08:19:10 +0000

In this blog post, we'll showcase two methods for installing the essential tools to develop software for an STM32 target without depending on an Integrated Development Environment (IDE).

We'll begin by examining the process of collecting and installing the necessary tools directly onto your host PC.
Next, we'll explore the creation of a containerized solution using the same toolset, leveraging DEM, our open-source Development Environment Manager.

Disclaimer:
This post is primarily tailored for x86_64 Linux users. However, Windows users can also follow along using Windows Subsystem for Linux (WSL).

Development Environment vs IDE

When starting development on a new target, the typical approach is to download the vendor-specific IDE, which comes with all the necessary tools preinstalled. While this method offers a quick start, it lacks flexibility. The IDE is a fixed environment, making it challenging to modify or update integrated tools later on.

Alternatively, opting for a Development Environment (Dev Env) allows us to assemble the tools we need individually, providing greater control over the development process. Unlike an IDE, a Dev Env consists of standalone tools tailored to our requirements.

One significant advantage of using a Dev Env is the freedom to choose your preferred editor for coding. This flexibility empowers developers to work with the tools and workflows that best suit their needs.

DEM

We'll be utilizing DEM to create our containerized Dev Env. DEM, which stands for Development Environment Manager, facilitates the creation and management of Dev Envs based on a containerized toolset. When working without an IDE, DEM serves as the glue between individual tools, enabling their assignment to the development project seamlessly.

If you're interested in delving deeper into the advantages of containerization, you can explore our article on Maximizing efficiency.

The toolset

To establish a functional Dev Env, we'll choose the following tools:

Build system: make
Toolchain: gnu-arm-none-eabi
Debugger: stlink-org

Dev Env creation - Two ways

I. Installing on the native host

The build system

The make build system is typically available by default on most major Linux distributions, requiring no additional actions.

The toolchain

Visit the official ARM website to download the GNU ARM toolchain.
Navigate to the appropriate section for your host's compatibility.
Download the archived file and extract it into the /opt/gnu directory using the command: tar -xf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz -C /opt/gnu/
Add the path of the toolchain binaries to your PATH environment variable to enable systemwide usage.
Verify the installation by running: arm-none-eabi-gcc --version

The debugger

The stlink-org is the open-source version of the STMicroelectronics STLINK Tools.

Follow the installation instructions outlined in the project's README file.
Verify the installation by running: st-info –version

II. Simple, containerized installation

To set up a containerized Dev Env, you'll need the following tools:

The Docker container engine to run containers.
The DEM to manage the Development Environment.
Python 3.10 or higher to run DEM.

Refer to the installation section of DEM’s README and follow the instructions.

Confirm the successful installation of DEM by running:

dem –version

Now, proceed with creating a Dev Env named "stm32" (or any preferred name) using the command:

dem create stm32

DEM will prompt you to select the desired tools for inclusion in the Dev Env. Choose from the available tool images:

axemsolutions/make_gnu-arm:13.2
axemsolutions/stlink-org:1.8.0

Selecting tool images to create the Development Environment

Finally, install the newly created Dev Env with:

dem install stm32

Containerized vs Native Dev Env

	Containerized Dev Env	Native Dev Env
Installation	A simple `dem install` command. The tool container images can be used immediately.	Manually, every time.
Scalability	The toolset can be modified by simply adding/removing tools with DEM.	Modifying the toolset involves many manual installation/uninstallation steps.
Interference	The tools live in their own isolated environment.	Tools can affect the behaviour of other tools and the host system.
Reproducibility	Tools are guaranteed to always work the same way in every host environment.	Differences in the OS and library versions between two dev PCs can result in different outputs of the same tools.

Simple reproducibility with DEM

A Dev Env can be assigned to a project with the following command:

dem assign {dev_env_name} {project_path}

This command saves the selected Development Environment descriptor to the project's root directory. Consequently, the descriptor can be added to the version control system. Employing this technique guarantees that all project contributors utilize the exact same Development Environment throughout the project's lifecycle.

Creating and Exporting Project-Specific Configurations in VS Code

Tamas — Fri, 20 Oct 2023 14:42:34 +0000

Based on the 2023 Stackoverflow survey Visual Studio Code remains the preferred IDE across all developers by 73.71%.
VS Code is renowned for being a lightweight yet powerful code editor, but its true potential shines when paired with the right extensions.
In this article, we'll explore how to boost your VS Code experience, particularly when you join a new project.

Although we emphasize the developers’ freedom to have their own toolset of choice, in this article we focus on the advantages of the unified development environments. It can be valuable when the team members use the same extensions with common configurations. This method can shorten the onboarding process, provide standardized debug configurations and code formatting settings, and can make pair programming to be a smoother activity.

The good news is that VS Code makes it easy to achieve this stable setup. You can export your installed plugin list to a file and effortlessly replicate it on another device.

Getting Started

Before we dive into the magic of sharing configurations, make sure you have VS Code installed and a few key extensions ready. These could include code linters, debugging tools, version control integrations, and language-specific extensions. You can explore and install these extensions directly from the VS Code marketplace.

Export your extensions with a simple command

To display the list of all the extensions with their versions currently installed in your VS Code environment, open a terminal window and execute the following command:

code --list-extensions --show-versions

Then run the following to save a .list file containing only the names of your installed extensions:

code --list-extensions > your-extensions.list

If you would like to export your extensions with versions, use the following command:

code --list-extensions --show-versions > your-extenstions.list

This action will save a .list file containing the names and versions of your installed extensions to your computer.

Example: Exported extension file with versions

Sharing your VS Code extensions

With your extensions exported, you're now ready to share it with your teammates or replicate it on another device.
Run the following command:

cat your-extensions.list | xargs -L 1 code --install-extension

By this command, VS Code automatically installs all extensions from your your-extensions.list file. The operational process of this command is as follows:
The ‘cat’ command will list your-extensions.list sequentially, meanwhile subsequently, the ‘xargs’ captures the current plugin name and triggers ‘code –install-extension’ command with that argument, and continues this process until it reaches the end of the .list file.

As a plus, using the command line to install VS Code extensions from a file offers a practical advantage. It seamlessly integrates into automated installer scripts and enables you to experiment with your friend's or coworker's configurations, enhancing your overall user experience. Importantly, when you install plugins from an external file, your existing plugins remain unaffected. You can easily update your plugins using the following command:

cat your-extensions.list | xargs -L 1 code --force --install-extension

Exporting your VS Code configuration

The settings.json file is a configuration file in VS Code that allows you to customize the behavior of the editor and its extensions.

On Windows, the settings.json file is typically located in %APPDATA%\Code\User\settings.json.
On macOS, it's in ~/Library/Application Support/Code/User/settings.json.
On Linux, you can find it at ~/.config/Code/User/settings.json.

The file is written in JSON (JavaScript Object Notation) format. It consists of key-value pairs that configure various aspects of VS Code.

Common Customizations:

You can customize settings related to the editor's behavior, such as indentation, word wrap, and font size.
You can configure settings for specific programming languages or file types.
You can set up preferences for extensions you've installed, including their keybindings and behavior.

Installing config for a different VS Code

On the recipient's end, they can import this settings.json configuration file by copying and replacing the original one or by merging the two files together.

Warning: Always make backups of your settings.json file before making significant changes, and refer to the official VS Code documentation for more details on configuration options and best practices.

Conclusion

Efficiency and consistency are the pillars of a productive coding environment, especially in a collaborative setting. With VS Code's ability to export and import extension configurations, you can ensure that everyone on your team has access to the same tools and settings, saving valuable time and streamlining your workflow.

So go ahead, empower your coding journey with this time-saving feature, and make the most out of the extensive VS Code extension library.

Calling All Python Enthusiasts: Join Us in Contributing to Our Open-Source Project!

Eszter Vezden — Fri, 20 Oct 2023 13:59:41 +0000

Are you passionate about Python and open-source development? We're thrilled to invite you to contribute to our Python open-source project!

Hacktoberfest 2023 is officially underway, and our open-source tool, DEM is part of this fantastic event. 🚀

The DEM is a command line tool that provides an easy, reproducible, and scalable way to set up Development Environments for embedded software development.

Learn more about DEM HERE.
Check out our GitHub repo.

Why Contribute?

Enhance your skills in Python
Learn more about containerization (especially Docker)
Learn about development platforms
Shape the future of embedded development tooling

Tech Stack
🐍 Python
🐳 Docker

Are you interested?
You can find our open issues HERE.

Don't forget to join our XM Community on Discord .

Python contributors wanted

janosmurai — Tue, 10 Oct 2023 12:23:24 +0000

Hacktoberfest 2023 is officially underway, and our open-source tool, DEM is part of this fantastic event. 🚀

The DEM is a command line tool that provides an easy, reproducible, and scalable way to set up Development Environments for embedded software development.
Find out more at our GitHub repo!

Are you looking for issues to implement completely new features? You are at the right place!

Why Contribute?

Enhance your skills in Python
Learn more about containerization (especially Docker)
Learn about development platforms
Shape the future of embedded development tooling

Tech Stack

🐍 Python
🐳 Docker

Are you interested?

Join our Discord or send me a DM!

Preptember - How to setup your project for Hacktoberfest

janosmurai — Tue, 10 Oct 2023 09:02:48 +0000

Every year in October open-source enthusiasts from all over the world gather together virtually to celebrate the community, by contributing to software that is freely available to use, redistribute, and modify. Many of us rely on open-source projects in our everyday lives, which are maintained by passionate people, but what do they get in return?

By Joining Hacktoberfest, contributors can learn new technologies, showcase their knowledge, build their professional network, and give back to the community.
Hacktoberfest is DigitalOcean’s annual event that encourages people to contribute to open source throughout October. The core values are:

Everyone is welcome
Quantity is fun, quality is key
Short-term action, long-term impact

DEM’s story

Development Environments created for implementing embedded software often include multiple different software tools. A great way to keep the toolset under control for the whole project lifecycle is to apply containerization techniques. Although containerization has many benefits, working on multiple projects can easily lead to several different container images.
At axem, we also reached a point where it became quite difficult to keep track of the tool images required for the projects. Hence we created a script that could be used to assign the images to the projects.
As we saw that containerization started to become quite popular in the embedded field, we decided to open up the script for the community. Our intention was to give back to those who helped us and countless others along the way. This was the beginning of the DEM.

We believe in the positive impact of having an outsider’s opinion, but our project is quite young and in this phase, it is hard to encourage people to collaborate. So we decided that we would like to nominate DEM as a project open for contribution in Hacktoberfest 2023.
Since opening the source, several new releases have been created from the DEM project, but all the implementations were done by the axem team, based on our inner processes. To be able to guide our new contributors properly, we had to make the DEM contributor-friendly.

Preparing the DEM GitHub repository

Nominating a project for Hacktoberfest doesn’t have any requirements, except maybe using the “hacktoberfest” topic and label in the repo. However, t*here are some best practices to make your issues more appealing* to potential contributors. Here are our two cents:

Create a Contribution Guideline that is easy to follow
Create a welcoming Code of Conduct
Choose some issues that look interesting, have a well-defined scope and clear description, and apply the “hacktoberfest” label to them
Create issue and PR templates for maintainability
In the README, highlight that the project is participating in the Hacktoberfest

If you just started your own project, the Open Source Guide is an excellent content on how to launch and grow an open-source project.

In the next chapters, I provide more details for each point mentioned above. As the DEM is hosted on GitHub, the descriptions will be specific to the GitHub platform.

Creating a Contribution Guideline

A Contribution Guideline has several essential purposes like maintaining consistency, clarifying expectations, and ensuring quality control.
First, we need to consider the specific goals of our project and decide on the types of contributions we are looking for.

In DEM we expect two kinds of contributions:

Creating a new issue: Here a contributor can file any feedback, bug report, or feature idea.
Submitting a PR: The implementation of a specific issue. Both code and no-code modifications.

After we defined the specific goals we had to decide on the format of the guideline. The most common format is a Markdown (.md) file named ‘CONTRIBUTING.md’.
Before creating this file we should be aware of the concept of special treatment for certain files within a GitHub repository. Some files are treated differently in a GitHub repository if they match a naming convention and are stored in either the root, the docs, or the .github directory. An example is the 'CONTRIBUTING.md' file. This way GitHub will recognize the file as a contribution guideline. To create the file you can follow GitHub’s step-by-step guide.

When filling in the details, for better understanding we broke down the guidelines into two main sections. After a short introduction, we described how to create and work on an issue in the DEM project. In the second part, we explained how one can start to work with the source files and test their modifications.

Creating a Code of Conduct

By accepting outside contributors, the community grows with the new members. To establish a set of clear guidelines and expectations for behavior and interactions among community members we created a Code of Conduct. Our goal is to make sure that every newcomer feels welcomed and appreciated. Maintaining a safe space can be a daunting task for project maintainers. Everyone has their own value system, so establishing the ground rules can reduce the occurrence and intensity of conflicts.
Writing a Code of Conduct from scratch is a challenging task. Fortunately, GitHub provides a well-written template that project maintainers can customize for their use case.
The Code of Conduct can be stored as another special file of GitHub, called the 'CODE_OF_CONDUCT.md'. Again, this file needs to be placed in either the root, the docs, or the .github directory, so GitHub will automatically recognize it and load it to the Code of Conduct label of the About section.

You can find a step-by-step guide to create a CoC here.

Preparing issues for open-source contribution

In Hacktoberfest, new contributors often embark on projects with which they may have little familiarity. Expecting them to understand the whole architecture, especially in the case of complex projects, is unrealistic. So it is the maintainers’ task to offer guidance and support in navigating these challenges. To make the journey smoother for both contributors and project maintainers, consider these tips and tricks when selecting issues that are most appealing to potential contributors:

Issue independence: Issues should not be interdependent or block other high-priority tasks. Project management can quickly become complex if contributors must wait for other tasks to be completed. On the flip side, issues that are reliant on unresolved dependencies may deter potential contributors.
Well-defined scope: Contributors are more likely to engage with an issue if its scope is well-defined and easily graspable. Issues with ambiguous or broad objectives can be intimidating for newcomers.
Detailed issue description: To facilitate understanding and engagement, it's crucial to craft detailed issue descriptions. A graphical representation like a flowchart or diagram can always help.
Utilize issue templates: Issue templates can significantly enhance the contributor experience. Templates provide a structured framework for issue reporting, ensuring that essential details are not overlooked. They make the overview of issues cleaner and more organized, helping contributors focus on the task at hand.
Link relevant documentation: Whenever possible, link relevant documentation to the issue. This includes references to project documentation, coding guidelines, and any relevant resources that can aid contributors in understanding the project's context and requirements. Accessible documentation encourages informed contributions.

If the list of issues is ready, the only thing left to do is to assign the “hacktoberfest” label to them. You can also use the “good first issue” label to help new contributors to find issues, which are easy to start with.

Creating templates

GitHub PR template

To maintain consistency and traceability in our DEM repository, we have added a PR template. This template includes the following key sections:

Checklist: There are certain requirements that must be fulfilled before a modification can be applied. Contributors can use a checklist to ensure that these requirements are fulfilled, marking them as completed. GitHub tracks the checklist's status, providing a clear indicator of progress.
Related issue: Here, we encourage contributors to reference the issue that the PR addresses. This link enhances our project's traceability significantly. Both our project management tools, Jira and GitHub issues, display linked PRs in the issues' properties, streamlining our workflow.
Type of change: To categorize PRs effectively, we've included properties related to the type of change being introduced. Contributors can specify whether the PR constitutes a bug fix, a new feature, a breaking change, or a documentation update.
Description: In this section, contributors provide an in-depth and detailed description of the proposed modifications. This description ensures that reviewers and other contributors have a comprehensive understanding of the changes being made
Test cases: We request that contributors outline how they have tested their modifications. This transparency in testing methods helps maintain the project's reliability and quality assurance.
Screenshots: Sometimes, visual aids in the form of screenshots can help the understanding of the behavior changes introduced by the PR. We encourage contributors to include such screenshots when applicable.

You can follow this step-by-step guide on how to create a PR template.

GitHub Issue template

A valuable way to contribute to DEM is by providing feedback by creating issues:

Bug report: We greatly appreciate reporting any instances of faulty behavior. Your bug reports help us identify and address issues promptly, improving the project's overall reliability.
Documentation change request: We try to keep the documentation as clear as possible and up-to-date. We welcome and encourage requests for documentation clarification. Your input helps us maintain accessible and accurate resources for all users.
Feature request: We value your innovative ideas for new features that could enhance DEM. Contributors are encouraged to share their feature requests, which contribute to the evolution and expansion of the project's capabilities.
Report a security vulnerability: While we have been fortunate not to encounter any security vulnerabilities thus far, we understand the importance of a secure environment. To address this concern, we have configured an email address to receive potential security vulnerability reports. We also provide guidance on how to file vulnerability reports using GitHub's built-in function, should the need ever arise.

You can apply GitHub’s built in function to create issue templates.

Update the README

Adding another label in the repo's About section may not sufficiently capture the attention of visitors. A well-crafted README on the other hand immediately conveys that the project is actively participating in Hacktoberfest.
In the DEM repository, we've added a general Hacktoberfest introduction, complete with links to the Contribution Guideline and the XM community. This approach ensures that visitors to the repository are promptly informed about our participation in this event and have easy access to relevant resources.

Building a community: XM

Communication is key for every successful collaboration. Providing multiple channels for contributors to reach out to each other and the maintainers can speed up the onboarding and help them to fit in.

We set up 3 channels:

Direct messages: We encourage contributors to reach out to us through direct messages on GitHub. Additionally, we have designated a specific email address that is actively monitored by all maintainers. This approach guarantees that contributors have accessible means to contact us directly for inquiries or assistance.
GitHub Discussions: Leveraging GitHub's built-in functionality, we offer a Q&A-like interface through GitHub Discussions. This platform serves as an open space for discussions, where contributors can seek answers to their queries, share insights, and engage with fellow collaborators.
The XM community: As part of axem's commitment to fostering an open and inclusive environment, we have established the XM community on Discord, dedicated to discussions related to embedded tooling and the ecosystem. Within this community, we have created a dedicated section, known as a Thread in Discord terminology, specifically tailored for Hacktoberfest discussions. This space provides a dynamic and real-time platform for contributors to connect, share ideas, and collaborate effectively.

Spreading the word

Nominating a repository in Hacktorberfest is as easy as adding the “hacktoberfest” or “hacktoberfest2023” topics in the About section. This way the repository will be visible when filtering for these topics, alongside the more than 133 thousand other ones matching the criteria… So standing out is quite difficult. Here are some tips that we found useful:

Use the right labeling for your repository and the issues
Share the repository in Hacktoberfest’s official Discord server
Look for contributors in related communities

By implementing these strategies, you can increase your repository's visibility for potential contributors during Hacktoberfest.

Checklist

As we get ready for DEM's first Hacktoberfest, we've learned valuable tips for preparing an open-source project. These include clear guidelines, communication, and engagement.
For an overall project health check, you can use GitHub's "health checklist" found under Insights -> Community Standards. It helps assess our readiness and adherence to open-source best practices.

Summary

Hacktoberfest is a celebration of the global open-source community, and by opening our doors to contributions, we aim to give back to this incredible ecosystem while fostering a welcoming environment for newcomers.

Open source is about collaboration, knowledge sharing, and community building. By taking these steps, we hope to make our DEM project a rewarding experience for both seasoned contributors and newcomers. As we embark on this Hacktoberfest journey, we look forward to witnessing the positive impact of collective effort, innovative ideas, and the spirit of open source. Together, we can build a stronger, more vibrant open-source ecosystem for everyone.

Happy hacking!

Illustrations were used from hacktoberfest.com

Join the Hottest Hacktoberfest Project of 2023 🚀

Eszter Vezden — Wed, 04 Oct 2023 09:07:48 +0000

axem is excited to join the Hacktoberfest community for the very first time! 🚀

About us:

We're focused on simplifying embedded development with our project, DEM (Development Environment Manager).
DEM is a command line tool that provides an easy, reproducible, and scalable way to set up containerized Development Environments

Useful links:

GitHub repository: https://github.com/axem-solutions/dem
And detailed documentation: https://axemsolutions.io/dem_doc/

Technology Used:

Python
Docker

Why Contribute?

Enhance your skills in Python
Learn more about containerization
Learn about development platforms
Shape the future of embedded development tooling

Team axem: XM Discord server

We're eager to receive your feedback and contributions to make DEM the go-to tool for embedded developers.
Don't forget to Star the project 🌟

Introducing XM: A New Embedded Tooling and DevOps Community

Eszter Vezden — Wed, 04 Oct 2023 08:48:20 +0000

I'm thrilled to announce the launch of XM, a vibrant new space dedicated to all things Embedded Tooling and DevOps. 🚀

XM Community is about empowering embedded developers, fostering innovation, and sharing insights on cutting-edge tooling and DevOps practices.

Whether you're a seasoned pro or just starting your journey, our community welcomes you to connect, learn, and grow together.

Join the XM Community Discord server today and be part of a dynamic network of embedded enthusiasts, where knowledge flows, ideas thrive, and collaboration knows no bounds.

Join here: XM Discord server

Creating an Optimized Development Environment beyond traditional IDEs

Eszter Vezden — Wed, 13 Sep 2023 15:24:32 +0000

What is an IDE?

Writing embedded software for other architectures than our host usually requires a lot of different tools.
The tools used for a specific project can be grouped together and they form a Development Environment. To make it simpler to work with these tools, they are typically bundled together into an IDE (Integrated Development Environment).

Example: Tools bundled into an IDE

Frequent problems

If you used IDEs before, you probably faced at least some of the following:

The error-prone and time-consuming setup process
Tools are hardcoded into IDEs
More instances of the same tool
Difficult IDE and tool version management
The IDEs enforce the use of their UI

One of the biggest disadvantages of IDEs

From the list above the second is a crucial problem if you want to create a well-managed infrastructure. When working with IDEs, you cannot easily use the underlying integrated tools separately. They usually depend on the IDE or each other, so standalone usage can be problematic and the whole IDE installation is required.

Problem #1: When navigating multiple projects

Let's say at your company the developers have to work on different projects simultaneously, switching between tools even daily.
If they want to keep all the needed tools on their PCs at the same time, more issues can occur:

A lot of duplications - storage-consuming
Tools can interfere
Version management of the tools is crazy

Re-installing an IDE every time to switch tools is time-consuming and let's face it: most developers don't like operational tasks.
To avoid the re-installation of whole IDEs somehow we have to reduce the dependencies between the tools and provide the possibility of standalone usage.

Problem #2: "It works on MY PC!"

Even if you are not project-based, you probably heard your co-worker say, "I don't know what's wrong with your installation. It works on my PC!"
When setting up your Development Environment the process is unpredictable. Two of the annoying scenarios:

Everything works smoothly on the first try. (But you already sense something will go wrong and the worst is you don't even see what. You start praying to the gods of tech.)
Random errors during development. It can be painful to discover after days of debugging that an environment variable had a different value for some obscure reason.

Isolating tools: Containerization

To eliminate these problems, the tools need to operate in their own isolated environments. A lightweight and fast solution for isolation is containerization.

A container is a completely isolated hosting environment, specifically tailored for the needs of the application it runs. The tools get built into their respective container images, and the way they communicate with the host system can be controlled. Containerization also ensures there is no interference between the tools. With this concept, consistent and scalable customized Development Environments can be built easily.

Tools used standalone, isolated in containers

Virtual Machine vs. Container

To understand the containerization concept it can be useful to compare it to the well-known virtual machines:

VM concept

Container concept

A VM runs as a standalone OS on top of the host OS
The container uses the shared resources of the host OS
Different lifetime: a VM is persistent, a container only exists while needed

Unveiling the Advantages

Now that we have all the tools in containers on our PC, after setting up the communication between them we have an optimized Development Environment. The advantages of an optimized DevEnv:

Easy and quick setup process
Tools are not hardcoded into IDEs anymore, standalone usage is possible
Keeping one instance of a tool is enough
The same tool with different versions can be present on our host
The developer is free to choose her favorite editor
No interference between the tools
Communication between the tool and the host can be controlled

Containerization gives flexibility, portability, reproducibility, and scalability to embedded software development and is a key when creating optimized Development Environments.

A little extra: Optimized Hybrid Development Environment

What is a Hybrid Development Environment?

We call a Development Environment Hybrid when the tools part of the DevEnv are not running in the same place. For example, a debugger is running on an SBC, or a build system is running in a cloud environment.

Containerization in a Hybrid Development Environment

After creating an optimized Development Environment, it's worth checking what other benefits can containerization bring to the table.
By combining a Hybrid DevEnv with containerization the following environment can be built easily:

Example: A Hybrid Development Environment with containerized tools

In this case, the client side of the chosen editor is running on the developer's PC, while the editor's server side, the build system, and the toolchain are deployed in the cloud. The project also has a Debugger or Flasher running on an SBC, depending on the state of development.

This example makes one of the greatest advantages of containerization clear: Containerized tools can move across physical and cloud environments easily. As the embedded development landscape continues to evolve, containerization is likely to become an even more significant player in enabling efficient and effective development practices for embedded systems.

Seamless embedded software development - The Benefits of Internal Developer Platforms

janosmurai — Mon, 17 Jul 2023 17:09:17 +0000

Embedded software development plays a crucial role in powering modern technology, enabling the functionality and intelligence of various devices and systems. As the complexity of embedded software continues to grow, companies in the industry are recognizing the importance of streamlining their development processes to improve their efficiency.

One effective method of achieving this is through the integration of the many development tools (e.g. build systems, toolchains, debuggers, etc.) into a manageable service, called an Internal Developer Platform (IDP).

The history of IDPs can be traced back to the growing complexity of software development processes within organizations. As software systems became more intricate and interdependent, the need for efficient collaboration, streamlined workflows, and standardized tools became evident. Internal Developer Platforms emerged as a solution to these challenges, providing a dedicated environment for developers to create, test, and deploy software applications.

Over time, these platforms evolved to offer enhanced capabilities, such as automated CI/CD pipelines, integrated development environments (IDEs), and shared libraries, enabling organizations to improve productivity, accelerate development cycles, and maintain consistency across their software projects.

This article delves into the importance of embedded software development organizations developing their IDPs and highlights several compelling reasons to do so. These include:

Boosting efficiency by reducing reliance on other teams
Standardized tools for standardized processes
Easy CI/CD management
Quick adaptation to changes
Safeguarding intellectual property through enhanced security measures

Boosted Efficiency

Minimizing distractions allows individuals to fully concentrate on tasks and optimize their productivity. An Internal Developer Platform consists of standardized Development Environments that can be self-served by software development team members. By providing the required tools on hand, IDPs can reduce the time spent on communication with IT/Ops and can enable teams to get up to speed more quickly.

Standardization

In safety-critical fields where embedded software is often used, complying with rigorous quality standards is mandatory. IDPs facilitate the establishment of standardized development processes across an organization by ensuring the consistency of the Development Environments. They can enforce quality control measures such as code reviews and automated testing, leading to higher software reliability.
The standards apply not only to the production code but also to the development tools. IDPs can ensure the utilization of the specified compliant tools throughout the development lifecycle, by providing them from a central, managed catalog.

Continuous Integration and Deployment (CI/CD)

Continuous Integration (CI) is already widely used in embedded software projects and Continuous Delivery (CD) is becoming more and more common for devices that can be updated online. CI/CD ensures faster delivery, reliable software releases, and better feedback for the developers. IDPs provide the necessary infrastructure and tools to implement the CI/CD pipelines. Integrating automated build processes, testing frameworks, and deployment pipelines into the IDP makes software delivery efficient and reliable.

Scalability and Adaptability

As embedded software projects grow in complexity, having a development platform that can scale accordingly becomes essential. IDPs can be customized and extended to accommodate the evolving needs of software development teams, ensuring they can meet the demands of their projects.

Security and Intellectual Property Protection

Embedded software development often involves sensitive intellectual property (IP) and security concerns. With an IDP, companies can establish secure development environments with role-based access controls (RBAC) to protect their valuable assets. IDPs can also incorporate security analysis tools and vulnerability assessments, ensuring that software is developed with security best practices in mind. Having a central tool catalog can also decrease security risks because the used tools are not downloaded from unknown sources.

How can DEM power your IDP?

Containerizing your development tools can help you maintain reliable and scalable environments. Storing the container images in a central catalog enforces integrity. An IDP based on containerized Development Environments can ensure that the right tools are used in every environment, like the developer’s PC or the CI/CD server. To simplify the process of obtaining tools for a specific project and maintaining a consistent Development Environment through the whole project life cycle, our open-source tool, the Development Environment Manager (DEM) comes in handy. With DEM, developers can set up a Development or CI/CD Environment with a single command and eliminate the notorious “It worked on my PC!” situation.

Key Points to Remember

Internal Developer Platforms offer embedded software development teams a range of advantages that contribute to improved collaboration, faster development cycles, higher quality software, and reduced time-to-market. If the developers can focus on their actual tasks instead of tweaking the infrastructure, they can stay in flow much longer, which leads to less frustration. Improved developer experience can have significant positive impacts on productivity, morale, and also reduce fluctuation.

DEV Community: axem

Tool Containerization Best Practices For Embedded Software Development

Creating a monocontainer

Problems with the monocontainer approach

Solution: Dedicated Tool Images

Separating the Tools for the Example Project

CppUTest

GNU Arm Toolchain

Stlink-org

The DEM Solution

The axem Open Tool Dockerfiles repo

Conclusion: A Containerized Future for Embedded Software Development

STM32 Development Without an IDE

Development Environment vs IDE

DEM

The toolset

Dev Env creation - Two ways

I. Installing on the native host

II. Simple, containerized installation

Containerized vs Native Dev Env

Simple reproducibility with DEM

Further reading

Creating and Exporting Project-Specific Configurations in VS Code

Getting Started

Export your extensions with a simple command

Sharing your VS Code extensions

Exporting your VS Code configuration

Installing config for a different VS Code

Conclusion

Calling All Python Enthusiasts: Join Us in Contributing to Our Open-Source Project!

Python contributors wanted

Why Contribute?

Tech Stack

Are you interested?

Preptember - How to setup your project for Hacktoberfest

DEM’s story

Preparing the DEM GitHub repository

Creating a Contribution Guideline

Creating a Code of Conduct

Preparing issues for open-source contribution

Creating templates

GitHub PR template

GitHub Issue template

Update the README

Building a community: XM

Spreading the word

Checklist

Summary

Join the Hottest Hacktoberfest Project of 2023 🚀

About us:

Useful links:

Technology Used:

Why Contribute?

Introducing XM: A New Embedded Tooling and DevOps Community

Creating an Optimized Development Environment beyond traditional IDEs

What is an IDE?

Frequent problems

One of the biggest disadvantages of IDEs

Problem #1: When navigating multiple projects

Problem #2: "It works on MY PC!"

Isolating tools: Containerization

Virtual Machine vs. Container

Unveiling the Advantages

A little extra: Optimized Hybrid Development Environment

What is a Hybrid Development Environment?

Containerization in a Hybrid Development Environment

Seamless embedded software development - The Benefits of Internal Developer Platforms

Boosted Efficiency

Standardization

Continuous Integration and Deployment (CI/CD)

Scalability and Adaptability

Security and Intellectual Property Protection

How can DEM power your IDP?

Key Points to Remember