DEV Community: Jan Stamer

The DDD Hamburger for Go

Jan Stamer — Sat, 06 Jan 2024 19:05:46 +0000

The DDD Hamburger is my favorite type of architecture for Go. Why? The DDD Hamburger beautifully combines the best of the hexagonal architecture of Domain-Driven-Design and layered architecture. Get to know the DDD Hamburger Architecture for Go and maybe it'll become your favorite too!

The DDD Hamburger Overview 🍔

The DDD Hamburger is a real Hamburger with well defined layers from top to bottom:

The upper Bun half 🍞
The upper bun half is the top of the Hamburger and is the presentation layer. The presentation layer contains your REST API and the web interface.

The Salad 🥗
Below the bun is the salad which is the application layer. The application layer contains the use case logic of your application.

The Meat 🥩
Next is the meat of your Hamburger, the domain layer. Just like the meat the domain layer is the most important part of your Hamburger. The domain layer contains you domain logic and all entities, aggregates and value objects of your domain.

The lower Bun half 🍞
The lower bun half is the infrastructure layer. The infrastructure layer contains concrete implementations of your repositories for a Postgres database. The infrastructure layer implements interfaces declared in the domain layer.

Got the idea? Great, so let's look into the details.

Go Example of the DDD Hamburger

Now we'll code a Go application using our DDD Hamburger. We'll use a simple time tracking example application with activities to show the practical Go implementation. New activities are added using a REST API and are then stored in a Postgres database.

The DDD Hamburger Architecture applied to Go is shown below. We'll go through all layers of the Hamburger in detail soon.

The Presentation Layer as upper Bun 🍞

The presentation layer contains the HTTP handlers for the REST API. The HTTP handlers like HandleCreateActivity create simple handler functions. All handlers for activities hang off a single struct ActivityRestHandlers, as the code below shows.

type ActivityRestHandlers struct {
    actitivityService  *ActitivityService
}

func (a ActivityRestHandlers) HandleCreateActivity() http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // ...
    }
}

The actual logic to create a new activity is handled by a service
ActivityService of the underlying application layer.

The HTTP handlers don't use the activity entity of the domain layer as JSON representation. They use their own model for JSON which contains the struct tags to serialize it to JSON properly, as you see below.

type activityModel struct {
    ID          string        `json:"id"`
    Start       string        `json:"start"`
    End         string        `json:"end"`
    Duration    durationModel `json:"duration"`
}

That way our presentation layer only depends on the application layer and the domain layer, not more. We allow relaxed layers, which means it's ok to skip the application layer and use stuff from the domain layer directly.

The Application Layer as Salad 🥗

The application layer contains service which implement the use cases of our application. One use case is to create an activity. This use case is implemented in the method CreateActivity that hangs of the application service struct ActitivityService.

type ActitivityService struct {
    repository ActivityRepository
}

func (a ActitivityService) CreateActivity(ctx context.Context, activity *Activity) (*Activity, error) {
    savedActivity, err := a.repository.InsertActivity(ctx, activity)
    // ... do more
    return savedActivity, nil
}

The application services uses the repository interface ActivityRepository for the use case. Yet it only knows the interface of the repository which is declared in the domain layer. The actual implementation of that interface is not of concern to the application layer.

The application services also handle the transaction boundaries, since one use case shall be handled in one single atomic transaction. E.g. the use case of creating an new project with an initial activity for it has to go through in one transaction although it'll use one repository for activities and one for projects.

The Domain Layer as Meat 🥩

The most important part is the domain layer which is the meat of our DDD hamburger. The domain layer contains the domain entity Activity, the logic of the domain like calculating the activity's duration and the interface of the activity repository.

// Activity Entity
type Activity struct {
    ID             uuid.UUID
    Start          time.Time
    End            time.Time
    Description    string
    ProjectID      uuid.UUID
}

// -- Domain Logic
// DurationDecimal is the activity duration as decimal (e.g. 0.75)
func (a *Activity) DurationDecimal() float64 {
    return a.duration().Minutes() / 60.0
}

// ActivityRepository
type ActivityRepository interface {
    InsertActivity(ctx context.Context, activity *Activity) (*Activity, error)
    // ... lot's more
}

The domain layer is the only layer that's not allowed to depend on other layers. It should also be implemented using mostly the Go standard library. That's why we neither use struct tags for json nor any database access code.

The Infrastructure Layer as lower Bun 🍞

The infrastructure layer contains the concrete implementation of the repository domain interface ActivityRepository in the struct DbActivityRepository. This repository implementation uses the Postgres driver pgx and plain SQL to store the activity in the database. It uses the database transaction from the context, since the transaction was initiated by the application service.

// DbActivityRepository is a repository for a SQL database
type DbActivityRepository struct {
    connPool *pgxpool.Pool
}

func (r *DbActivityRepository) InsertActivity(ctx context.Context, activity *Activity) (*Activity, error) {
    tx := ctx.Value(shared.ContextKeyTx).(pgx.Tx)
    _, err := tx.Exec(
        ctx,
        `INSERT INTO activities 
           (activity_id, start_time, end_time, description, project_id) 
         VALUES 
           ($1, $2, $3, $4, $5)`,
        activity.ID,
        activity.Start,
        activity.End,
        activity.Description,
        activity.ProjectID,
    )
    if err != nil {
        return nil, err
    }
    return activity, nil
}

The infrastructure layer depends on the domain layer and may use all entities, aggregates and repository interfaces from the domain layer. But from the domain layer only.

Assemble the Burger in the Main Function

No we have meat, salad and bun lying before us as single pieces. It's time to make a proper Hamburger out of these pieces. We assemble our Hamburger in the main function, as you see below.

To wire the dependencies correctly together we work from bottom to top:

First we create a new instance of of the db activity repository DbActivityRepository and pass in the database connection pool.
Next we create the application service ActivityService and pass in the repository.
Now we create the ActivityRestHandlers and pass in the application services. We now register the HTTP handler functions with the HTTP router.

The code to assemble our DDD Hamburger architecture is like this:

func main() {
    // ...

    // Infrastructure Layer with concrete repository
    repository := tracking.NewDbActivityRepository(connectionPool)

    // Application Layer with service
    appService := tracking.NewActivityService(repository)

    // Presentation Layer with handlers
    restHandlers := tracking.NewActivityRestHandlers(appService)
    router.HandleFunc("/api/activity", restHandlers.HandleCreateActivity())
}

The code to assemble our Hamburger is plain, simple and very easy to understand. I like that, and it's all I usually need.

Package Structure for the DDD Hamburger

One question remains: Which structure of our Go packages is best suited for the DDD Hamburger?

I usually start out with a single package for all layers. So a single package tracking with files activity_rest.go for the rest handlers, activity_service.go for application services, the domain layer in activity_domain.go and the database repository in activity_repository_db.go.

A next step is to separate out all layers as separate packages, except the domain layer. So we'd have a root package tracking. The root package contains the domain layer. The root package has sub package for each layer like application, infrastructure, presentation. Why the domain layer in the root package? That way we can use the domain layer with it's proper name. So if we'd use the domain entity Activity somewhere, the code would read tracking.Activity which is very nice to read.

Whichever package structure is best depends on your project. I'd suggest you start small and easy and adjust it as your project grows over time.

Wrap Up of the DDD Hamburger 🍔

The DDD Hamburger is a layered architecture based fully on Domain Driven Design. It's very easy to understand and follow. That's why the DDD Hamburger is my favorite architectural style. In general and especially in Go.

Using the DDD Hamburger for your Go application is pretty straightforward, as you've seen. You can start out small yet you're able to grow as needed.

See an application of the DDD Hamburger below:

Baralga / baralga-app

Simple and lightweight time tracking for individuals and teams, for the cloud in the cloud.

Baralga

Multi user time tracking application with web frontend and API.

User Guide

Keyboard Shortcuts

Track Activities

Shortcut	Action
Alt + Shift + n	Add Activity
Alt + Shift + p	Manage Projects

Report Activities

Shortcut	Action
Shift + Arrow Left	Show previous Timespan
Shift + Arrow Down	Show current Timespan
Shift + Arrow Right	Show next Timespan

Administration

Accessing the Web User Interface

The web user interface is available at http://localhost:8080/. You can log in as administrator with admin/adm1n or as user with user1/us3r.

Configuration

The backend is configured using the following environment variables:

Environment Variable	Default Value	Description
`BARALGA_DB`	`postgres://postgres:postgres@localhost:5432/baralga`	PostgreSQL Connection string for database
`BARALGA_DBMAXCONNS`	`3`	Maximum number of database connections in pool.
`PORT`	`8080`	http server port
`BARALGA_WEBROOT`	`http://localhost:8080`	Web server root
`BARALGA_JWTSECRET`	`secret`	Random secret for JWT generation
`BARALGA_CSRFSECRET`	`CSRFsecret`	Random secret for CSRF protection
`BARALGA_ENV`	`dev`	use `production` for production mode
`BARALGA_SMTPSERVERNAME`

…

View on GitHub

Credits

The DDD Hamburger Architecture was coined by Henning Schwentner, so thanks for that. Another great influence for structuring the HTTP handlers came from Mat Ryer.

Simplest WebServer for Angular in 6 Lines

Jan Stamer — Mon, 17 Jul 2023 17:43:06 +0000

How do you serve your Angular app? Learn to build a web server for your Angular app in 6 lines of code. Throw in 8 more lines and you have a multi stage Docker build for your web server along with it.

Angular Web Server in 6 Lines

6 lines of code, that's all it takes for a simple Angular WebServer powered by Caddy:

:4200 {
        root * /usr/share/caddy
        encode gzip
        try_files {path} /index.html
        file_server
}

Multi Stage Docker Build in 8 Lines

Now let's throw in another 8 lines of code to get a multi stage Docker build for our Angular frontend.

As a result you'll get a Caddy powered container of your Angular app.

# 1. Frontend Builder
FROM node:18 as builder
WORKDIR /app
ADD . /app
RUN npm ci && npm run build

# 2. Frontend Container
FROM caddy:2.6.4-alpine
EXPOSE 4200
COPY Caddyfile /etc/caddy/Caddyfile
COPY --from=builder /app/dist/frontend/* /usr/share/caddy

Check it out live and working in our Github repo below (folder frontend). Have fun and enjoy!!

crossnative / cloudland-rps

Cloud native Rock, Paper, Scissors Workshop @CloudLand

Hello Cloudland! - Workshop Rock, Paper, Scissors

See documentation...

OpenAPI Backend Specifcation

Look here...

View on GitHub

Go Integration Tests using Testcontainers

Jan Stamer — Mon, 29 Aug 2022 07:45:04 +0000

Your application uses a database like Postgres? So how do you test your persistence layer to ensure it's working properly with a real Postgres database? Right, you need to test against a real Postgres. Since that test requires external infrastructure it's an integration test. You'll learn how easy it is to write integration tests for your Go application using Testcontainers and Docker.

Integration Test Setup

Our application stores users in a Postgres database. It uses the struct UserRepository with a method FindByUsername that uses plain SQL to find a user by username. We will write an integration test running against a real Postgres in Docker for the method FindByUsername.

The integration test for the FindByUsername of our UserRepository looks like:

func TestUserRepository(t *testing.T) {
    // Setup database
    dbContainer, connPool := SetupTestDatabase()
    defer dbContainer.Terminate(context.Background())

    // Create user repository
    repository := NewUserRepository(connPool)

    // Run tests against db
    t.Run("FindExistingUserByUsername", func(t *testing.T) {
        adminUser, err := repository.FindByUsername(
            context.Background(),
            "admin",
        )

        is.NoErr(err)
        is.Equal(adminUser.Username, "admin")
    })
}

First the database is set up. Then a new UserRepository is created for the test with a reference to the connection pool of the database connPool. No we run the method to test userRepository.FindByUsername(ctx, "admin") and verify the result. But wait, where did that database container come from? Right, we'll set that up using Testcontainers and Docker.

Database Setup using Testcontainers

We set up the PostgreSQL database in a Docker container using the Testcontainers library.

As a first step we create a testcontainers.ContainerRequest where we set the Docker image to postgres:14 with exposed port 5432/tcp. The database name as well as username and password are set using environment variables. And to make sure the test only starts when the database container is up and running we wait for it using the WaitingFor option with wait.ForListeningPort("5432/tcp").

Now as second step we start the requested container.

Finally in step 3 we use host and port of the running database container in the connection string for the database with fmt.Sprintf("postgres://postgres:postgres@%v:%v/testdb", host, port.Port()). Now we connect with pgxpool.Connect(context.Background(), dbURI).

The whole method SetupTestDatabase to set up the PostgreSQL container is (errors omitted):

func SetupTestDatabase() (testcontainers.Container, *pgxpool.Pool) {
    // 1. Create PostgreSQL container request
    containerReq := testcontainers.ContainerRequest{
        Image:        "postgres:latest",
        ExposedPorts: []string{"5432/tcp"},
        WaitingFor:   wait.ForListeningPort("5432/tcp"),
        Env: map[string]string{
            "POSTGRES_DB":       "testdb",
            "POSTGRES_PASSWORD": "postgres",
            "POSTGRES_USER":     "postgres",
        },
    }

    // 2. Start PostgreSQL container
    dbContainer, _ := testcontainers.GenericContainer(
        context.Background(),
        testcontainers.GenericContainerRequest{
            ContainerRequest: containerReq,
            Started:          true,
    })

    // 3.1 Get host and port of PostgreSQL container
    host, _ := dbContainer.Host(context.Background())
    port, _ := dbContainer.MappedPort(context.Background(), "5432")

    // 3.2 Create db connection string and connect
    dbURI := fmt.Sprintf("postgres://postgres:postgres@%v:%v/testdb", host, port.Port())
    connPool, _ := pgxpool.Connect(context.Background(), dbURI)

    return dbContainer, connPool
}

Notice that we make sure the PostgreSQL container is terminated after our integration tests with defer dbContainer.Terminate(context.Background()).

Adding Database Migrations

So far our test starts out with an empty database. That's not very useful since we need the database tables of our application. In our example we need the table users. We will now set up our database using golang-migrate.

We add the database migrations to the SetupTestDatabase() method by adding the call MigrateDb(dbURI).

func SetupTestDatabase() (testcontainers.Container, *pgxpool.Pool) {
    // ... (see before)

    // 3.2 Create db connection string and connect
    dbURI := fmt.Sprintf("postgres://postgres:postgres@%v:%v/testdb", host, port.Port())
    MigrateDb(dbURI)
    connPool, _ := pgxpool.Connect(context.Background(), dbURI)

    return dbContainer, connPool
}

The method MigrateDb(dbURI) applies the database migrations to the database using golang-migrate. The migration scripts are read from the directory migrations which is embedded into the binary of our application.

//go:embed migrations
var migrations embed.FS

func MigrateDb(dbURI string) error {
    source, _ := iofs.New(migrations, "migrations")

    m, _ := migrate.NewWithSourceInstance("iofs", source, strings.Replace(dbURI, "postgres://", "pgx5://", 1))
    defer m.Close()

    err = m.Up()
    if err != nil && !errors.Is(err, migrate.ErrNoChange) {
        return err
    }

    return nil
}

Wrap Up

We have a working setup for integration tests against a real Postgres database running in Docker using Testcontainers. We can use this setup for integration tests of our persistence layer. But that's not all it's good for.

This setup is a great way for all kinds of integration tests that need infrastructure. E.g. a test that send emails to an mail server running in docker, as in mail_resource_smtp_test.go.

remast / go-for-testcontainers

Demo for Testcontainers in Go

Go for Testcontainers

Sample code for my post Go for Testcontainers.

View on GitHub

Local Dev Setup with automatic HTTPS + WebSockets in 1 Line

Jan Stamer — Sun, 07 Feb 2021 15:30:46 +0000

Let's set up a local web server for development that supports automatic HTTPs and WebSockets in just 1 line of code. No problem thanks to the awesome Caddyserver.

Preparations

Set up the local hostname

For automatic HTTPs you need to set up a local hostname first. We will use remast.local. To make your local development machine available at that hostname you need to add the hostname to your local hosts file.

So locate the hosts file on your machine and add the following line:

127.0.0.1   remast.local

Here's where you find the hosts file in your operating system.

Operating System	Location of hosts file
Windows 10	`C:\Windows\System32\drivers\etc\hosts`
Linux	`/etc/hosts`
Mac OS X	`/private/etc/hosts`

Now verify by testing to reach the hostname via ping with ping remast.local. If that takes you to 127.0.0.1 everything is fine and you can proceed.

Install Caddyserver

Now install Caddyserver on your machine. If you're on Windows and use the Chocolatey package manager you can do that with choco install caddy. For other plattforms check the Caddyserver install docs.

Starting the local dev server with HTTPs and WebSockets

Now off we go with our local dev server with automatic HTTPs and WebSocket support. Here's the only one line you need:

caddy reverse-proxy --from remast.local --to 127.0.0.1:8080

This one line will fire up your local Caddyserver which will automaticlally start HTTPs and proxy all requests including WebSockets. How cool is that?!

Using the Caddyfile

Of course you can always save your configuration in file called Caddyfile, with the following contents:

remast.local {
    reverse_proxy 127.0.0.1:8080
}

What's next?

Once you've made your first steps Caddyserver there's no way back. Caddy is really awesome for both development and production!

Turbo Streams powered by Go WebSockets

Jan Stamer — Wed, 30 Dec 2020 21:04:31 +0000

Turbo is a new way to build greate web frontends fully powered by html markup and without any JavaScript. That Turbo is what powers Basecamps new mail service HeyHey.

At the heart of Turbo are Streams which provides realtime updates over HTTP or WebSocket.

With Turbo Streams you just sent dom updates over the wire as plain html. Like so:

<turbo-stream action="append" target="dom_id">
  <template>
    Content to append to container designated with the dom_id.
  </template>
</turbo-stream>

Then the html sent over in the template element is appended to the dom element with id dom_id. Just so simple. Of course you can also replace or remove html from the dom the same way by using the actions repace and remove.

For all you Gophers out there keen on using it, I've got you covered! Here's a simple example for using Turbos Streams in Go
with the Gorilla WebSocket toolkit.

remast / go_websocket_turbo

Simple example for using Turbos Streams in Go with the Gorilla WebSocket toolkit.

Go Example for TurboStreams over WebSockets

Simple example for using Turbos Streams in Go with the Gorilla WebSocket toolkit.

Run the sample using the following command:

$ go run *.go

To use the chat example, open http://localhost:8080/ in your browser.

Frontend

The frontend connects to the Turbo Stream using plain JavaScript like:

<script src="https://cdn.jsdelivr.net/npm/@hotwired/turbo@8.0.4/dist/turbo.es2017-umd.min.js" ></script>
<script>
Turbo.connectStreamSource(new WebSocket("ws://" + document.location.host + "/ws"));
</script>

After that the frontend is connected to the Turbo Stream and get's all messages. Every chat message is appended to the dom element with id board.

This should work with html markup too but I have not gotten it working yet.

Server

The server receives the new chat message via web socket. Then it wraps the message as Turbo Stream…

View on GitHub

5 Great Go Resources

Jan Stamer — Fri, 24 Jul 2020 13:57:32 +0000

Here are 5 great resources for all you Gophers out there:

Awesome Go

avelino / awesome-go

A curated list of awesome Go frameworks, libraries and software

Awesome Go

We use the Golang Bridge community Slack for instant communication, follow the form here to join.

Sponsorships:

Special thanks to

Your app, enterprise-ready. Start selling to enterprise customers with just a few lines of code. ^{Add Single Sign-On (and more) in minutes instead of months.}

Awesome Go has no monthly fee, but we have employees who work hard to keep it running. With money raised, we can repay the effort of each person involved! You can see how we calculate our billing and distribution as it is open to the entire community. Want to be a supporter of the project click here.

A curated list of awesome Go frameworks, libraries, and software. Inspired by awesome-python.

Contributing:

Please take a quick gander at the contribution guidelines first. Thanks to all contributors; you rock!

If you see a package or project here that is no…

View on GitHub

Go Training Courses and Material by Ardanlabs

ardanlabs / gotraining

Go Training Class Material :

Go Training

Review our different courses and material

Learn More

Reach out about corporate training events, open enrollment live training sessions, and on-demand learning options.

Ardan Labs (www.ardanlabs.com)
hello@ardanlabs.com

To attend any of our high-performance tranings check out this link:
https://www.ardanlabs.com/training

Index

Purchase Video / Book

The entire training class has been recorded to be made available to those who can't have the class taught at their company or who can't attend a conference. This is the entire class material.

education.ardanlabs.com

There is a book for all of the material in the class.

Ultimate Go Notebook

Our Experience

We have taught Go to thousands of developers all around the world since 2014. There is no other…

View on GitHub

Idiomatic Go Resources

Idiomatic Go Resources. Every time the discussion comes up… | by Damian Gryski | Medium

Damian Gryski ・ Jul 31, 2020 ・
Medium

Security Related Resources for Go

guardrailsio / awesome-golang-security

Awesome Golang Security resources 🕶🔐

A curated list of awesome golang Security related resources.

List inspired by the awesome list thing.

Supported by: GuardRails.io

Tools

Web Framework Hardening

nosurf - CSRF protection middleware for Go.
gorilla/csrf - Provides Cross-Site Request Forgery (CSRF) prevention middleware for Go web applications & services.
gorilla/securecookie - Encodes and decodes authenticated and optionally encrypted cookie values for Go web applications.
secure - Secure is an HTTP middleware for Go that facilitates most of your security needs for web applications.
unindexed - A drop-in replacement for http.Dir which disables directory indexing.
beego-security-headers - beego framework filter for easy security headers management.

Libraries

paseto - Platform-Agnostic Security Tokens implementation in GO (Golang).
hsts - Go HTTP Strict Transport Security library.
jwt-go - Golang implementation of JSON Web Tokens (JWT).
httprobe - Take a list of domains and probe for working HTTP and HTTPS servers.

Static Code Analysis

…

View on GitHub

Style Guides for Go

dgryski / awesome-go-style

A collection of Go style guides

This is a collection of style guides for Go.

Be sure to read about writing engineering guidelines before trying to adopt one of these wholesale.

(For more commentary on these, see my blog post Idiomatic Go Resources)

The Classics

Googler Talks

Go Wiki Pages

Non-Googlers

Corporate/Project-specific Style Guildes

General tips for Code Reviews:

My…

View on GitHub

Using SonarCloud with Github Actions and Maven

Jan Stamer — Sat, 07 Mar 2020 21:40:33 +0000

In this post you will will learn how to analyse your Java Maven project with SonarCloud using Github Actions.

Starting point is a simple Java project with a Maven build. First we'll use SonarCloud to analyze our source code from our local dev environment. Then we'll use Github Actions to run the Maven build. So finally we have a fully functional ci pipeline which builds and analyzes our code using Github Actions.

Set up SonarCloud

Step 1: Create a Project

In order to use SonarCloud you need to create an account and set up a project. So first create an account and log in. Now you can create a new project here or using the '+' button. A project in SonarCloud must belong to an organization. SonarCloud automatically imports your Github organizations. So you can use any of your Github organizations or use the default organization by your Github user name.

After you've created your project, your project has an organization key and a project key. You'll need both to run an SonarCloud analysis. You can always look up organization key and project key from the dashboard
of your project like shown below.

Step 2: Generate a SonarCloud Token

Now we'll set up a secure token as authentication for SonarCloud. Generate a new token from the tab 'security' in your account settings (which is here). Make sure to store the token since you'll only see it right after you've greated it. Now you're all set up to run a first analysis.

Run SonarCloud analysis locally using Maven

You can run the SonarCloud analysis using maven. The organization key, project key and the generated token must be passed to the Sonar Maven Plugin as well as the url for SonarCloud. Replace <GENERATED_TOKEN> with the SonarCloud token you generated in the previous step. So the command is:

mvn sonar:sonar \
   -Dsonar.projectKey=baralga \
   -Dsonar.organization=baralga \
   -Dsonar.host.url=https://sonarcloud.io \
   -Dsonar.login=<GENERATED_TOKEN>

After you ran the analysis the results will shortly be online in the SonarCloud dashboard at https://sonarcloud.io/dashboard?id=<projectKey>. The dashboard for our sample project baralga is available at https://sonarcloud.io/dashboard?id=baralga.

Run SonarCloud analysis using Github Actions

Now we will use Github Actions to run the SonarCloud analysis from our ci pipeline. We'll use Maven for that like we did before. We set up Github Action that runs the SonarClound analysis using Maven. Like before we pass organization key and project key as parameters. Additionally we need to provide the SonarCloud token and the Github Token.

The token for SonarCloud is stored as a encrypted secret as described here. We can access it in our Github Action with ${{ secrets.SONAR_TOKEN }}. The Github Token is already provided by Github Actions itself and we can access it with ${{ secrets.GITHUB_TOKEN }}.

And here's the complete Github Action workflow. Save it in the file .github\workflows\sonar.yml and off you go.

name: SonarCloud
on:
  push:
    branches:
      - master
jobs:
  build:
    runs-on: ubuntu-16.04
    steps:
    - uses: actions/checkout@v1
    - name: Set up JDK
      uses: actions/setup-java@v1
      with:
        java-version: '11'
    - name: Analyze with SonarCloud
      run: ./mvnw -B verify sonar:sonar -Dsonar.projectKey=baralga -Dsonar.organization=baralga -Dsonar.host.url=https://sonarcloud.io -Dsonar.login=$SONAR_TOKEN
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Now every build of your Github Actions pipeline analyzes the code using SonarCloud. That looks like below or see it live in action in the Baralga Actions.

Topping it off with Code Coverage

As last step we add the code coverage of our unit tests to SonarCloud.

Step 1: Calculate Test Coverage with Jacoco

We use Jacoco to calculate the code coverage of our tests. For that we add the jacoco-maven-plugin to our pom.xml:

<plugins>
  ...
  <plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.5</version>
    <executions>
      <execution>
        <id>prepare-agent</id>
        <goals>
          <goal>prepare-agent</goal>
        </goals>
      </execution>
      <execution>
        <id>report</id>
          <phase>prepare-package</phase>
          <goals>
            <goal>report</goal>
        </goals>
      </execution>
    </executions>
  </plugin>
<plugins>

We tell SonarCloud where to find the calculated code coverage using the parameter -Dsonar.coverage.jacoco.xmlReportPaths=${project.build.directory}/site/jacoco/jacoco.xml for our Maven build. After the next build the code coverage will show up in SonarCloud.

Summary

Step by step we introduced SonarCloud to analyze our code within our ci pipeline using Github Actions. Whenever the ci pipeline runs, the code is analyzed using SonarCloud and the results and metrics are available
in the SonarCloud dashboard. You can find a working example at baralga.

Baralga / baralga

Simple and lightweight time tracking for individuals and teams.

Go for SonarCloud with Github Actions

Jan Stamer — Sun, 03 Nov 2019 12:36:00 +0000

Go for SonarCloud with Github Actions

Learn the basics of analyzing a Go project with SonarQube in my post Go for SonarQube. In this post I'll show you how to use Github Actions to analyze your Go project with SonarCloud. SonarCloud offers SonarQube as a service.

Go for SonarQube

Jan Stamer ・ Aug 8 '19

#go #codequality #architecture #cleancode

Metrics like lines of code or test coverage are great to track and improve the quality of your source code. SonarQube can calculate these metrics for your project and track how they evolve over time. Since SonarQube natively supports Go it's a great fit to calculate metrics fo your Go project.

Set up a Build with Github Actions

We start by setting up a basic build pipeline for our Go project using Github Actions. We use the Github's official Go action setup-go for our build. So we create our pipeline for Github Actions in the file .github/workflows/build.yml with the following content:

on: push
name: Main Workflow
jobs:
  build:
    name: Build, Test and Analyze
    runs-on: ubuntu-latest
    steps:
      - name: Setup Go
        uses: actions/setup-go@v3
        with:
          go-version: '1.19'
      - name: Clone Repository
        uses: actions/checkout@master
      - name: Build and Test
        run: make test

Now Github runs our pipeline on every push to the git repository. The pipeline set's up an Go environment, clones the repository and finally builds and tests with make test. The tests also calculates the code coverage of the tests and save it in bin/cov.out.

Run the SonarCloud Analysis as Step

Now we can add the SonarCloud analysis as step in our pipeline using the sonarcloud-github-action. But before that you need to register for SonarCloud. You can use your Github account to sign up.

The step to add the SonarCloud analysis is:

      - name: Analyze with SonarCloud
        uses: sonarsource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

The step analyzes our Go code using the sonar-scanner tool.

The SonarCloud Action needs two environment variables. The first one is GITHUB_TOKEN which is already provided by Github (see Virtual environments for GitHub Actions). The second one is the SONAR_TOKEN to authenticate the Github Action with SonarCloud.

To generate the access token SONAR_TOKEN log into SonarCloud. Now click on your profile and then go to 'My Account' and then 'Security'. Or go directly to account/security. Generate your access token for SonarCloud here. The access token is provided to the build pipeline as a secret environment variable. Go to your repository settings in Github. Then to 'Secrets' and add a new secret with name SONAR_TOKEN and use the generated SonarCloud access token as value, as shown below.

Our new pipeline to build, test and analyze the source code with SonarCloud is shown below.

on: push
name: Main Workflow
jobs:
  build:
    name: Build, Test and Analyze
    runs-on: ubuntu-latest
    steps:
      - name: Setup Go
        uses: actions/setup-go@v3
        with:
          go-version: '1.19'
      - name: Clone Repository
        uses: actions/checkout@master
      - name: Build and Test
        run: make test
      - name: Analyze with SonarCloud
        uses: sonarsource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Add Code Coverage to SonarCloud

SonarCloud is able to analyze our source code. Yet it's not able to pick up the code coverage of our tests which is stored in the file bin/cov.out. And this is a bit tricky. The sonar-scanner looks for the code coverage in file /home/runner/work/service_sonar/service_sonar/bin/cov.out. That's where it is after the tests. The problem is the sonar-scanner tool is executed with docker and mounts the current directory as a volume like docker run --workdir /github/workspace [...] -v "/home/runner/work/service_sonar/service_sonar":"/github/workspace". Within the docker run there is no file /home/runner/work/service_sonar/service_sonar/bin/cov.out but the path of that file within docker is actually /github/workspace/bin/cov.out. So we need to use that path as sonar.go.coverage.reportPaths in the SonarQube settings file sonar-project.properties. The full sonar-project.properties for Github Actions are shown below:

# Github organization linked to sonarcloud
sonar.organization=remast

# Project key from sonarcloud dashboard for Github Action, otherwise pick a project key you like
sonar.projectKey=de.red6:service_sonar

sonar.projectName=service_sonar
sonar.projectVersion=1.0.0

sonar.sources=.
sonar.exclusions=**/*_test.go,**/vendor/**,**/testdata/*

sonar.tests=.
sonar.test.inclusions=**/*_test.go
sonar.test.exclusions=**/vendor/**
sonar.go.coverage.reportPaths=/github/workspace/bin/cov.out

A live example of the pipeline can be found at service_sonar (branch feature/gh-pipeline-simplify).

Our project in the SonarCloud Dashboard

After the first SonarCloud analysis you can see the results on the SonarCloud project dashboard shown below.

Check out dashboard of the sample project at https://sonarcloud.io/dashboard?id=de.red6%3Aservice_sonar.

Run the SonarCloud Analysis as Job

Currently our pipeline has two main steps. The first step is build and test and the second step is the SonarCloud analysis. In big projects the SonarClould analysis can take up to several minutes. Since steps are executed synchronously one after the other the pipeline can only continue once SonarCloud is done analyzing. But what about our integration tests? Say we want to run some integration tests after the build. We want to give feedback about the integration tests as soon as possible. This means we don't want to wait for the SonarCloud analysis but proceed to the integration tests right after build and test. This is easily possible if we separate out the SonarCloud analysis as a separate job.

Github Actions runs jobs in a workflow in parallel. So we can split our pipeline in three jobs. The first job is build and test. Then we have a second job to for the SonarCloud analysis and a third job that runs the integration tests. The job to build and test has to run first. After that both the job for the SonarCloud analysis and the integration tests can run in parallel.

First pipeline draft

The first draft of our improved pipeline shown below. It has the three jobs build, sonarCloudTrigger and integrationTest. The jobs sonarCloudTrigger and integrationTest both depend on the job build which is declared with needs: build.

on: push
name: Main Workflow
jobs:
  build:
    name: Compile and Test
    runs-on: ubuntu-latest
    steps:
      - name: Clone Repository
        uses: actions/checkout@master
      - name: Setup go
        uses: actions/setup-go@v1
        with:
          go-version: '1.13'
      - run: make test

  sonarCloudTrigger:
    needs: build
    name: SonarCloud Trigger
    runs-on: ubuntu-latest
    steps:
      - name: Clone Repository
        uses: actions/checkout@master
      - name: Analyze with SonarCloud
        uses: sonarsource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

  integrationTest:
    needs: build
    name: Integration Test
    runs-on: ubuntu-latest
    steps:
      - run: echo Should run integration tests.

The pipeline is executed the way we wanted. The job build runs first and after that sonarCloudTrigger and integrationTest run in parallel. It has only one flaw though. Do you see it? If you check the logs you will notice that SonarCloud is not able to read the code coverage of the unit tests. SonarCloud looks for the code coverage in the file bin/cov.out. This file is only present in the build job but not in the job sonarCloudTrigger. So we need to transfer the file from one job to the other.

Revised pipeline with Code Coverage

To transfer the file bin/cov.out from the build job to the sonarCloudTriggerJob we'll use the actions upload-artifact and download-artifact. We add the action upload-artifact to the build job and download-artifact to the sonarCloudTriggerJob job like below:

on: push
name: Main Workflow
jobs:
  build:
    name: Compile and Test
    runs-on: ubuntu-latest
    steps:
      - name: Clone Repository
        uses: actions/checkout@master
      - name: Setup go
        uses: actions/setup-go@v1
        with:
          go-version: '1.13'
      - run: make test
      - name: Archive code coverage results
        uses: actions/upload-artifact@v1
        with:
          name: code-coverage-report
          path: bin

  sonarCloudTrigger:
    needs: build
    name: SonarCloud Trigger
    runs-on: ubuntu-latest
    steps:
      - name: Clone Repository
        uses: actions/checkout@master
      - name: Download code coverage results
        uses: actions/download-artifact@v1
        with:
          name: code-coverage-report
          path: bin
      - name: Analyze with SonarCloud
        uses: sonarsource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

  integrationTest:
    needs: build
    name: Integration Test
    runs-on: ubuntu-latest
    steps:
      - run: echo Should run integration tests.

A live example of the pipeline can be found at service_sonar (branch master).

Add a SonarCloud Badge

We can add a SonarCloud badge to our project to quickly show the SonarCloud status of our project from within the readme. To create SonarCloud badge go to your SonarCloud project dashboard and click 'Get project badges'. You can choose between three badges as shown below.

Wrap Up

Now you have learned how to analyse Go code using SonarCloud from a Github Action pipeline. By using a separate job for the SonarCloud analysis we are able to run the time-consuming analysis in parallel to other important tasks like integration tests or deployment.

All code is provided as a running example in the Github project service_sonar.

remast / service_sonar

Example for analyzing Go code with SonarQube (including Github Action).

The Coders Bucket List

Jan Stamer — Tue, 22 Oct 2019 18:04:54 +0000

A bucket list is a list of things you ought to do before you kick the bucket and die. Here's a bucket list for every coder out there. Check it out and tackle them one at a time!

Compile your own Linux Kernel

The linux kernel is the heart of the Linux operating system and is open source. Compile your own linux kernel to get a deeper understanding of how linux, open source software and the kernel itself work.

Set up a Build and Deployment Pipeline

Set up your own pipeline to build and deploy your code. The pipeline should compile your code, create an artifact like a docker container and deploy it on a remote machine. You can build the pipeline in Jenkins, Gitlab CI, Travis CI or using Github Actions.

Mentor a Junior Dev

Mentor a junior developer will teach both you and your mentee a lot. You can share your knowledge and give your mentee a great start in the job. You'll see that also you will benefit by looking at things from a different angle and explaining things you've been using and doing for quite some time.

Install and run stuff on FreeBSD

Install a unix distribution like FreeBSD on a machine and run some stuff on it. Real unixes are quite different from linux as you will notice. Yet unixes are great and very stable server operating systems.

Give a Talk

Share experiences or knowledge with others by giving a talk. This talk can be for your team only, within your company of even at a conference. Think carefully about what you want to tell and how. Learning and practicing
to give a talk will greatly improve your communication skills.

Contribute to Open Source

Contributing to an Open Source project that you like. You can contribute not only code but also documentation, translations or tests. Some projects have issues that are specially suited for newbies. E.g. in Spring these issues are tagged ideal-for-contribution.

Give a Training

Give a training in our outside your company. Prepare the training yourself with background information and hands on practical excercises and tasks. If you can give the training more than once - even better. You'll experience
that every training is different and all participants will experience the training differently.

Write a Compiler

Compilers is where program languages start. Write you own little compiler and you will gain lots of insights into the inner workings of programming languages.

Learn a Functional Programming Language

Learning to program in a pure functional programming lanuage like Haskell or Elm will change the way you think about programming. So check it out even it won't become your primary language.

Bring a Website to Production

Code a website using plain HTML and CSS and bring it to production. Register a domain name and make your website available under that domain. Even better use web server like Apache, nginx or Caddy to serve the website.

Develop a mobile app (and release it)

Develop your own mobile application for Android or iOS and release it in the app store. Make your app communicate with backend services and use device functionalities like geolocation via GPS.

Learn to use vim and emacs

The classic text editors are vim and emacs. So take the time to learn basic text editing in both vim and emacs. Once mastered it's a skill you can use for the next 30 years.

Mess up Production real bad

When you develop production code it's only a matter of time. One day you will write code that messes up production. But don't you worry we've all been there and we all make mistakes. So fix it, learn from and grow from it.

Fix Production on Pager Duty

You are on pager duty and shit happens. Maybe some hardware breaks down, a 3rd party service is not available any more or the configuration is messed up. There will be a day when you dig into the issue and find the root cause. Hang into it and fix production. Enjoy the feeling when everything is running smoothly again.

Write a JavaScript Single Page Application

Single Page JavaScript Applications (SPAs) are one very important way to develop a web frontend. Write a JavaScript SPA application to get to know how SPAs are built and distributed. You can use a JavaScript framework like Vue.js, Angular or React to get you started.

Write a Server Rendered App

Building a web frontend as a server rendered app is a classic but still very valid approach to develop a web frontend. So write a server rendered web frontend in the language of your choice. For PHP you might use Laravel, for Java Spring Boot & MVC or Buffalo for Go.

Learn Shell Scripting

Shell scripting is quick way to script and automate tasks on any Linux and Unix machine. Learn to write a script for the Bash shell. Make use of classic tools for shell scripting like awk, sed or cut. If you want to top it off write your script in vim or emacs (see above).

Hack in a Capture the Flag Contest

Improve your security skills and learn how to hack in a capture the flag contest. Read all about it here:

What is CTF and how to get started!

atan ・ Mar 28 '19

#security #ctf #beginners

Go for the REST

Jan Stamer — Fri, 06 Sep 2019 18:29:40 +0000

The Go programming language is a great fit for RESTful web applications. Picking the right Go web framework to start with is not an easy task. Lucky enough it is a task solved already. Pick the Go web framework buffalo and off you go. Not convinced yet? Learn how to build a RESTful web application with buffalo.

Buffalo is a whole ecosystem to develop web applications in Go. Buffalo combines routing, templating and testing to build web applications based on the famous model-view-controller pattern. The foundation is a powerful tooling with hot reload, a generator for views and controller and tools for common tasks like compressing JavaScript or database migration. Buffalo combines existing libraries like the Gorilla web toolkit with best practices of Go web development. With buffalo comes a uniform and consistent project structure covering packages and directories up until handling the database session. So let's get started.

Installing Buffalo

We'll use the new Go modules for dependency management. So let's activate that using the environment variable GO111MODULE=on. Now we can install buffalo with go get -u -v github.com/gobuffalo/buffalo/buffalo. Executing the command buffalo in your shell should display the help.

Create and run the RESTful web application

Now we create our first buffalo RESTful web application named rowdy. The command buffalo new rowdy --api creates a new RESTful buffalo web application (without frontend) in the directory rowdy. We can run the application with the command buffalo dev (from directory rowdy). The application is available at http://localhost:3000. But wait, that doesn't work! The generated application needs a running database. so we start a PostgreSQL database with docker run --name rowdy_db -e POSTGRES_PASSWORD=postgres -d postgres. Now you should be able to start the application and access the REST API under the web root saying "Welcome to Buffalo".

You can keep the application running while you develop. Buffalo will automatically pick up changes in sources and rebuild and restart the application as needed.

Guide through the App

Entry point of the buffalo application is the main.go in the root directory of our application. The main.go is the main package of the application and starts the application by executing the method Serve of the buffalo struct buffalo.App. The buffalo app itself is created in the app.go which is in the package actions.

The app.go is where the music plays. In the app.go all routes and middlewares are registered. For every route an action is registered. An action is the controller layer of the MVC pattern. For the moment we only have one action which is the HomeHandler. With app.GET("/", HomeHandler) the action HomeHandler is registered in the app.go for the path /. The HomeHandler itself is declared in home.go in the package actions, and the associated test is in home_test.go.

The package model is for the model layer of MVC. Buffalo assumes the model layer is the persistence layer of the application running against a relational database like postgres. This model layer is based on buffalo pop, a active record inspired persistence layer for Go.

The directory migrations is all about maintaining the schema of the relational database. Here scripts for the database migration framework fizz are stored. But we'll get to that later.

Then there are the directories fixtures, config and grifts. Fixtures are scripts to create database content for tests. Config is for the configuration of the buffalo code generator. Grifts are small scripts to automate common tasks like listing all routes.

To complete our guide we look at the generated configuration files. The most important config file of our application is the database.yml with the database credentials for several environments. Then there is the Dockerfile to create a docker container of our app.

Create a Health Check Action

Now let's create our first action, a health check at route /health/check. We generate the skeleton of our health check action using the buffalo code generator by running buffalo generate action health check --skip-template. That generated an action with name HealthCheck. The first parameter of the generator command is the name of the action and the second parameter check is the name of the handler. Since we build a RESTful API we don't need a HTML template for our action and suppress generating one with the parameter --skip-template. Buffalo created the files health.go and health_test.go in the directory actions. The generator also registered our new action for the path /health/check in the app.go.

A buffalo action needs to implement the interface buffalo.Handler. The terms action and handler are both synonymously used for the controller layer of the MVC pattern. A look at the home.go shows the generated code:

func HealthCheck(c buffalo.Context) error {
    return c.Render(200, r.HTML("health/check.html"))
}

The generated handler tries to render the template health/check.html as HTML. Since we build a RESTful API we don't have templates so this can't work. We change the implementation to c.Render(200, r.String("Up and running!")) so that it returns a plain text string.

func HealthCheck(c buffalo.Context) error {
    return c.Render(200, r.String("Up and running!"))
}

Test the Health Check

Now we need to fix the test for our health check. The whole buffalo application is available it the test. In the test you can run requests against the application or even access the database it's running against of the HTTP session.

To test the health check we run request path /health/check and verify the HTTP status code. The code to do that is:

package actions

func (as *ActionSuite) Test_Health_Check() {
    result := as.HTML("/health").Get()

    as.Equal(200, result.Code)
    as.Contains(result.Body.String(), "Up and running")
}

More about Actions and Routes

When configuring routes you can use placeholders and regular expressions in the path like app.GET("/account/{category}/{id:[0-9]+}", AccountHandler). Since that's base on the Gorilla web toolkit you can make full use of the Gorilla multiplexer.

In the handler itself you have full access to the buffalo.Context. With the buffalo context you can render JSON or HTML and access path and query parameters with them methods c.Params() and c.Param(string). You can also acess the logger and log health check requests with c.Logger().Info("Health checked by: %v", c.Request().Host).

To get an overview of all registered routes use the command buffalo routes which prints:

METHOD | PATH           | ALIASES | NAME            | HANDLER
------ | ----           | ------- | ----            | -------
GET    | /              |         | rootPath        | rowdy/actions.HomeHandler
GET    | /health/check/ |         | healthCheckPath | rowdy/actions.HealthCheck

CRUD Resources

To make it easy to develop CRUD functionality buffalo has resources. A buffalo resource always contains the actions Create, Show, Update, Destroy and List with standardized paths in the router. Resources are always RESTful and never use templates.

Now we create a resource to manage concerts with buffalo generate resource concert. Buffalo created several files for us. The implemntation of the concert resource in the files concert.go in the package actions, along with the tests in concert_test.go. A database mapping for pop in concert.go in the package models including test. And last but not least in the directory migrations two fizz scripts to create a table for the concerts in our database schema. The concerts resources is registered in the app.go using app.Resource("/concerts", ConcertsResource{}).

Looking at the output of buffalo routes we see all the routes of our new concert resource.

METHOD | PATH                    | ALIASES | NAME            | HANDLER
------ | ----                    | ------- | ----            | -------
GET    | /                       |         | rootPath        | rowdy/actions.HomeHandler
GET    | /concerts/              |         | concertsPath    | rowdy/actions.ConcertsResource.List
POST   | /concerts/              |         | concertsPath    | rowdy/actions.ConcertsResource.Create
GET    | /concerts/{concert_id}/ |         | concertPath     | rowdy/actions.ConcertsResource.Show
PUT    | /concerts/{concert_id}/ |         | concertPath     | rowdy/actions.ConcertsResource.Update
DELETE | /concerts/{concert_id}/ |         | concertPath     | rowdy/actions.ConcertsResource.Destroy
GET    | /health/check/          |         | healthCheckPath | rowdy/actions.HealthCheck

Let's now look into the generated handler method Show. We see that buffalo gets the database connection from the buffalo context with c.Value("tx").(*pop.Connection). Then buffalo loads the concert model from the database using pop. The id of the concert model is provided as a path parameter. Finally the loaded concert model is returned with c.Render(200, r.Auto(c, concert)). Depending on the negotiated content-type this is either as JSON or as XML.

// GET /concerts/{concert_id}
func (v ConcertsResource) Show(c buffalo.Context) error {
    // Get the DB connection from the context
    tx, ok := c.Value("tx").(*pop.Connection)
    if !ok {
        return errors.WithStack(errors.New("no transaction found"))
    }

    // Allocate an empty Concert
    concert := &models.Concert{}

    // To find the Concert the parameter concert_id is used.
    if err := tx.Find(concert, c.Param("concert_id")); err != nil {
        return c.Error(404, err)
    }

    return c.Render(200, r.Auto(c, concert))
}

Error Handling

Handling errors properly can be tricky. But buffalo has you covered. Errors generally occur in handlers. Handlers have two options of dealing with them:

Return a normal Go error from the handler like errors.New("Boom!"). The application then returns HTTP status code 500 for internal server error.
Use the bufallo context method Error to handle like return context.Error(501, errors.New("That blew up!")). That way the applications returns the custom status code 501.

Either way the application keeps running and the error is logged and returned either as JSON or XML. If buffalo runs in dev mode (environment variable GO_ENV="development" or start with buffalo dev) this will contain a stacktrace of the error.

Middlewares

Buffalo applications feature many best practices in Go web development. This is mainly featured by various middleware. A middleware is code that runs befor or after a HTTP handler (in buffalo called action). Middlewares are a general concept of Go web development. A good read about the concept of middleware is Making and Using HTTP Middleware by Alex Edwards.

Buffalo comes with lots of usefule middlewares integrated into the buffalo.App right from the start. E.g. middleware that prevents your app from crashing after a panic or an error. In our generated app middleware is registered in actions/app.go. One example is app.Use(forceSSL) which registers a middleware that redirects HTTP requests to HTTPS.

It's also possible to skip middleware for specific routes. We could use this to skip authentication middleware for our health check route. The command buffalo task middleware shows the registered middlewares for every route.

Configuration

A buffalo app is configured using environment variables as proposed in the twelve-factor app. The configuration is powered by the buffalop package envy.

There are predefined environment variables like GO_ENV for the current environment (e.g. development, test, production) or HOST (localhost) and PORT (3000). You can also use custom variables. You can use an optional environment variable with envy.Get("CUSTOM_SETTING", "default-value"). Or a mandatory variable with envy.MustGet("MY_REQUIRED_OPTION") which results in an error if the variable is not set.

You can also read environment variables from the file .env. Buffalo looks for a file named .env in the current directory and evaluates it. This nicely integrates with direnv if you create a .envrc with only the line dotenv (see .env files). You can use this to have buffalo read your local development settings which of course you should never check in.

Plugins

Buffalo can be extended using plugins. The database framework pop is implemented as a buffalo plugin. There are also plugins for kubernetes or Azure. The buffalo website contains a list of available plugins.

Build the App

To build the app run buffalo build. Buffalo compiles the app and creates a standalone binary rowdy in the directory build.

Wrap up the REST

We learned how to build a RESTful application with the buffalo framework that adheres to the MVC pattern. The model layer consists of structs mapped against a PostgreSQL database using pop. Actions and resources make up the controller layer. We created a simple health check action and a CRUD resource for concerts. We have not used the view layer yet since all we need is serializing structs to XML or JSON. We learned how to handle error, how to deal with configuration and how to run a production build in the CI environment.

Buffalo comes with a uniform and consistent structure for web applications in go, follows best practices of Go web development and covers the full lifecycle from development to production.

Yet there's still so much that we could't cover. Buffalo has a lot to offer to build frontends. Or you can use buffalos tasks or background workers. All this is covered by the great buffalo documentation.

And of course stay tuned because I'll spread more buffalo love here soon!

Soft Coder / 3x3 des Daily Standup

Jan Stamer — Mon, 26 Aug 2019 19:22:33 +0000

Egal ob Kanban oder Scrum, täglich grüßt das Daily Standup. Das Team hat 15 Minuten, um die wichtigsten Informationen von gestern und heute auszutauschen. Du hast 30 Sekunden. Nutze sie.

Das folgende 3x3 des Daily Standup hilft Dir dabei. Mit 9 Tipps zum wichtigsten Meeting der agilen Entwicklung.

Bereite Dich vor

Vorbereitung ist alles, auch beim Daily Standup. Also bereite Dich vor. Diese 3 Dinge helfen Dir dabei:

#1 Komme pünktlich

Komme pünktlich. Nichts ist ärgerlicher als 10 Leute die 5 Minuten auf eine Person warten müssen, die zu spät kommt. Jeden Morgen. Also komme pünktlich. Startet das Daily um 10:00 Uhr setze Dir eine Erinnerung auf 9:55 Uhr. Dann hast Du 5 Minuten, um einen Kaffee zu holen, auf die Toilette zu gehen oder einfach nur um Dich zu sammeln. Um Punkt 10 Uhr stehst Du bereit und der Rest des Teams auch. Dann kann‘s direkt losgehen und Ihr seid auch pünktlich um 10:15 Uhr durch.

#2 Überlege was Du sagen willst

Überlege Dir was Du sagen willst. 30 Sekunden für 1 Tag Arbeit ist wenig. Also überlege Dir vorab was Du sagen willst und was nicht. Am wichtigsten sind die Basics des Daily Standup. Also woran hast Du gestern gearbeitet, woran arbeitest Du heute und was behindert Dich gerade. Aber denke auch an andere Dinge wie Deinen Urlaub nächste Woche. Mach Dir einen Spickzettel mit Notizen falls Dir das hilft an alles zu Denken was Du sagen willst.

#3 Kopf frei fürs Daily

Löse Dich von dem woran Du gerade noch gearbeitet hast. So hast Du den Kopf frei fürs Daily und kannst Dich voll auf Dich und Deine Team Mitglieder konzentrieren. Lass doch Dein Handy am Platz, die nächsten 15 Minuten brauchst Du es nicht. Sei präsent und wachsam im Daily Standup. Dein Team weiß das zu schätzen.

Bringe Deinen Inhalt unter

In Deinen 30 Sekunden musst Du Deinen Inhalt unterbringen. 30 Sekunden, einmal und am Stück. Für Dich. Nutze diese 30 Sekunden. Informiere das Team darüber was Du gestern getan hast, und was heute ansteht. Halte alle über Problem, Fragen und Entscheidungen auf dem Laufenden. Dazu musst Du Dich fokussieren, folgende 3 Tipps helfen Dir dabei:

#4 Gib einen aussagekräftigen Status.

Gib einen aussagekräftigen Status. Sage was Du getan hast und welchen Status Du damit erreicht hast. Sage konkret was gelaufen ist, aber ohne Details. Lass Ticket Nummern weg, wenn Sie nicht allen geläufig sind. Vermeide Phrasen und Füllwörter sowie schwammige Aussagen. Zum Beispiel: „Gestern habe ich die DDL Skripte zur Erweiterung der Tabelle UMSATZ um die Spalte UUID erstellt. Heute morgen habe ich die fertigen DDL Skripte auf der Testdatenbank cd34 eingespielt.“

#5 Sage konkret was Deine nächsten Schritte sind.

Sage konkret was Deine nächsten Schritte sind. Arbeite heute an den wichtigsten nächsten Schritten, um Deinen Task zu erledigen. Sage konkret welche Schritte das sind. Also „Nach dem Daily analysiere ich den Fehler bei der IBAN Prüfung aus der Fast Lane. Heute Nachmittag mache ich das Review der Datenbank Änderung von Michael.“ Beziehe Dich morgen wieder auf diese Aussagen. Dann sieht das Team immer wo Du stehst und was Du noch vor Dir hast.

#6 Sprich offene Fragen und Entscheidungen an.

Sprich offene Fragen und Entscheidungen an. Sind noch Fragen oder Entscheidungen offen die Dich behindern, führe aktiv Lösungen und Entscheidungen herbei. „Der Name des Datenbank Schemas ist noch unklar. Das klären wir heute um 11 Uhr gemeinsam, im Termin mit dem Architekten und einem Datenbank Admin.“ Nimm Dir am Morgen Zeit, um einen Termin für eine Entscheidung zu organisieren oder schreibe dem Fachbereich eine E-Mail mit Deinen Fragen. Erledige diese Dinge vor dem Standup. Dann weiß das Team im Standup das Du alles in die Wege geleitet hast um die Entscheidung herbeizuführen. Dann weiß das Team im Standup das Du die Entscheidung auf den Weg gebracht hast.

Achte auf Deine Sprache

Nutze die Sprache, um Dich zu fokussieren und Deine Inhalte rüberzubringen. Es hängt von Deiner Sprache ab ob Deine Inhalte beim Team ankommen oder nicht. Hier drei handfeste Tipps, um Deine Sprache im Daily zu verbessern:

#7 Mache kurze, knappe Sätze.

Mache kurze und knappe Sätze. Sage nicht zu viel auf einmal. „Ich habe das Review der SQL Query gemacht. Und den Merge Request in den master übernommen. Jetzt habe ich wieder Luft. Ich schnappe mir gleich ein neues Ticket.“

#8 Sprich aktiv und konkret

Sprich aktiv und konkret. Sage was Du tun wirst und wann Du es tun wirst. Formuliere immer so das klar ist was zu tun ist und wer das tut. Bestenfalls noch bis wann das getan ist. „Ich schreibe noch einen Unit Test fürs Löschen der Überweisung. Heute Nachmittag werde ich damit fertig.“ Oder „Ich bin auf Angela aus dem QA Team zugegangen. Sie wird den Test bis Ende der Woche durchführen.“

#9 Sprich deutlich und mache Pausen.

Sprich deutlich und mache Pausen. Gerade wenn die Zeit knapp ist: Lass Dir Zeit beim Sprechen und mache Pausen. Konzentriere Dich auf die wesentlichen Informationen. Dann hast Du auch in 30 Sekunden genug Zeit langsam und deutlich zu sprechen. „Ich habe gestern Nachmittag das Review abgeschlossen. Heute kümmere ich mich um den Vorfall aus Produktion.“

Daily Standup, Daily Practice

Nutze diese 9 Tipps für Dich und Dein Team. Arbeite an Dir Tag für Tag, Daily für Daily. Betrachte das Daily Standup als täglichen Elevator Pitch. Diesen Elevator Pitch hältst Du jeden Tag. Mache den Elevator Pitch jeden Tag ein kleines Stück besser. Probiere mal etwas Neues aus und experimentiere. Finde heraus was Dir liegt, und was für Dich funktioniert. Wage Dich dabei aus Deiner Komfortzone heraus. Probiere etwas aus was Dir schwer fällt oder für Dich neu ist.

Gibt es jemand im Team zu dem Du einen persönlichen Draht hast? Frage ob die Person Dir Feedback geben kann. Oder Ihr macht das gegenseitig. Beziehe Dein Team ein und bitte es in der Retro um Feedback.

So lernst Du jeden Tag etwas dazu. Was Du dazu lernst hilft Dir nicht nur im Daily Standup sondern in vielen beruflichen und privaten Situationen.

Dieser Artikel erschien ursprünglich auf medium.

Wir schreiben Code

Jan Stamer — Sat, 10 Aug 2019 15:10:34 +0000

Wir Software Entwickler lesen und schreiben viel. Aber unser Genre ist nicht Belletristik und nicht Liebesromane. Wir lesen Code. In vielen Sprachen. Wir lesen und schreiben XML, SQL, Java oder JavaScript. Der eine bevorzugt Java, die andere PHP. Wenn wir schreiben sind wir genau und präzise, das müssen wir. Jedes Wort ist wichtig. Auch einzelne Zeichen können entscheiden über Gewinn oder Verlust, manchmal auch über Tod oder Leben. Deswegen müssen wir genau sein, genau schreiben ohne Missverständnisse oder Kompromisse zuzulassen.

Wie ein Richter in seiner Urteilsbegründung müssen wir eine Aufgabe auseinandernehmen, zerpflücken, und von allen Seiten betrachten. Was für den Richter Verordnungen und Paragraphen sind, sind für uns Anforderungen und technische Rahmenbedingungen. Der Richter stützt sich auf Gesetze, wo es keine Gesetze gibt, kann er nicht richten. Wir stützen uns auf Anforderungen und technische Rahmenbedingungen. Wir können keinen Code schreiben, wenn nicht klar ist, was der Code tun soll. Wir können keinen Code schreiben, der technische Unmöglichkeiten überwindet. Unseren Code schreiben wir wie Anwälte und Richter — klar, präzise und ohne Kompromisse zuzulassen. Aber das ist nicht immer so.

Manchmal müssen wir wie Schriftsteller weit ausholen und viel Fantasie zulassen. Dann müssen wir einfach anfangen zu schreiben. Mit Idee aber ohne Plan. Wir beginnen zu schreiben, aber wir kennen das Ende noch nicht. Erst im Laufe des Schreibens merken wir, was wir schreiben wollen. Am Anfang haben wir nur den leeren Bildschirm vor uns, mit einem Anfang aber ohne Ende. In einem Punkt sind wir schlechter dran als der Schriftsteller. Unsere Geschichte wird nie fertig. Sie geht immer weiter und weiter. Und nicht nur das. Nicht mal auf den Anfang können wir uns verlassen. Jederzeit kann der Anfang über den Haufen geworfen werden, ein Mittelteil kommt hinzu oder eine neue Hauptfigur wird eingefügt. Je besser und erfolgreicher unsere Geschichte ist desto schneller passiert das.

Manchmal müssen wir exakt und präzise sein. Manchmal müssen wir einfach drauflos schreiben, vorpreschen ohne genau zu Wissen wohin. Wir müssen erkennen wann es Zeit ist exakt und präzise zu sein und wann es Zeit ist draufloszuschreiben. Das sagt uns keiner.

Wir schreiben nie alleine. Immer schreiben wir unsere Geschichte mit anderen zusammen. Mal mit vielen, mal mit wenigen. Wenn wir es gut machen merkt das niemand. Keiner der die Software nutzt soll erkennen wer einen Paragraphen, ein Kapitel oder einen Band geschrieben hat. Dann sind wir erfolgreich.

Doch schreiben wir anders als der Schriftsteller oder der Jurist. Was wir schreiben sind Anweisungen an die Maschine. Die müssen korrekt sein, sie müssen der Maschine exakte Befehle geben. Die Maschine hat keine Intelligenz, alles was sie tut haben wir für sie aufgeschrieben. Aber noch mehr als das.

Was wir schreiben muss lesbar sein. Jederzeit müssen andere an jedem Punkt die Geschichte weiterschreiben können. Dazu müssen sie rasch verstehen worum es geht, sie müssen den Plot und die Hauptpersonen erkennen können. Ja sie müssen auch in der Lage sein, die Geschichte in der Art und Weise weiterzuschreiben, wie es andere zuvor getan haben. Nur dann wird es eine Geschichte und nicht viele kurze Geschichten aneinander gereiht.

Unter keiner unserer Geschichten steht unser Name. Vom Pophit kennt jeder die Band, vom Bestseller den Autor. Uns kennt keiner. Das ist auch gut so. Wir wollen nicht bekannt sein, nicht gefeiert werden. Wenn unsere Arbeit ankommt, unsere Software ihren Dienst gut verrichtet sind wir glücklich. Meistens.

Dieser Artikel erschien ursprünglich auf medium.

DEV Community: Jan Stamer

The DDD Hamburger for Go

The DDD Hamburger Overview 🍔

Go Example of the DDD Hamburger

The Presentation Layer as upper Bun 🍞

The Application Layer as Salad 🥗

The Domain Layer as Meat 🥩

The Infrastructure Layer as lower Bun 🍞

Assemble the Burger in the Main Function

Package Structure for the DDD Hamburger

Wrap Up of the DDD Hamburger 🍔

Baralga / baralga-app

Simple and lightweight time tracking for individuals and teams, for the cloud in the cloud.

Baralga

User Guide

Keyboard Shortcuts

Track Activities

Report Activities

Administration

Accessing the Web User Interface

Configuration

Credits

Simplest WebServer for Angular in 6 Lines

Angular Web Server in 6 Lines

Multi Stage Docker Build in 8 Lines

crossnative / cloudland-rps

Cloud native Rock, Paper, Scissors Workshop @CloudLand

Hello Cloudland! - Workshop Rock, Paper, Scissors

Contents

OpenAPI Backend Specifcation

Go Integration Tests using Testcontainers

Integration Test Setup

Database Setup using Testcontainers

Adding Database Migrations

Wrap Up

remast / go-for-testcontainers

Demo for Testcontainers in Go

Go for Testcontainers

Local Dev Setup with automatic HTTPS + WebSockets in 1 Line

Preparations

Set up the local hostname

Install Caddyserver

Starting the local dev server with HTTPs and WebSockets

Using the Caddyfile

What's next?

Turbo Streams powered by Go WebSockets

remast / go_websocket_turbo

Simple example for using Turbos Streams in Go with the Gorilla WebSocket toolkit.

Go Example for TurboStreams over WebSockets

Frontend

Server

5 Great Go Resources

Awesome Go

avelino / awesome-go

A curated list of awesome Go frameworks, libraries and software

Awesome Go

Go Training Courses and Material by Ardanlabs

ardanlabs / gotraining

Go Training Class Material :

Go Training

Learn More

Index

Purchase Video / Book

Our Experience

Idiomatic Go Resources

Idiomatic Go Resources. Every time the discussion comes up… | by Damian Gryski | Medium

Damian Gryski ・ Jul 31, 2020 ・ Medium

Security Related Resources for Go

guardrailsio / awesome-golang-security

Awesome Golang Security resources 🕶🔐

Contents

Tools

Web Framework Hardening

Libraries

Static Code Analysis

Style Guides for Go

dgryski / awesome-go-style

A collection of Go style guides

The Classics

Googler Talks

Damian Gryski ・ Jul 31, 2020 ・
Medium