DEV Community: Thibault NORMAND

Bridging Security and Reliability

Thibault NORMAND — Tue, 02 Jun 2026 15:45:42 +0000

Using threat modelling to make system dependability observable, testable, and actionable

Executive summary

Security and Reliability address system degradation. Security addresses degradation from intentional actions, such as denial-of-service attacks, while reliability addresses degradation from failure, load, dependency behaviour, operational change, or complexity. The underlying engineering question is the same: which critical system property can degrade, how would users experience that degradation, how would we detect it, and what controls would prevent, contain, or recover from it?

This document proposes a practical way to bridge the two disciplines: anchor analysis on Critical User Journeys, express expected behaviour through SLOs and SLIs, use RAMSS to ensure coverage across dependability dimensions, and adapt PASTA-style threat modelling to reliability scenarios. The goal is not to merge Security and Reliability into one generic practice. The goal is to reuse the strongest habits of each discipline: security's adversarial modelling and reliability's production-oriented measurement, validation, and recovery loops.

The most useful outcome is a shared model of degradation scenarios. A degradation scenario links a critical user journey to a concrete reliability or security threat, the system weakness that makes it possible, the signal that would detect it, the objective it would violate, and the mitigation or experiment that would validate the control. This makes risk easier to discuss with engineering teams because it connects abstract concerns to user impact, SLO burn, business loss, and testable remediation.

1. The problem: two disciplines, one degradation model

After working in both Reliability and Security, I found that the two domains share much in common: both focus on objectives, weaknesses, control effectiveness, incident response, prioritisation, and residual risk, but often use different rituals, terminology, metrics, and boundaries.

This separation causes friction. Security may identify abuse paths without tying them to reliability outcomes; reliability may find failure modes without Security's adversarial modelling. This leads to duplicated effort and missed coverage. Different root causes, like bot traffic spikes or dependency outages, may still degrade the same Critical User Journey.

The distinction between the disciplines still matters. Security generally models intentional behaviour and assumes an adaptive opponent. Reliability generally models failure, load, complexity, and recovery in production systems. But those differences should be treated as complementary perspectives rather than organisational boundaries. A mature dependability practice should ask both questions: what could fail, and how could the system be forced to fail?

2. Thesis: model degradation from the user journey outward

This document argues that anchoring reliability engineering in security-style threat modelling, focused on Critical User Journeys and SLOs, makes reliability risks explicit and actionable. Linking threat modelling with SLOs turns abstract reliability concerns into specific, testable scenarios.

This approach is useful because many reliability conversations fail at the same point: they identify a possible weakness, but the remediation is not prioritised because the impact is unclear, the signal is weak, or the owner is unconvinced. A threat-model-style workflow forces the discussion to become specific: which journey is affected, which objective is at risk, which failure path is plausible, which existing controls reduce the risk, which signal proves the risk is materialising, and which experiment validates the mitigation?

3. Principles for integrating Security and Reliability

Start with the user journey, not the component. A database, queue, cache, or service is important only because it supports a user or service journey. The journey defines the impact of degradation.
Treat previous incidents as hostile scenarios. Incident replays are a reliability analogue to adversarial simulation. They show how real systems break and where controls failed or were missing.
Measure controls, not only the absence of failure. A lack of alerts does not prove security, and a lack of incidents does not prove reliability. Good SLIs should measure the system property or control that matters.
Prefer testable scenarios over abstract risk statements. A scenario that can be validated through load testing, chaos engineering, gamedays, abuse simulation, or incident replay is more useful than a risk that only exists in a document.
Preserve domain expertise. The objective is collaboration, not dilution. Security should keep its knowledge of adversarial behaviour and abuse paths; Reliability should keep its knowledge of production behaviour, recovery, and SLO management.

4. Critical User Journeys as the shared anchor

A Critical User Journey (CUJ) is a high-impact workflow that expresses what the system must allow a user, customer, service, or operator to accomplish. At the product level, a CUJ reflects the end-user experience. At the service level, it reflects the experience of internal or external clients who depend on the service. In both cases, the CUJ provides the context required to define meaningful objectives.

CUJs bridge Security and Reliability. Both disciplines can reason about degradation affecting the same journey: Reliability considers failures and outages, while Security considers abuse and compromise. The shared anchor prevents both teams from siloed control optimisation.

From each CUJ, teams should derive SLOs and SLIs. SLOs describe the expected service level for the journey. SLIs provide evidence that the journey's healthy or degrading. The important design choice is to define SLIs that reflect the journey and its controls, not only infrastructure-level symptoms.

5. RAMSS: a coverage model for dependability

To avoid designing observability around only one dimension of dependability, I adapt the RAMS framework by adding Security, resulting in RAMSS: Reliability, Availability, Maintainability, Safety, and Security. 'Reliability' means uniform performance to specification, 'Availability' means readiness for correct service, 'Maintainability' means ease of repair or modification, 'Safety' means prevention of harm, and 'Security' means protection against intentional attacks. The purpose of RAMSS is not to create more metrics. It is to force an explicit discussion about which system properties matter for a given CUJ and which signals prove that those properties are protected.

Reliability	The ability to perform the expected function correctly over time.	Does the journey complete correctly under expected conditions?
Availability	The ability to remain reachable and usable when needed.	Can users or dependent services access the journey when they need it?
Maintainability	The ability to repair, change, roll back, or operate the system safely and quickly.	Can teams recover or modify the system without prolonged degradation?
Safety	The ability to avoid unacceptable harm to people, assets, customers, or the business during the system lifecycle.	Can failures occur without causing unacceptable harm or irreversible impact?
Security	The ability to resist, detect, contain, and recover from intentional misuse or attack.	Can the journey withstand abuse or compromise attempts without violating its objectives?

6. Worked example: completing a payment in a shop

The following example shows how the method works end-to-end. The CUJ is: a customer completes a payment in the shop.

A possible SLO for this journey could be: 99.95% of legitimate payment attempts are authorised or declined with a correct, user-visible outcome within the expected latency window over a 30-day period. The exact target should be adapted to product requirements, but the SLO must be user-centred: the customer either pays successfully or receives a correct and timely decline. Silent failure, duplicate charge risk, unexplained timeout, or prolonged pending state are all forms of degradation.

The same CUJ can degrade due to reliability or security issues. For example, payment provider timeouts trigger retries, causing checkout failures, while bot-driven abuse campaigns trigger fraud controls, both resulting in failed or untrustworthy payment journeys.

Reliability	Payment outcome correctness rate	Percentage of payment attempts that reach a correct terminal state: authorised, declined, cancelled, or safely expired.
Availability	Checkout payment endpoint availability	Percentage of time the payment path is reachable and able to accept legitimate payment attempts.
Maintainability	Rollback and recovery time for payment degradation	Time required to detect, mitigate, and recover from a payment-path regression, including rollback readiness.
Safety	Unsafe financial outcome rate	Number or rate of payment failures that create duplicate charge risk, incorrect settlement, loss of funds, or unresolved customer impact.
Security	Abuse-resilient payment completion rate	Percentage of legitimate payment attempts completed while abuse controls, fraud decisions, and rate limits are functioning within defined thresholds.

The Security SLI deliberately does not measure “payments without alerts.” Absence of alerts is weak evidence: the system may simply be blind. A better signal measures whether legitimate users can complete the journey while the relevant security controls are operating correctly. This correlates the security signal with user impact and control effectiveness.

7. Reliability-adapted PASTA

PASTA, the Process for Attack Simulation and Threat Analysis, is a structured threat modelling approach. It is organised around objectives, system decomposition, threat analysis, weakness analysis, modelling, impact, and mitigation. For reliability, the identical structure can be adapted to model failure and degradation paths. In this context, 'system decomposition' means breaking a system into components for analysis, while 'threat analysis' examines potential paths to failure or attack. The adapted process below should be treated as the main workflow.

1. Define objectives	Identify business goals, critical user journeys, and SLOs.	Payment CUJ: legitimate customers can complete payment with correct outcome and acceptable latency.
2. Define technical scope	List systems, boundaries, dependencies, and ownership.	Checkout service, payment orchestration, fraud decisioning, provider APIs, queues, database, feature flags.
3. Decompose the system	Map request flow, state transitions, data dependencies, and critical paths.	Customer submits payment; checkout calls fraud service; payment provider authorises; order state is committed.
4. Identify degradation threats	List plausible reliability and security scenarios that could degrade the CUJ.	Provider timeout, retry storm, schema drift, bad rollout, bot-driven traffic spike, fraud service latency, throttling.
5. Analyse weaknesses and controls	Map scenarios to existing controls and gaps.	Idempotency, circuit breakers, rate limits, backoff, canaries, rollback, abuse detection, dashboards, runbooks.
6. Model impact and prioritise	Rank by SLO impact, blast radius, likelihood, detectability, and confidence in mitigation.	Retry storm has high SLO burn and high blast radius if idempotency and backoff are incomplete.
7. Validate and improve	Test mitigations through load tests, chaos experiments, abuse simulation, gamedays, or incident replay.	Replay provider timeout; verify retries do not amplify load; confirm alerts fire and rollback path is clear.

This scoring should not rely on a generic DREAD-style score without context. For reliability, prioritisation is stronger when it uses SLO impact, expected error-budget burn, blast radius, incident history, operational complexity, detectability, and mitigation confidence. These criteria are closer to how reliability risk is experienced in production.

8. Incident replay as reliability threat modelling

Previous incidents are one of the best sources of reliability threats. Instead of treating an incident only as a postmortem, teams can replay it as if it were an adversarial scenario. This means extracting the invariants that made the incident possible: which assumption failed, which dependency behaved unexpectedly, which control was missing, which signal appeared too late, and which mitigation would have broken the cascade.

For example, suppose a storage dependency experienced a cascading failure after a software update. A reliability threat model would break the incident into nodes: rollout begins, incompatible version reaches multiple nodes, replication lag increases, client retries amplify load, error rate crosses the SLO threshold, and recovery is delayed because rollback ownership is unclear. Each node becomes a place to ask: what control would prevent this step, detect it earlier, contain its blast radius, or accelerate recovery?

The result is a reusable artefact: a failure tree or degradation path that engineers, security reviewers, reliability engineers, and product owners can understand. It makes the risk concrete and gives teams a basis for targeted validation by canary policy, staggered rollout, dependency isolation, alert tuning, or gameday exercises.

9. Intended outcomes

The value of this approach should be measured, not assumed. A successful adoption must produce practical improvements such as:

A clearer inventory of CUJs and the SLOs that protect them.
More reliability and security risks are expressed as concrete degradation scenarios.
Better alignment between security controls, reliability controls, and user-visible outcomes.
Higher-quality SLIs that measure control effectiveness rather than only symptoms or absence of alerts.
More targeted validation by load tests, chaos experiments, abuse simulation, pentests, and gamedays.
Fewer unowned risks because each scenario is tied to a journey, objective, control, signal, and mitigation owner.
Improved incident learning because previous incidents become reusable threat models instead of isolated historical records.

10. What this means for resilience

Resilience is often discussed as if it belonged only to reliability, but it applies across dependability dimensions. A system can be resilient to load, dependency failure, abuse, compromise, operational error, privacy degradation, or serviceability problems. The key question is: resilient against degradation of which system property?

In this framing, resilience is the ability to preserve or recover an acceptable system objective after degradation begins. For reliability, that may mean absorbing load spikes or entering a degraded mode that protects the most important CUJ. For security, it may mean containing a denial-of-service attempt, preserving legitimate traffic, rotating compromised credentials, or recovering trusted state. In both cases, resilience is not an abstract virtue. It is a designed and tested capability tied to a specific objective.

11. Recommended pilot

The next step should be a small pilot, not an organisation-wide rollout. Select one high-impact CUJ with known reliability and security relevance, such as checkout, ingestion, authentication, search, or alert delivery. Assemble a small group of service owners, reliability engineers, security engineers, and product stakeholders.

The pilot should produce five artefacts:

A CUJ definition and the SLOs that represent an acceptable user experience.
A RAMSS coverage table showing which dependability dimensions matter for that journey.
A reliability-adapted PASTA model listing degradation scenarios, weaknesses, controls, and gaps.
A validation plan using at least one experiment: load test, chaos experiment, incident replay, abuse simulation, pentest, or gameday.
A remediation plan with owners, priority, expected SLO impact, and follow-up review date.

The pilot should also define success criteria before it begins. Useful criteria include the number of previously untracked degradation scenarios discovered, the number of weak or missing SLIs improved, the number of mitigations accepted by service owners, the reduction in time needed to reason about incident recurrence, and the quality of validation evidence produced. Without success criteria, the pilot risks becoming another process exercise rather than a proof of value.

12. Scaling without creating process drag

If the pilot works, scaling should be incremental. The main risk is turning a useful thinking tool into a heavyweight review process that teams avoid. To prevent that, the method should be standardised lightly: provide templates, examples, facilitation guidance, and common scoring criteria, but leave teams room to adapt the workflow to their architecture and operating model.

Standardise vocabulary. Agree on definitions for CUJ, SLO, SLI, degradation scenario, control, mitigation, and validation.
Create reusable templates. Provide a short RAMSS table and a concise PASTA worksheet that teams can complete during design reviews or incident follow-ups.
Train through examples. Use real incidents and real CUJs rather than abstract exercises.
Appoint domain champions. Security and reliability champions can help teams apply the method without centralising all decisions.
Automate where useful. Tooling can help map services to SLOs, dashboards, dependencies, incidents, and ownership, but automation should support the model rather than replace engineering judgment.

Conclusion

Security and Reliability should not be treated as identical disciplines, but they should be treated as complementary ways to reason about system dependability. Security brings adversarial thinking, abuse modelling, and control analysis. Reliability brings production feedback, SLO discipline, recovery practices, and validation by operational testing. When both are anchored to Critical User Journeys, they can describe different paths to the same user-visible degradation and prioritise work using shared evidence.

The practical recommendation is to start with one CUJ, define its objectives, plot its RAMSS dimensions, model degradation scenarios using a reliability-adapted PASTA workflow, and validate the most important mitigations. The outcome should be more than a document. It should be a better engineering conversation: specific, evidence-based, tied to user impact, and actionable enough to improve the system.

The Identity Puzzle: the Crucial Difference Between Access Tokens and ID Tokens

Thibault NORMAND — Sun, 23 Jun 2024 11:38:47 +0000

In the real world

Let's start with a real-world analogy. Imagine a flight ticket you previously bought as an access token authorising you to board the plane for your trip. On the other hand, an ID card functions as your ID token, a document that proves an authority has authenticated your identity. Another example could be a concert ticket, your access token to the event, and your driver's license, your ID token.

Just as you can't board a plane with only your ID card without buying a flight ticket, you can't use the ticket to prove your identity. Like cards and tickets, both tokens are created to serve specific purposes.

In the context of our analogy, the access token is your right to board the plane, while the ID token is your proof of identity. Using the person's name of the flight ticket bearer and proving that the name identifies you makes your intent to board the plane legit and authenticated.

ID vs Access

In the digital world, ID tokens are your digital identity, a secure container that holds all the information you agree to share with the identity authority. They're a digital version of your ID card. Just as you present your ID card to authenticate your identity, the entity initiating the authentication process uses ID tokens to retrieve authenticated identity details.

Access tokens are not just proof of authorisation but the key to your digital world. They contain identity references for the user and the software authorised to access a specific service. This proof of authorisation is forged by evaluating the identity intent, is valid for a certain period, and is closely linked to service usage only.

The ID tokens concern the user authentication process, demonstrating how the system knows you, while Access tokens concern an authorisation decision proving that the intent has been authorised.

How are authenticated ID cards?

When you present your ID card on request, it tends to authenticate you as the identity claimed by your ID card, representing proof of ownership. The verifier checks if the ID card looks legitimate to forge a proof of authority and compares the associated picture with you in person, acting as proof of life.

In essence, the ID card "distributed" authentication process is based on the following:

It looks legit (valid symbols, colours, format, etc.)
The picture looks like the bearer's face
The card has not expired

The term 'distributed' here refers to the fact that the authentication process is not solely dependent on one factor but rather a combination of factors, making it more secure and reliable.

To build trust around an ID card, you must create a card that looks legit to bypass the proof of authority, change the picture to ensure the holder looks like the associated picture, and ensure it's not expired.

How are authenticated ID tokens?

In the digital world where ID cards are translated as ID tokens, you will reduce the 'distributed' authentication process to 'it looks legit', meaning the token is cryptographically signed with a key from a trusted authority who delivered the ID token.
In other words, the authentication process is 'distributed' because it relies on multiple factors, such as the token's legitimacy and the authority verifier trust that issued it, to verify the user's identity.

The authentication process uses:

A cryptographic signature verification
The token is usable (expiration, type, etc.)

To build trust around an ID token, you must ensure that you can sign it with one of the private keys associated with public key trust. By definition, an ID token is vulnerable to bearer spoofing, as it's not possible to provide an equivalent to a picture matching check like for an ID card to authenticate the bearer.

The purpose of ID tokens is to authenticate an identity by trusting the authority who generated it.

How are authenticated Access tokens?

Access tokens are opaque tokens, meaning they should not have any meaning for the software that received them as proof of authorisation.
The access token represents a sealed authorisation decision valid for a given time and associated with a validated intent.

The authentication process uses:

Proof of authority knowledge
A cryptographic signature verification
An identity cryptographic binding
The token is usable (expiration, type, etc.)
The token matches the expected intent

To build trust around an opaque Access Token, you must ensure the token is known by the authority that forged it. Using digital signature verification to prove the token provenance only proves that the private key used to sign the token is the same as the authority. This weakness is why many providers don't use rich tokens but simple pseudo-random strings as access tokens. To mirror the ID card picture-based comparison check, an Access Token can have an identity binding to confirm that the access token owner and the client using it are legitimate. The token is also subject to acceptance time and intent validation to ensure that it is used in the appropriate context that represents the proof of authorisation.

The purpose of an access token is to authenticate an authorisation decision.

Why is using ID tokens as proof of authorisation, not a good idea?

Let's consider that you are authorising people based on their ID card information only:

How would you prove to the boarding control that you paid for this flight?
What can happen if someone steals your ID card and looks like you?

Building an authorisation model solely based on claims identified in the ID Tokens can immediately expose you to the risk of identity spoofing and authorisation bypass. We saw that ID Tokens act as proof of authentication and don't serve an authorisation purpose. When you use ID Tokens as service access authorisation tokens, you open your service to identity impersonation. Secondly, it forces you to distribute your authorisation policy to each service that consumes your ID Tokens as you evaluate the access decision based on the presented identity just in time.

This vulnerability occurs when you lack 'bearer authentication', similar to the picture on your ID card. Bearer authentication means that the token holder is considered the legitimate bearer of the token, which is why it's called bearer authorisation. This flaw is similar to API key authentication, which is, in fact, an API Key bearer authorisation. The identity is not proven but trusted by data associated with the provided token. Merely presenting API keys or ID Tokens is enough to authenticate as a legitimate bearer, posing significant security risks that you must be aware of.

For example, if someone gets hold of your API Key or ID token, they could pretend to be you or access your private data without permission. This emphasises the significance of comprehending and safeguarding your tokens and using the correct type for the intended purpose.

Conclusion

A lot of the confusion arises from the fact that we can utilise the same technical encoding for both tokens in the digital realm. However, even if we use JWT for both, with shared claims, each token is constructed for a specific purpose.

Use ID Tokens for authentication-related use cases only

Transfer identity from authority to client
Transfer identity from client to another authority for federation
Exchange your identity to another token

Use Access Tokens for authorisation-related use cases only

Access a service/resource
Represent an intent authorisation decision

In conclusion, understanding the crucial difference between access and ID tokens is essential for securely navigating the digital world. Access tokens serve as proof of authorisation and are linked to specific services, while ID tokens authenticate a user's identity and are equivalent to digital ID cards. Both tokens undergo authentication processes, with access tokens focused on sealed authorisation decisions and ID tokens utilising cryptographic signatures to verify legitimacy. By grasping the distinct roles of these tokens, individuals and organisations can enhance security and data protection in the digital space.

How I write micro-service (part 3)

Thibault NORMAND — Thu, 06 Jun 2019 10:29:13 +0000

Repositories as Hexagonal architecture defines this component is a persisence adapter. It's a technical implementation of a models provider. It could be:

A local provider: database, file;
A remote provider: another micro-service.

I'm used to splitting repository implementations in a dedicated package according to technical backend used.

$ mkdir internal/repositories
$ touch internal/repositories/api.go
$ mkdir internal/repositories/pkg
$ mkdir internal/repositories/pkg/postgresql
$ mkdir internal/repositories/pkg/mongodb

The filesystem should be like it follows:

 + internal
   + helpers
     - id.go
     - password.go
     - time.go
   + models
     - user.go
   + repositories
     - api.go
     + pkg
       + postgresql
       + mongodb

Adapter Contract

But before starting to implements the persistence adapter, and in order to comply with the dependency loose coupling of The Clean Architecture, we have to declare the adapter interface first. So that all implementations must be compliant with.

For example api.go, must contain only repository contract.

package repositories

import (
    "context"

    "go.zenithar.org/pkg/db"
    "go.zenithar.org/spotigraph/internal/models"
)

// UserSearchFilter represents user entity collection search criteria
type UserSearchFilter struct {
    UserID    string
    // Principal hash value must be used
    Principal string
}

// User describes user repository contract
type User interface {
    // Create a user
    Create(ctx context.Context, entity *models.User) error
    // Retrieve an user
    Get(ctx context.Context, id string) (*models.User, error)
    // Update selectively an user instance
    Update(ctx context.Context, entity *models.User) error
    // Delete an user instance
    Delete(ctx context.Context, id string) error
    // Search for user entities using filter, pagination and sorts
    Search(ctx context.Context, filter *UserSearchFilter, p *db.Pagination, s *db.SortParameters) ([]*models.User, int, error)
    // Retrieve an user by principal hash value
    FindByPrincipal(ctx context.Context, principal string) (*models.User, error)
}

Don't forget to pass context.Context as far as possible in your calls, in case of request cancellation bound to HTTP request.
When the caller cancel the HTTP request, the SQL request must be cancelled too, etc.

Our repository implementations must handle entities so that principal has to be hashed before using it (for example FindByPrincipal)

Unit Tests

For testing, I've added this go compiler generate step in api.go, to build User mocks from interface type definition.

//go:generate mockgen -destination test/mock/user.gen.go -package mock go.zenithar.org/spotigraph/internal/repositories User

Repository mocks will be used by service unit tests.

Integration Tests

These tests are executed using a real backend.

For example, the User integration test generator creates a full test that create, read, update, delete User using the persistence adapter implementation.

Integration tests will be evicted from default test target using the build tag integration

//+build integration

package specs

import (
    "context"
    "testing"

    "github.com/google/go-cmp/cmp"
    . "github.com/onsi/gomega"

    "go.zenithar.org/pkg/db"
    "go.zenithar.org/spotigraph/internal/models"
    "go.zenithar.org/spotigraph/internal/repositories"
)

// User returns user repositories full test scenario builder
func User(underTest repositories.User) func(*testing.T) {
    return func(t *testing.T) {
        // Flag this test that could be run in parallel
        t.Parallel()

        // Prepare gomega matcher
        g:= NewGomegaWithT(t)

        // Stub context
        ctx:= context.Background()

        // Prepare a new entity
        created:= models.NewUser("toto@foo.com")
        g.Expect(created).ToNot(BeNil(), "Newly created entity should not be nil")

        // Create the entity using repository
        err:= underTest.Create(ctx, created)
        g.Expect(err).To(BeNil(), "Error creation should be nil")

        // -------------------------------------------------------------------------

        // Retrieve by id from repository
        saved, err:= underTest.Get(ctx, created.ID)
        g.Expect(err).To(BeNil(), "Retrieval error should be nil")
        g.Expect(saved).ToNot(BeNil(), "Saved entity should not be nil")

        // Compare objects
        g.Expect(cmp.Equal(created, saved)).To(BeTrue(), "Saved and Created should be equals")

        // Retrieve by non-existent id
        nonExistent, err:= underTest.Get(ctx, "non-existent-id")
        g.Expect(err).ToNot(BeNil(), "Error should be raised on non-existent entity")
        g.Expect(err).To(Equal(db.ErrNoResult), "Error ErrNoResult should be raised")
        g.Expect(nonExistent).To(BeNil(), "Non-existent entity should be nil")

        // Retrieve by principal
        savedPrincipal, err:= underTest.FindByPrincipal(ctx, created.Principal)
        g.Expect(err).To(BeNil(), "Retrieval error should be nil")
        g.Expect(savedPrincipal).ToNot(BeNil(), "Saved entity should not be nil")

        // Compare objects
        g.Expect(cmp.Equal(created, savedPrincipal)).To(BeTrue(), "SavedPrincipal and Created should be equals")

        // -------------------------------------------------------------------------

        // Update an entity
        saved, err = underTest.Get(ctx, created.ID)
        g.Expect(err).To(BeNil(), "Retrieval error should be nil")
        g.Expect(saved).ToNot(BeNil(), "Saved entity should not be nil")

        // Update properties

        // Update with repository
        err = underTest.Update(ctx, saved)
        g.Expect(err).To(BeNil(), "Update error should be nil")

        // Retrieve from repository to check updated properties
        updated, err:= underTest.Get(ctx, created.ID)
        g.Expect(err).To(BeNil(), "Retrieval error should be nil")
        g.Expect(updated).ToNot(BeNil(), "Saved entity should not be nil")

        // Compare objects
        g.Expect(cmp.Equal(created, updated)).To(BeTrue(), "Saved and Updated should be equals")

        // -------------------------------------------------------------------------

        // Remove an entity
        err = underTest.Delete(ctx, created.ID)
        g.Expect(err).To(BeNil(), "Removal error should be nil")

        // Retrieve from repository to check deletion
        deleted, err:= underTest.Get(ctx, created.ID)
        g.Expect(err).ToNot(BeNil(), "Deletion error should not be nil")
        g.Expect(err).To(Equal(db.ErrNoResult), "Error ErrNoResult should be raised")
        g.Expect(deleted).To(BeNil(), "Deleted entity should be nil")

        // Remove a non-existent entity
        err = underTest.Delete(ctx, "non-existent-id")
        g.Expect(err).ToNot(BeNil(), "Removal error should not be nil")
        g.Expect(err).To(Equal(db.ErrNoModification), "Error ErrNoModification should be raised")
    }
}

Persistence implementations

During many years, I have enhanced, rebuild approximatively 30 times my own framework.
All implementation are using it as foundation, you are free to pickup what you need form it.

go.zenithar.org/pkg

It's multi-sub-module Golang repository, so you don't need to pull the entire repository, everything is
splitted and versionned independently in a minimal package.

PostgreSQL

First of all, we need to prepare the PostgreSQL package.

$ mkdir internal/repositories/pkg/postgresql/migrations
$ touch internal/repositories/pkg/postgresql/setup.go
$ touch internal/repositories/pkg/postgresql/table_user.go

Schema migrations handling

I'm use to put all the database specific preparation setup in a file nammed setup.go.
This is used to declares all table name constants, but also database migrations that will be embedded in the final
atifact using packr.

Why packr ? Heu because ... not go-bin-data ... ?

package postgresql

import (
    "github.com/gobuffalo/packr"
    "github.com/google/wire"
    "github.com/jmoiron/sqlx"
    "golang.org/x/xerrors"

    // Load postgresql drivers
    _ "github.com/jackc/pgx"
    _ "github.com/jackc/pgx/pgtype"
    _ "github.com/jackc/pgx/stdlib"
    _ "github.com/lib/pq"

    migrate "github.com/rubenv/sql-migrate"
    db "go.zenithar.org/pkg/db/adapter/postgresql"
)

// ----------------------------------------------------------

var (
    // UserTableName represents users collection name
    UserTableName = "users"
)

// ----------------------------------------------------------

//go:generate packr

// migrations contains all schema migrations
var migrations = &migrate.PackrMigrationSource{
    Box: packr.NewBox("./migrations"),
}

// CreateSchemas create or updates the current database schema
func CreateSchemas(conn *sqlx.DB) (int, error) {
    // Migrate schema
    migrate.SetTable("schema_migration")

    // Run the schema migration from embedded resource
    n, err := migrate.Exec(conn.DB, conn.DriverName(), migrations, migrate.Up)
    if err != nil {
        return 0, xerrors.Errorf("postgresql: could not migrate sql schema, applied %d migrations :%w", n, err)
    }

    return n, nil
}

And database schema migrations like it follows :

-- +migrate Up
CREATE TABLE IF NOT EXISTS chapters (
    id          VARCHAR(32) NOT NULL PRIMARY KEY,
    name        VARCHAR(50) NOT NULL,
    meta        JSON        NOT NULL,
    leader_id   VARCHAR(32) NOT NULL,
    member_ids  JSON        NOT NULL
);

-- +migrate Down
DROP TABLE chapters;

There are 2 "annotations" for migration direction, it is used by your migration tools to apply the correct
SQL script according the upgrade or downgrade process.

Meticulous people should have noticed that I load 2 different PostgreSQL drivers (pq, and pgx).

The rule of internal database migrations says :

Use automigration for integration tests only
Never try to apply automatically migrations on service starts, in case of a cluster there will have code running not compatible with the updated persistence schema
Know what you are doing

User Persistence Adapter

For example as User persistence adapter for PostgreSQL in table_user.go in internal/repositories/pkg/postgresql

package postgresql

import (
    "context"
    "strings"

    "go.zenithar.org/pkg/db"
    "go.zenithar.org/pkg/db/adapter/postgresql"
    "go.zenithar.org/spotigraph/internal/models"
    "go.zenithar.org/spotigraph/internal/repositories"

    sq "github.com/Masterminds/squirrel"
    "github.com/jmoiron/sqlx"
    "golang.org/x/xerrors"
)

type pgUserRepository struct {
    adapter *postgresql.Default
}

// NewUserRepository returns a Postgresql user management repository instance
func NewUserRepository(session *sqlx.DB) repositories.User {
    // Default columns to retrieve
    defaultColumns := []string{"user_id", "principal", "secret", "creation_date"}

    // Sortable columns for criteria
    sortableColumns := []string{"user_id", "principal", "creation_date"}

    // Initialize repository
    return &pgUserRepository{
        adapter: postgresql.NewCRUDTable(session, "", UserTableName, defaultColumns, sortableColumns),
    }
}

// -----------------------------------------------------------------------------

func (r *pgUserRepository) Create(ctx context.Context, entity *models.User) error {
    // Validate entity first
    if err := entity.Validate(); err != nil {
        return xerrors.Errorf("postgresql: unable to validate entity, creation aborted : %w", err)
    }

    return r.adapter.Create(ctx, entity)
}

func (r *pgUserRepository) Get(ctx context.Context, realmID, id string) (*models.User, error) {
    var entity models.User

    // Delegate to my own postgresql adpater
    if err := r.adapter.WhereAndFetchOne(ctx, map[string]interface{}{
        "user_id":  id,
    }, &entity); err != nil {
        return nil, xerrors.Errorf("postgresql: unable to query database: %w", err)
    }

    return &entity, nil
}

func (r *pgUserRepository) Update(ctx context.Context, entity *models.User) error {
    // Validate entity first
    if err := entity.Validate(); err != nil {
        return xerrors.Errorf("postgresql: unable to validate entity, update aborted : %w", err)
    }

    // Only update allowed attributes, never, never, did I say never ?
    // always the full object.
    if err := r.adapter.Update(ctx, map[string]interface{}{
        "secret": entity.Secret,
    }, map[string]interface{}{
        "user_id":  entity.ID,
    }); err != nil {
        return xerrors.Errorf("postgresql: unable to update entity: %w", err)
    }

    // No error
    return nil
}

func (r *pgUserRepository) Delete(ctx context.Context, realmID, id string) error {

    // Delegate to adapter
    if err := r.adapter.RemoveOne(ctx, map[string]interface{}{
        "user_id":  id,
    }); err != nil {
        return xerrors.Errorf("postgresql: unable to remove entity: %w", err)
    }

    // No error
    return nil
}

func (r *pgUserRepository) Search(ctx context.Context, filter *repositories.UserSearchFilter, pagination *db.Pagination, sortParams *db.SortParameters) ([]*models.User, int, error) {
    var results []*models.User

    // Delegate to adapter
    count, err := r.adapter.Search(ctx, r.buildFilter(filter), pagination, sortParams, &results)
    if err != nil {
        return nil, count, xerrors.Errorf("postgresql: unable to list entities: %w", err)
    }

    // Standardize the error on no result to be coherent with all implementations
    if len(results) == 0 {
        return results, count, db.ErrNoResult
    }

    // Return results and total count
    return results, count, nil
}

func (r *pgUserRepository) FindByPrincipal(ctx context.Context, realmID string, principal string) (*models.User, error) {
    var entity models.User

    // Delegate to adapter
    if err := r.adapter.WhereAndFetchOne(ctx, map[string]interface{}{
        "principal": principal,
    }, &entity); err != nil {
        return nil, xerrors.Errorf("postgresql: unable to retrieve entity: %w", err)
    }

    return &entity, nil
}

// -----------------------------------------------------------------------------

func (r *pgUserRepository) buildFilter(filter *repositories.UserSearchFilter) interface{} {
    if filter != nil {
        clauses := sq.Eq{
            "1": "1",
        }

        if len(strings.TrimSpace(filter.UserID)) > 0 {
            clauses["user_id"] = filter.UserID
        }
        if len(strings.TrimSpace(filter.Principal)) > 0 {
            clauses["principal"] = filter.Principal
        }

        return clauses
    }

    return nil
}

Squad Persistence Adapter

If the model can't fit directly in the persistence adapter you have to translate it as needed

With given table schema :

-- +migrate Up
CREATE TABLE IF NOT EXISTS squads (
    id                  VARCHAR(32) NOT NULL PRIMARY KEY,
    name                VARCHAR(50) NOT NULL,
    meta                JSON        NOT NULL,
    product_owner_id    VARCHAR(32) NOT NULL,
    member_ids          JSON        NOT NULL
);

-- +migrate Down
DROP TABLE squads;

In this case, I choose to use JSON column type to handle Squad:members association and Squad:metadata. So I had to translate member_ids array as a JSON object before writing to the database, a reading from JSON to an array when decoding.

My persistence adapter must transform the model before each SQL operations.

package postgresql

import (
    "context"
    "encoding/json"
    "strings"

    api "go.zenithar.org/pkg/db"
    db "go.zenithar.org/pkg/db/adapter/postgresql"
    "go.zenithar.org/spotigraph/internal/models"
    "go.zenithar.org/spotigraph/internal/repositories"

    sq "github.com/Masterminds/squirrel"
    "github.com/jmoiron/sqlx"
    "github.com/pkg/errors"
)

type pgSquadRepository struct {
    adapter *db.Default
}

// NewSquadRepository returns an initialized PostgreSQL repository for squads
func NewSquadRepository(cfg *db.Configuration, session *sqlx.DB) repositories.Squad {
    // Defines allowed columns
    defaultColumns := []string{
        "id", "name", "meta", "product_owner_id", "member_ids",
    }

    // Sortable columns
    sortableColumns := []string{
        "name", "product_owner_id",
    }

    return &pgSquadRepository{
        adapter: db.NewCRUDTable(session, "", SquadTableName, defaultColumns, sortableColumns),
    }
}

// ------------------------------------------------------------

type sqlSquad struct {
    ID             string `db:"id"`
    Name           string `db:"name"`
    Meta           string `db:"meta"`
    ProductOwnerID string `db:"product_owner_id"`
    MemberIDs      string `db:"member_ids"`
}

func toSquadSQL(entity *models.Squad) (*sqlSquad, error) {
    meta, err := json.Marshal(entity.Meta)
    if err != nil {
        return nil, errors.WithStack(err)
    }

    members, err := json.Marshal(entity.MemberIDs)
    if err != nil {
        return nil, errors.WithStack(err)
    }

    return &sqlSquad{
        ID:             entity.ID,
        Name:           entity.Name,
        Meta:           string(meta),
        MemberIDs:      string(members),
        ProductOwnerID: entity.ProductOwnerID,
    }, nil
}

func (dto *sqlSquad) ToEntity() (*models.Squad, error) {
    entity := &models.Squad{
        ID:             dto.ID,
        Name:           dto.Name,
        ProductOwnerID: dto.ProductOwnerID,
    }

    // Decode JSON columns

    // Metadata
    err := json.Unmarshal([]byte(dto.Meta), &entity.Meta)
    if err != nil {
        return nil, errors.WithStack(err)
    }

    // Membership
    err = json.Unmarshal([]byte(dto.MemberIDs), &entity.MemberIDs)
    if err != nil {
        return nil, errors.WithStack(err)
    }

    return entity, nil
}

// ------------------------------------------------------------

func (r *pgSquadRepository) Create(ctx context.Context, entity *models.Squad) error {
    // Validate entity first
    if err := entity.Validate(); err != nil {
        return err
    }

    // Convert to DTO
    data, err := toSquadSQL(entity)
    if err != nil {
        return err
    }

    return r.adapter.Create(ctx, data)
}

func (r *pgSquadRepository) Get(ctx context.Context, id string) (*models.Squad, error) {
    var entity sqlSquad

    if err := r.adapter.WhereAndFetchOne(ctx, map[string]interface{}{
        "id": id,
    }, &entity); err != nil {
        return nil, err
    }

    return entity.ToEntity()
}

func (r *pgSquadRepository) Update(ctx context.Context, entity *models.Squad) error {
    // Validate entity first
    if err := entity.Validate(); err != nil {
        return err
    }

    // Intermediary DTO
    obj, err := toSquadSQL(entity)
    if err != nil {
        return err
    }

    return r.adapter.Update(ctx, map[string]interface{}{
        "name":             obj.Name,
        "meta":             obj.Meta,
        "product_owner_id": obj.ProductOwnerID,
        "member_ids":       obj.MemberIDs,
    }, map[string]interface{}{
        "id": entity.ID,
    })
}

MongoDB

Another example with a NoSQL persistence adapter.

package mongodb

import (
    "context"

    mongowrapper "github.com/opencensus-integrations/gomongowrapper"
    api "go.zenithar.org/pkg/db"
    db "go.zenithar.org/pkg/db/adapter/mongodb"
    "go.zenithar.org/spotigraph/internal/models"
    "go.zenithar.org/spotigraph/internal/repositories"
)

type mgoSquadRepository struct {
    adapter *db.Default
}

// NewSquadRepository returns an initialized MongoDB repository for squads
func NewSquadRepository(cfg *db.Configuration, session *mongowrapper.WrappedClient) repositories.Squad {
    return &mgoSquadRepository{
        adapter: db.NewCRUDTable(session, cfg.DatabaseName, SquadTableName),
    }
}

// ------------------------------------------------------------

func (r *mgoSquadRepository) Create(ctx context.Context, entity *models.Squad) error {
    // Validate entity first
    if err := entity.Validate(); err != nil {
        return err
    }

    return r.adapter.Insert(ctx, entity)
}

func (r *mgoSquadRepository) Get(ctx context.Context, id string) (*models.Squad, error) {
    var entity models.Squad

    if err := r.adapter.WhereAndFetchOne(ctx, map[string]interface{}{
        "id": id,
    }, &entity); err != nil {
        return nil, err
    }

    return &entity, nil
}

func (r *mgoSquadRepository) Update(ctx context.Context, entity *models.Squad) error {
    // Validate entity first
    if err := entity.Validate(); err != nil {
        return err
    }

    return r.adapter.Update(ctx, map[string]interface{}{
        "name":             entity.Name,
        "meta":             entity.Meta,
        "product_owner_id": entity.ProductOwnerID,
    }, map[string]interface{}{
        "id": entity.ID,
    })
}

func (r *mgoSquadRepository) Delete(ctx context.Context, id string) error {
    return r.adapter.Delete(ctx, id)
}

func (r *mgoSquadRepository) FindByName(ctx context.Context, name string) (*models.Squad, error) {
    var entity models.Squad

    if err := r.adapter.WhereAndFetchOne(ctx, map[string]interface{}{
        "name": name,
    }, &entity); err != nil {
        return nil, err
    }

    return &entity, nil
}

func (r *mgoSquadRepository) AddMembers(ctx context.Context, id string, users ...*models.User) error {
    // Retrieve squad entity
    entity, err := r.Get(ctx, id)
    if err != nil {
        return err
    }

    // Add user as members
    for _, u := range users {
        entity.AddMember(u)
    }

    // Update members only
    return r.adapter.Update(ctx, map[string]interface{}{
        "member_ids": entity.MemberIDs,
    }, map[string]interface{}{
        "id": entity.ID,
    })
}

func (r *mgoSquadRepository) RemoveMembers(ctx context.Context, id string, users ...*models.User) error {
    // Retrieve squad entity
    entity, err := r.Get(ctx, id)
    if err != nil {
        return err
    }

    // Remove user from members
    for _, u := range users {
        entity.RemoveMember(u)
    }

    // Update members
    return r.adapter.Update(ctx, map[string]interface{}{
        "member_ids": entity.MemberIDs,
    }, map[string]interface{}{
        "id": entity.ID,
    })
}

Never update a full object without controlling each keys, you must setup update function to update only updatable attributes.

Remote Adapter

Your adapter could be an external service which provides data and is called via a transport protocol (HTTP, gRPC, etc.)

Mocks could be used to simulate remote access during tests.

Running integration tests

In order to run integration tests with Golang, you must prepare a TestMain . This runner is responsible for building all related persistence adapter instance according to requested command line flag.

The test specification will be used to generate the full scenario test by passing the persistence adapter to the generator.

Obviously, all persistence adapter implementations should have the same behavior, validated by your test suite.

// +build integration

package integration

import (
    "context"
    "flag"
    "fmt"
    "math/rand"
    "os"
    "strings"
    "testing"
    "time"

    "go.uber.org/zap"
    "go.zenithar.org/pkg/log"
    "go.zenithar.org/pkg/testing/containers/database"
    "go.zenithar.org/spotigraph/internal/version"
)

var databases = flag.String("databases", "postgresql", "Repositories backend to use, splitted with a coma ','. Example: postgresql,mongodb,rethinkdb")

func init() {
    flag.Parse()

    ctx := context.Background()

    // Prepare logger
    log.Setup(ctx, &log.Options{
        Debug:     true,
        AppName:   "spotigraph-integration-tests",
        AppID:     "123456",
        Version:   version.Version,
        Revision:  version.Revision,
        SentryDSN: "",
    })

    // Initialize random seed
    rand.Seed(time.Now().UTC().Unix())

    // Set UTC for all time
    time.Local = time.UTC
}

func testMainWrapper(m *testing.M) int {
    if testing.Short() {
        fmt.Println("Skipping integration tests")
        return 0
    }

    log.Bg().Info("Initializing test DB for integration test (disable with `go test -short`)")

    ctx := context.Background()
    backends := strings.Split(strings.ToLower(*databases), ",")

    for _, back := range backends {
        switch back {
        case "postgresql":
            // Initialize postgresql
            cancel, err := postgreSQLConnection(ctx)
            if err != nil {
                log.Bg().Fatal("Unable to initialize repositories", zap.Error(err))
            }
            defer func() {
                cancel()
            }()
        default:
            log.Bg().Fatal("Unsupported backend", zap.String("backend", back))
        }
    }

    defer func() {
        database.KillAll(ctx)
    }()

    return m.Run()
}

// TestMain is the test entrypoint
func TestMain(m *testing.M) {
    os.Exit(testMainWrapper(m))
}

It will initialize a Docker container running a PostgreSQL server using ory-am/dockertest. It will automate the database server deployment when executing the tests in local and could be used also on a remote database instance (when managing database execution with your CI pipeline for example).

package database

import (
    "fmt"
    "log"

    // Load driver if not already done
    _ "github.com/lib/pq"

    "github.com/dchest/uniuri"
    dockertest "gopkg.in/ory-am/dockertest.v3"

    "go.zenithar.org/pkg/testing/containers"
)

var (
    // PostgreSQLVersion defines version to use
    PostgreSQLVersion = "10"
)

// PostgreSQLContainer represents database container handler
type postgreSQLContainer struct {
    Name     string
    pool     *dockertest.Pool
    resource *dockertest.Resource
    config   *Configuration
}

// NewPostgresContainer initialize a PostgreSQL server in a docker container
func newPostgresContainer(pool *dockertest.Pool) *postgreSQLContainer {

    var (
        databaseName = fmt.Sprintf("test-%s", uniuri.NewLen(8))
        databaseUser = fmt.Sprintf("user-%s", uniuri.NewLen(8))
        password     = uniuri.NewLen(32)
    )

    // Initialize a PostgreSQL server
    resource, err := pool.Run("postgres", PostgreSQLVersion, []string{
        fmt.Sprintf("POSTGRES_PASSWORD=%s", password),
        fmt.Sprintf("POSTGRES_DB=%s", databaseName),
        fmt.Sprintf("POSTGRES_USER=%s", databaseUser),
    })
    if err != nil {
        log.Fatalf("Could not start resource: %s", err)
    }

    // Prepare connection string
    connectionString := fmt.Sprintf("postgres://%s:%s@localhost:%s/%s?sslmode=disable", databaseUser, password, resource.GetPort("5432/tcp"), databaseName)

    // Retrieve container name
    containerName := containers.GetName(resource)

    // Return container information
    return &postgreSQLContainer{
        Name:     containerName,
        pool:     pool,
        resource: resource,
        config: &Configuration{
            ConnectionString: connectionString,
            Password:         password,
            DatabaseName:     databaseName,
            DatabaseUser:     databaseUser,
        },
    }
}

// -------------------------------------------------------------------

// Close the container
func (container *postgreSQLContainer) Close() error {
    log.Printf("Postgres (%v): shutting down", container.Name)
    return container.pool.Purge(container.resource)
}

// Configuration return database settings
func (container *postgreSQLContainer) Configuration() *Configuration {
    return container.config
}

The connection builder that try to connect and build the database schema by running migrations.

// +build integration

package integration

import (
    "context"

    "github.com/pkg/errors"
    "go.uber.org/zap"

    "go.zenithar.org/pkg/log"
    "go.zenithar.org/pkg/testing/containers/database"
    "go.zenithar.org/spotigraph/internal/repositories/pkg/postgresql"
)

func postgreSQLConnection(ctx context.Context) (func(), error) {
    // Initialize connection and/or container
    conn, _, err := database.ConnectToPostgreSQL(ctx)
    if err != nil {
        return nil, errors.Wrap(err, "unable to initialize database server")
    }

    // Try to contact server
    if err = conn.Ping(); err != nil {
        return nil, errors.Wrap(err, "unable to contact database")
    }

    // Migrate schema
    n, err := postgresql.CreateSchemas(conn)
    if err != nil {
        return nil, errors.Wrap(err, "unable to initialize database schema")
    }

    // Log migration
    log.For(ctx).Info("Applyied migrations to database", zap.Int("level", n))

    // Build repositories
    userRepositories["postgresql"] = postgresql.NewUserRepository(nil, conn)

    // Return result
    return func() {
        log.SafeClose(conn, "unable to close connection")
    }, nil
}

Complete integration test suite could be found here - https://github.com/Zenithar/go-spotigraph/blob/master/internal/repositories/test/integration/main_test.go

Conclusion

At this point, you must be able to execute all test suites on your domain as persistence adapters. You should be able to create business service by using these adapters via the interface, NOT DIRECTLY via adapter instance. All persistence adapters are constraint by contract defined in the api.go, from the public side, all adapter user must fit and stick with this interface to be sure to have port and interface in the persistence layer. By doing this you will fit with Hexagonal architecture principles.

In the next post, we will start to prepare the service by declaring a protocol, and contracts to be used by dispatchers to communicate with business services.

References

How I write micro-service (part 2)

Thibault NORMAND — Thu, 06 Jun 2019 09:27:47 +0000

We are about to prepare a Golang project according to Clean and Hexagonal Architecture principles.

TL;DR - Code repository on Github - <https://github.com/Zenithar/go-spotigraph

Domain

In order to manipulate concepts, we need to implements them. Let's use Spotify agile terminology.

User is a system identity
- User has an id
- User has a principal to describe user private identity (email, LDAP dn, etc.)
Squad is a group of User working together on a product
- Squad has an id
- Squad has a label
- Squad has a Product Owner, referenced by an User:id
Guild is a group of User according to their affinities
- Guild has an id
- Guild has a label
- Guild has members, a collection of User:id
Tribe is a group of Squad working on common project
- Tribe has an id
- Tribe has a label
- Tribe has members, a collection of User:id
Chapter is a group of User according to their skill-sets
- Chapter has an id
- Chapter has a label
- Chapter has members, a collection of User:id
- Chapter has a Chapter Leader, referenced by an User:id

Core Services

I assume from here that you know how to work with Golang - https://tour.golang.org

First, I create an internal package used to create the complete project, which will be executed in dedicated runnable artifacts.

$ mkdir spotigraph
$ go mod init go.zenithar.org/spotigraph
$ mkdir internal

internal package will not be accessible from outside of the project package.

 + internal
   + helpers
     - id.go
     - password.go
     - time.go
   + models
     - user.go

Helpers

Helpers should contain additional functions for models, such as:

Password encoding, verification and policy upgrades (Algorithm deprecation)
ID generation and validation
Time function indirection for time drive tests

$ mkdir internal/helpers

Random ID Generation

For example, id.go should contain all logic used to generate and verify ID syntax.

package helpers

import (
    "github.com/dchest/uniuri"
    validation "github.com/go-ozzo/ozzo-validation"
    "github.com/go-ozzo/ozzo-validation/is"
)

// IDGeneratedLength defines the length of the id string
const IDGeneratedLength = 32

// IDGeneratorFunc returns a randomly generated string useable as identifier
var IDGeneratorFunc = func() string {
    return uniuri.NewLen(IDGeneratedLength)
}

// IDValidationRules describes identifier contract for syntaxic validation
var IDValidationRules = []validation.Rule{
    validation.Required,
    validation.Length(IDGeneratedLength, IDGeneratedLength),
    is.Alphanumeric,
}

I like to use ozzo-validation because it's easy to create composable validation checks

Distributed Sequencial Number Generation

Sequencial numbers are really difficult in a distributed environment, if so really want
a sequencial number from a counter, you only have 2 ways to do :

Unique source of truth (Sequence in SAME database)
Use one of sony/big/*snow*flake algorithm

Personnaly I use bigflake due to the fact that it could be used with more than 1024 concurrent generators, the result encoded in Base62 should be quite short.

You should have a look to my side project KornFlake

Principal hashing

Another example with principal handling, you don't need to store it as plain text, think about privacy, database leaks, nobody is perfect ... So you could hash the principal before storing it. It's not the database role to hash the principal, it's part of your requirements to be privacy compliant.

We don't have to store the real principal, just use the hash of it as proof. The hash will be computed on user login
and stored. If, no, when the database will leak on the Internet, it should be more difficult to reverse the principal
from database record (PrincipalHash), due to the fact that we use an additional salt (sort of) used server wide.

package helpers

import (
    "encoding/base64"

    "golang.org/x/crypto/blake2b"
)

// Default hash key for principal hash function, must be different in each environment.
var principalHashKey = []byte(`7EcP%Sm5=Wgoce5Sb"%[E.<&xG8t5soYU$CzdIMTgK@^4i(Zo|)LoDB'!g"R2]8$`)

// PrincipalHashFunc return the principal hashed using Blake2b keyed algorithm
var PrincipalHashFunc = func(principal string) string {
    // Prepare hasher
    hasher, err:= blake2b.New512(principalHashKey)
    if err!= nil {
        panic(err)
    }

    // Append principal
    _, err = hasher.Write([]byte(principal))
    if err!= nil {
        panic(err)
    }

    // Return base64 hash value of the principal hash
    return base64.RawStdEncoding.EncodeToString(hasher.Sum(nil))
}

// SetPrincipalHashKey used to set the key of hash function
func SetPrincipalHashKey(key []byte) {
    if len(key)!= 64 {
        panic("Principal hash key length must be 64bytes long.")
    }

    principalHashKey = key
}

principalHashKey is hard-coded for default behavior, you must call exported function SetPrincipalHashKey to update the hash key.

Don't use simple hash simple (without key) because in this case, the secret is only based on the hash algorithm you use!

In the special case, in which you need the real value of the principal, you must think first, "do I really need it ?" or "do I really use it ?".
For example, you need it to find the user in the table, think "No hash is sufficient", if the principal is an email and you decide
to use principal column as email storage, you will not be able to restore the real value from the hash. So the only thing you can do is to
encrypt the email BUT not in the same table, and maybe not handled by the same microservice.

You should split the Account model containing all system information (principal hash, password hash, dates) and Profile model in a
dedicated. Because you also have to introduce consent in Profile attributes usages with authorizations. Maybe Attribute Based Encryption
should do the work well, but today it is quite young.

The principalHashKey should be different for each service, but same for services that needs to work together, if you use hash value a
grouping key. If you use the real principal for external function calls, you could use different key.

This is like Pairwise Authentication Context for OIDC extention by Paypal. Real identity is only hold by the caller, and every consumer
service has their own internal identifier, to prevent database cross verification. But sometimes it's a needed feature.

Time indirection

In order to set time during tests, you could use time.Now function indirection by declaring an alias and use it everywhere.

package helpers

// TimeFunc is used for time based tests
var TimeFunc = time.Now

For more complex clock mocking, I advise you to consider github.com/jonboulle/clockwork.

Password

Password has to be carefully processed when you decide to store it.

More information about my password encoder nammed Butcher

The main with password storage is not only the secret and mystic method you use to hash it
but also time consideration and cpu work needed to process the hash, in order to prevent time-attack
password recovery.

In a life of software, also password encoding policy could change in time, algoritm deprecation,
higher security requirements. You have to consider this before starting to store password in your database.
I have made library to handle password storage and gives the ability to the library to temporary use
the "old" password policy encoding to verify the password, then produce the hash with the new policy.

In fact, you don't have to ask to all your users to change their password in case of password encoding
policy changes.

For paranoids, like me, you could also encrypt the hash result using a server-wide key, with Enveloppe Encryption
method, this will reduce to mostly null the chance to recover the password. Because you have an header in the string
that helps you to know which algorithm and settings are used to encode.

And for ultimate paranoids, like me also, you put password based derivation function in a dedicated distributed micro-service, with
service authentication and transport encryption obviously, with that only service wil know the pepper and salt in cleartext.
It's also helps you to distribute the load between service instances.

Memory could be allocated and wiped after cleartext password usages - Secure software enclave for storage of sensitive information in memory.

package helpers

import (
    "context"
    "fmt"
    "sync"

    "go.zenithar.org/pkg/log"
    "go.zenithar.org/butcher"
    "go.zenithar.org/butcher/hasher"
    "github.com/trustelem/zxcvbn"
)

var (
    once sync.Once

    // butcher hasher instance
    butch *butcher.Butcher

    // PasswordPepper is added to password derivation during hash in order to
    // prevent salt/password bruteforce
    // Do not change this!! It will makes all stored password unrecoverable
    pepperSeed = []byte("TuxV%AqKB0|gxjEB!vc~]T8Hf[q|xgS('3S<IEnqOv:jF&F8}+pur)N@DHYulF#")
)

const (
    // PasswordQualityScoreThreshold is the password quality threshold
    PasswordQualityScoreThreshold = 3
)

// Initialize butcher hasher
func init() {
    once.Do(func() {
        var err error
        if butch, err = butcher.New(
            butcher.WithAlgorithm(hasher.ScryptBlake2b512),
            butcher.WithPepper(pepperSeed),
        ); err!= nil {
            log.Bg().Fatal("Unable to initialize butcher hasher")
        }
    })
}

// SetPasswordPepperSeed used to set the peppering parameter for butcher
func SetPasswordPepperSeed(key []byte) {
    if len(key)!= 64 {
        panic("Password peppering seed length must be 64bytes long.")
    }

    pepperSeed = key
}

// PasswordHasherFunc is the function used for password hashing
var PasswordHasherFunc = func(password string) (string, error) {
    return butcher.Hash([]byte(password))
}

// PasswordVerifierFunc is the function used to check password hash besides the cleartext one
var PasswordVerifierFunc = func(hash string, cleartext string) (bool, error) {
    return butcher.Verify([]byte(hash), []byte(cleartext))
}

// CheckPasswordPolicyFunc is the function used to check password storage policy and
// call the assigner function to set the new encoded password using the updated 
// password storage policy, if applicable.
var CheckPasswordPolicyFunc = func(hash string, cleartext string, assigner func(string) error) (bool, error) {
    if butcher.NeedsUpgrade([]byte(hash)) {
        if err:= assigner(cleartext); err!= nil {
            return false, err
        }
    }
    return true, nil
}

// CheckPasswordQualityFunc is the function used to check password quality regarding security constraints
var CheckPasswordQualityFunc = func(cleartext string) error {
    quality:= zxcvbn.PasswordStrength(cleartext, nil)
    if quality.Score < PasswordQualityScoreThreshold {
        return fmt.Errorf("Password quality insufficient, try to complexify it (score: %d/%d)", quality.Score, PasswordQualityScoreThreshold)
    }
    return nil
}

I use zxvbn as password strength evaluator (github.com/trustelem/zxcvbn)

Models

$ mkdir internal/models

For example user.go is defined as follows:

package models

import (
    "fmt"
    "time"

    "go.zenithar.org/spotigraph/internal/helpers"

    validation "github.com/go-ozzo/ozzo-validation"
    "github.com/go-ozzo/ozzo-validation/is"
)

// User describes user attributes holder
type User struct {
    // Internal identifier generated with custom strategy
    ID                 string    `bson:"user_id" db:"id"`
    // Principal must be a valid external principal value (Email, LDAP DN, etc.)
    Principal          string    `bson:"principal" db:"principal"`
    // Create date of the current user
    Created            time.Time `bson:"created" db:"created"`
    // PasswordModifiedAt is the date when the secret was modified
    PasswordModifiedAt time.Time `bson:"password_modified_at" db:"password_modified_at"`
    // Secret is the password cryptographic hash
    Secret             string    `bson:"secret" db:"secret"`
}

// NewUser returns an user instance
func NewUser(principal string) *User {
    return &User{
        // Generate an identity from ID helper
        ID:        helpers.IDGeneratorFunc(),
        // Hash the given principal using helper
        Principal: helpers.PrincipalHashFunc(principal),
        // Set the creation date using the time function helper
        Created:   helpers.TimeFunc(),
    }
}

Password management

Password is not persistence adapter specific, it's an entity attribute of the User model.

// SetPassword updates the password hash of the given account
func (u *User) SetPassword(password string) (err error) {
    // Check password quality
    err = helpers.CheckPasswordQualityFunc(password)
    if err!= nil {
        return err
    }

    // Generate password hash
    u.Secret, err = helpers.PasswordHasherFunc(password)
    if err!= nil {
        return err
    }

    // Update last modified date
    u.PasswordModifiedAt = helpers.TimeFunc()

    // No error
    return nil
}

// VerifyPassword checks if given password matches the hash
func (u *User) VerifyPassword(password string) (bool, error) {
    // Validate password hash using constant time comparison
    valid, err:= helpers.PasswordVerifierFunc(u.Secret, password)
    if!valid || err!= nil {
        return false, err
    }

    // Check if password need upgrades
    return helpers.CheckPasswordPolicyFunc(u.PasswordHash, password, func(pwd string) error {
        return u.SetPassword(pwd)
    })
}

SetPassword is used to update password hash using the password helpers, the given password is evaluated according to password complexity policy, hashed using scrypt-blake2b-512 algorithm.
VerifyPassword is used to verify the given clear-text password with the local encoded one. If the password encoding strategy changed since the password storage, the password is verified using the last password encoding strategy, then updated to latest one if password match. If the encoded password is modified, User:passwordHash will be updated using the callback.

The password encoding update strategy is mandatory if you want to be able to update the password encoding policy without asking everyone to change their password.

Validation

For User validation, let's implement a Validate() error function using ozzo-validation.

// Validate entity contraints
func (u *User) Validate() error {
    return validation.ValidateStruct(u,
        // User must have a valid id
        validation.Field(&u.ID, helpers.IDValidationRules...),
        // User must have a principal with valid printable ASCII characters as value
        validation.Field(&u.Principal, validation.Required, is.PrintableASCII),
    )
}

This method will be called from persistence adapters on creation, and update requests

By adding logic in models, you must add unit tests to check for specification compliance.

package models_test

import (
    "testing"

    . "github.com/onsi/gomega"

    "go.zenithar.org/spotigraph/internal/models"
)

func TestUserValidation(t *testing.T) {

    // Prepare the testscase list
    tcl := []struct {
        name      string
        principal string
        expectErr bool
    }{
        // Failed testcases
        {"blank principal", "", true},
        {"1234", "1234", true},

        // Valid testcases
        {"valid email", "toto@foo.com", false},
    }

    for _, tc := range tcl {
        t.Run(tc.name, func(t *testing.T) {
            // Flag test to be run in parallel
            t.Parallel()

            // Matcher library
            g:= NewGomegaWithT(t)

            // Initialize object to be tested
            underTest:= models.NewUser(f.principal)
            g.Expect(underTest).ToNot(BeNil(), "Entity should not be nil")

            // Function to test
            if err := underTest.Validate(); err!= nil {
                // Error occurs, check if not expected
                if !f.expectErr {
                    t.Errorf("Validation error should not be raised, %v raised", err)
                }
            } else {
                // No error, check if expected
                if f.expectErr {
                    t.Error("Validation error should be raised")
                }
            }
        })
    }
}

Note the table driven test pattern very useful to decorelate test data from test cases (https://dave.cheney.net/2019/05/07/prefer-table-driven-tests).

Conclusion

At this point, you should have defined all your domain entities, what your project is going to handle.
Don't think yet as "How I could store it in my database" because database model could differ from Entities, so create your
domain first as a plain object tree then apply repository patterns.

In the next post, we will see how I implement persistence adapters using models.

References

How I write micro-service (part 1)

Thibault NORMAND — Thu, 06 Jun 2019 09:23:25 +0000

During my software developer experience, I have seen many bad practices as a code reviewer and collected a lot of tips to try to build better products. I want to focus on a specific part in order to prepare a enhanced Golang project template merging all these tips and practices.

What is a micro-service (again)?

My own personal definition

A micro-service is an internal autonomous observable immutable scalable self-contained unit of deployment of a non-monolithic style architecture. Each micro-service is responsible partially or completely of a part of a business problem, orchestrated and exposed by a Service Gateway.

A micro-service doesn't means micro "code" infrastructure, because if you consider a micro-service like a microcode, you will be in a world of pico-service where each service is designed to run roughly one Assembly opcode. I'm hearing tech early-adop-hipsters while writing: "Oh dude, such a good idea, we could have a distributed multi-architecture assembly services that could run on Kubernetes".

An opcode is an instruction executed by your CPU

Consider micro-services as deployment unit, built and running to achieve a more complex service. Responsibilities have been shared for deployment reasons (heavy load, sometimes re-usability). So don't design micro-services without a full vision of your business service. I'm not saying to rollback your code to that gorgeous monolith, but consider that splitting it in micro-service should be driven by technical requirements (load balancing, etc.), not just for fancy hype reasons.

Micro-services add network between problems!

And all micro-services are part of a micro-service style architecture that serve an identified target to reach.

Splitting them and contact them via an external transport, just add external transport problems to the business problems that you are trying to solve (Connection Resiliency / Explosion, Distributed Concurrency, etc.)

Once again, you must know what to do BEFORE how to do it!

Architectural patterns

Clean code is not written by following a set of rules. You don’t become a software craftsman by learning a list of heuristics. Professionalism and craftsmanship come from values that drive disciplines. Robert C. Martin, Clean Code: A Handbook of Agile Software Craftsmanship

Architecture patterns are frameworks in which, you must understand all concepts, but use only what you need.

In software development, architecture should not be the most "hot" point to solve first, you should be focused on features to add value to your product. So that in order to save times, many architectural patterns exist to be able to think about "what I want to do", instead of "how am I going to organize my code".

This is the main role of the software architect to balance between adding features and keep code maintainable in time.

Clean Architecture

The "Uncle Bob" Clean Architecture (Robert C. Martin) is an architectural toolbox to describes and organize software component concepts and how they are communicating between them.

The main objective of this architecture pattern is to design software with very low coupling, independent of technical implementation and fully testable.

Entities, contains no business logic only intrinsic validation;
Use Cases, contains business logic completely independent of technical implementation;
Interface Adapters, contains bridges between external world via presenter (HTTP, gRPC, Subscriber) and User Cases;
Frameworks & Drivers, contains all technical implementations (Database repositories, Security controls, etc.)

All layers are bound by the dependency rule. The dependency rule tells us that inner layers should not depend on outer layers. That is, our business logic and application logic should not be pegged to the Presenter, UI, databases, or other elements. Nothing from an external layer can be mentioned by the code of an internal layer.

Hexagonal Architecture

Allow an application to equally be driven by users, programs, automated test or batch scripts, and to be developed and tested in isolation from its eventual run-time devices and databases.

The main objective of this architecture is to isolate business services and rules from technical environment using ports and adapters.

When you starts a project, you have to make technical choice that could reveal, in time, to be a bad choice (project death, license changes, etc.), when not isolating these implementation from your code, you will not be able to switch quickly to a new implementation.

For example, you want to build software using Function as a service pattern, but you stick invocation with AWS Lambda, using internally AWS service objects, when you would like to migrate to GCP or other, your code will be completely and tightly coupled to AWS instead of having defined technically independent business logic and provided an AWS Lambda adapter.

By isolating business from technical dependency, you can test by using mock and stub. You can also start your project without waiting for completion of all parts, just contract (adapters) are needed. You can also start testing your code without deploying the full stack, no need to wait for final artifact to be available and built. You can test each feature, layer by layer.

By isolating your project from infrastructure, you can have a quick demonstration product; you can run your product as a simple HTTP server in a docker container by changing an adapter, then deploy as a Lambda on AWS as production target. Don't think that using AWS to test your code is mandatory as unit test level, even as integration level, you don't have to test AWS Lamda invocation but your own code.

By isolating persistence, you can produce stub repositories to simulate data providers, in order to create the first PoC (for example) and mocking business service.

Hexagonal architecture products can communicate between them using adapters. Think about micro-service architecture style, it's completely applicable, if you consider each micro-service as its own hexagonal architecture product with transport communication layer as an adapter between them (HTTP, gRPC, Pub/Sub).

Consider architecture patterns as toolboxes or guidelines, never as source of truth to apply them in any situation.

Conclusion

I know, we have to work faster every day and quality (also security) is the first feature to be dropped. By being focused on business problems, and use architectural patterns as tools, you will have free time, that is generally spent by reinventing the wheel, again, and again, to produce demonstrable value and makes happy maintainers and auditors.

Consider the maintainer as a complete psychopath who knows where you live and want to kill you ... while you are writing code.

References

Original post from my blog

DEV Community: Thibault NORMAND

Bridging Security and Reliability

Executive summary

1. The problem: two disciplines, one degradation model

2. Thesis: model degradation from the user journey outward

3. Principles for integrating Security and Reliability

4. Critical User Journeys as the shared anchor

5. RAMSS: a coverage model for dependability

6. Worked example: completing a payment in a shop

7. Reliability-adapted PASTA

8. Incident replay as reliability threat modelling

9. Intended outcomes

10. What this means for resilience

11. Recommended pilot

12. Scaling without creating process drag

Conclusion

The Identity Puzzle: the Crucial Difference Between Access Tokens and ID Tokens

In the real world

ID vs Access

How are authenticated ID cards?

How are authenticated ID tokens?

How are authenticated Access tokens?

Why is using ID tokens as proof of authorisation, not a good idea?

Conclusion

More about this topic

How I write micro-service (part 3)

Adapter Contract

Unit Tests

Integration Tests

Persistence implementations

PostgreSQL

Schema migrations handling

User Persistence Adapter

Squad Persistence Adapter

MongoDB

Remote Adapter

Running integration tests

Conclusion

References

How I write micro-service (part 2)

Domain

Core Services

Helpers

Random ID Generation

Distributed Sequencial Number Generation

Principal hashing

Time indirection

Password

Models

Password management

Validation

Conclusion

References

How I write micro-service (part 1)

What is a micro-service (again)?

My own personal definition

Architectural patterns

Clean Architecture

Hexagonal Architecture

Conclusion

References